Navigation system and navigation method suitable for visually impaired user

Abstract

The invention relates to a navigation system and method for visually impaired users. The navigation system comprises: a navigation module for deriving a navigation path from a starting position to a preset destination; the guiding vehicle is used for guiding the visually impaired user to move to a preset destination based on the exported navigation path, and further providing navigation guidance for the visually impaired user through movement; the navigation module is further configured to, in response to detecting an obstacle during navigation of the guided vehicle, adjust a navigation path of the guided vehicle so as to avoid the detected obstacle while following the navigation path.

Description

Technical Field

[0001] The present disclosure relates to a navigation system for a user and a method for navigating a user, and in particular, but not exclusively, to a navigation system suitable for a visually impaired user and a method for navigating a visually impaired user. Background Art

[0002] Traditional road guidance tools for the visually impaired include canes and guide dogs to provide navigation and guidance for the visually impaired. Such tools can improve navigation for the visually impaired, but can be difficult to use or, in the case of guide dogs, require high training and maintenance costs. Summary of the invention

[0003] According to a first aspect of the present disclosure, a navigation system is provided, the navigation system being used for navigating a visually impaired user, the navigation system comprising:

[0004] A navigation module, configured to derive a navigation path from a starting location to a preset destination; and

[0005] a guiding vehicle, configured to guide the visually impaired user to move toward the preset destination based on the derived navigation path, thereby providing navigation guidance to the visually impaired user through the movement;

[0006] The navigation module is further configured to adjust the navigation path of the guide vehicle in response to detecting an obstacle during navigation of the guide vehicle so as to avoid the detected obstacle while following the navigation path.

[0007] According to a first aspect, the navigation module is configured to detect one or more dimensions of the obstacle adjacent to the guide vehicle.

[0008] According to a first aspect, the navigation module is used to determine a safety zone in the absence of the detected obstacle.

[0009] According to the first aspect, the navigation system further comprises: an obstacle sensing module for detecting the obstacle present in the derived navigation path during the navigation, and the navigation module is further configured to receive a signal associated with the detection of the obstacle from the obstacle sensing module.

[0010] According to a first aspect, the obstacle sensing module comprises a depth camera for capturing a three-dimensional image from the perspective of the guide vehicle.

[0011] According to the first aspect, the obstacle sensing module further comprises a laser radar (LiDAR) for capturing a full-angle plan view representing the position of the obstacle relative to the guide vehicle.

[0012] According to the first aspect, the obstacle sensing module is movable together with the guide vehicle.

[0013] According to the first aspect, the navigation system further comprises: a user control module for receiving an input associated with the preset destination.

[0014] According to the first aspect, the navigation system further comprises: a signal communication module for transmitting signals between the navigation module and the user control module embedded in the handheld device.

[0015] According to a first aspect, the navigation path is derived based on a pre-recorded navigation path in a storage device.

[0016] According to a first aspect, the guide vehicle comprises a base movable by one or more wheels, each of the wheels being driven by a respective servomotor.

[0017] According to the first aspect, the navigation system further comprises: a mileage sensor for determining the position of each of the wheels, and the navigation module is further configured to determine the current position of the guide vehicle relative to the preset destination.

[0018] According to the first aspect, the guided vehicle further comprises a handle portion extending from the base, whereby the visually impaired user makes physical contact with the guide vehicle via the handle portion in use.

[0019] According to the first aspect, the handle portion further comprises a touch sensing unit for receiving input associated with an instruction of the visually impaired user.

[0020] According to the first aspect, the handle portion further comprises a vibration member for providing at least one of tactile feedback and audio feedback representing an event to the visually impaired user through the physical contact between the visually impaired user and the guide vehicle.

[0021] According to a second aspect of the present disclosure, a method for navigating a visually impaired user is provided. The method comprises the following steps:

[0022] deriving a navigation path for guiding the vehicle based on the one or more parameters;

[0023] Based on the derived navigation path, the guide vehicle guides the visually impaired user to move toward a preset destination, thereby providing navigation guidance to the visually impaired user through the movement;

[0024] receiving, during navigation of the guide vehicle, another signal associated with detection of an obstacle; and

[0025] In response to detecting the obstacle during navigation of the lead vehicle, a navigation path of the lead vehicle is adjusted to avoid the detected obstacle while following the navigation path.

[0026] According to the second aspect, the method further comprises the following steps:

[0027] receiving another signal associated with detection of another obstacle during navigation of the lead vehicle based on the adjusted navigation path of the lead vehicle; and

[0028] In response to the signal associated with the detected obstacle during navigation of the lead vehicle, the adjusted navigation path of the lead vehicle is further adjusted to avoid the detected another obstacle.

[0029] According to the second aspect, the method further comprises: recording the actual navigation path of the lead vehicle.

[0030] According to a second aspect, the navigation path is derived based on an actual navigation path recorded in a previous navigation.

[0031] According to the second aspect, the method further comprises presenting at least one of tactile feedback and audible feedback representing an event to the visually impaired user through the physical contact between the visually impaired user and the guide vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Embodiments of the present disclosure will be described below by way of example with reference to the accompanying drawings, in which:

[0033] Figure 1 is a block diagram of a navigation system according to an embodiment of the present disclosure;

[0034] Figure 2A A top perspective view of a guide vehicle or guide robot of a navigation system according to an embodiment of the present disclosure is shown;

[0035] Figure 2B A bottom perspective view of a guide vehicle or guide robot of a navigation system according to an embodiment of the present disclosure is shown;

[0036] Figure 3 To show Figure 1 A schematic diagram of the workflow of the navigation system;

[0037] Figure 4 To show Figure 1 Schematic diagram of the operational workflow of the navigation system, wherein a multimodal sensing priority management and processing algorithm is implemented;

[0038] Figure 5is a block diagram of a navigation system according to an optional embodiment of the present disclosure; and

[0039] Figure 6 To show Figure 5 Schematic diagram of the operational workflow of the navigation system, in which the multimodal sensing priority management and processing algorithms are implemented. DETAILED DESCRIPTION

[0040] The inventors have designed a safer walking guidance system for visually impaired people that combines the function of a guide dog with a user's target destination. The system includes a navigation control module and a small guide vehicle or guide robot for avoiding obstacles and executing walking commands. Therefore, this new road guidance tool can improve the convenience of visually impaired users. In this article, the term "visually impaired person or user" can include people with visual impairment, partial visual impairment, or people with one or more various sensory impairments or cognitive impairments that prevent them from visually processing, understanding, or interacting with their environment. These people may have visual impairments, hearing impairments, sensory impairments, or physical or mental impairments or diseases, which may prevent them from moving around in their surroundings.

[0041] refer to Figure 1 , shows an embodiment of a navigation system 10 suitable for visually impaired users. The system 10 includes: a navigation module 110, which is used to derive a navigation path from a starting position to a preset destination; and a guide vehicle 200, which is used to guide the visually impaired user to move to the preset destination based on the derived navigation path, thereby providing navigation guidance to the visually impaired user through movement. The navigation module 110 is also used to adjust the navigation path of the guide vehicle 200 in response to detecting an obstacle during navigation of the guide vehicle 200, so as to avoid the detected obstacle in the process of following the navigation path.

[0042] As used herein, the term "obstacle" includes any type of obstacle, such as, but not limited to, a person, an animal, a plant, an object with a regular or irregular shape, an area or terrain, an elevated object, a wall, a pit or a visually impaired user that may obstruct the navigation path of the guided vehicle. The term "obstacle" also includes any stationary and moving obstacles.

[0043] like Figure 1 , a schematic diagram of a navigation system 10 and a user 12 (e.g., a visually impaired user) interacting with the system 10 is shown. The navigation system 10 may be implemented as a guide vehicle 200 in which a computing device 100 is embedded. In addition, an additional computing device 30 is provided in signal communication with the guide vehicle 200 for transmitting one or more inputs associated with destination information to the navigation system 10.

[0044] refer to Figure 1Further details of the overall architecture of the navigation system 10 according to an exemplary embodiment of the present disclosure are obtained. In essence, the computing device 100 includes a navigation module 110 for providing navigation directions to the guide vehicle 200. The navigation module 110 can calculate a navigation path from a current location to a destination and control the movement of the guide vehicle 200 toward the destination. The navigation module 110 can also control the guide vehicle 200 in real time to avoid obstacles in front of the guide vehicle 200.

[0045] Preferably, the navigation module 110 is embedded in the guide vehicle 200 and can move with the guide vehicle 200. The navigation module 110 includes an obstacle avoidance and navigation control module (NCM) for processing sensor data associated with obstacles around the guide vehicle 200 and determining a steering angle and a speed of the guide vehicle 200. For example, the navigation module 110 also includes a computing module 120, a light detection and ranging (LiDAR) module 130, a depth camera 140, and a mileage sensor 150.

[0046] The computing module 120 includes appropriate components necessary to receive, store and execute appropriate computer instructions. These components may include: a processing unit, including a central processing unit (CPU), a math co-processing unit (Math Co-processing Unit, Math Processor), a graphics processing unit (Graphics Processing Unit, GPU) or a tensor processing unit (TPU) for tensor or multi-dimensional array calculation or manipulation operations; read-only memory (ROM); random access memory (RAM); input / output (I / O) devices, such as disk drives); and user interfaces (such as keyboards or touch screens). The processing unit can be a single processor to provide the combined functionality of multiple processors. In this example embodiment, the computing module 120 is configured to receive data associated with the device 10 and the environment measured by an external sensing unit.

[0047] The obstacle avoidance and navigation control module (NCM) 110 may also include one or more sensing units to capture multiple images or capture video streams within a predetermined time period. Preferably, the sensing unit may be mounted on the guide vehicle 200 so that the sensing unit can capture images or videos of objects near the guide vehicle 200. The sensing unit is used to communicate signals with the processing unit of the computing module 120, so that the computing module 120 is configured to receive the recorded images or videos from the sensing unit and process the images or videos in real time.

[0048] For example, an optical measurement module 130 is provided, which is operable to generate a virtual map representing obstacles in the vicinity of the guide vehicle 200. In addition, an image capture module 140 is provided, which is operable to capture images of distant objects relative to the guide vehicle 200.

[0049] In an exemplary embodiment, the optical measurement module 130 may be a 2D LiDAR sensing unit 130 that measures the distance to an obstacle by irradiating the obstacle with a laser and analyzing the reflected laser. For example, the LiDAR sensing unit 130 is used to emit a laser beam with pulses based on a rotational motion, and collects the horizontal distance to the obstacle to retrieve data for the X-axis and Y-axis. In this way, a full-angle 360° plane view of the guided vehicle 200 relative to the vertical Z axis is obtained. Subsequently, the 360° plane view is processed by the computing module 120 to detect adjacent obstacles and walls. Therefore, the computing module 120 can process the 360° data to plan a safe path for guiding the vehicle 200 to travel.

[0050] In an example embodiment, the image capture module 140 may be a depth camera 140. For example, the depth camera 140 is used to capture a front 3D view of the guide vehicle 200 in order to measure the distance between the guide vehicle 200 and one or more obstacles in front. The front 3D view is then processed by the computing module 120 to detect objects with irregular shapes and determine a safe zone in front of the guide vehicle 200 to travel. In addition, the computing module 120 may also determine the height of overhead objects and avoid obstacles at head height.

[0051] In addition, the obstacle avoidance and navigation control module (NCM) 110 may also include an odometer module, which includes one or more odometer sensors 150 to detect the rotational displacement of the wheels of the guide vehicle 200. For example, the odometer sensor 150 may include one or more encoder sensors for detecting the rotation rate of one or more wheels of the guide vehicle 200.

[0052] For example, an encoder may track the number or rotations traveled by each wheel of the lead vehicle 200 and send the data to the computing module 120. Preferably, the mileage module may communicate with one or more encoder sensors disposed on each drive motor to determine the distance and direction traveled by each wheel, and in turn transmit the rotational distance and rotational direction of each wheel to the computing module 120. By calculating the motor rotations, the gearbox ratio (i.e., the transmission ratio), and the outer diameter of the wheel, the computing module 120 may process and determine the rotational distance traveled by each wheel and the position of the lead vehicle 200 relative to the origin and destination.

[0053] Advantageously, the computing module 120 can calibrate the data obtained from the various sensing units and detect differences therebetween. For example, due to an uneven surface, the wheels of the guide vehicle 200 have traveled an additional rotation distance, and the mileage sensor 150 can track that the guide vehicle 200 has traveled to a specific coordinate on the virtual map that is located at a distance substantially away from the coordinates and real-time navigation information obtained from the LiDAR sensing unit 130 and the depth camera 140. The computing module 120 can determine that the coordinates of the mileage sensor 150 are abnormal and inconsistent with the navigation information from the LiDAR sensing unit 130 and the depth camera 140. The computing module 120 can in turn correct the error caused by the mileage sensor 150 and calibrate the current position of the guide vehicle 200.

[0054] The computing device 100 may also include: other input devices, such as an Ethernet port, a USB port, etc.; a display, such as a liquid crystal display, a light-emitting display, or any other suitable display; and a communication link (i.e., a communication interface). For example, the obstacle avoidance and navigation control module (NCM) 110 may further include a microcontroller (MCU) 160 for signal communication with the handle portion on the guide vehicle 200.

[0055] The computing module 120 may include instructions that may be stored in ROM, RAM, or a disk drive and executed by a processing unit. One or more communication interfaces (i.e., one or more communication links) may be provided, which may connect one or more computing devices, such as servers, personal computers, terminals, wireless or handheld computing devices, Internet of Things (IoT) devices, smart devices, and edge computing devices in different ways. At least one of the multiple communication links may be connected to an external computing network via a telephone line or other type of communication link. The communication interface is configured to allow data communication over any suitable communication network using any suitable protocol (e.g., Wi-Fi, Bluetooth, 4G, 5G, or any other suitable protocol).

[0056] In an exemplary embodiment, the communication interface may be a communication module (CM) 170 for communicating with the obstacle avoidance and navigation control module (NCM) 110. The communication module (CM) 170 is also used to communicate with the additional computing device 30. Therefore, the communication module (CM) 170 is a bridge between the computing device 30 and the obstacle avoidance and navigation control module (NCM) 110.

[0057] For example, the communication module (CM) 170 may include two key units: a microcontroller (MCU) 180 and a Bluetooth (BLE) communication module 190. The microcontroller (MCU) 180 is used to control and coordinate with the obstacle avoidance and navigation control module (NCM) 110, while the BLE module 190 is used to handle Bluetooth (BLE) communication with the computing device 30. Specifically, the MCU 180 controls the sleep and wake-up cycles for communicating with various modules. The BLE data received from each module will be processed to generate a communication protocol and provide an interface to handle events from the obstacle avoidance and navigation control module (NCM) 110. The BLE communication module 190 is used to communicate between the computing device 30 and the obstacle avoidance and navigation control module (NCM) 110. The BLE communication module 190 processes the BLE data packets to the microcontroller 180.

[0058] In an example embodiment, the computing device 30 may be a handheld computing device such as a tablet, a smart phone, a laptop, or other personal computing device. The computing device 30 may execute an application program (App) to implement various functions defined by the application program. Specifically, the computing device 30 includes a software application program (i.e., App) stored in a memory unit (e.g., ROM 104 or RAM 106 or another memory unit). The software application program includes computer-readable and executable instructions. The computing device 30 is configured to execute the instructions so that the processor performs one or more functions defined in the instructions. For example, the application program may control the processor to provide one or more navigation path planning functions.

[0059] refer to Figure 1 In an exemplary embodiment of the present invention, the computing device 30 is a handheld computing device, such as a smart phone pre-installed with a navigation mobile app, for accessing the navigation system 10 once executed. The smart phone 30 includes a location database 32 storing map data (e.g., location information of multiple destinations on a virtual map). For example, the location database 32 may store a pre-built map and multiple destination IDs associated with one or more corresponding locations on the pre-built map. The smart phone 30 also includes a BLE communication module for communicating with the BLE communication module 190 of the computing device 100.

[0060] Preferably, the smartphone 30 may also include a sound sensing unit 34, such as a microphone that captures sound data associated with the interaction between the user and the device 10. The smartphone 30 may also include a speaker unit for providing audible information to the user.

[0061] In use, the smart phone 30 is paired with the guide vehicle 200 through the Bluetooth connection with the communication module (CM) 170 when running the navigation mobile application, and the user can execute the software application to access the system 10 and input one or more commands through the software application. For example, the user can input the target destination by voice, and the smart phone 30 converts the input voice into a destination ID and sends the corresponding destination ID to the guide vehicle 200 through the NCM 110. Once the guide vehicle 200 arrives at the destination, the smart phone 30 can remind the user through the sound of the speaker unit or the vibration on the handle portion of the guide vehicle 200.

[0062] Reference Figure 2A to 2B , showing Figure 1 An exemplary embodiment of a navigation system 10 is shown in which at least some electronic components, such as NCM 110, are embedded as part of a guided vehicle or guided robot 200.

[0063] In this embodiment, the guide robot 200 can be a guide vehicle or a guide dog. The guide vehicle 200 includes two basic parts: a base 210 and a handle 220 extending from the base 210. Preferably, a plurality of sensors for object avoidance and scene analysis are provided on the base 210, and one or more user communication interfaces are provided on the handle 220. Preferably, the lidar module 130 and the depth camera 140 are externally located on the housing of the base 210 and exposed to the environment. The lidar module 130 and the depth camera 140 are each located at an elevated position relative to the ground without being visually obstructed. The depth camera 140 is located at a further elevated position relative to the lidar module 130.

[0064] The LiDAR guide vehicle 200 has a body 210 having a pair of front wheels 212a and 212b and a pair of rear wheels 212c and 212d, each of which is operably connected to the body 210 to drive the guide vehicle 200 along a surface. The front wheels 212a and 212b and the rear wheels 212c and 212d are driven by respective servo motors 214, so that the four wheels 212a, 212b, 212c, and 212d can be steered at different angles to turn around corners or obstacles.

[0065] Preferably, one or more gearbox devices (not shown) are further provided between the wheels 212a, 212b, 212c and 212d and the servo motor 214 for mechanically transmitting the rotational force from the servo motor 214 to the wheels 212a, 212b, 212c and 212d. Optionally, the front wheels 212a and 212b may be mechanically connected to a first gearbox, while the rear wheels may be mechanically connected to a second gearbox for operating different gearbox ratios.

[0066] The guide vehicle 200 includes a battery module 216 for providing power to the servo motor 214. When the servo motor 214 is powered by the battery module 216, for example, upon receiving a command from a user via the handle 220, the servo motor 214 will rotate and thereby actuate the wheels 212a, 212b, 212c and 212d through the gearbox.

[0067] Preferably, one or more brakes (not shown) are also provided to interrupt the rotation of each wheel 212a, 212b, 212c, and 212d in the event of an emergency. For example, if the computing module 120 determines that the distance between the lead vehicle 200 and the approaching obstacle exceeds a predetermined threshold and it is not possible to avoid the collision only by turning at different angles and gradually decelerating, the wheels 212a, 212b, 212c, and 212d can be stopped immediately by the brakes to avoid hitting the obstacle.

[0068] Referring again to FIG. 2, there is shown Figure 1 1 is an example of a mileage sensor 150 for implementation in the guide vehicle 200. In this exemplary embodiment, the mileage sensor 150 is implemented into each servo motor 214 that actuates the respective front and rear wheels 212a, 212b, 212c, and 212d of the guide vehicle 200. The mileage sensor 150 is used to measure the number of revolutions of the front wheels 212a and the rear wheels 212b, 212c, and 212d, and the mileage sensor 150 is implemented to operate with the front wheels 212a, 212b, 212c, and 212d. In turn, the processing unit of the calculation module 120 can estimate the distance traveled by the guide vehicle 200 based on the detected number of revolutions of each respective wheel 212a, 212b, 212c, and 212d, the corresponding gear ratio of the gear device, and the circumference of the wheels 212a, 212b, 212c, and 212d.

[0069] In an exemplary embodiment, the mileage sensor 150 includes an encoder sensor for accurately determining the rotational position of the servo motor 214 and determining the relative movement of the wheels 212a, 212b, 212c, and 212d. For example, one or more encoders can be disposed on the motor housing of each servo motor 214, and a magnetic component can be disposed on the shaft of the servo motor 214, preferably adjacent to the encoder sensor. Thus, the encoder sensor can determine the angular movement of the adjacent magnetic component.

[0070] In addition, the guide vehicle 200 can also turn at different angles to turn around corners or obstacles. To achieve this, in at least one pair of wheels 212a, 212b and 212c, 212d, the opposing left and right wheels can be driven by two separate servo motors 214 so that the opposing wheels can rotate at different speeds or in opposite directions. Therefore, the computing module 210 can determine the direction and turning speed of the guide vehicle 200 along the surface based on the difference in rotation speed measured by each odometer sensor 150.

[0071] In this exemplary embodiment, the guide vehicle 200 also includes a handle 220 that can be held by the user so that the guide vehicle 200 can navigate and guide the user to move from one location to another. Preferably, the handle 220 can also be used to allow the user to provide travel instructions to the guide vehicle 200 so as to travel a predetermined path and / or provide the user with information associated with the detection of obstacles.

[0072] The handle 220 may also include a holding portion 222 embedded with a touch sensor or button 224. The handle 220 may include a vibration motor to provide tactile and audio feedback to generate an alarm and serve as a control panel to give commands such as start and stop. When an alarm is received from the holding portion 222, the user may tap the touch sensor or button 224 to send an instruction to the guide vehicle 200. For example, a single tap by the user may indicate confirmation of the request by the microcontroller (MCU) 160, while a quick double tap by the user may indicate cancellation of the request by the microcontroller (MCU) 160.

[0073] Preferably, the handle 220 may further include an elongated shaft 226, so that from an ergonomic point of view, the holding portion 222 becomes easier to reach by a visually impaired user. Optionally, the elongated shaft 226 is retractable, so that the elevated position of the holder portion 222 is adjustable to accommodate users with different heights.

[0074] The navigation system 10 according to an exemplary embodiment of the present disclosure can operate in at least two different operation modes: navigation mode ( Figure 3 and 4 ) and training mode, i.e. path following mode ( Figure 6 ). The navigation mode can navigate the user to a preset destination, while the training mode can train new users to learn how to use the road guide vehicle or guide robot of the present invention.

[0075] refer to Figure 3, shows an embodiment of a method 300 for navigating a visually impaired user 12. The method 300 includes the following steps: deriving a navigation path for a guide vehicle 200 based on one or more parameters; guiding the visually impaired user 12 to move toward a preset destination based on the derived navigation path by the guide vehicle 200, thereby providing navigation guidance to the visually impaired user 12 through movement; receiving another signal associated with the detection of an obstacle during navigation of the guide vehicle 200; and in response to detecting an obstacle during navigation of the guide vehicle 200, adjusting the navigation path of the guide vehicle 200 so as to avoid the detected obstacle while following the navigation path.

[0076] Reference Figure 3 and 4 , further describing the operating mode of an exemplary embodiment of system 10.

[0077] In navigation mode, Figure 1 As shown, the user can input a preset destination from a mobile app pre-installed on the smart phone 30. The user can input the destination through the user interface of the mobile app or by voice. The mobile app is connected to the guide vehicle 200 via Bluetooth communication. The destination will be sent to the guide vehicle 200, and the guide vehicle 200 will move to the destination along the planned navigation path.

[0078] refer to Figure 3, the method 300 starts with step 310. Step 310 includes: inputting a destination ID by a user through the smart phone 30. The guide vehicle 200 receives the destination and waits for a user input signal from the handle 220 for confirmation. Upon receiving the confirmation signal, the guide vehicle 200 will find the destination point on the pre-built map. Step 320 includes: deriving a navigation path for the guide vehicle 200 based on one or more parameters. If the destination is found on the map, the guide vehicle 200 will plan a navigation path to the destination. Step 330 includes: reminding the user through a touch sensor or button 224 on the handle 220. If the destination is not found on the map, the guide vehicle 200 will trigger a vibration motor on the holding portion 222 to remind the user. Step 340 includes: detecting one or more obstacles during navigation of the guide vehicle 200. If an obstacle is detected on the navigation path, the guide vehicle 200 will repeat step 320 and re-plan the path to avoid the obstacle. Otherwise, the guide vehicle 200 will move along the path and reach the destination. Step 350 includes determining the current position of the guide vehicle 200 relative to the preset destination. If the guide vehicle 200 reaches the destination, the guide vehicle 200 will trigger the vibration motor on the holding portion 222 to alert the user and report the arrival through the mobile app (step 330). Otherwise, the guide vehicle 200 will repeat step 320 and plan another path to the destination again.

[0079] In some scenarios, the planned navigation path is too ideal and obstacles do not exist in the virtual map. In other scenarios, the planned navigation path is not realistic and must be refined in time to avoid any collision with obstacles in the real world.

[0080] In one embodiment, an object avoidance and scene analysis algorithm is provided. The pathfinding solution includes a vision module for object detection, obstacle avoidance, and path planning. Initially, a frontal 3D view of the guide vehicle 200 is captured by the depth camera 140. This view is used to detect objects with irregular shapes and to determine a safe area ahead of travel. In addition, it is used to determine the height of overhead objects and avoid obstacles at head height. Subsequently, a 360° plan view is obtained through the 2D LiDAR 130. This view is used to detect obstacles and walls. The 360° data is used to plan a safe path to travel.

[0081] In an alternative embodiment, a multi-modal sensing priority management and processing algorithm is also provided, in which various data are captured and calculated at different priorities by the calculation module 120. For example, the calculation module 120 can process to determine a global planner and then refine it with a local planner in real time.

[0082] Initially, the computation module 120 may process to derive movement instructions, i.e., a path plan for guiding the user from the current location to the destination via a complete path. The computation module 120 will then adjust the planned complete path based on information from the local map generated by the obstacle avoidance analysis module 110. Specifically, the obstacle avoidance analysis module 110 receives information from the LiDAR 130 and the depth camera 140 to detect any obstacles in the vicinity of the guide vehicle 200. The guide vehicle 200 will then adjust its planned path based on the obstacle avoidance algorithm.

[0083] For example, an elastic band, such as a deformable collision-free path, is initially formed by the global planner. Based on the real-time simulation captured by the LiDAR 130 and the depth camera 140, the navigation module 110 will further adjust the collision-free path locally. For example, the initial shape of the elastic band will be reduced to a shorter and smoother path that maintains sufficient clearance from detected obstacles. During the navigation movement of the guide vehicle 200, the user can also set another destination on the map, and the guide vehicle 200 will re-plan a new path to reach the destination.

[0084] refer to Figure 4 , method 400 starts with step 410. Step 410 includes: a destination ID is input by a user through a smart phone 30. The guide vehicle 200 receives the destination and waits for a user input signal from the handle 220 for confirmation. Step 412 includes: receiving a touch sensor input from the handle 220. Step 414 includes: receiving a destination ID and a confirmation signal. The guide vehicle 200 will find the destination point on a pre-built map. Step 420 includes: based on one or more parameters, deriving a navigation path for the guide vehicle 200; if the destination is found on the map, the guide vehicle 200 will plan a navigation path to the destination. Step 430 includes: reminding the user. If the destination is not found on the map, the guide vehicle 200 will trigger an alarm to notify the user through the handle 220 or the smart phone 30.

[0085] Step 440 includes: detecting one or more obstacles during navigation of the guide vehicle 200. In the exemplary embodiment, obstacle detection is achieved by processing data received by various sensors (such as the depth camera 140 and the 2D LiDAR 130). Step 442 includes: capturing a front 3D view of the guide vehicle 200 through the depth camera 140. Step 444 includes: detecting a 360° plan view by the 2D LiDAR 130. Step 446 includes: estimating the location and size of the obstacle based on the data captured by the 2D LiDAR 130 and the depth camera 140. If an obstacle is detected on the navigation path, the guide vehicle 200 will repeat step 420 and re-plan the path to avoid the obstacle. Otherwise, the guide vehicle 200 will move along the path and reach the destination. Step 450 includes: determining the current position of the guide vehicle 200 relative to the preset destination. If the guide vehicle 200 reaches the target, the guide vehicle 200 will trigger an alarm to notify the user and report the arrival through the mobile app (step 430). Otherwise, the lead vehicle 200 will repeat step 420 and again plan another path to the destination.

[0086] refer to Figure 5 Further details of the overall architecture of the navigation system 10a according to an alternative embodiment of the present disclosure are obtained. In essence, the navigation system 10a includes a navigation module 110a. The functions of the navigation module 110a are similar to Figure 1 The navigation module 110a is embedded in the guide vehicle 200. The navigation module 110a includes main components such as a computing module 120a, a laser radar module 130a, a depth camera 140a, a mileage sensor 150a, and a microcontroller (MCU) 160a, each of which has a function similar to that of Figure 1 Corresponding components of the navigation system 10 .

[0087] However, in this exemplary embodiment, the computing module 120a does not calculate a default navigation path. The navigation system 10a also does not include a microcontroller (MCU) 170 and a Bluetooth (BLE) communication module 180. Instead, the computing module 120a can communicate signals with a remote storage or server 50 via the Internet 40. For example, the computing module 120a can load a pre-recorded navigation path from a remote storage or server 50 via the Internet 40. The navigation path is further refined based on the real-time reading of the lidar module 130a and the depth camera 140. Preferably, the refined navigation path is recorded by the navigation module 110a and uploaded to the remote storage or server 50 for future reference.

[0088] Reference Figure 6 , further describing the operating mode of an example embodiment of system 10a.

[0089] In the path following mode, the user can set a fixed navigation path that the guide vehicle 200 will follow. At the same time, the server 50 will record the movement path of the guide vehicle 200. The guide vehicle 200 can load the stored path locally or download the recorded path from the remote memory or server 50. When the user triggers the touch sensor or button 224, the guide vehicle 200 will move along the path.

[0090] refer to Figure 6 , the method 600 begins with step 610. Step 610 includes: receiving a recorded path via the Internet 40. The guide vehicle 200 receives the destination and waits for a user input signal from the handle 220 for confirmation. Step 612 includes: receiving a touch sensor input from the handle 210. Step 614 includes: receiving the recorded path and the confirmation signal. The guide vehicle 200 will issue a command to load a navigation path from a pre-established map. Step 620 includes: loading a navigation path for the guide vehicle 200 and executing the loaded navigation path until the guide vehicle 200 arrives at the destination. Step 630 includes: determining the current position of the guide vehicle 200 relative to the preset destination. If the guide vehicle 200 arrives at the destination, the guide vehicle 200 will alert the user and report the arrival (step 640). If the current position is not the destination, the guide vehicle 200 will determine the nearest point on the navigation path (step 650).

[0091] Step 660 includes: detecting one or more obstacles during navigation of the guide vehicle 200. In this exemplary embodiment, obstacle detection is achieved by processing data received by certain different sensors (such as the depth camera 140 and the 2D lidar 130). Step 662 includes: capturing a front 3D view of the guide vehicle 200 through the depth camera 140a. Step 664 includes: detecting a 360° plan view through the 2D lidar 130a. Step 666 includes: estimating the position and size of the obstacle based on the data captured by the 2D lidar 130a and the depth camera 140a. If an obstacle is detected on the navigation path, the guide vehicle 200 will stop or re-plan the navigation path to avoid the obstacle (step 670). Step 680 includes: moving a predetermined distance, such as 1 meter, along the planned path. Subsequently, the guide vehicle 200 will repeat step 620 and execute the loaded navigation path until the guide vehicle 200 reaches the target.

[0092] Although not required, the embodiments described with reference to the accompanying drawings may be implemented as an application programming interface (API) or a series of libraries used by developers, or may be included in another software application, such as a terminal or personal computer operating system or a portable computing device operating system. Generally, since program modules include routines, programs, objects, components, and data files that help perform specific functions, those skilled in the art will understand that the functionality of a software application may be distributed among multiple routines, objects, or components to achieve the same functionality required herein.

[0093] It should also be understood that in the case where the methods and systems of the present disclosure are implemented in whole or in part by a computing system, any suitable computing system architecture may be used. Computing systems may include tablet computers, wearable devices, smart phones, IoT devices, edge computing devices, stand-alone computers, network computers, cloud-based computing devices, and dedicated hardware devices. Where the terms "computing system" and "computing device" are used, these terms are intended to cover any suitable configuration of computer hardware capable of implementing the described functionality.

[0094] It will be appreciated by those skilled in the art that various changes and / or modifications may be made to the present disclosure as shown in the specific embodiments without departing from the spirit or scope of the present disclosure as broadly described. Therefore, the present embodiments are to be considered in all aspects as illustrative and not restrictive.

[0095] Unless otherwise indicated, any reference to prior art contained herein should not be taken as an admission that the information is common general knowledge.

Claims

1. A navigation system, characterized in that: The navigation system is used for navigating visually impaired users, and the navigation system comprises: A navigation module, configured to derive a navigation path from a starting location to a preset destination; and a guiding vehicle, configured to guide the visually impaired user to move toward the preset destination based on the derived navigation path, thereby providing navigation guidance to the visually impaired user through the movement; The navigation module is further configured to adjust the navigation path of the guide vehicle in response to detecting an obstacle during navigation of the guide vehicle so as to avoid the detected obstacle while following the navigation path.

2. The navigation system according to claim 1, characterized in that: in, The navigation module is configured to detect one or more dimensions of the obstacle adjacent to the lead vehicle.

3. The navigation system according to claim 1, characterized in that: in, The navigation module is used to determine a safety zone in the absence of the detected obstacle.

4. The navigation system according to claim 3, characterized in that: Further including: An obstacle sensing module is used to detect the obstacle present in the derived navigation path during the navigation, and the navigation module is then used to receive a signal associated with the detection of the obstacle from the obstacle sensing module.

5. The navigation system according to claim 4, characterized in that: in, The obstacle sensing module includes a depth camera for capturing a three-dimensional image from the perspective of the guide vehicle.

6. The navigation system according to claim 5, characterized in that: in, The obstacle sensing module further includes a laser radar for capturing a full-angle plan view representing the position of the obstacle relative to the guide vehicle.

7. The navigation system according to claim 6, characterized in that: in, The obstacle sensing module may move together with the guide vehicle.

8. The navigation system according to claim 1, characterized in that: Further including: The user control module is used to receive input associated with the preset destination.

9. The navigation system according to claim 8, characterized in that: Further including: A signal communication module is used to transmit signals between the navigation module and the user control module embedded in the handheld device.

10. The navigation system according to claim 1, characterized in that: in, The navigation path is derived based on a pre-recorded navigation path in a storage device.

11. The navigation system according to claim 1, characterized in that: in, The guide vehicle comprises a base movable by one or more wheels, each of the wheels being driven by a respective servo motor.

12. The navigation system according to claim 11, characterized in that: Further including: The mileage sensor is used to determine the position of each wheel, and the navigation module is used to determine the current position of the guide vehicle relative to the preset destination.

13. The navigation system according to claim 11, characterized in that: in, The guided vehicle further comprises a handle portion extending from the base, whereby the visually impaired user in use physically contacts the guided vehicle via the handle portion.

14. The navigation system according to claim 13, characterized in that: in, The handle portion further includes a touch sensing unit for receiving input associated with an instruction of the visually impaired user.

15. The navigation system according to claim 13, characterized in that: in, The handle portion further includes a vibration member for providing at least one of tactile feedback and audible feedback indicative of an event to the visually impaired user through the physical contact between the visually impaired user and the guide vehicle.

16. A method for navigating a visually impaired user, characterized in that: The following steps are involved: deriving a navigation path for guiding the vehicle based on the one or more parameters; Based on the derived navigation path, the guide vehicle guides the visually impaired user to move toward a preset destination, thereby providing navigation guidance to the visually impaired user through the movement; receiving, during navigation of the guided vehicle, another signal associated with detection of an obstacle; as well as In response to detecting the obstacle during navigation of the lead vehicle, a navigation path of the lead vehicle is adjusted to avoid the detected obstacle while following the navigation path.

17. The method according to claim 16, characterized in that Further comprising the steps of: receiving another signal associated with detection of another obstacle during navigation of the lead vehicle based on the adjusted navigation path of the lead vehicle; as well as In response to the signal associated with the detected obstacle during navigation of the lead vehicle, the adjusted navigation path of the lead vehicle is further adjusted to avoid the detected another obstacle.

18. The method according to claim 17, characterized in that Further including: The actual navigation path of the guide vehicle is recorded.

19. The method according to claim 18, characterized in that in, The navigation path is derived based on the actual navigation path recorded in the previous navigation.

20. The method according to claim 16, characterized in that Further comprising the steps of: At least one of tactile feedback and audible feedback representing an event is presented to the visually impaired user through the physical contact between the visually impaired user and the guide vehicle.

Images