Electronic guide dog based on deep learning
By introducing deep learning-based technology into electronic guide dogs, combining multi-sensor fusion and convolutional neural networks, the existing electronic guide dogs have been solved, and the effects of fast response and efficient navigation have been achieved.
Patent Information
- Application Number
- CN202411514558.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-03-14
AI Technical Summary
Existing electronic guide dogs require a large amount of environmental data collection when planning routes, resulting in low response speed, short service life and low retention rate, making them unable to serve the visually impaired population widely.
The electronic guide dog based on deep learning is adopted, combining the central control module, ranging module, intelligent navigation module, intelligent learning module and speech recognition module, and using multi-sensor fusion technology, SLAM technology, convolutional neural network and end-to-end speech recognition model to achieve fast response and efficient navigation.
The navigation planning is completed in a short time, reducing the risk of the original plan failure caused by changes in the external environment, and improving the response speed and service capabilities of the electronic guide dog.
Smart Images
Figure CN119606719A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent electronic devices, and in particular to an electronic guide dog based on deep learning. Background Art
[0002] At present, guide dogs face high training costs, short service life, strict selection, high prices and extremely low retention rates, which leads to their inability to widely serve the visually impaired. Therefore, electronic guide dogs are the best choice to solve the travel problems of visually impaired people.
[0003] Currently, the common electronic guide dogs on the market only have ultrasonic detection function, and use ultrasonic ranging to achieve the guiding function. Some electronic guide dogs can also realize route planning and guiding functions, but this kind of electronic guide dogs need to collect a large amount of surrounding environment data before planning each route, resulting in low response speed. Summary of the invention
[0004] To this end, the present invention provides an electronic guide dog based on deep learning to solve the above-mentioned problems in the prior art.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] According to a first aspect of the present invention, an electronic guide dog based on deep learning includes a central control module, which is connected to a power module, a data transmission module, a distance measurement module, an intelligent navigation module, an intelligent learning module, and a speech recognition module, wherein:
[0007] The power module provides power for the operation of each module. The ranging module detects the distance and direction of nearby obstacles and draws a digital image of the obstacle distance in the current area environment.
[0008] The speech recognition module builds an end-to-end speech recognition model to accurately understand the blind person's voice commands and achieve rapid response;
[0009] The intelligent navigation module calculates the best path in the current regional environment based on the obstacle distance data image of the current regional environment provided by the ranging module, and collects the map of the current regional environment and the best path map in the current regional environment into the historical action data, and stores them as the historical regional environment map and the regional environment best path map;
[0010] The intelligent learning module introduces convolutional neural networks to process historical regional environment maps in historical action data, and realizes rapid response from input commands of the speech recognition module to output commands of the intelligent navigation module;
[0011] The data transmission module is used to transmit the historical action data of the voice recognition module and the intelligent navigation module to the cloud.
[0012] Furthermore, the ranging module includes an ultrasonic ranging unit, an infrared detection unit, a lidar ranging unit, and a camera unit, wherein the camera unit captures images within the current area environment, and combines the distance data provided by the ultrasonic ranging unit, the infrared detection unit, and the lidar ranging unit to draw an obstacle distance data image of the current area environment.
[0013] Furthermore, the intelligent navigation module adopts multi-sensor fusion technology, SLAM technology, Dijkstra and A* path planning algorithms.
[0014] Furthermore, the specific steps of the intelligent learning module operation include:
[0015] S1: obtaining the frequently appearing starting point-destination information in the speech recognition module and the historical area environment map in the corresponding historical action data in the intelligent navigation module;
[0016] S2: Input the historical regional environmental map into the input layer;
[0017] S3: The convolution layer uses multiple convolution kernels to slide on the historical regional environment map to extract local features;
[0018] S4: The pooling layer uses the maximum pooling method to downsample the input feature map, thereby reducing the number of model parameters and computational complexity. The pooling layer can help enhance the spatial invariance of the model;
[0019] S5: Multi-layer stacking, which stacks the convolutional layers and pooling layers of multiple historical regional environment maps to gradually extract higher-level features;
[0020] S6: The fully connected layer converts the extracted feature maps into classification labels of the same location;
[0021] S7: The output layer outputs the final classification label for matching by the intelligent navigation module.
[0022] Furthermore, the central control module is also connected to the wristband control module.
[0023] Furthermore, the central control module is also connected to the headphone connection module.
[0024] Furthermore, the central control module is also connected to the voice playing module for playing voice to prompt the surrounding people.
[0025] Furthermore, the central control module is also connected to the anti-loss module, which includes a GPS positioning data acquisition unit, and the GPS positioning unit built into the bracelet in the bracelet control module.
[0026] Furthermore, the working steps of the anti-loss module are:
[0027] When the electronic guide dog is more than 5 meters away from the bracelet, the anti-loss module will feed back the distance information to the central control module, and the intelligent navigation module will start running, scanning the surrounding environment and patrolling automatically.
[0028] Furthermore, the central control module is also connected to the alarm and warning module.
[0029] The present invention has the following advantages:
[0030] The present invention can provide rapid drawing of navigation routes in historical areas according to the user's usage history, and introduce convolutional neural networks to process historical area environmental maps in historical action data, thereby achieving rapid response from input commands of the voice recognition module to output commands of the intelligent navigation module, thereby completing planning in a short time and reducing the risk of the original plan becoming invalid due to changes in the external environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A module connection diagram of an electronic guide dog based on deep learning provided by the present invention.
[0032] Figure 2 This is a flowchart of the operation of an electronic guide dog based on deep learning provided by the present invention. DETAILED DESCRIPTION
[0033] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] Example 1
[0035] like Figure 1 to Figure 2 As shown, an electronic guide dog based on deep learning in an embodiment of the first aspect of the present invention includes a central control module, which is connected to a power module, a data transmission module, a distance measurement module, an intelligent navigation module, an intelligent learning module, and a voice recognition module, wherein the power module provides electrical energy for the operation of each module, the distance measurement module is used to detect the distance and direction of nearby obstacles to prompt the user to avoid the obstacles, and the distance measurement module includes an ultrasonic distance measurement unit, an infrared detection unit, a laser radar distance measurement unit, and a camera unit, wherein the camera unit captures an image in the current regional environment, and combines the distance data provided by the ultrasonic distance measurement unit, the infrared detection unit, and the laser radar distance measurement unit to draw an obstacle distance data image of the current regional environment;
[0036] The speech recognition module can build an end-to-end speech recognition model, accurately understand the blind person's voice commands, such as start, stop, set destination, etc., and can respond quickly within 1 second;
[0037] The end-to-end speech recognition model is mainly composed of the front-end feature extractor, CTC branch and Attention branch;
[0038] The front-end feature extractor is responsible for reducing the amount of calculation, while the CTC branch and the Attention branch are used to process input and output sequences of different lengths respectively. During the training process, the model learns how to map from audio features to text through a large amount of annotated data. After the training is completed, the model can receive new audio input and convert it into text in real time.
[0039] The intelligent navigation module uses multi-sensor fusion technology, SLAM technology, Dijkstra, and A* path planning algorithms to effectively cope with various complex environments, including:
[0040] The multi-sensor fusion technology combines the obstacle distance data image of the current regional environment drawn by the ranging module, uses the SLAM technology to build a map of the current regional environment, and realizes autonomous positioning. Finally, the Dijkstra and A* path planning algorithms are used to comprehensively obtain the best path in the current regional environment.
[0041] After the navigation is completed, the map of the current regional environment and the best path map of the current regional environment are collected into the historical action data and stored as the historical regional environment map and the regional environment best path map;
[0042] SLAM technology, Dijkstra, and A* path planning algorithms are all existing technologies and will not be described in detail here;
[0043] The intelligent learning module collects the historical action data of the speech recognition module and the intelligent navigation module, and introduces the convolutional neural network (CNN) based on the frequently appearing starting point-destination information in the speech recognition module and the corresponding historical action data in the intelligent navigation module to process the historical regional environment map in the historical action data. After processing, it can effectively extract the spatial features of the image, reduce the number of parameters, and improve the calculation efficiency, thereby realizing the rapid response from the input command of the speech recognition module to the output command of the intelligent navigation module;
[0044] The specific steps of the intelligent learning module operation include:
[0045] S1: obtaining the frequently appearing starting point-destination information in the speech recognition module and the historical area environment map in the corresponding historical action data in the intelligent navigation module;
[0046] S2: Input the historical regional environmental map into the input layer;
[0047] S3: The convolution layer uses multiple convolution kernels (filters) to slide on the historical regional environment map to extract local features;
[0048] S4: The pooling layer uses the maximum pooling method to downsample the input feature map, thereby reducing the number of model parameters and computational complexity. The pooling layer can help enhance the spatial invariance of the model;
[0049] S5: Multi-layer stacking, which stacks the convolutional layers and pooling layers of multiple historical regional environment maps to gradually extract higher-level features;
[0050] S6: The fully connected layer converts the extracted feature maps into classification labels of the same location;
[0051] S7: The output layer outputs the final classification label for matching by the intelligent navigation module.
[0052] The data transmission module is used to transmit the historical action data of the speech recognition module and the intelligent navigation module to the cloud, thereby realizing the construction and training of the model using the remote terminal.
[0053] Example 2
[0054] like Figure 1 and Figure 2 As shown, an electronic guide dog based on deep learning includes all the contents of Example 1. In addition, the central control module is also connected to the bracelet control module to provide bracelet control, thereby realizing button control and providing alternative operation methods for use by visually impaired and language-impaired people, wherein the bracelet can also monitor the wearer's vital sign data.
[0055] Example 3
[0056] like Figure 1 and Figure 2 As shown, an electronic guide dog based on deep learning includes all the contents of Example 2. In addition, the central control module is also connected to the earphone connection module, so that after the central control module receives the voice or case command, the successful command output by the central control module can be repeated through the earphone for the user to confirm again, and at the same time, the voice repetition effect can be avoided from being affected by the noise of the external environment.
[0057] Example 4
[0058] like Figure 1 and Figure 2 As shown, an electronic guide dog based on deep learning includes all the contents of Example 3. In addition, the central control module is also connected to the voice playback module to play voice to prompt the surrounding people to avoid the surrounding people as politely as possible.
[0059] Example 5
[0060] like Figure 1 and Figure 2 As shown, an electronic guide dog based on deep learning includes all the contents of Example 4. In addition, the central control module is also connected to the anti-lost module, and the anti-lost module includes a GPS positioning data acquisition unit, which cooperates with the GPS positioning unit built into the wristband in the wristband control module. When the electronic guide dog body is far away from the wristband position, the anti-lost module feeds back to the central control module to enable the intelligent navigation module to operate, thereby moving to the wristband position;
[0061] In a specific embodiment, when the electronic guide dog body is more than 5 meters away from the bracelet, the anti-loss module feeds back the distance information to the central control module, and the intelligent navigation module runs, scans the surrounding environment, and automatically patrols.
[0062] Example 6
[0063] like Figure 1 and Figure 2 As shown, an electronic guide dog based on deep learning includes all the contents of Example 5. In addition, the central control module is also connected to the alarm and alert module, and cooperates with the wearer's vital sign data monitored by the bracelet to monitor the user's physical condition in real time, and issue timely warnings and alarms.
Claims
1. An electronic guide dog based on deep learning, characterized in that: It includes a central control module, which is connected with a power module, a data transmission module, a distance measurement module, an intelligent navigation module, an intelligent learning module, and a voice recognition module, wherein: The power module provides power for the operation of each module. The ranging module detects the distance and direction of nearby obstacles and draws a digital image of the obstacle distance in the current area environment. The speech recognition module builds an end-to-end speech recognition model to accurately understand the blind person's voice commands and achieve rapid response; The intelligent navigation module calculates the best path in the current regional environment based on the obstacle distance data image of the current regional environment provided by the ranging module, and collects the map of the current regional environment and the best path map in the current regional environment into the historical action data, and stores them as the historical regional environment map and the regional environment best path map; The intelligent learning module introduces convolutional neural networks to process historical regional environment maps in historical action data, and realizes rapid response from input commands of the speech recognition module to output commands of the intelligent navigation module; The data transmission module is used to transmit the historical action data of the voice recognition module and the intelligent navigation module to the cloud.
2. The electronic guide dog based on deep learning according to claim 1, characterized in that: The ranging module includes an ultrasonic ranging unit, an infrared detection unit, a lidar ranging unit, and a camera unit. The camera unit captures images within the current area environment, and combines the distance data provided by the ultrasonic ranging unit, the infrared detection unit, and the lidar ranging unit to draw an obstacle distance data image of the current area environment.
3. The electronic guide dog based on deep learning according to claim 1, characterized in that: The intelligent navigation module adopts multi-sensor fusion technology, SLAM technology, Dijkstra and A* path planning algorithms.
4. The electronic guide dog based on deep learning according to claim 1, characterized in that: The specific steps of the intelligent learning module operation include: S1: obtaining the frequently appearing starting point-destination information in the speech recognition module and the historical area environment map in the corresponding historical action data in the intelligent navigation module; S2: Input the historical regional environmental map into the input layer; S3: The convolution layer uses multiple convolution kernels to slide on the historical regional environment map to extract local features; S4: The pooling layer uses the maximum pooling method to downsample the input feature map, thereby reducing the number of model parameters and computational complexity. The pooling layer can help enhance the spatial invariance of the model; S5: Multi-layer stacking, which stacks the convolutional layers and pooling layers of multiple historical regional environment maps to gradually extract higher-level features; S6: The fully connected layer converts the extracted feature maps into classification labels of the same location; S7: The output layer outputs the final classification label for matching by the intelligent navigation module.
5. The electronic guide dog based on deep learning according to claim 1, characterized in that: The central control module is also connected to the bracelet control module.
6. The electronic guide dog based on deep learning according to claim 5, characterized in that: The central control module is also connected to the headphone connection module.
7. The electronic guide dog based on deep learning according to claim 1, characterized in that: The central control module is also connected to the voice playback module to play voice to prompt the surrounding people.
8. The electronic guide dog based on deep learning according to claim 5, characterized in that: The central control module is also connected to the anti-loss module, which includes a GPS positioning data acquisition unit, and the GPS positioning unit built into the bracelet in the bracelet control module.
9. The electronic guide dog based on deep learning according to claim 8, characterized in that: The working steps of the anti-loss module are: When the electronic guide dog is more than 5 meters away from the bracelet, the anti-loss module will feed back the distance information to the central control module, and the intelligent navigation module will start running, scanning the surrounding environment and patrolling automatically.
10. The electronic guide dog based on deep learning according to claim 5, characterized in that: The central control module is also connected to the alarm and warning module.