Auxiliary terminal system for visually impaired people

Abstract

The invention provides an auxiliary terminal system for visually impaired people. The system comprises a mobile phone terminal, a service platform and at least one auxiliary terminal device, the mobile phone terminal is used for inputting a voice or character instruction of a user, sending position information and travel information of the user to the service platform and establishing communication with the auxiliary terminal equipment; the service platform is used for searching the auxiliary terminal device closest to the mobile phone terminal according to the position information and the travel information sent by the mobile phone terminal, and planning a travel route for the user according to the obstacle information sent by the closest auxiliary terminal device; and the auxiliary terminal equipment is used for sending information of obstacles around the auxiliary terminal equipment, receiving the travel route planned by the service platform and performing voice broadcast. Through the system, the visual impaired people can be ensured to go out safely.

Description

Technical Field

[0001] The present invention relates to the field of auxiliary terminals, and in particular to an auxiliary terminal system for visually impaired persons. Background Art

[0002] Visual impairment, also known as visual disability or visual impairment, generally includes two categories: blindness and low vision.

[0003] In order to build an urban environment that is friendly to the visually impaired and ensure that they can travel and live safely, independently and conveniently, systematic social support is needed, including the improvement of the physical barrier-free environment and the enhancement of humanistic barrier-free awareness, etc., aiming to improve the accessibility of visually impaired people in urban spaces, and help them obtain rich environmental information through multi-sensory design dimensions such as touch and hearing, so that they can explore the city more confidently.

[0004] However, existing assistive devices cannot help the visually impaired avoid obstacles, and in environments such as underground passages, device signals will be blocked and reflected, causing certain difficulties for their travel, and in severe cases, even causing physical harm. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an integrated, multi-scenario applicable visually impaired auxiliary terminal system. The user inputs travel information on the mobile phone, and the mobile phone uploads its own location information and travel information to the service platform. The service platform sends the planned route to the auxiliary device terminal closest to the visually impaired person based on the mobile phone's own location information sent by the mobile phone. The auxiliary device terminal performs voice broadcast to ensure the safe travel of the visually impaired.

[0006] To achieve the above-mentioned purpose of the invention, the present invention provides an auxiliary terminal system for visually impaired persons, the system comprising: a mobile phone terminal, a service platform and at least one auxiliary terminal device;

[0007] The mobile phone terminal is used to input the user's voice or text commands, send the user's location information and travel information to the service platform, and establish communication with the auxiliary terminal device;

[0008] The service platform is used to find the auxiliary terminal device closest to the mobile phone according to the location information and travel information sent by the mobile phone, and plan a travel route for the user according to the obstacle information sent by the nearest auxiliary terminal device;

[0009] The auxiliary terminal device is used to send obstacle information around itself, receive travel routes planned by the service platform, and perform voice broadcasting.

[0010] Furthermore, the user's location information includes GPS positioning and the direction and speed of travel obtained by the mobile phone's gyroscope.

[0011] Furthermore, the auxiliary terminal device includes: a communication module, a positioning module, a data acquisition module and a voice module;

[0012] The communication module is used to establish communication with the mobile phone through the Beacon broadcast protocol and establish network communication with the service platform;

[0013] The positioning module is used to obtain Beidou positioning and GPS positioning of the auxiliary terminal device when the auxiliary terminal device is placed outdoors; when the auxiliary terminal device is placed indoors, it obtains Beacon positioning of the mobile phone;

[0014] The data acquisition module is used to obtain obstacle information around itself through the laser radar and upload it to the service platform;

[0015] The voice module is used for voice broadcasting and adjusting the volume of the broadcasting sound according to the decibel of the on-site sound.

[0016] Furthermore, the communication module includes a low-power Bluetooth communication module, which sends a UU ID to the surroundings at regular intervals, and the application software in the mobile phone displays the signal strength between the mobile phone and the auxiliary terminal device after receiving the UU ID;

[0017] The communication module also includes 4G, 5G and Wi F i communication modules, and the 4G, 5G and Wi F i communication modules are used to establish network communication with the service platform.

[0018] Furthermore, when the auxiliary terminal device is placed indoors, obtaining the Beacon positioning of the mobile phone includes:

[0019] When the application software in the mobile phone receives the UU ID sent by the auxiliary terminal device, the signal strength between the mobile phone and the auxiliary terminal device is represented by the RSS I value. The application software in the mobile phone pre-establishes a mapping relationship between the RSS I value and the distance value, and converts the RSS I value into the distance between the mobile phone and the auxiliary terminal device;

[0020] A circle is drawn with the coordinates of the auxiliary terminal device as the center and the distance between the mobile terminal and the auxiliary terminal device as the radius. When three auxiliary terminal devices are connected to the mobile terminal, three circles intersecting at one point are obtained, and the intersection point is the coordinates of the mobile terminal.

[0021] Furthermore, the data acquisition module is used to obtain obstacle information around itself through laser radar and upload it to the service platform, including:

[0022] The laser radar includes but is not limited to mechanical laser radar, semi-solid laser radar and solid-state laser radar. The laser radar serves as the auxiliary terminal device body, and the communication module, positioning module and voice module are installed in the body;

[0023] The laser radar emits laser beams and / or millimeter waves to the surrounding environment through a transmitter. When the laser beams and / or millimeter waves encounter obstacles, they are reflected to the receiver. The data acquisition module obtains the direction and height of the obstacle based on the time interval between sending and receiving signals and the direction of transmission, and uploads it to the service platform.

[0024] Furthermore, the laser radar simultaneously emits a laser beam and a millimeter wave to the surrounding environment, uses the surrounding obstacle information identified by the laser beam as the first obstacle information, and uses the surrounding obstacle information identified by the millimeter wave as the second obstacle information, wherein the surrounding obstacle information includes but is not limited to: the distance, direction, height, speed, posture, and shape of the obstacle;

[0025] If the first obstacle information is consistent with the second obstacle information, a multi-target tracking technology is used to assign an ID number to all obstacles in the first obstacle information, track the obstacles according to their shape characteristics and movement characteristics at the previous and next moments, and plan a travel route for the user;

[0026] If the first obstacle information is inconsistent with the second obstacle information, a multi-target tracking technology is used to track all obstacles in the second obstacle information and plan a travel route for the user.

[0027] Furthermore, after obtaining the location information, travel information sent by the mobile phone and the obstacle information sent by the auxiliary terminal device, the service platform calculates the distance between the mobile phone and the obstacle and the distance between the mobile phone and the auxiliary terminal device in real time to plan a travel route for the user.

[0028] Furthermore, the voice module obtains weather information of the location of the auxiliary device after the communication module establishes network communication with the mobile phone and the service platform;

[0029] The voice module is used to broadcast weather information, travel routes planned by the service platform, and information about surrounding obstacles, and adjust the volume of the broadcast according to the decibel level of the on-site sound.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. It solves the problem of the difficulty of integrating existing auxiliary equipment, saving the labor cost of installing equipment and the subsequent maintenance cost;

[0032] 2. Applicable to multiple scenarios, even underground passages with weak signals;

[0033] 3. It can help the visually impaired avoid obstacles and ensure travel safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 is a schematic diagram of the structure of an auxiliary terminal system for visually impaired persons provided by an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of the structure of an auxiliary terminal device provided by an embodiment of the present invention;

[0037] Figure 3 is a flow chart of dynamic switching of network communication provided by an embodiment of the present invention;

[0038] Figure 4 is a flow chart of a multi-target tracking algorithm provided by an embodiment of the present invention;

[0039] Figure 5 This is a flow chart of sensor data fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The enumerated embodiments are only used to explain the present invention and are not used to limit the scope of the present invention.

[0041] Reference Figure 1 This embodiment provides a schematic diagram of the structure of an auxiliary terminal system for visually impaired persons. This embodiment can be applied to assisting visually impaired persons in traveling. Figure 1 As shown, the system includes at least: a mobile phone, a service platform, at least one auxiliary terminal device, an external data source and other intelligent systems.

[0042] It should be noted that the system in the embodiment of the present invention adopts a modular design, and each functional module is connected through a standardized interface (such as USB, UART, SPI) to support future upgrades and expansions. At the same time, the system provides an open API interface, which can be seamlessly integrated with the smart city transportation system to form a more extensive smart assisted travel network, providing a safer and more convenient travel experience for the visually impaired.

[0043] In addition, the auxiliary terminal device in the embodiment of the present invention can be fixedly installed on the bus and move with the bus. Preferably, it is installed near the door reflector or the front bumper of the vehicle to facilitate visually impaired people to find the door position.

[0044] It can also be fixedly installed in public places, such as public toilets, pharmacies, shopping malls, and entrances and exits of underground passages, etc.

[0045] The power supply of auxiliary terminal equipment can be connected to the bus power supply. For fixed-installed auxiliary terminal equipment, various methods such as solar power supply and battery power supply can be used.

[0046] The mobile phone is used to input the user's voice or text commands, send the user's location information and travel information to the service platform, and establish communication with the auxiliary terminal device.

[0047] Specifically, the user's location information includes GPS positioning and the direction and speed of travel obtained by the mobile phone's gyroscope. The visually impaired person sends voice or text instructions on the mobile phone APP, and the mobile phone APP performs voice recognition and converts it into text as travel information. The travel information includes a designated travel route or a designated travel destination. After the visually impaired person sends instructions on the mobile phone APP, the mobile phone sends the user's location information and travel information to the service platform. The service platform searches for the auxiliary terminal device closest to the user's location, and controls the auxiliary terminal device to broadcast voice, instructing the user to move so that the mobile phone enters the broadcast range of the auxiliary terminal device.

[0048] The service platform is used to find the auxiliary terminal device closest to the mobile phone based on the location information and travel information sent by the mobile phone, and plan the travel route for the user based on the obstacle information sent by the nearest auxiliary terminal device.

[0049] It should be noted that the service platform serves as the background of the mobile APP to communicate with the mobile phone, and also serves as an Internet of Things platform to remotely control the Internet of Things devices.

[0050] Auxiliary terminal equipment, used to send information about obstacles around itself, receive travel routes planned by the service platform, and make voice broadcasts.

[0051] Specifically, taking bus travel as an example, the visually impaired person uses voice or text to inform the backend of the vehicle information they need to take through the mobile APP, such as: I want to go to the train station. The service platform filters out the bus information closest to the station based on the voice command issued and the current station location of the visually impaired person, and sends the real-time vehicle information to the mobile phone interface of the visually impaired person; if the bus route is specified, only the real-time information of the current route and the arrival prompt are displayed. If the location is specified, the real-time information of all passing routes and the arrival prompt of the bus must be displayed, and the auxiliary terminal device must perform voice broadcast.

[0052] Visually impaired people make reservations for the public buses they need to take through the mobile phone APP. The service platform behind the APP starts to search for the location of the corresponding bus. When the bus reaches the waiting area, the APP can connect to the auxiliary terminal device and send a voice command to play the route, triggering the auxiliary terminal device to broadcast the voice of entering the station. Furthermore, the auxiliary terminal device can be triggered to broadcast the voice of entering the station again by shaking the mobile phone. The visually impaired people can find the vehicle they need to take according to the voice prompts and get on the bus by themselves.

[0053] The biggest problem for visually impaired people when taking public transportation is that they cannot find the bus they need to take. When multiple vehicles enter the same bus stop, they cannot know which line the vehicles belong to. For this purpose, a unique identification UU ID of the auxiliary terminal device is established and bound to the license plate number of the bus. After the mobile phone receives the UU ID sent by the auxiliary terminal device, the signal strength between the mobile phone and the auxiliary terminal device is displayed, and the voice broadcast of the auxiliary terminal device is triggered.

[0054] Combination Figure 2 , a schematic diagram of the structure of an auxiliary terminal device provided by an embodiment of the present invention, such as Figure 2 As shown, the auxiliary terminal device includes: a communication module, a positioning module, a data acquisition module and a voice module.

[0055] Specifically, the auxiliary terminal device is a low-power integrated device that integrates a communication module, a positioning module, a data acquisition module and a voice module on a circuit board.

[0056] The communication module is used to establish communication with the mobile phone through the Beacon broadcast protocol and to establish network communication with the service platform. The communication module includes a low-power Bluetooth communication module, which sends UU ID to the surrounding at regular intervals. After receiving the UU ID, the application software in the mobile phone displays the signal strength between the mobile phone and the auxiliary terminal device; the communication module also includes 4G, 5G and Wi F i communication modules, which are used to establish network communication with the service platform.

[0057] Combination Figure 3, this embodiment provides a flow chart of a dynamic switching of network communication. Figure 3 As shown, the auxiliary terminal device establishes communication with the service platform through Wi-Fi. When the Wi-Fi signal strength decreases, it switches to 4G / 5G signal. When all networks are interrupted, the data is temporarily stored locally, and data transmission is resumed after the network is restored.

[0058] The communication module in this embodiment is designed with a dynamic network switching mechanism, which can automatically select the best communication channel (4G, 5G or WiFi) according to the real-time network signal strength. When the network signal is unstable, the system will automatically save the unuploaded data and immediately retransmit the data after the signal is restored to ensure the integrity and accuracy of the travel data.

[0059] In addition, to protect user privacy, the embodiment of the present invention uses AES-256 encryption technology to encrypt all transmitted data, and uses SSL / TLS protocol to protect data security during communication. The service platform also sets up a strict access control mechanism, allowing only authenticated users and devices to access sensitive information, thereby ensuring that data is not obtained by unauthorized third parties.

[0060] It should be noted that since underground passages are indoors, GPS signals will be blocked or reflected by the walls and cannot enter the room, resulting in the failure to receive satellite signals, forming a signal coverage weakness, causing inaccurate positioning and large errors.

[0061] Therefore, Beacon technology is used to enable auxiliary terminal devices to have low-power Bluetooth communication capabilities.

[0062] Auxiliary terminal devices are installed at the entrance and exit of the underground passage, and the entrance and exit prompt information is recorded in advance. When the visually impaired pass the entrance of the underground passage, the application software on the mobile phone can trigger the auxiliary positioning device to make a voice announcement: "There is an underground passage entrance straight ahead", and the visually impaired can find the entrance according to the voice announcement. After entering the underground passage, auxiliary positioning devices will also be installed at each turning point and exit. The visually impaired can shake the mobile phone to trigger the auxiliary terminal device to broadcast the voice again, so as to know the turning point information and exit location.

[0063] Beacon is a positioning technology used indoors. Since the GPS signal is weak indoors, and the auxiliary terminal system needs the location information of the mobile phone and the auxiliary terminal device, Beacon is used to obtain the location information of the mobile phone indoors.

[0064] Beacon is a broadcast protocol based on the low-power Bluetooth protocol. Devices equipped with low-power Bluetooth communication function use the low-power Bluetooth communication function to send the device's unique universal unique identifier UUID to the surrounding area. UUID is a specific form of identifier. The application software that receives the UUID will respond accordingly based on the UUID. UUID is a 128-bit ID.

[0065] The auxiliary terminal device broadcasts to the surrounding environment at regular intervals, and the mobile phone is installed with corresponding application software to receive the UU ID and respond accordingly according to the UU ID. For example, the distance between devices can be estimated by the change of the RSSI value.

[0066] The positioning module is used to obtain the Beidou positioning and GPS positioning of the auxiliary terminal device when the auxiliary terminal device is placed outdoors; when the auxiliary terminal device is placed indoors, it is used to obtain the Beacon positioning of the mobile phone.

[0067] When the auxiliary terminal device is placed indoors, obtain the Beacon positioning of the mobile phone, including:

[0068] When the application software in the mobile phone receives the UU ID sent by the auxiliary terminal device, the signal strength between the mobile phone and the auxiliary terminal device is represented by the RSS I value. The application software in the mobile phone pre-establishes a mapping relationship between the RSS I value and the distance value, and converts the RSS I value into the distance between the mobile phone and the auxiliary terminal device;

[0069] A circle is drawn with the coordinates of the auxiliary terminal device as the center and the distance between the mobile terminal and the auxiliary terminal device as the radius. When three auxiliary terminal devices are connected to the mobile terminal, three circles intersecting at one point are obtained, and the intersection point is the coordinates of the mobile terminal.

[0070] In the embodiment of the present invention, a mapping relationship between the RSSI value and the distance value is constructed in advance to estimate the distance between the mobile phone and the auxiliary terminal device.

[0071] The data acquisition module is used to obtain obstacle information around it through lidar and upload it to the service platform.

[0072] Laser radar includes but is not limited to mechanical laser radar, semi-solid laser radar and solid-state laser radar. The laser radar serves as an auxiliary terminal device body, and the communication module, positioning module and voice module are installed in the body.

[0073] Mechanical laser radar uses a motor to drive the entire optical-mechanical structure to rotate 360°, achieving a full range of scanning of the surrounding environment to form a point cloud. After the chip-based integrated transceiver module and the 360° rotating scanning module, a new type of mechanical laser radar with high performance, high reliability and better cost can be produced. Semi-solid laser radar changes the direction of the laser through an internal moving reflector. Whether it is a mechanical laser radar or a semi-solid laser radar, the transceiver module is equipped with a scanning module that performs mechanical movement to scan the surrounding environment. Solid-state laser radar uses multiple laser emitting units to form an emitting array, and the emission angle of the laser beam is changed by adjusting the phase difference of each unit in the emitting array.

[0074] The laser radar emits laser beams and / or millimeter waves to the surrounding environment through the transmitter. When the laser beams and / or millimeter waves encounter obstacles, they are reflected to the receiver. The data acquisition module obtains the direction and height of the obstacle based on the time interval between sending and receiving signals and the direction of transmission, and uploads it to the service platform. The data reflected by the laser beam is used as the laser radar data, and the data reflected by the millimeter wave is used as the millimeter wave radar data.

[0075] The wavelength of laser is very short, which can achieve extremely high resolution. In contrast, millimeter wave radar transmits electromagnetic waves with a frequency between microwave and infrared. These wavelengths are shorter than centimeters but longer than microwaves. They can work stably in different weather and lighting conditions, have strong penetration, and are not easily affected by environmental factors such as rain and fog. Through the scanning of laser and millimeter waves, obstacles can be scanned in different weather conditions and the scanning accuracy can be guaranteed.

[0076] It should be noted that LiDAR is a radar system that uses laser beams to detect the position, speed and other characteristic quantities of a target. Its working principle is to transmit a detection signal or laser beam to the target, and then compare the received signal reflected from the target or the target echo with the transmitted signal. After appropriate processing, relevant information about the target can be obtained, such as target distance, direction, height, speed, attitude, and even shape parameters, so as to detect, track and identify the target.

[0077] Furthermore, the laser radar simultaneously emits a laser beam and a millimeter wave to the surrounding environment, and uses the surrounding obstacle information identified by the laser beam as the first obstacle information, and uses the surrounding obstacle information identified by the millimeter wave as the second obstacle information. The surrounding obstacle information includes but is not limited to: the distance, direction, height, speed, posture, and shape of the obstacle;

[0078] If the first obstacle information is consistent with the second obstacle information, a multi-target tracking technology is used to assign an ID number to all obstacles in the first obstacle information, track the obstacles according to their shape characteristics and movement characteristics at the previous and next moments, and plan a travel route for the user;

[0079] If the first obstacle information is inconsistent with the second obstacle information, a multi-target tracking technology is used to track all obstacles in the second obstacle information and plan a travel route for the user.

[0080] It should be noted that multi-target tracking technology can predict the movement of obstacles based on the shape and motion characteristics of obstacles at previous and subsequent moments. Shape characteristics include the posture and shape of obstacles at different moments, and motion characteristics include the distance, position, height, speed, etc. Multi-target tracking technology can quickly detect and track objects, even in the presence of occlusion, lighting changes, background, motion blur, etc.

[0081] Furthermore, the auxiliary terminal device adopts multi-target tracking technology, including:

[0082] An extended Kalman filter is set inside the auxiliary terminal device to perform multi-target tracking;

[0083] The auxiliary terminal device fuses the lidar data and millimeter-wave radar data through the extended Kalman filter, identifies and tracks the dynamic and static obstacles in the route of the visually impaired person, and predicts the position of the obstacle at the next moment through the extended Kalman filter.

[0084] Combination Figure 4 , a flowchart of a multi-target tracking algorithm provided in this embodiment. Figure 4 As shown, after the auxiliary terminal device fuses the lidar data and millimeter-wave radar data through the extended Kalman filter, it performs initial target detection and assigns a unique ID to each target. It then uses the extended Kalman filter to predict the target's position at the next moment, and measures and updates the target's position when the next moment arrives. It establishes associated tracking based on the changes in the target's position before and after, and finally outputs the updated obstacle position and motion trajectory.

[0085] It should be noted that the Extended Kalman Filter (EKF) is a state estimation algorithm that efficiently processes nonlinear systems. It approximates the system dynamics and observation models by linearizing the nonlinear model, thereby allowing the use of the Kalman filter framework.

[0086] The EKF predicts the position of the obstacle at the next moment and adjusts the path based on its motion characteristics to ensure user safety.

[0087] Furthermore, the auxiliary terminal device fuses the lidar data and the millimeter-wave radar data through an extended Kalman filter, including:

[0088] First, the sensor data is preprocessed to filter out noise signals. Then, the weight of each sensor is combined and the proportion of data fusion is adjusted in real time according to the dynamic changes of the environment, and finally an accurate obstacle recognition result is generated.

[0089] Combination Figure 5 , this embodiment provides a flow chart of sensor data fusion. Figure 5 As shown in the figure, before the lidar data and the millimeter-wave radar data enter the extended Kalman filter, they are first preprocessed to filter out the noise signals in the data, and then the features of the preprocessed data are extracted and input into the extended Kalman filter. Finally, the extended Kalman filter outputs the position, shape and movement direction of the obstacle.

[0090] When the auxiliary terminal device is placed on the bus, the positioning module obtains the Beidou positioning and GPS positioning of the auxiliary terminal device.

[0091] When the bus stops moving, the lidar transmitter can emit a millimeter wave and a laser beam in the same direction successively.

[0092] If the distances to obstacles measured by millimeter waves and laser are inconsistent, it means that the laser was affected by rain, dust, haze, snow, etc. during the emission process, causing the laser to complete reflection in advance. Especially in the case of heavy rain, the laser cannot pass through the rain to reach the obstacle, causing abnormal obstacle scanning. At this time, the obstacle information scanned by millimeter waves shall prevail.

[0093] If the distances are consistent, the obstacle information from the laser scan shall prevail. Laser scanning has high precision and resolution and can capture tiny objects and details.

[0094] When the bus starts moving, the lidar stops working;

[0095] When the auxiliary terminal device is placed indoors, the lidar can emit laser beams in all directions through the transmitter to achieve high-precision obstacle scanning.

[0096] Furthermore, after obtaining the location information, travel information sent by the mobile phone and the obstacle information sent by the auxiliary terminal device, the service platform calculates the distance between the mobile phone and the obstacle and the distance between the mobile phone and the auxiliary terminal device in real time to plan travel routes for users.

[0097] Optionally, after the communication module establishes network communication with the mobile phone and the service platform, the voice module can obtain weather information of the location of the auxiliary device, and the weather information includes: sunny, rainy, snowy, foggy, etc.

[0098] The voice module broadcasts weather information, travel routes planned by the service platform, and information about surrounding obstacles, and adjusts the volume of the broadcast according to the decibel level of the on-site sound.

[0099] After the data collection module uploads the distance and direction of the obstacle to the service platform, the service platform calculates the distance between the mobile phone and the obstacle and the distance between the mobile phone and the auxiliary terminal device in real time, and plans the travel route for the user. The voice module uses a speaker to ensure that the blind can hear the voice within a range of 20 meters. It broadcasts not only each section of the path, but also the surrounding obstacles.

[0100] In the embodiment of the present invention, travel information is inputted on a mobile phone, and the mobile phone uploads its own location information and travel information to a service platform. The service platform sends the planned route to the auxiliary device terminal closest to the visually impaired person based on the location information sent by the mobile phone. The auxiliary device terminal scans the surrounding obstacles and uploads them to the service platform. The service platform plans a route for the user and sends a voice broadcast to the auxiliary device terminal to ensure the travel safety of the visually impaired person.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An auxiliary terminal system for visually impaired persons, characterized in that: The system includes: a mobile phone terminal, a service platform and at least one auxiliary terminal device; The mobile phone terminal is used to input the user's voice or text commands, send the user's location information and travel information to the service platform, and establish communication with the auxiliary terminal device; The service platform is used to find the auxiliary terminal device closest to the mobile phone according to the location information and travel information sent by the mobile phone, and plan a travel route for the user according to the obstacle information sent by the nearest auxiliary terminal device; The auxiliary terminal device is used to send obstacle information around itself, receive travel routes planned by the service platform, and perform voice broadcasting.

2. The auxiliary terminal system for visually impaired persons according to claim 1, characterized in that: The user's location information includes GPS positioning and the direction and speed obtained by the mobile phone gyroscope.

3. The auxiliary terminal system for visually impaired persons according to claim 1, characterized in that: The auxiliary terminal device includes: a communication module, a positioning module, a data acquisition module and a voice module; The communication module is used to establish communication with the mobile phone through the Beacon broadcast protocol and establish network communication with the service platform; The positioning module is used to obtain Beidou positioning and GPS positioning of the auxiliary terminal device when the auxiliary terminal device is placed outdoors; when the auxiliary terminal device is placed indoors, it obtains Beacon positioning of the mobile phone; The data acquisition module is used to obtain obstacle information around itself through the laser radar and upload it to the service platform; The voice module is used for voice broadcasting and adjusting the volume of the broadcasting sound according to the decibel of the on-site sound.

4. The auxiliary terminal system for visually impaired persons according to claim 3, characterized in that: The communication module includes a low-power Bluetooth communication module, which sends a UUID to the surroundings at regular intervals. After receiving the UUID, the application software in the mobile phone displays the signal strength between the mobile phone and the auxiliary terminal device; The communication module also includes 4G, 5G and WiFi communication modules, and the 4G, 5G and WiFi communication modules are used to establish network communication with the service platform.

5. The auxiliary terminal system for visually impaired persons according to claim 4, characterized in that: When the auxiliary terminal device is placed indoors, obtaining the Beacon positioning of the mobile phone includes: When the application software in the mobile phone receives the UUID sent by the auxiliary terminal device, the signal strength between the mobile phone and the auxiliary terminal device is represented by the RSSI value. The application software in the mobile phone pre-establishes a mapping relationship between the RSSI value and the distance value, and converts the RSSI value into the distance between the mobile phone and the auxiliary terminal device; A circle is drawn with the coordinates of the auxiliary terminal device as the center and the distance between the mobile terminal and the auxiliary terminal device as the radius. When three auxiliary terminal devices are connected to the mobile terminal, three circles intersecting at one point are obtained, and the intersection point is the coordinates of the mobile terminal.

6. The auxiliary terminal system for visually impaired persons according to claim 3, characterized in that: The data acquisition module is used to obtain obstacle information around itself through the laser radar and upload it to the service platform, including: The laser radar includes but is not limited to mechanical laser radar, semi-solid laser radar and solid-state laser radar. The laser radar serves as the auxiliary terminal device body, and the communication module, positioning module and voice module are installed in the body; The laser radar emits laser beams and / or millimeter waves to the surrounding environment through a transmitter. When the laser beams and / or millimeter waves encounter obstacles, they are reflected to the receiver. The data acquisition module obtains the direction and height of the obstacle based on the time interval between sending and receiving signals and the direction of transmission, and uploads it to the service platform.

7. The auxiliary terminal system for visually impaired persons according to claim 6, characterized in that: The laser radar simultaneously emits a laser beam and a millimeter wave to the surrounding environment, uses the surrounding obstacle information identified by the laser beam as the first obstacle information, and uses the surrounding obstacle information identified by the millimeter wave as the second obstacle information, wherein the surrounding obstacle information includes but is not limited to: the distance, direction, height, speed, posture, and shape of the obstacle; If the first obstacle information is consistent with the second obstacle information, a multi-target tracking technology is used to assign an ID number to all obstacles in the first obstacle information, track the obstacles according to their shape characteristics and movement characteristics at the previous and next moments, and plan a travel route for the user; If the first obstacle information is inconsistent with the second obstacle information, a multi-target tracking technology is used to track all obstacles in the second obstacle information and plan a travel route for the user.

8. The auxiliary terminal system for visually impaired persons according to claim 7, characterized in that: After obtaining the location information, travel information and obstacle information sent by the mobile phone and the auxiliary terminal device, the service platform calculates the distance between the mobile phone and the obstacle and the distance between the mobile phone and the auxiliary terminal device in real time to plan a travel route for the user.

9. The auxiliary terminal system for visually impaired persons according to claim 8, characterized in that: The voice module obtains weather information of the location of the auxiliary device after the communication module establishes network communication with the mobile phone and the service platform; The voice module is used to broadcast weather information, travel routes planned by the service platform, and information about surrounding obstacles, and adjust the volume of the broadcast according to the decibel level of the on-site sound.

Images