Smart guide glasses based on obstacle monitoring and reminder

Through the data collection, processing and communication module of smart guide glasses, real-time monitoring and reminding of obstacles is realized, solving the problem that traditional blind guide tools cannot be warned in advance, and improving the travel safety and navigation reliability of blind people.

CN116549218BActive Publication Date: 2024-12-03JIANGXI HENGBIDA IND CO LTD

Patent Information

Application Number
CN202310531389.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-12-03
Estimated Expiration
2043-05-12

AI Technical Summary

Technical Problem

Traditional blind guide canes and smart blind navigation glasses have limited detection range in obstacle detection, which cannot be warning in advance, resulting in low safety for blind people travel. The existing voice interaction mode cannot be applied in environments where voice cannot be freely emitted.

Method used

Design an intelligent blind glasses based on obstacle monitoring and reminder. The environment video and distance information are obtained through the data acquisition module. The data processing module generates reminder audio, the voice broadcast module provides real-time reminders, and the intelligent communication module cannot freely issue voices. Remote navigation guidance in the environment.

Benefits of technology

Improve navigation reliability and user experience of navigation glasses, ensuring safe travel for blind people, especially in environments where voice is not free to be emitted.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116549218B_ABST
    Figure CN116549218B_ABST
Patent Text Reader

Abstract

The present invention provides a kind of intelligent blind guide glasses based on obstacle monitoring and reminder, including: glasses body, and arranged on the glasses body: a data acquisition module is used to collect the current environment information of the user based on the shooting module, obtain the environment video, and measure the distance between the user and the living body or obstacle in front based on the TOF module, and generate distance information; a data processing module is used to process the distance information, generate reminder audio, and at the same time, process the environment video to obtain the video to be sent; an intelligent communication module is used to send the video to be sent to the network terminal processor and receive remote navigation information; a voice broadcast module is used to play the reminder audio; a central processing module is used to complete the whole process control of obstacle monitoring and reminder. It is convenient to carry out remote navigation guidance in various environments where the user is inconvenient or cannot freely make voice, improve the navigation reliability of navigation glasses, improve the user experience, and ensure the safety of users.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of blind guide equipment, and in particular to a pair of intelligent blind guide glasses based on obstacle monitoring and reminder. Background Art

[0002] The blind are a special group in society. The biggest problem they face in daily life is independent travel. They often need auxiliary tools to predict road obstacles. Traditional auxiliary tools, such as guide canes, are limited in detection range. Blind people can only sense obstacles when they are near them, which poses a great risk.

[0003] Guide glasses are born to provide a better auxiliary tool for the convenience and safety of blind people. The current intelligent navigation glasses for the blind provide navigation and positioning functions for the blind through distance measurement, navigation positioning, route guidance, voice prompts and other designs. Due to the small size, integration and simplicity of wearable electronic devices such as guide glasses, the traditional human-computer interaction model based on keyboards and screens can no longer adapt to guide glasses. The existing human-computer interaction mode based on voice is also limited in the actual application environment and cannot be applied to various environments where people cannot speak freely. Therefore, the present invention proposes a kind of intelligent guide glasses based on obstacle monitoring and reminders. Summary of the invention

[0004] The present invention provides a pair of intelligent blind-sighted glasses based on obstacle monitoring and reminder, which are used to solve the above problems. The present invention collects the current environmental information of the user through a data acquisition module, generates an environmental video to provide a remote navigation basis for the guardian, provides a judgment basis for the monitoring of obstacles, and obtains the distance information of the living body or the obstacle. The data processing module processes the collected distance information and the environmental video, generates a reminder audio for navigation, and plays the reminder audio through a voice broadcast module to complete real-time obstacle monitoring and reminder, and provide real-time navigation for the user. At the same time, according to the video to be sent obtained by processing the environmental video, the video to be sent is sent to a network terminal processor through an intelligent communication module, and remote navigation information is received, which is convenient for remote navigation guidance in various environments where the user is inconvenient or cannot freely make a voice, improves the navigation reliability of the navigation glasses, improves the user experience, and ensures the safety of the user.

[0005] The present invention provides an intelligent blind-guiding glasses based on obstacle monitoring and reminder, characterized in that it comprises: a glasses body, and a data acquisition module, a data processing module, a voice broadcast module, an intelligent communication module and a central processing module arranged on the glasses body;

[0006] The data acquisition module is used to collect the user's current environment information based on the shooting module to obtain the environment video, and to measure the distance from the user to the living body or obstacle in front based on the TOF module and generate distance information;

[0007] The data processing module is used to process the distance information, generate a reminder audio, and process the environment video to obtain the video to be sent;

[0008] An intelligent communication module, used to send the video to be sent to the network terminal processor and receive remote navigation information;

[0009] Voice broadcast module, used to play reminder audio;

[0010] The central processing module is used to complete the entire process control of obstacle monitoring and reminders.

[0011] Preferably, a data acquisition module in a smart blind guide glasses based on obstacle monitoring and reminder includes:

[0012] The environment detection submodule is used to perform binocular vision shooting of the current environment based on the shooting module after the user arrives in a new environment, obtain binocular vision video, and integrate the binocular vision video to obtain the environment video;

[0013] The distance detection submodule is used to calculate the distance between the living object or obstacle in front and the user based on the detection laser emitted by the TOF module transmitting end to the living object or obstacle in front, and the detection feedback light received by the TOF module receiving end, obtaining the detection laser return time, and generating distance information.

[0014] Preferably, in a smart blind guide glasses based on obstacle monitoring and reminder, an environment detection submodule comprises:

[0015] A video processing unit is used to perform video framing on the binocular vision video to obtain two synchronously framed videos, and to compare the two synchronously framed videos frame by frame to obtain frame difference features;

[0016] A difference determination unit is used to mark the same object in each video frame based on the frame difference feature, mark the screen of the same object, and determine the shooting angle difference of the corresponding screen of each synchronous video frame of the two synchronous framed videos according to the screen marking result;

[0017] A stereoscopic fusion unit, used for fusing video images of the same object in each synchronous video frame of the synchronous frame video based on the difference in shooting angles to obtain a stereoscopic image;

[0018] A panoramic processing unit, used to determine the visual range difference between single-vision shooting based on frame difference features, take any single-vision shooting picture as a standard picture, and visually expand the standard picture based on the visual range difference to obtain a panoramic picture;

[0019] The image fusion unit is used to fuse the panoramic image with the stereoscopic image based on the corresponding relationship between the video frames of the panoramic image and the stereoscopic image to obtain a stereoscopic video frame image, and to splice all the stereoscopic video frames to obtain an environmental video.

[0020] Preferably, in a smart guide glasses based on obstacle monitoring and reminder, the distance detection submodule includes:

[0021] The obstacle judgment unit is used to obtain the real-time environment picture based on the shooting module, and the image recognition model recognizes the real-time environment picture to determine whether there are living things or obstacles in the user's current forward range;

[0022] An angle determination unit is used to obtain a stereoscopic video frame corresponding to the real-time image when it is determined that there is a living body or an obstacle in the current forward range of the user, mark the observation center of the stereoscopic video frame, and determine the distance angle between each living body or obstacle and the user in the stereoscopic video frame based on the observation center;

[0023] A sequence generation unit, used to determine the relative position of each living body or obstacle and the user in the stereoscopic video frame using the observation center as a reference point, and generate a detection distance sequence based on the relative position;

[0024] A detection trigger unit is used to generate a detection angle sequence based on a distance detection sequence. The TOF module transmitter emits a detection laser to a living body or an obstacle according to the detection angle sequence.

[0025] A detection feedback unit, used to receive the detection feedback light based on the receiving end of the TOF module and obtain the return time of the detection laser;

[0026] The distance calculation unit is used to detect the laser return time and calculate the distance between each living body or obstacle in front and the user to generate distance information.

[0027] Preferably, a data processing module in a smart guide glasses based on obstacle monitoring and reminder includes: a distance processing submodule for processing distance information and generating reminder audio;

[0028] The distance processing submodule comprises:

[0029] A first judgment unit is used to analyze the distance information, obtain distance feature data, and judge whether the living body or obstacle in front affects the user's current walking according to the distance feature data and the user's walking habits;

[0030] The distance feature data includes the relative angle and relative distance between the living body or obstacle in front and the user;

[0031] A first processing unit, configured to generate a trajectory change strategy when it is determined that a living body or obstacle ahead has an impact on the user's current walking;

[0032] The voice conversion unit is used to convert the distance feature data and the trajectory change strategy into a template voice based on the voice broadcast template, and generate a reminder audio based on the template voice.

[0033] Preferably, the data processing module in the intelligent blind guide glasses based on obstacle monitoring and reminder further includes: a video processing submodule, which is used to process the environmental video to obtain the video to be sent;

[0034] The video processing submodule comprises:

[0035] A second processing unit is used to extract the passable road features of the current environment based on the environment video, and mark the video road for passage according to the passable road features;

[0036] The video refinement element is used to determine the useful video frames of the environment video based on the traffic mark, remove the useless video frames, and perform video splicing on the useful video frames to obtain the final video to be sent.

[0037] Preferably, in a smart blind guide glasses based on obstacle monitoring and reminder, the first judgment unit includes:

[0038] The first prediction subunit obtains the user's historical walking record, calculates the user's walking speed, and determines the user's walking step length;

[0039] At the same time, multiple normal walking video records are intercepted from the historical walking data, and the walking trajectory details are described based on the multiple normal walking video records to obtain the walking routes of multiple users;

[0040] The second prediction subunit is used to determine the sway range of the user on each walking route according to the walking routes of multiple users, and calculate the average sway range to obtain the user's habitual walking safety range;

[0041] The data generating subunit is used to generate the user's walking habit data according to the user's walking speed, walking step length and habitual walking safety range.

[0042] Preferably, in the intelligent blind guide glasses based on obstacle monitoring and reminder, the first judgment unit further includes:

[0043] The influence judgment subunit is used to predict the walking radiation range of the user in the current environment based on the current environment characteristics and the walking habit data of the user, and when the living body or obstacle in front is within the walking radiation range, determine that the living body or obstacle in front has an impact on the current walking of the user;

[0044] Otherwise, it is determined that the living body or obstacle in front has no impact on the user's current walking.

[0045] Preferably, in a smart blind guide glasses based on obstacle monitoring and reminder, the video processing submodule further includes:

[0046] A scene identification unit, used for obtaining scene features of the current environment based on the video to be sent, comparing the scene features with preset features corresponding to the preset speech-banning scene, and determining whether the current environment is a speech-banning environment, and if so, determining that the video to be sent is a first-category video;

[0047] If not, obtain the environmental parameters of the current environment, and determine whether the current environment is a hidden danger environment based on the environmental parameters;

[0048] Among them, the environmental parameters include environmental noise value and environmental chaos prediction value;

[0049] When the environmental noise value or the environmental chaos prediction value is greater than or equal to the corresponding preset threshold, it is determined that the current environment is a hidden danger environment, and the video to be sent is a first-category video;

[0050] Otherwise, it is determined that the current environment is not a hidden danger environment, and the video to be sent is the second type of video;

[0051] The scene warning unit is used to add a danger signal to the video to be sent when the video to be sent is a first type of video.

[0052] Preferably, an intelligent communication module in an intelligent blind guide glasses based on obstacle monitoring and reminder includes:

[0053] A data sending unit is used to send the video to be sent to the network terminal processor, and when the video to be sent carries a danger signal, send an early warning notification to the guardian to remind the guardian to check the monitoring video;

[0054] The signal receiving unit is used to receive the remote navigation information of the guardian, and drive the voice broadcast module to start the earphone broadcast mode for remote voice broadcast through the central processing module. Compared with the prior art, the present invention has at least the following beneficial effects:

[0055] The present invention ensures that the data acquisition module, the data processing module, the voice broadcast module and the intelligent communication module can cooperate with each other to complete the navigation task for the blind and realize obstacle monitoring and reminder through the control of the central processing module.

[0056] The present invention collects the user's current environmental information through a data acquisition module, generates an environmental video to provide a remote navigation basis for the guardian, provides a judgment basis for obstacle monitoring, and obtains living objects or obstacles. The data processing module processes the collected distance information and environmental video, generates a reminder audio for navigation, and plays the reminder audio through a voice broadcast module to complete real-time obstacle monitoring and reminders, providing real-time navigation for the user. At the same time, according to the video to be sent obtained by processing the environmental video, the video to be sent is sent to the network terminal processor through the intelligent communication module, and remote navigation information is received, which is convenient for remote navigation guidance in various environments where the user is inconvenient or cannot freely speak, thereby improving the navigation reliability of the navigation glasses, improving the user experience, and ensuring the safety of the user.

[0057] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0058] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0060] Figure 1 A schematic diagram of intelligent blind-guiding glasses based on obstacle monitoring and reminder according to the present invention;

[0061] Figure 2 A schematic diagram of a data acquisition module of intelligent blind guide glasses based on obstacle monitoring and reminder according to the present invention;

[0062] Figure 3 A schematic diagram of a data processing module of intelligent blind guide glasses based on obstacle monitoring and reminder according to the present invention;

[0063] Figure 4 The present invention is a schematic diagram of an intelligent communication module of intelligent guide glasses based on obstacle monitoring and reminder. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] Embodiment 1:

[0066] A kind of intelligent blind guide glasses based on obstacle monitoring and reminder, such as Figure 1 As shown, it includes: a glasses body, and a data acquisition module, a data processing module, a voice broadcast module, an intelligent communication module and a central processing module arranged on the glasses body;

[0067] The data acquisition module is used to collect the user's current environment information based on the shooting module to obtain the environment video, and to measure the distance from the user to the living body or obstacle in front based on the TOF module and generate distance information;

[0068] The data processing module is used to process the distance information, generate a reminder audio, and process the environment video to obtain the video to be sent;

[0069] An intelligent communication module, used to send the video to be sent to the network terminal processor and receive remote navigation information;

[0070] Voice broadcast module, used to play reminder audio;

[0071] The central processing module is used to complete the entire process control of obstacle monitoring and reminders.

[0072] In this embodiment, there are two groups of shooting modules and two groups of TOF module measurements, which are respectively arranged on the temples of the smart guide glasses. Both can be rotated within a certain angle to expand the collection range of environmental information and ensure the accuracy of the distance detection between obstacles and users.

[0073] TOF (Time of Flight) refers to the time of flight method, which is a distance measurement method that measures the distance to an object by utilizing the time difference between the irradiated wave and the reflected wave with an accuracy of centimeters.

[0074] In this embodiment, the distance information includes the relative position information between the living body or obstacle and the user and the actual distance information.

[0075] In this embodiment, the environment video refers to the video of the current environment.

[0076] In this embodiment, the reminder audio is used to generate a voice for navigating the user according to the distance information.

[0077] In this embodiment, the video to be sent is a pre-processed environmental video sent to the user's guardian, and the guardian can view it through the network terminal processor to perform remote auxiliary navigation for the user.

[0078] In this embodiment, the reminder audio is played in two channels to ensure that the user can hear the complete playback audio with both ears, and to ensure that the guide eyes can complete the reminder in a noisy environment, thereby providing the user with correct road guidance.

[0079] The beneficial effects of the above embodiments are as follows: the present invention ensures that the data acquisition module, the data processing module, the voice broadcast module and the intelligent communication module can cooperate with each other to complete the navigation task for the blind and realize obstacle monitoring and reminder through the control of the central processing module; the present invention collects the current environmental information of the user through the data acquisition module, generates an environmental video to provide a remote navigation basis for the guardian, and provides a judgment basis for the monitoring of obstacles, and obtains the living body or obstacles. The data processing module processes the collected distance information and environmental video, generates a reminder audio for navigation, and plays the reminder audio through the voice broadcast module to complete real-time obstacle monitoring and reminder, and provide real-time navigation for the user. At the same time, according to the processing of the environmental video, the video to be sent is obtained, and the video to be sent is sent to the network terminal processor through the intelligent communication module, and the remote navigation information is received, which is convenient for remote navigation guidance in various environments where the user is inconvenient or cannot freely make voice, improves the navigation reliability of the navigation glasses, improves the user experience, and ensures the safety of the user.

[0080] Embodiment 2:

[0081] Based on Example 1, the data acquisition module, such as Figure 2 As shown, including:

[0082] The environment detection submodule is used to perform binocular vision shooting of the current environment based on the shooting module after the user arrives in a new environment, obtain binocular vision video, and integrate the binocular vision video to obtain the environment video;

[0083] The distance detection submodule is used to calculate the distance between the living object or obstacle in front and the user based on the detection laser emitted by the TOF module transmitting end to the living object or obstacle in front, and the detection feedback light received by the TOF module receiving end, obtaining the detection laser return time, and generating distance information.

[0084] In this embodiment, the binocular vision video refers to two videos captured by two shooting modules on the temples respectively, including two left and right monocular vision videos.

[0085] Beneficial effects of the above embodiments: The present invention collects the user's current environmental information through a data acquisition module, generates an environmental video to provide a remote navigation basis for the guardian, and provides a judgment basis for obstacle monitoring.

[0086] Embodiment 3:

[0087] Based on Example 2, the environment detection submodule includes:

[0088] A video processing unit is used to perform video framing on the binocular vision video to obtain two synchronously framed videos, and to compare the two synchronously framed videos frame by frame to obtain frame difference features;

[0089] A difference determination unit is used to mark the same object in each video frame based on the frame difference feature, mark the screen of the same object, and determine the shooting angle difference of the corresponding screen of each synchronous video frame of the two synchronous framed videos according to the screen marking result;

[0090] A stereoscopic fusion unit, used for fusing video images of the same object in each synchronous video frame of the synchronous frame video based on the difference in shooting angles to obtain a stereoscopic image;

[0091] A panoramic processing unit, used to determine the visual range difference between single-vision shooting based on frame difference features, take any single-vision shooting picture as a standard picture, and visually expand the standard picture based on the visual range difference to obtain a panoramic picture;

[0092] The image fusion unit is used to fuse the panoramic image with the stereoscopic image based on the corresponding relationship between the video frames of the panoramic image and the stereoscopic image to obtain a stereoscopic video frame image, and to splice all the stereoscopic video frames to obtain an environmental video.

[0093] In this embodiment, the synchronous framed video refers to the monocular vision video after frame division, and the monocular vision video refers to the video of any one vision of the binocular vision video.

[0094] In this embodiment, the frame difference feature refers to the picture differences and similarities between the corresponding video frames of two synchronous frame-by-frame videos.

[0095] In this embodiment, the picture marking refers to marking the same object on the picture corresponding to the video frames at the same time.

[0096] In this embodiment, the synchronous video frames refer to video frames of two synchronous frame-based videos shot at the same time point.

[0097] In this embodiment, the shooting angle difference refers to the difference in visual angles of two synchronous frame-based videos.

[0098] In this embodiment, the stereoscopic image refers to a stereoscopic image of the same object in a synchronous video frame.

[0099] In this embodiment, the visual range difference refers to the difference in viewing angle and visual range due to the different distribution of the left and right shooting components.

[0100] In this embodiment, visual expansion refers to supplementing the standard picture with a visible picture based on the standard picture according to the difference in visual range between another monocular visual picture and the monocular visual picture corresponding to the standard picture.

[0101] In this embodiment, the panoramic picture refers to the complete picture that can be seen in the binocular vision video.

[0102] In this embodiment, the stereoscopic video frame refers to a stereoscopic panoramic picture corresponding to the current video frame.

[0103] The beneficial effects of the above embodiments are as follows: the present invention obtains a binocular vision video through a shooting group, performs video framing on the binocular vision video to obtain two synchronously framed videos, performs frame-by-frame comparison on the two synchronously framed videos to obtain frame difference features, obtains a stereoscopic image and a panoramic picture of the corresponding video frame according to the frame difference features, and finally, based on the video frame correspondence between the panoramic picture and the stereoscopic image, fuses the panoramic picture and the stereoscopic image to obtain a stereoscopic video frame picture, and splices all the stereoscopic video frame pictures to obtain an environmental video, using a shooting component to replace a real human eye, thereby ensuring that the environmental video captured by the shooting component is closer to the observation video of a real human eye, and providing reliable data for obstacle monitoring and remote navigation guidance of the guardian.

[0104] Embodiment 4:

[0105] Based on Example 2, the distance detection submodule includes:

[0106] The obstacle judgment unit is used to obtain the real-time environment picture based on the shooting module, and the image recognition model recognizes the real-time environment picture to determine whether there are living things or obstacles in the user's current forward range;

[0107] An angle determination unit is used to obtain a stereoscopic video frame corresponding to the real-time image when it is determined that there is a living body or an obstacle in the current forward range of the user, mark the observation center of the stereoscopic video frame, and determine the distance angle between each living body or obstacle and the user in the stereoscopic video frame based on the observation center;

[0108] A sequence generation unit, used to determine the relative position of each living body or obstacle and the user in the stereoscopic video frame using the observation center as a reference point, and generate a detection distance sequence based on the relative position;

[0109] A detection trigger unit is used to generate a detection angle sequence based on a distance detection sequence. The TOF module transmitter emits a detection laser to a living body or an obstacle according to the detection angle sequence.

[0110] A detection feedback unit, used to receive the detection feedback light based on the receiving end of the TOF module and obtain the return time of the detection laser;

[0111] The distance calculation unit is used to detect the laser return time and calculate the relative distance between each living body or obstacle in front and the user:

[0112]

[0113] Where L0 represents the relative distance between the currently detected living body or obstacle and the user; t1 represents the detection laser return time of the TOF module set on the left; t2 represents the detection laser return time of the TOF module set on the right; c represents the speed of light, which is 3×10 8 m / s; E A is the emission energy of the detection light at the transmitting end of the TOF module; E B1 Indicates the feedback energy of the detection feedback light received by the receiving end of the TOF module arranged on the left side; E B2 represents the feedback energy of the detection feedback light received by the receiving end of the TOF module arranged on the right; θ1 represents the detection angle of the TOF module arranged on the left; θ2 represents the detection angle of the TOF module arranged on the left; Indicates detection error;

[0114] Based on the distance calculation result, distance information is generated.

[0115] In this embodiment, the distance angle refers to the relative angle between each living object or obstacle and the user in the stereoscopic video frame. A dividing line is drawn through the observation center to divide the stereoscopic video frame into equal left and right images. The angle between each living object or obstacle and the straight line is the relative angle between each living object or obstacle and the user.

[0116] In this embodiment, the relative position refers to the relative position between each living object or obstacle and the user in the stereoscopic video frame, which can roughly represent the distance between the living object or obstacle and the user. The detection distance sequence is the distance sequence detected according to the distance, and the closer the user is, the closer the distance is.

[0117] In this embodiment, the detection angle sequence refers to a sequence obtained by sorting the angles at which the TOF module emits the detection laser according to the obstacle monitoring order corresponding to the detection distance sequence.

[0118] In this embodiment, the feedback light refers to the laser reflected back after the detection feedback light contacts an obstacle (including a living body or an obstacle).

[0119] Beneficial effects of the above embodiments: The present invention obtains real-time environmental images based on the shooting module, and the image recognition model recognizes the real-time environmental images to determine whether there are living things or obstacles in the current forward range of the user, and only judges the obstacles within the effective range, avoiding useless operations, reducing the data processing pressure of the eyeglass navigation system, and improving the accuracy of data processing. And determine the relative position of the user and the obstacle, so as to determine the detection laser emission sequence of the TOF module, improve the output efficiency of the obstacle monitoring results, and at the same time, the order of obstacles is also guaranteed to ensure the orderliness of monitoring, and the nearby detection of obstacles improves the user's passage safety to a certain extent.

[0120] Embodiment 5:

[0121] Based on Example 1, the data processing module, such as Figure 3 As shown, it includes: a distance processing submodule, which is used to process the distance information and generate a reminder audio;

[0122] The distance processing submodule comprises:

[0123] A first judgment unit is used to analyze the distance information, obtain distance feature data, and judge whether the living body or obstacle in front affects the user's current walking according to the distance feature data and the user's walking habits;

[0124] The distance feature data includes the relative angle and relative distance between the living body or obstacle in front and the user;

[0125] A first processing unit, configured to generate a trajectory change strategy when the predicted living body or obstacle ahead affects the current walking of the user;

[0126] The voice conversion unit is used to convert the distance feature data and the trajectory change strategy into a template voice based on the voice broadcast template, and generate a reminder audio based on the template voice.

[0127] In this embodiment, the trajectory change strategy refers to a method for the user to change the route he / she is walking on based on the current walking.

[0128] In this embodiment, the voice broadcast template is a standard template for guiding the blind glasses to play the reminder voice. The template voice refers to the voice waiting to be played that is processed according to the voice broadcast template Jining standard.

[0129] The beneficial effects of the above embodiments are as follows: the present invention analyzes distance information, obtains distance feature data, and judges whether the living body or obstacle in front has an impact on the user's current walking according to the distance feature data and the user's walking habits, and is used to generate a trajectory change strategy when the predicted living body or obstacle in front has an impact on the user's current walking, and converts the distance feature data and the trajectory change strategy into a template voice based on a voice broadcast template, and generates a reminder audio based on the template voice to achieve the purpose of obstacle reminder and road navigation, thereby reducing the blind's dependence on infrastructure such as blind paths and expanding the range of scenarios in which the blind can walk freely.

[0130] Embodiment 6:

[0131] Based on Example 1, the data processing module, such as Figure 3 As shown, it also includes: a video processing submodule, which is used to process the environmental video to obtain the video to be sent;

[0132] The video processing submodule comprises:

[0133] A second processing unit is used to extract the passable road features of the current environment based on the environment video, and mark the video road for passage according to the passable road features;

[0134] The video refinement element is used to determine the useful video frames of the environment video based on the traffic mark, remove the useless video frames, and perform video splicing on the useful video frames to obtain the final video to be sent.

[0135] In this embodiment, the passable road characteristics refer to the distribution characteristics of the roads that pedestrians can walk on in the current environment.

[0136] In this embodiment, the pass mark refers to marking the passable roads in the environment video.

[0137] In this embodiment, the useful video frame refers to the video frame that includes the current environment's passable roads and their traffic conditions; the useless video frame refers to the video frame that does not include the current environment's passable roads and their traffic conditions.

[0138] The beneficial effects of the above embodiments: the present invention extracts the passable road features of the current environment based on the environmental video, marks the video roads according to the passable road features, determines the useful video frames of the environmental video by the pass marks, eliminates useless video frames, and splices the useful video frames to obtain the final video to be sent, thereby improving the effectiveness of the road passage guidance of the video to be sent, improving the efficiency of the guardian in obtaining effective information, and ensuring that in various environments where it is inconvenient or impossible for the user to speak freely, the guardian can quickly provide the user with reliable navigation guidance, thereby improving the user experience.

[0139] Embodiment 7:

[0140] Based on Embodiment 5, the first judging unit includes:

[0141] The first prediction subunit obtains the user's historical walking record, calculates the user's walking speed, and determines the user's walking step length;

[0142] At the same time, multiple normal walking video records are intercepted from the historical walking data, and the walking trajectory details are described based on the multiple normal walking video records to obtain the walking routes of multiple users;

[0143] The second prediction subunit is used to determine the sway range of the user on each walking route according to the walking routes of multiple users, and calculate the average sway range to obtain the user's habitual walking safety range;

[0144] The data generating subunit is used to generate the user's walking habit data according to the user's walking speed, walking step length and habitual walking safety range.

[0145] In this embodiment, the walking step length refers to the distance that a user can walk in one step.

[0146] In this embodiment, the normal walking video record refers to a video segment in which the user walks normally without changing the walking track.

[0147] In this embodiment, the walking trajectory detail description refers to describing the position change of each step of the user in the normal walking video record.

[0148] In this embodiment, the user's walking route refers to the walking trajectory obtained by describing the walking trajectory details in each normal walking video record.

[0149] In this embodiment, the swing range refers to the width of the change in the landing position of the left and right feet when a person walks normally due to the change in the landing location.

[0150] In this embodiment, the customary walking safety range refers to the distance corresponding to safe passage when the user normally passes by an obstacle without being hit.

[0151] The beneficial effects of the above embodiments: the present invention determines the user's walking speed, walking step length and habitual walking safety range based on the user's historical walking records, generates the user's walking habit data, facilitates more accurate judgment of whether the obstacles ahead affect the user's safe passage, provides the user with reliable obstacle reminders, and improves the navigation safety of the smart guide glasses.

[0152] Embodiment 8:

[0153] Based on Embodiment 5, the first determining unit further includes:

[0154] The influence judgment subunit is used to predict the walking radiation range of the user in the current environment based on the current environment characteristics and the walking habit data of the user, and when the living body or obstacle in front is within the walking radiation range, determine that the living body or obstacle in front has an impact on the current walking of the user;

[0155] Otherwise, it is determined that the living body or obstacle in front has no impact on the user's current walking.

[0156] In this embodiment, the walking radiation range refers to the occupied range in the left and right directions when the user walks normally.

[0157] The beneficial effects of the above embodiments are as follows: the present invention predicts the walking radiation range of the user in the current environment based on the current environmental characteristics and the user's walking habit data, determines whether the living objects or obstacles in front have an impact on the user's current walking, realizes obstacle monitoring, reduces the probability of the user being injured due to restricted movement, and improves the user's travel safety.

[0158] Embodiment 9:

[0159] Based on Example 6, the video processing submodule further includes:

[0160] A scene identification unit, used for obtaining scene features of the current environment based on the video to be sent, comparing the scene features with preset features corresponding to the preset speech-banning scene, and determining whether the current environment is a speech-banning environment, and if so, determining that the video to be sent is a first-category video;

[0161] If not, obtain the environmental parameters of the current environment, and determine whether the current environment is a hidden danger environment based on the environmental parameters; wherein the environmental parameters include the environmental noise value and the environmental chaos prediction value;

[0162] When the environmental noise value or the environmental chaos prediction value is greater than or equal to the corresponding preset threshold, it is determined that the current environment is a hidden danger environment, and the video to be sent is a first-category video;

[0163] Otherwise, it is determined that the current environment is not a hidden danger environment, and the video to be sent is the second type of video;

[0164] The scene warning unit is used to generate a user danger signal when the video to be sent is a first-class video.

[0165] In this embodiment, the scene feature refers to the characteristics of the scene corresponding to the current environment.

[0166] In this embodiment, the silent scene refers to a scene where it is inconvenient to speak or speak loudly, such as a lecture scene.

[0167] In this embodiment, the first type of video refers to the video to be sent when the user is currently in a silent scene or a hidden dangerous environment.

[0168] In this embodiment, the hidden danger environment refers to an environment with a high noise value where the user may not be able to hear the warning voice clearly, or an environment with a complex environment where the difficulty of passing is high.

[0169] In this embodiment, the second type of video refers to a video to be sent that allows the user to pass through the guide glasses normally and does not hide any danger.

[0170] In this embodiment, the danger signal refers to an early warning signal indicating that the user currently needs remote assistance from a guardian.

[0171] The beneficial effects of the above embodiments: the present invention obtains the scene characteristics of the current environment according to the video to be sent, determines whether the current environment is a voice-disabled environment, and classifies the video to be sent, and when the current environment is a non-voice-disabled environment, obtains the environmental parameters of the current environment, and based on the environmental parameters, determines whether the current environment is a hidden danger environment, judges the danger of the non-language-disabled environment, obtains the classification result of the video to be sent, intelligently distinguishes the current environment of the user, and when the video to be sent is a first-category video, adds a danger signal to the video to be sent, to ensure that the guardian can promptly discover the user's current predicament, promptly check the monitoring video, and provide the user with timely remote navigation guidance.

[0172] Embodiment 10:

[0173] Based on Example 1, the intelligent communication module, such as Figure 4 As shown, including:

[0174] A data sending unit is used to send the video to be sent to the network terminal processor, and when the video to be sent carries a danger signal, send an early warning notification to the guardian to remind the guardian to check the monitoring video;

[0175] The signal receiving unit is used to receive the remote navigation information of the guardian, and drive the voice broadcast module through the central processing module to turn on the earphone broadcast mode for remote voice playback.

[0176] In this embodiment, the monitoring video refers to the video to be sent to the network terminal processor.

[0177] The beneficial effects of the above embodiments are as follows: the present invention sends the video to be sent to the network terminal processor, and when the video to be sent carries a danger signal, sends an early warning notification to the guardian, reminding the guardian to check the monitoring video, ensuring that the guardian can promptly discover the user's current predicament, promptly check the monitoring video, and provide the user with timely remote navigation guidance, thereby realizing the automatic danger alarm function to a certain extent and improving the user's travel safety; the remote navigation information of the guardian is received through the signal receiving unit, and the voice broadcast module is driven by the central processing module to start the headphone broadcast mode for remote voice playback, ensuring that the remote navigation voice transmission can be completed in a silent environment without affecting others.

[0178] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A smart blind guide glasses based on obstacle monitoring and reminder, characterized in that: include: The glasses body, and the data acquisition module, data processing module, voice broadcast module, intelligent communication module and central processing module arranged on the glasses body; The data acquisition module is used to collect the user's current environment information based on the shooting module to obtain the environment video, and to measure the distance from the user to the living body or obstacle in front based on the TOF module and generate distance information; The data processing module is used to process the distance information, generate a reminder audio, and process the environment video to obtain the video to be sent; An intelligent communication module, used to send the video to be sent to the network terminal processor and receive remote navigation information; Voice broadcast module, used to play reminder audio; Central processing module, used to complete the whole process control of obstacle monitoring and reminder; The data processing module includes: a video processing submodule, which is used to process the environment video to obtain the video to be sent; The video processing submodule comprises: A second processing unit is used to extract the passable road features of the current environment based on the environment video, and mark the video road for passage according to the passable road features; A video refinement element is used to determine useful video frames of the environment video based on the traffic mark, remove useless video frames, and perform video splicing on the useful video frames to obtain the final video to be sent; The video processing submodule includes: A scene identification unit, used for obtaining scene features of the current environment based on the video to be sent, comparing the scene features with preset features corresponding to the preset speech-banning scene, and determining whether the current environment is a speech-banning environment, and if so, determining that the video to be sent is a first-category video; If not, obtain the environmental parameters of the current environment, and determine whether the current environment is a hidden danger environment based on the environmental parameters; Among them, the environmental parameters include environmental noise value and environmental chaos prediction value; When the environmental noise value or the environmental chaos prediction value is greater than or equal to the corresponding preset threshold, it is determined that the current environment is a hidden danger environment, and the video to be sent is a first-category video; Otherwise, it is determined that the current environment is not a hidden danger environment, and the video to be sent is the second type of video; A scene warning unit, used for adding a danger signal to the video to be sent when the video to be sent is a first type of video; The video to be sent is sent to the network terminal processor through the intelligent communication module, and the remote navigation information is received. The intelligent communication module includes: a data sending unit, which is used to send the video to be sent to the network terminal processor, and when the video to be sent carries a danger signal, send an early warning notification to the guardian to remind the guardian to check the monitoring video; The signal receiving unit is used to receive the remote navigation information of the guardian, and drive the voice broadcast module through the central processing module to turn on the earphone broadcast mode for remote voice playback.

2. The intelligent blind guide glasses based on obstacle monitoring and reminder according to claim 1, characterized in that: Data acquisition module, including: The environment detection submodule is used to perform binocular vision shooting of the current environment based on the shooting module after the user arrives in a new environment, obtain binocular vision video, and integrate the binocular vision video to obtain the environment video; The distance detection submodule is used to calculate the distance between the living object or obstacle in front and the user based on the detection laser emitted by the TOF module transmitting end to the living object or obstacle in front, and the detection feedback light received by the TOF module receiving end, obtaining the detection laser return time, and generating distance information.

3. The intelligent blind-guiding glasses based on obstacle monitoring and reminder according to claim 2, characterized in that: Environmental detection submodule, including: A video processing unit is used to perform video framing on the binocular vision video to obtain two synchronously framed videos, and to compare the two synchronously framed videos frame by frame to obtain frame difference features; A difference determination unit is used to mark the same object in each video frame based on the frame difference feature, mark the screen of the same object, and determine the shooting angle difference of the corresponding screen of each synchronous video frame of the two synchronous framed videos according to the screen marking result; A stereoscopic fusion unit, used for fusing video images of the same object in each synchronous video frame of the synchronous frame video based on the difference in shooting angles to obtain a stereoscopic image; A panoramic processing unit, used to determine the visual range difference between single-vision shooting based on frame difference features, take any single-vision shooting picture as a standard picture, and visually expand the standard picture based on the visual range difference to obtain a panoramic picture; The image fusion unit is used to fuse the panoramic image with the stereoscopic image based on the corresponding relationship between the video frames of the panoramic image and the stereoscopic image to obtain a stereoscopic video frame image, and to splice all the stereoscopic video frames to obtain an environmental video.

4. The intelligent blind-guiding glasses based on obstacle monitoring and reminder according to claim 2, characterized in that: Distance detection submodule, including: The obstacle judgment unit is used to obtain the real-time environment picture based on the shooting module, and the image recognition model recognizes the real-time environment picture to determine whether there are living things or obstacles in the user's current forward range; An angle determination unit is used to obtain a stereoscopic video frame corresponding to the real-time image when it is determined that there is a living body or an obstacle in the current forward range of the user, mark the observation center of the stereoscopic video frame, and determine the distance angle between each living body or obstacle and the user in the stereoscopic video frame based on the observation center; A sequence generation unit, used to determine the relative position of each living body or obstacle and the user in the stereoscopic video frame using the observation center as a reference point, and generate a detection distance sequence based on the relative position; A detection trigger unit is used to generate a detection angle sequence based on a distance detection sequence. The TOF module transmitter emits a detection laser to a living body or an obstacle according to the detection angle sequence. A detection feedback unit, used to receive the detection feedback light based on the receiving end of the TOF module and obtain the return time of the detection laser; The distance calculation unit is used to detect the laser return time and calculate the distance between each living body or obstacle in front and the user to generate distance information.

5. The intelligent blind-guiding glasses based on obstacle monitoring and reminder according to claim 1, characterized in that: The data processing module includes: a distance processing submodule, which is used to process the distance information and generate a reminder audio; The distance processing submodule comprises: A first judgment unit is used to analyze the distance information, obtain distance feature data, and judge whether the living body or obstacle in front affects the user's current walking according to the distance feature data and the user's walking habits; The distance feature data includes the relative angle and relative distance between the living body or obstacle in front and the user; A first processing unit, configured to generate a trajectory change strategy when it is determined that a living body or obstacle ahead has an impact on the user's current walking; The voice conversion unit is used to convert the distance feature data and the trajectory change strategy into a template voice based on the voice broadcast template, and generate a reminder audio based on the template voice.

6. The intelligent blind-guiding glasses based on obstacle monitoring and reminder according to claim 5, characterized in that: The first judging unit comprises: The first prediction subunit obtains the user's historical walking record, calculates the user's walking speed, and determines the user's walking step length; At the same time, multiple normal walking video records are intercepted from the historical walking data, and the walking trajectory details are described based on the multiple normal walking video records to obtain the walking routes of multiple users; The second prediction subunit is used to determine the sway range of the user on each walking route according to the walking routes of multiple users, and calculate the average sway range to obtain the user's habitual walking safety range; The data generating subunit is used to generate the user's walking habit data according to the user's walking speed, walking step length and habitual walking safety range.

7. The intelligent blind-guiding glasses based on obstacle monitoring and reminder according to claim 5, characterized in that: The first judging unit further includes: The influence judgment subunit is used to predict the walking radiation range of the user in the current environment based on the current environment characteristics and the walking habit data of the user, and when the living body or obstacle in front is within the walking radiation range, determine that the living body or obstacle in front has an impact on the current walking of the user; Otherwise, it is determined that the living body or obstacle in front has no impact on the user's current walking.

Citation Information

Patent Citations

  • Blind autonomous navigation method based on stereoscopic vision and information fusion

    CN101701828A

  • Wearable intelligent sensing blind guiding system

    CN112932910A

  • Blind guiding glasses and blind guiding system thereof

    CN210750149U

Cited By

  • Blind sidewalk keeping control method of intelligent blind guiding glasses

    CN121857273A