Intelligent blind guiding glasses, intelligent blind guiding system and intelligent blind guiding method

Abstract

The invention provides intelligent blind guiding glasses, an intelligent blind guiding system and an intelligent blind guiding method. The pair of intelligent blind guiding glasses comprises a glasses frame main body, a detection module, a control module and a voice guiding module, the detection module comprises an obstacle avoidance sensor and an audio and video sensor which are arranged in the middle of the glasses frame; the obstacle avoidance sensor and the audio and video sensor are respectively connected with the first control panel and / or the second control panel through the flexible circuit board; the control module controls the audio and video sensor to start image acquisition based on the received voice starting instruction, transmits distance data received from the obstacle avoidance sensor and video image data received from the audio and video sensor to the user terminal, and receives audio guide data generated based on the distance data and the video image data from the user terminal; and the voice guide module control module receives and plays the audio guide data so as to guide the blind to advance. The intelligent blind guiding glasses are novel in structure and easy to use, and the user experience of blind guiding users can be effectively improved.

Description

Technical Field

[0001] This application belongs to the field of data analysis technology, and in particular relates to a smart guide glasses, a smart guide system, and a smart guide method. Background Technology

[0002] Blind people face numerous difficulties in daily life, such as limited mobility and access to information. These difficulties create significant obstacles for them in their lives, work, and social participation. Currently, blind people mainly rely on canes and guide dogs for mobility. Canes have drawbacks such as a small detection range, low detection efficiency, and the tendency to overlook obstacles at heights. Guide dogs require extensive training, are expensive, and are troublesome to care for and maintain.

[0003] Therefore, in order to improve the quality of life and social participation of blind people, it has become an urgent need to develop assistive devices and guidance methods suitable for blind people. Summary of the Invention

[0004] This application provides intelligent guide glasses, an intelligent guide system, and an intelligent guide method to improve the user experience for guide users.

[0005] In a first aspect, embodiments of this application provide intelligent guide glasses for the blind, comprising: a frame body, a detection module, a control module, and a voice guidance module; wherein: the frame body includes a frame and a first temple and a second temple connected to both sides of the frame; the detection module includes an obstacle avoidance sensor and an audio-visual sensor disposed in the middle of the frame; the control module includes a first control board and a second control board respectively installed inside the first temple and the second temple, and at least one function button installed on the outer surface of the first temple and the second temple; wherein the obstacle avoidance sensor and the audio-visual sensor are respectively connected to the first control board and / or the second control board via a flexible circuit board; the control module controls the audio-visual sensor to start image acquisition based on a received voice start command, transmits distance data received from the obstacle avoidance sensor and video image data received from the audio-visual sensor to a user terminal, and receives audio guidance data generated based on the distance data and the video image data from the user terminal; the voice guidance module is connected to the control module, receives and plays the audio guidance data from the control module to guide the blind person forward.

[0006] In one implementation of the first aspect, a first control board is provided at the front end of the interior of the first temple, and a first battery module connected to the first control board is provided at the rear end; a second control board is provided at the front end of the interior of the second temple, and a second battery module connected to the second control board is provided at the rear end; wherein, a magnetic charging interface for charging the first battery module and the second battery module is provided at the position corresponding to the first control board or the position corresponding to the second control board of the first temple.

[0007] In one implementation of the first aspect, the first control board is a camera function control board for controlling the audio and video sensor to acquire images, and the second control board is a voice function control board for controlling the playback of the audio guidance data.

[0008] In one implementation of the first aspect, the voice guidance module includes an earphone connected to the second control board; the second temple is provided with an earphone hole for the earphone cable to pass through, and a magnet is provided at the bottom of the outer surface of the second temple to attract the earbud portion of the earphone.

[0009] In one implementation of the first aspect, the function buttons include: at least one first temple function button disposed on the outside of the first temple and electrically connected to the first control board, and at least one second temple function button disposed on the outside of the second temple and electrically connected to the second control board.

[0010] In one implementation of the first aspect, the frame is provided with a frame signal slot for accommodating the flexible circuit board, the first temple is provided with a first temple signal slot for accommodating the flexible circuit board, and the second temple is provided with a second temple signal slot for accommodating the flexible circuit board; the frame signal slot, the first temple signal slot, and the second temple signal slot are connected to each other.

[0011] In one implementation of the first aspect, the frame has at least one sound transmission hole in the middle, and a sound-transmitting membrane is attached to the sound transmission hole.

[0012] In one implementation of the first aspect, the control module further includes: a GPS positioning module; the control module receives GPS data from the GPS positioning module and transmits the GPS data to the user terminal, so that the user terminal can generate the audio guidance data in combination with the GPS data.

[0013] Secondly, embodiments of this application provide an intelligent guidance system for the blind, comprising: intelligent guidance glasses as described above and a user terminal wirelessly communicating with the intelligent guidance glasses; the user terminal is configured to: identify obstacles in an image based on video image data received from the intelligent guidance glasses, and generate obstacle identification result broadcast data; generate corresponding distance broadcast data based on distance data received from the intelligent guidance glasses and a preset distance alarm level; generate navigation data based on the video image data and GPS data received from the intelligent guidance glasses; and generate audio guidance data by combining the obstacle identification result broadcast data, the distance broadcast data, and the navigation data, and feed it back to the intelligent guidance glasses, so that the intelligent guidance glasses can broadcast prompts for obstacles around the blind person, broadcast prompts for the distance of obstacles, and broadcast prompts for navigation routes.

[0014] Thirdly, embodiments of this application provide an intelligent guidance method for the blind, applied to the intelligent guidance glasses described above, comprising: responding to receiving a voice activation command, powering on the intelligent guidance glasses and activating the audio-visual sensors in the intelligent guidance glasses; transmitting distance data collected by the obstacle avoidance sensor, video image data collected by the audio-visual sensors, and GPS data to a user terminal; receiving audio guidance data generated by the user terminal based on the distance data, the video image data, and the GPS data from the user terminal, and broadcasting the audio guidance data to provide prompts for obstacles around the blind person, to provide prompts for the distance to obstacles, and to provide navigation prompts for the travel route.

[0015] The intelligent guide glasses, intelligent guide system, and intelligent guide method provided in this application have the following beneficial effects:

[0016] In this application, blind users can control the use of guide glasses via voice. The guide glasses provide voice prompts for obstacles around the blind user, as well as for the distance to obstacles and for navigation of the route. They can also provide remote guidance services to the blind. Moreover, the intelligent guide glasses of this application have a novel structure and are easy to use, which can effectively improve the user experience of guide glasses users. Attached Figure Description

[0017] Figure 1 The diagram shown is an application illustration of the smart guide glasses of this application in one embodiment.

[0018] Figure 2 The image shown is a front view of one embodiment of the smart guide glasses of this application.

[0019] Figure 3 The image shown is a right view of one embodiment of the smart guide glasses of this application.

[0020] Figure 4 The image shown is a left view of one embodiment of the smart guide glasses of this application.

[0021] Figure 5 The diagram shown illustrates the interaction principle of the intelligent guide system for the visually impaired in one embodiment of this application.

[0022] Figure 6 The diagram shown is an example of an interaction process in one embodiment of the intelligent guide system for the visually impaired according to this application.

[0023] Figure 7 The diagram shown is a schematic block diagram of a user terminal in one embodiment of the intelligent guide system for the visually impaired according to this application.

[0024] Figure 8 The flowchart shown is an embodiment of the intelligent guide method for the visually impaired according to this application.

[0025] Component designation explanation

[0026] 100 Smart Guide Glasses

[0027] 101. Frame body

[0028] 102 Detection Module

[0029] 103 Control Module

[0030] 104 Voice Guidance Module

[0031] 110 Eyeglass Frames

[0032] 111 Obstacle Avoidance Sensor

[0033] 112 Audio / Video Sensors

[0034] 113 Frame Signal Slot

[0035] 120 First temple

[0036] 121 First Control Board

[0037] 122 First Battery Module

[0038] 123, 124 First temple function buttons

[0039] 125 headphone jack

[0040] 126 headphone cable

[0041] 127 magnets

[0042] 130 Second temple

[0043] 131 Second Control Board

[0044] 132 Second Battery Module

[0045] 133, 134 Second temple function buttons

[0046] 135 Magnetic Charging Interface

[0047] 200 user terminals

[0048] 201 processor

[0049] 202 Memory

[0050] 2021 Operating System

[0051] 2022 Applications

[0052] 203 Network Interface

[0053] 204 bus system

[0054] 205 User Interface

[0055] S100~S300 Steps Detailed Implementation

[0056] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0057] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, "first control board" and "second control board" are used only to distinguish different control boards and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0058] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0059] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0060] The following will refer to the appendices in the embodiments of this application. Figure 1 To be continued Figure 8 This application provides a detailed description of the smart guide glasses, smart guide system, and smart guide method in the embodiments of this application.

[0061] This embodiment provides a smart guide glasses for the blind. Figure 1 The diagram shown illustrates the application of the smart guide glasses of this application in one embodiment. Figure 1 As shown, in this embodiment, the smart guide glasses 100 are wirelessly connected to a user terminal 200. The user terminal 200 can be, but is not limited to, a smartphone, tablet, desktop computer, or laptop. Specifically, the application on the user terminal 200 can be software or an application running on a user-controlled terminal device or computing device, such as a personal computer (PC) client, a web client accessed via a browser, an application (APP) client running on a mobile terminal, or a cloud platform console. This application does not impose any specific limitations. With the assistance of the user terminal 200, the smart guide glasses 100 provide guidance services for the visually impaired.

[0062] like Figure 1 As shown, the intelligent guide glasses 100 includes a frame body, a detection module, a control module, and a voice guidance module. It should be understood that the module division in this embodiment is illustrative and represents only a logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0063] Specifically, in this embodiment, the frame body includes a frame and a first temple and a second temple connected to both sides of the frame; the detection module includes an obstacle avoidance sensor and an audio-visual sensor disposed in the middle of the frame; the control module includes a first control board and a second control board respectively installed inside the first temple and the second temple, and at least one function button installed on the outer surface of the first temple and the second temple; wherein, the obstacle avoidance sensor and the audio-visual sensor are respectively connected to the first control board and / or the second control board through a flexible circuit board; the control module controls the audio-visual sensor to start image acquisition based on a received voice start command, transmits the distance data received from the obstacle avoidance sensor and the video image data received from the audio-visual sensor to the user terminal, and receives audio guidance data generated based on the distance data and the video image data from the user terminal; the voice guidance module is connected to the control module, receives and plays the audio guidance data from the control module to guide the blind person forward.

[0064] Specifically, in this embodiment, as Figures 2 to 4 As shown, the smart guide glasses 100 includes a frame 110 and a first temple 120 and a second temple 130 connected to both sides of the frame 110. The frame 110, the first temple 120, and the second temple 130 adopt a master-slave structure design with a main body and a cover plate. The connecting parts are fixed by multiple buckles and screws, and the positions of the devices (the physical hardware structure of the detection module and the control module) placed inside the frame 110, the first temple 120, and the second temple 130 are fixed by positioning holes and positioning slots. The advantage of this is that the main structure of the smart guide glasses 100 is supported and not easily deformed.

[0065] Figure 2 The image shown is a front view of the smart guide glasses 100 of this application in one embodiment. Figure 2 As shown, an obstacle avoidance sensor 111 and an audio / video sensor 112 are disposed in the middle of the frame 110. The smart guide glasses 100 collects obstacle data through the obstacle avoidance sensor 111 and collects audio / video data through the audio / video sensor 112. The obstacle avoidance sensor 111 is, but is not limited to, an ultrasonic obstacle avoidance sensor, which measures distance by sending ultrasonic signals and receiving their echoes. The audio / video sensor 112 refers to a device capable of capturing sound and images. In this embodiment, the audio / video sensor 112 includes an audio sensor and a video sensor. For example, the audio sensor is a microphone or microphone array; the video sensor is a camera or webcam, which captures images through a photosensitive element such as a CCD or CMOS sensor.

[0066] Specifically, the obstacle avoidance sensor 111 uses an ultrasonic obstacle avoidance sensor for detection. Preferably, the obstacle avoidance sensor 111 is a radar. The frame 110 is provided with a radar mounting hole, and the radar is installed in the radar mounting hole. The smart guide glasses 100 uses ultrasonic obstacle detection. After the obstacle avoidance sensor 111 emits ultrasonic waves, they are reflected back when they encounter an obstacle. After receiving the reflected signal, the obstacle avoidance sensor 111 calculates the distance from the obstacle to the obstacle avoidance sensor 111 based on the sound propagation time, thereby obtaining the distance between the smart guide glasses 100 and the obstacle.

[0067] In one possible implementation, the audio / video sensor 112 includes a microphone and a camera.

[0068] In one possible implementation, voice wake-up and voice / video functions are used to control the smart guide glasses 100. For example, calling out "Xiaoguang Xiaoguang" wakes up the smart guide glasses 100 and puts it into voice recognition mode. The smart guide glasses 100 then announces "Please speak," prompting the user to operate the smart guide glasses 100 using voice commands. For example, saying "Open video" will cause the smart guide glasses 100 to enter video capture and playback mode.

[0069] The frame 110 has two sensor detection holes located at the center of its outer side, for mounting an obstacle avoidance sensor 111 and an audio / video sensor 112, respectively. In this embodiment, the sensor detection holes at the center of the outer side of the frame 110 allow for diverse frame styles, and the microphone is positioned closest to the sound source, effectively preventing echoes.

[0070] In one possible implementation, a dust cover is installed on the sensor detection hole to prevent dust, dirt, and other particles from entering the sensor. The dust cover not only prevents dust but also increases the lifespan of the obstacle avoidance sensor 111 and the audio / video sensor 112.

[0071] In one possible implementation, the frame 110 has at least one sound-transmitting hole in the center (e.g., corresponding to the microphone position) for mounting a microphone or improving sound transmission. The microphone can be installed in the sound-transmitting hole on the eyeglass frame. This design allows the wearer to more easily receive and transmit sound. Exemplarily, the cover plate of the frame 110 has three rows of sound-transmitting holes. Each sound-transmitting hole is fitted with a sound-permeable membrane, a polymer film material that provides waterproofing while maintaining clear sound quality. The sound-permeable membrane transmits sound through its microporous structure; these micropores are smaller than water molecules, thus preventing moisture ingress while allowing sound to pass through. Therefore, in this embodiment, the sound-permeable membrane provides dust and water protection while ensuring unobstructed sound transmission.

[0072] Figure 3 The image shown is a right-side view of one embodiment of the smart guide glasses 100 of this application. Figure 3 As shown, a first control board 121 is provided at the front end inside the first temple 120, and a first battery module 122 connected to the first control board 121 is provided at the rear end. Figure 4 The image shown is a left view of one embodiment of the smart guide glasses 100 of this application. Figure 4 As shown, a second control board 131 is provided at the front end inside the second temple 130, and a second battery module 132 connected to the second control board 131 is provided at the rear end; that is, in the smart guide glasses 100 of this embodiment, an obstacle avoidance sensor 111 and an audio-visual sensor 112 are placed in the middle of the frame 110, and functional modules are provided on the two temples to control the functions of the smart guide glasses 100.

[0073] In this embodiment, the obstacle avoidance sensor 111 and the audio / video sensor 112 are respectively connected to the first control board 121 and / or the second control board 131 via a flexible circuit board. That is, the sensor data is transmitted to the first control board 121 and the second control board 131 inside the two temples via the flexible circuit board. The flexible circuit board and sensors can be designed to adapt to the shape of the smart guide glasses, making the smart guide glasses more suitable for the user's needs. Furthermore, the flexible circuit board allows the smart guide glasses to be thinner, more flexible, and more stretchable.

[0074] In one possible implementation, the frame 110 is provided with a frame signal slot 113 for accommodating the flexible circuit board, the first temple 120 is provided with a first temple 120 signal slot for accommodating the flexible circuit board, and the second temple 130 is provided with a second temple 130 signal slot for accommodating the flexible circuit board; wherein the frame signal slot 113, the first temple 120 signal slot, and the second temple 130 signal slot are connected.

[0075] In this embodiment, the top of the frame 110 has a cover plate corresponding to the position of the frame signal groove 113, and the cover plate of the frame 110 locks the frame signal groove 113.

[0076] In one possible implementation, the frame signal slot 113 is located inside the upper edge of the frame 110. That is, the flexible circuit board is placed in the signal slot on the upper edge of the frame 110 and secured with a cover plate.

[0077] In this embodiment, the first temple 120 has a first temple 120 cover plate corresponding to the position of the first temple 120 signal slot, and the first temple 120 cover plate locks the first temple 120 signal slot; the second temple 130 has a second temple 130 cover plate corresponding to the position of the second temple 130 signal slot, and the second temple 130 cover plate locks the second temple 130 signal slot.

[0078] The flexible circuit board is connected to the first control board 121 in the first temple 120 and the second control board 131 in the second temple 130. After the flexible circuit board is installed in the signal slot of the first temple 120, the cover plate of the first temple 120 is used to lock the signal slot of the first temple 120. After the flexible circuit board is installed in the signal slot of the second temple 130, the cover plate of the second temple 130 is used to lock the signal slot of the second temple 130.

[0079] The first temple 120 and the second temple 130 are bent relative to each other via a pivot to fold the smart guide glasses 100. The flexible circuit board is installed in the frame signal slot 113, the first temple 120 signal slot, and the second temple 130 signal slot. During installation, the flexible circuit board passes through the pivots on the first temple 120 and the second temple 130, respectively. The pivot holes are aligned with the mounting screws for fixation. The flexible circuit board passes through the pivots on the temples to achieve circuit connection while maintaining the foldability of the temples.

[0080] In this way, the signal wires of the smart guide glasses 100 are concealed inside the housing of the smart guide glasses 100, improving the safety and aesthetics of the smart guide glasses 100.

[0081] In one possible implementation, the first control board 121 is a camera function control board for controlling the audio and video sensor 112 to acquire images, and the second control board 131 is a voice function control board for playing corresponding guide voice based on the data acquired by the obstacle avoidance sensor 111 and the audio and video sensor 112.

[0082] In one possible implementation, such as Figure 3 As shown, at least one first temple function button 123 and a first temple function button 124, which are electrically connected to the first control board 121, are provided on the outside of the first temple 120; as Figure 4 As shown, at least one second temple 130 function button 133 and second temple function button 134 are provided on the outside of the second temple 130 and are electrically connected to the second control board 131.

[0083] In one possible implementation, the first temple function button 123, the first temple function button 124, the second temple function button 133, and the second temple function button 134 are touch-sensitive to realize multiple controls on the smart guide glasses 100. This can effectively reduce the complexity of controlling the smart guide glasses 100 and improve the flexibility of appearance design.

[0084] The first temple function buttons 123 and 124 are not limited to music playback control buttons. For example, there may be two first temple function buttons 123 and 124, which are respectively the previous track control button and the next track control button. In this way, the smart guide glasses 100 can freely control playback when playing audio information or receiving audio playback information from the user terminal 200.

[0085] In one possible implementation, the first control board 121 is connected to an earphone cable 126, and the second temple 130 is provided with an earphone hole 125 through which the earphone cable 126 passes. In this embodiment, for example, the earphone hole 125 is located diagonally below the first temple 120.

[0086] In one possible implementation, a magnet 127 is provided at the bottom of the outer surface of the second temple 130 to attract the earpiece of the earphone. The second control board 131 is equipped with an earphone, which receives audio through the earpiece. The earpiece is led out through a through hole at the lower edge of the temple, and the magnet 127 below the second control board 131 can attract the earpiece. This reduces interference with the voice signal and facilitates voice recognition. When the earpiece is not in use, the temple magnetically holds the earpiece, enhancing the overall aesthetics of the smart guide glasses 100. In this embodiment, by providing a magnet at the bottom of the outer surface of the temple, the earpiece of the earphone can be easily fixed to the smart guide glasses. This allows the user to easily store the earphone on the glasses when not in use, preventing the earpiece from being lost or soiled. The use of magnets also helps improve user safety. For example, during sports or outdoor activities, the user can quickly fix the earpiece to the glasses using magnetic force, preventing the earpiece from being lost due to large movements.

[0087] Among them, the function buttons 133 and 134 of the second temple 130 are not limited to playback control and power switch control buttons. For example, there are two function buttons 133 and 134 of the second temple 130, which are the play / stop control button and the power switch control button, respectively.

[0088] In this embodiment, the first control board 121 and the second control board 131 on the temple are connected by a flexible circuit board, share two rechargeable lithium batteries, and charge the rechargeable lithium batteries to increase the battery life of the smart guide glasses 100.

[0089] In one possible implementation, the first battery module 122 and the second battery module 132 are button batteries or rechargeable batteries, wherein the rechargeable battery can be removed for charging, and the rechargeable battery is a rechargeable lithium battery. That is, the rechargeable lithium batteries placed at the tips of the first temple 120 and the second temple 130 power the first control board 121 and the second control board 131 on the temples. This makes signal routing easier and the structure simpler. Furthermore, since the first battery module 122 and the second battery module 132 use rechargeable lithium batteries, the smart guide glasses 100 can be charged at any time, ensuring its continuous operation.

[0090] In one possible implementation, the rechargeable lithium battery can be directly charged via the first control board 121 or the second control board 131. Specifically, this embodiment further includes a magnetic charging interface 135 for charging the first battery module 122 and the second battery module 132. The magnetic charging interface 135 is disposed at the position where the first temple 120 is located corresponding to the first control board 121 or the position where the second temple 130 is located corresponding to the second control board 131.

[0091] The magnetic charging interface 135 is a type-c magnetic charging interface 135. The magnetic head is inserted into the type-c charging port, and a USB charging cable with a magnetic head is used for charging, which can reduce the complexity of the charging connection.

[0092] In one possible implementation, the first control board 121 or the second control board 131 is equipped with a wireless communication module. The smart guide glasses 100 transmits the detection data of the obstacle avoidance sensor 111 and the audio-visual sensor 112 to the user terminal 200 through the wireless communication module, and receives voice signals for guidance from the user terminal 200.

[0093] In this embodiment, the wireless communication module includes 4G, 5G, Wi-Fi, and Bluetooth modules.

[0094] The first control board 121 and / or the second control board collect obstacle avoidance data and audio / video data, and transmit them to the user terminal 200 via wireless transmission technology (BT, Wi-Fi). The user terminal 200 processes the received data, generates voice guidance information, and feeds back the voice results to the wearer of the smart guide glasses 100 via wireless transmission technology.

[0095] Taking a mobile phone as an example, the wireless communication module establishes a wireless transmission process with the user terminal 200 as follows: The mobile APP establishes a Wifi-Direct (also known as Wi-Fi P2P) connection and then scans for the smart guide glasses 100 via Bluetooth. After the Bluetooth connection is successful, the mobile APP transmits the Wifi name and password of Wifi-Direct to the smart guide glasses 100 via Bluetooth. The smart guide glasses 100 connects to Wifi-Direct according to the received Wifi name and password, thereby realizing the mutual connection and data transmission between the mobile APP and the smart guide glasses 100.

[0096] In this embodiment, the smart guide glasses 100 can convert information about images and obstacles into speech and play it.

[0097] Preferably, the smart guide glasses 100 can transmit image and obstacle information to the user terminal 200. The user terminal 200 converts the image and obstacle information in front of the glasses into voice prompts and returns them to the smart guide glasses 100 via wireless technology. For example, the obstacle avoidance sensor 111 of the smart guide glasses 100 detects the distance to an obstacle and transmits the data to the user terminal 200 APP via the Bluetooth module of the smart guide glasses 100. The user terminal 200 APP analyzes and processes the received data and plays a prompt sound through the Bluetooth headset of the smart guide glasses 100 to remind the user of the obstacle information. If the obstacle is within 1 meter, the smart guide glasses 100 emits a beeping alarm. If the obstacle is between 1 meter and 3.5 meters, the smart guide glasses 100 reports the actual distance of the obstacle in decimeters, for example, if the obstacle is 2.0 meters, the smart guide glasses 100 will announce 20. When the distance exceeds 3.5 meters, the user terminal 200APP uses Wi-Fi to wirelessly transmit and call the smart guide glasses 100 camera module to collect images in front. Through image recognition, it analyzes what the obstacle is and plays sound through Bluetooth headphones. The wearer receives prompts through the earpiece, thus effectively guiding the blind (or visually impaired) in their activities.

[0098] In addition, the smart guide glasses 100 can also acquire GPS data and transmit it to the user terminal 200 APP via public networks (4G, 5G, Wi-Fi). The assistant can then use the user terminal APP to provide voice guidance or assistance to the person being assisted.

[0099] In addition, the recipient wears smart guide glasses 100 connected to a user terminal app. An assistant establishes a voice connection with the recipient, and the smart guide glasses 100 automatically answer the call. During the voice call, the recipient uses the smart guide glasses 100's camera to capture an image of the area in front of them and transmits the data to the assistant via a public network (4G, 5G, Wi-Fi). The recipient automatically sends GPS data to the assistant's user terminal app. The assistant uses the received GPS data to help the recipient navigate, providing guidance or assistance through voice calls, video images, and GPS data navigation.

[0100] For example, the camera of the smart guide glasses 100 captures and obtains image information of the area in front, sending it to the assistant's user terminal 200 APP for image recognition. The obstacle avoidance detection module detects the distance to obstacles in front and sends the data to the assistant's user terminal 200 APP. The assistant's user terminal 200 APP processes the image information and obstacle distance information. If the obstacle is within 1 meter, it emits a beeping alarm. If the obstacle is between 1 and 3.5 meters, it reports the actual distance of the obstacle in decimeters, for example, 2.0 meters, broadcasting "20". If it exceeds 3.5 meters, the assistant's user terminal 200 APP analyzes what kind of object the obstacle is using its image recognition function. Sound is then played through the smart glasses' earphone module to provide the user with useful information.

[0101] In this embodiment, the smart guide glasses 100 collects obstacle avoidance data and audio / video data, and transmits it wirelessly to the assistant's user terminal 200 APP. The assistant's user terminal 200 APP processes the received data and feeds back the results to the smart guide glasses 100 via wireless transmission. This simplifies the device connection complexity and facilitates use and maintenance.

[0102] This embodiment also provides an intelligent guide system for the visually impaired. Figure 5 The diagram shown illustrates the interaction principle of the intelligent guide system for the visually impaired in one embodiment of this application. Figure 5 As shown, the intelligent guide system in this embodiment includes: the intelligent guide glasses 100 as described above and the user terminal 200 that wirelessly communicates with the intelligent guide glasses 100.

[0103] The functions of the smart guide glasses 100 have been described in detail above and will not be repeated here. Specifically, the user terminal 200 is used to: identify obstacles in the video image data received from the smart guide glasses 100 and generate obstacle identification result broadcast data; generate corresponding distance broadcast data based on distance data received from the smart guide glasses 100 and a preset distance alarm level; generate navigation data based on the video image data and GPS data received from the smart guide glasses 100; and generate audio guidance data by combining the obstacle identification result broadcast data, the distance broadcast data, and the navigation data, and feed it back to the smart guide glasses 100, so that the smart guide glasses 100 can broadcast prompts for obstacles around the blind person, broadcast prompts for the distance of obstacles, and broadcast navigation prompts for the travel route.

[0104] In addition, the user terminal 200 is also used to establish voice calls with the smart guide glasses 100 and provide guide services to the user through the received distance data, video image data and GPS data.

[0105] Figure 6 The diagram illustrates an example of an interaction process in one embodiment of the intelligent guidance system of this application. The intelligent guidance glasses 100 transmit image information acquired by a camera to a user's mobile app for image recognition. Exemplarily, the user app uses a trained deep neural network model to identify objects in the image. These objects include, but are not limited to, banknotes, traffic lights, zebra crossings, and other common household items. Through image recognition, corresponding voice prompts are generated for the intelligent guidance glasses 100, informing the user of obstacles ahead, thereby assisting the blind user in safe travel. The intelligent guidance glasses 100 acquires distance information through obstacle avoidance detection and sends it to the mobile app for distance measurement. Based on the distance measurement, corresponding voice prompts are generated for the intelligent guidance glasses 100, informing the user of the distance to obstacles ahead, thereby assisting the blind user in safe travel. In this embodiment, the environment in front of and around the blind person is detected by both image recognition and distance data from the obstacle avoidance sensor 111. The user interacts with the mobile APP through the smart guide glasses 100 by voice, and feedback is given to the user through the earphone in the smart guide glasses 100. Communication is convenient and there is little interference with the voice signal. This further improves the accuracy and safety of obstacle avoidance, while also effectively improving the user experience.

[0106] Figure 7 The diagram shown is a schematic block diagram of a user terminal in one embodiment of the intelligent guidance system for the visually impaired according to this application. Figure 7As shown, user terminal 200 includes at least one processor 201, memory 202, at least one network interface 203, and user interface 205. The various components in the terminal are coupled together via bus system 404. It is understood that bus system 204 is used to implement communication between these components. In addition to a data bus, bus system 204 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 7 The general will label all buses as bus systems.

[0107] The user interface 205 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0108] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0109] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the user terminal 200. Examples of this data include: any executable program for operation on the user terminal 200, such as the operating system 2021 and application programs 2022; the operating system 2021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 2022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The implementation of the XX method provided in this embodiment of the invention may be included in the application program 2022.

[0110] The methods disclosed in the above embodiments of the present invention can be applied to processor 201, or implemented by processor 201. Processor 201 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 201 or by instructions in software form. The processor 201 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 201 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 201 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0111] In an exemplary embodiment, the user terminal 200 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0112] like Figure 8 As shown, this embodiment also provides an intelligent guide method for the blind, applied to the intelligent guide glasses 100 as described above, including the following steps S100 to S300.

[0113] Step S100: In response to receiving a voice start command, power on the smart guide glasses 100 and start the audio and video sensor 112 in the smart guide glasses 100;

[0114] Step S200: The distance data collected by the obstacle avoidance sensor 111, the video image data collected by the audio and video sensor 112, and the GPS data are transmitted to the user terminal 200.

[0115] Step S300: Receive audio guidance data generated by the user terminal 200 based on the distance data, the video image data, and the GPS data from the user terminal 200, and broadcast the audio guidance data to provide prompts about obstacles around the blind person, to provide prompts about the distance to obstacles, and to provide navigation prompts about the travel route.

[0116] In this embodiment, the implementation principle of the intelligent guide method is the same as that of the intelligent guide system described above, and the implementation principle of the intelligent guide method will not be repeated here.

[0117] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 8 The method shown or the method in the user terminal 200.

[0118] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 8 The method shown or the method in the user terminal 200.

[0119] As used in this specification, the terms "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0120] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0121] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and method described above can be referred to the corresponding process in the aforementioned smart guide glasses 100, and will not be repeated here.

[0122] In the embodiments provided in this application, it should be understood that the disclosed systems, modules, terminals, and methods can be implemented in other ways. For example, the module and terminal embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between terminals or modules may be electrical, mechanical, or other forms.

[0123] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0124] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0125] In the above embodiments, the functions of each functional module can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable terminal. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0126] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0128] In summary, this application provides a method for blind users to use guide glasses controlled by voice. The glasses provide voice prompts regarding obstacles in the user's surroundings, distances to obstacles, and navigation routes. Furthermore, it can remotely provide guidance services to the blind. Moreover, the intelligent guide glasses of this application have a novel structure and are easy to use, effectively improving the user experience. Therefore, this application effectively overcomes the shortcomings of existing technologies and possesses high industrial applicability.

[0129] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A smart guide glasses for the blind, characterized in that, include: The frame body, detection module, control module, and voice guidance module; among which: The main body of the eyeglass frame includes a frame and a first temple and a second temple connected to both sides of the frame; The detection module includes an obstacle avoidance sensor and an audio / video sensor disposed in the middle of the frame; The control module includes a first control board and a second control board respectively installed inside the first temple and the second temple, and at least one function button installed on the outer surface of the first temple and the second temple; wherein the obstacle avoidance sensor and the audio-visual sensor are respectively connected to the first control board and / or the second control board via a flexible circuit board; the control module controls the audio-visual sensor to start image acquisition based on a received voice start command, transmits the distance data received from the obstacle avoidance sensor and the video image data received from the audio-visual sensor to the user terminal, and receives audio guidance data generated based on the distance data and the video image data from the user terminal; The voice guidance module is connected to the control module, receives and plays the audio guidance data from the control module to guide the blind person forward.

2. The intelligent guide glasses according to claim 1, characterized in that, The first temple has a first control board at its front end and a first battery module connected to the first control board at its rear end; the second temple has a second control board at its front end and a second battery module connected to the second control board at its rear end; wherein, the first temple or the second temple is provided with a magnetic charging interface for charging the first battery module and the second battery module at the position corresponding to the first control board.

3. The intelligent guide glasses according to claim 1, characterized in that, The first control board is a camera function control board for controlling the acquisition of images by the audio and video sensor, and the second control board is a voice function control board for controlling the playback of the audio guidance data.

4. The intelligent guide glasses according to claim 3, characterized in that, The voice guidance module includes an earphone connected to the second control board; the second temple is provided with an earphone hole for the earphone cable to pass through, and a magnet is provided at the bottom of the outer surface of the second temple to attract the earbud part of the earphone.

5. The intelligent guide glasses according to claim 1 or 3, characterized in that, The function buttons include: at least one first temple function button disposed on the outside of the first temple and electrically connected to the first control board, and at least one second temple function button disposed on the outside of the second temple and electrically connected to the second control board.

6. The intelligent guide glasses according to claim 1, characterized in that, The frame is provided with a frame signal slot for accommodating the flexible circuit board, the first temple is provided with a first temple signal slot for accommodating the flexible circuit board, and the second temple is provided with a second temple signal slot for accommodating the flexible circuit board; the frame signal slot, the first temple signal slot, and the second temple signal slot are connected to each other.

7. The intelligent guide glasses according to claim 1, characterized in that, The frame has at least one sound transmission hole in the middle, and a sound-transmitting membrane is attached to the sound transmission hole.

8. The intelligent guide glasses according to claim 1, characterized in that, Also includes: The control module receives GPS data from the GPS positioning module and transmits the GPS data to the user terminal, so that the user terminal can combine the GPS data to generate the audio guidance data.

9. An intelligent guide system for the visually impaired, characterized in that, include: The smart guide glasses as described in any one of claims 1 to 8 and the user terminal that wirelessly communicates with the smart guide glasses; The user terminal is used for: Based on the video image data received from the smart guide glasses, obstacles in the image are identified, and obstacle identification result broadcast data is generated; Based on the distance data received from the smart guide glasses and the preset distance alarm level, corresponding distance broadcast data is generated; Navigation data is generated based on the video image data and GPS data received from the smart guide glasses; The audio guidance data is generated by combining the obstacle recognition result broadcast data, the distance broadcast data, and the navigation data, and fed back to the smart guide glasses so that the smart guide glasses can broadcast prompts for obstacles around the blind person, broadcast prompts for the distance of obstacles, and broadcast prompts for navigation routes.

10. A smart guide glasses method, applied to the smart guide glasses as described in any one of claims 1 to 8, characterized in that, include: In response to receiving a voice start command, the smart guide glasses are powered on and activated, and the audio and video sensors in the smart guide glasses are also activated. The distance data collected by the obstacle avoidance sensor, the video image data collected by the audio and video sensor, and the GPS data are transmitted to the user terminal. The system receives audio guidance data generated by the user terminal based on the distance data, the video image data, and the GPS data, and broadcasts the audio guidance data to provide prompts about obstacles around the blind person, to provide prompts about the distance to obstacles, and to provide navigation prompts for the travel route.

Images