Smart glasses

Abstract

This invention relates to a smart glasses designed to help visually impaired individuals in their lives, security activities and guidance issues, wherein it comprises at least one camera module (1) fixed on the inside of the frame of the glasses, which informs users by processing images of living beings and objects in the environment and creates alert signals to inform the user's distance relative to the living being and the object by determining the distance, mobile application coded into the camera module (1) microcontroller and containing map module, camera module and security module, power unit (2) providing electrical energy supply to the glasses via battery (4), at least one battery charging module (5) fixed in the power unit (2), internal or external audio receiver transmitting input sound waves to the camera module (1) by converting thereof into electrical signals, and a speaker transmitting electrical signals to the user by converting thereof into output sound waves.

Description

[0001] SMART GLASSES

[0002] Technical Field:

[0003] This invention relates to a smart glasses designed specifically to help visually impaired individuals in their lives, safety activities and guidance issues.

[0004] State of the Art:

[0005] Visually impaired individuals face various difficulties in their daily lives. These difficulties negatively affect the life experiences of the individual. It is possible for visually impaired individuals to go from one place to another in public without the need for anyone. Today, visually impaired individuals commonly meet these needs by means of a cane. However, these canes, which are in widespread use, provide limited information regarding the environment. They are especially insufficient for perceiving the obstacles above the waist level.

[0006] With the developing technologies, interest has increased on different smart device attempts which can replace a cane used by visually impaired individuals. In this context, smart canes, smart glasses, wearable technological tools have been developed. These technologies are generally used as a navigation device for visually impaired individuals to identify obstacles that may arise and to guide individuals from one point to another safely and independently.

[0007] In utility model application no. TR201913953 smart glasses are described. The invention relates to a smart glasses designed for hearing-impaired individuals who are deaf from birth or who have lost their hearing later, which allows the user to be alerted by detecting various sounds such as car horns and human voices, thus reducing the difficulties faced by hearing-impaired individuals in daily life and improving their quality of life.

[0008] Patent Application No. TR201720215 describes a smart glasses for the colorblind. It is a smart glasses for color-blind individuals which provides safe driving by identifying and / or notifying with an alert the color of traffic lights to drivers with color blindness by using image processing technologies, wherein it comprises a body, at least one camera which detects the images of traffic lights by being connected to the body, a processor which processes the incoming data with image processing technologies and converts it into alerts and / or notifications, a transparent screen which transmits visual alerts from the processor to the driver, and / or a warning module which transmits incoming audible and / or sensory alerts.

[0009] In the above-mentioned application, the object of the glasses is to support the safe driving of colorblind drivers and to reduce error rates by using image processing techniques. However, it is not possible for an individual who is completely visually impaired to determine the destination location as a pedestrian or to provide the proximity-distance detection of objects with the glasses in said application.

[0010] As a result, there is a need for a new technology which can overcome the disadvantages mentioned above.

[0011] Description of the Invention:

[0012] The invention relates to a smart glasses designed specifically to help visually impaired individuals in their lives, safety activities and guidance issues.

[0013] The main object of the invention is to enable the user to recognize objects indoors or outdoors and to transmit the distance to the object to the user, allowing the user to move comfortably in the environment. For example, when the user wants to go to the restroom at home, with its guidance feature, it will be possible to reach the desired location more easily.

[0014] Another object of the invention is that the user can easily get location information for the desired location without consulting anyone using technology, and at the same time, get directions for the destination via Google maps. A voice assistant is activated for directions and guidance is provided at every step. Also, since the navigation module works in coordination with the camera module, object detection and distance measurement between the object and the user are also performed in order to avoid any accidents on the road while reaching the relevant location, and information flow is provided to the user audibly. The mobile application in the system can also detect objects and read text with the phone's camera. With the map module, getting directions to the desired location is enabled. Through user login, the opportunity to obtain information about the people with whom the user of smart glasses frequently communicates is provided through the mobile application. The mobile application is open to development and updating. In this way, extra functions can be added without changing the design of smart glasses.

[0015] The smart system (smart glasses and mobile application) brings along important responsibilities in terms of cyber security. It provides access to users' personal and sensitive data. Certain measures have been taken to prevent this. On a regular basis, the system will be audited for cyber security and authorized persons will be notified in case of an abnormality. During the discovery stage of cyber security, a network scan will be performed.

[0016] The security module in the invention has a special intrusion detection system. Prior to the installation of this detection system, DDoS, Man-in-the-Middle (MITM), Wi-Fi Eavesdropping, Replay, Web, Mobile, Spoofing, etc. intrusions were carried out on the system, their impact on the system was examined and source code analysis was performed. For detection, protocols suitable for SMTP and the operating system were used in the information stage and the object was to report suspicious network behavior to the authorized expert.

[0017] The object is that the glasses of the invention can be used for 4-5 hours by means of a battery. The power unit to be added to the left arm of the glasses consists of a 3000 mAh battery and a TP4056 charging module. The camera module connection with the power unit is designed to be easily pluggable with the use of pogo pins. The 3D printed glasses frames and arms are designed for the user to be comfortable, light and to use the modules located thereon in the most efficient way. It is ensured that it is an ergonomic product.

[0018] With the easy fastening of the parts that make up the invention to each other, it is easy to install, and thanks to the short assembly time, costs are low. Also, the invention has a solid structure. Description of the Drawings:

[0019] The invention will be described with reference to the accompanying figures, so that the features of the invention will be more clearly understood and appreciated, however it is not intended to limit the invention to these particular embodiments. On the contrary, it is intended all alternatives, modifications and equivalences that may be included in the field of the invention as defined by the accompanying claims are within the scope. It should be understood that the details shown are for the sole purpose of illustrating preferred embodiments of the present invention and are intended to provide the most useful and easily understandable description of both the embodiment of the methods and the rules and conceptual features of the invention. In the drawings,

[0020] Fig. 1 shows the front view of the glasses of the invention.

[0021] Fig. 2 shows the side-section view of the glasses of the invention.

[0022] Fig. 3 shows the top view of the glasses of the invention.

[0023] Fig. 4 shows the rear view of the glasses of the invention.

[0024] Fig. 5 shows the block diagram of the Map (Navigation) module.

[0025] Fig. 6 shows the block diagram of the camera module.

[0026] Fig. 7 shows the block diagram of the mobile application cyber security software.

[0027] The figures which will help understand this invention are numbered as indicated in the accompanying drawing and are given below with their names.

[0028] Description of the References:

[0029] 1. Camera module

[0030] 2. Power unit

[0031] 2.1. Female pogo pin

[0032] 3. Male pogo pin

[0033] 4. Battery

[0034] 5. Battery charging module

[0035] 6. Battery charging socket

[0036] 100. Turning on the map module

[0037] 101. Asking for location information

[0038] 102. Asking for directions 103. Connecting to Google maps

[0039] 104. Asking for location information again

[0040] 200. Turning on the camera module

[0041] 201. Processing

[0042] 202. Providing information

[0043] 300. Security module

[0044] 301. Scanning foreign networks

[0045] 302. Detecting smart system

[0046] 303. Returning to the first step

[0047] 304. Determining the type of attack

[0048] 305. Copying network traffic

[0049] 306. Evaluating according to deep learning method

[0050] 307. Observing in real time

[0051] 308. Returning to the copy network traffic step

[0052] 309. Returning to the first step

[0053] 310. Alerting the system authority

[0054] 311. Returning to the step of observing

[0055] Detailed Description of the Invention:

[0056] The parts that make up the glasses of the invention are a camera module (1), a power unit (2), a male pogo pin (3), a battery (4), a battery charging module (5) and a battery charging socket (6). The power unit (2) contains a female pogo pin (2.1) within and a battery charging socket (6) on top.

[0057] Figs. 1-4 show the glasses structure of the invention. The glasses frame is preferably printed from the 3D printer and the camera module (1) is placed between the lenses. The camera module (1) is an ESP32 based microcontroller board and contains a camera module. This circuit board has wide wireless communication capability and also has image and video processing capabilities. With the mobile application coded into the ESP32, it provides the user with voice assistant, navigation, object detection, distance measurement between the object or person, text reading, sharing the location of the smart glasses user with people they frequently communicate with, and emergency support call features. The glasses are electrically powered by a battery (4). The battery (4) is located in the power unit (2). The glasses also contain an audio receiver, which works internally to the ESP32 card or externally to the card, converting incoming sound waves into electrical signals, allowing the ESP32 mobile application to process them. As a result of the processing, electrical signals are converted into output sound waves through the speaker and transmitted to the user for feedback to the user. Here, it is also possible to connect headphones instead of speakers, and it will be possible to connect headphones to the ESP32 device via Bluetooth. The battery charging module (5) fixed within the power unit (2) and the battery charging socket (6) behind the power unit (2) are associated to each other, and battery (4) charging will be possible from this part. The energy flow between the glasses and the power unit (2) will be provided by pogo pins. The female pogo pin (2.1) is located in the power unit (2), while the male pogo pin (3) is located on the glasses and energy flow is provided as a result of the interlocking of the pogo pins. The camera module (1) will be powered via the cable flow inside the glasses.

[0058] Figs. 5-7 show the block diagrams of the mobile application, coded in the camera module (1) microcontroller and including the map module, camera module and security module parts.

[0059] In the Fig. 5, the block diagram of the map module (navigation) is provided. When the user turns on the map module (100), the voice assistant asks the question, "Where will you be heading?", in other words, location information will be asked (101). The map module can be turned on via the application or by voice command. Then, the map will be searched once the location is told by the user. The search result is transmitted to the user. Then the voice assistant asks the user the question, "Would you like to get directions?", in other words, it will ask the user whether to receive location directions (102). If the user says "Yes", the mobile application will be connected to Google maps for directions (103). If the answer is "No", the location information will be asked again (104). The GPS module can be used to obtain location information.

[0060] In the Fig. 6, the block diagram of the camera module is provided. The camera module can be turned on with a voice command (200) or can be used with a touch / key via the mobile application. When the user enters the camera module (1), the images coming from this module will be processed (201) and object detection and distance determination will be made. The information of the results will be notified to the user audibly (202). In the Fig. 7, the block diagram of the security module is provided. The security module is coded into the mobile application to prevent any cyber threats to the glasses that may come from cyberspace. The security module works as follows: firstly, foreign networks within the network domain of the glasses are scanned (301), smart systems, if any, are detected as a result of the scan (302), if the smart system is not detected, return is made to first step to repeat the scanning process of foreign networks until it is detected (303), if the smart system is detected, the attack type is determined (304) (these attack types can be any of the following: DDoS, Man-in-the-Middle (MITM), WiFi Eavesdropping, Replay, Web, Mobile, Spoofing, etc.), if the attack is successful, the network traffic is copied (305), the copied network traffic is evaluated according to the deep learning method (306) on one hand and on the other hand, it is observed in real time (307), if the observed network traffic does not have the necessary features of the attack types, return is made to network traffic copy (305) step (308), if there is an attack that cannot be evaluated within the deep learning method return is made to first step (309), if the attack is detected, the system authority is alerted (310), and if it is not an attack, return is made to the step of observing in real time (307) (311). During the evaluation according to the deep learning method, it is taken into account whether the copied network traffic is sufficient for deep learning, whether the attack type has the necessary features and whether the deep learning method has been successfully applied.

Claims

CLAIMS1. A smart glasses designed for visually impaired individuals, characterized in that it comprises:- at least one camera module (1) fixed on the inside of the frame of the glasses, which informs users by processing images of living beings and objects in the environment and creates alert signals to inform the user's distance relative to the living being and the object by determining the distance,- mobile application coded into the camera module (1) microcontroller and containing map module, camera module and security module,- power unit (2) providing electrical energy supply to the glasses via battery (4),- at least one battery charging module (5) fixed in the power unit (2),- internal or external audio receiver transmitting input sound waves to the camera module (1) by converting thereof into electrical signals, and- a speaker transmitting electrical signals to the user by converting thereof into output sound waves.

2. The smart glasses of claim 1 , characterized in that it has a female pogo pin (2.1) and a male pogo pin (3) that interlock with each other to ensure the electric flow between the power unit (2) and the glasses.

3. The power unit (2) of claim 1, characterized in that it has at least one battery charging socket (6) formed on the back side.

4. The operation method of the map module located in the mobile application integrated into the camera module (1) microcontroller, characterized in that it comprises the process steps of:- turning on the map module by voice command (100),- asking the user the location information (101),- asking the user whether to receive location directions according to the location address given by the user (102),- connecting to Google maps via the application (103) when directions are wanted, otherwise asking for location information again (104).

5. The operation method of the camera module located in the mobile application integrated into the camera module (1) microcontroller, characterized in that it comprises the process steps of:- turning on the camera module by voice command (200),- processing of incoming images via the camera module (1) (201),- detecting the objects of the processed images and the determining distances thereof and providing information to the user audibly (202).

6. The operation method of the security module located in the mobile application integrated into the camera module (1) microcontroller, characterized in that it comprises the process steps of:- scanning (301) for foreign networks within the network domain of the glasses,- detecting the smart system as a result of scanning (302),- if the smart system is not detected, returning to the first step (303),- if the smart system is detected, determining the type of attack (304),- copying network traffic if the attack is successful (305),- evaluating copied network traffic according to deep learning method (306),- observing the copied network traffic in real time (307),- in the case of the attack types of the observed network traffic not having the required features, returning to the copy network traffic (305) step (308),- returning to the first step due to an attack that cannot be evaluated within the framework of the deep learning method (309),- in the case of detection of an attack, alerting the system authority (310), if it is not an attack, returning to the step of observing in real time (307) (311).

Images