Intelligent glasses and working method thereof

Abstract

The invention relates to intelligent glasses and a working method thereof. The intelligent glasses comprise a glasses frame, and lenses, a first micro optical camera, a second micro optical camera, a microprocessor and a projector are arranged on the glasses frame; the first micro optical camera and the second micro optical camera are respectively arranged on two sides of the glasses frame, are connected to the microprocessor and are used for respectively acquiring PPG signals of two side faces of a user and sending the PPG signals to the microprocessor; the microprocessor is used for carrying out pulse waveform analysis and calculation according to the collected PPG signals of the faces of the two sides of the user to obtain the pulse conduction speed, and a blood pressure value is obtained through a calibration model and deep learning calculation; the projector is connected to the microprocessor, corresponds to the lens and is used for displaying the blood pressure value obtained through calculation on the lens. During use, effective detection can be achieved without clinging to the skin of a user, interference of user activities and ambient light is avoided, the detection result is high in precision, and real-time blood pressure state information can be provided.

Description

Technical Field

[0001] The present invention relates to the technical field of physiological health monitoring, and in particular to a pair of smart glasses and a working method thereof. Background Art

[0002] Hypertension is a common chronic disease characterized by persistently elevated arterial blood pressure, generally defined as a systolic blood pressure (blood pressure when the heart contracts) greater than or equal to 140 millimeters of mercury (mmHg) and / or a diastolic blood pressure (blood pressure when the heart relaxes) greater than or equal to 90 mmHg at rest.

[0003] Blood pressure measurement devices on the market are usually large in size, poor in portability, lack of timeliness and dynamic feedback functions; some monitoring devices are expensive and difficult to popularize in the market; at the same time, due to the lack of interpretation of monitoring data and professional guidance, users are anxious. With the development of remote monitoring technology or remote health monitoring technology, health monitoring technology is constantly changing, and hypertensive patients and medical service providers can be more proactive in health management. At the same time, wearable blood pressure monitoring devices provide more timely and portable functions for health diagnosis of hypertensive people or healthy people.

[0004] The prior art discloses a pair of smart glasses, comprising a frame for holding two eyepieces, the frame having two nose pads for being placed on a user's nose, and two arms for being placed on two ears of the user; a sensor mounted on at least one of the nose pads or the arms and configured to collect optical signals from the user's perfused tissue; and a processor configured to obtain a waveform from the optical signal and determine a cardiovascular parameter based on the waveform.

[0005] It has the following technical problems: It uses a photoplethysmography (PPG) sensor, which needs to be in close contact with the user's skin for effective detection, affecting the user's overall comfort. It is also easily affected by user activities and ambient light, resulting in poor signal quality and reduced accuracy of detection results. Summary of the invention

[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide a pair of smart glasses and a working method thereof, which can realize blood pressure detection with high accuracy and improve the overall comfort of the user.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: A pair of smart glasses, comprising a frame, on which are arranged lenses, a first micro optical camera, a second micro optical camera, a microprocessor and a projector; The first micro optical camera and the second micro optical camera are respectively arranged on both sides of the frame and connected to the microprocessor, and are used to respectively collect PPG signals of both sides of the user's face and send them to the microprocessor; The microprocessor is used to analyze the pulse waveform and calculate the pulse conduction velocity based on the PPG signals collected from both sides of the user's face, and obtain the blood pressure value through calibration model and deep learning calculation; The projector is connected to the microprocessor and arranged corresponding to the lens, and is used for displaying the calculated blood pressure value on the lens.

[0008] Furthermore, a third micro optical camera is provided in the middle of the frame, and the third micro optical camera is connected to the microprocessor for photographing and identifying user gestures or external objects.

[0009] Furthermore, two nose pads are arranged on the inner side of the frame, and the third micro optical camera is arranged between the two nose pads.

[0010] Furthermore, the frame is connected to a bone conduction earphone module, and the bone conduction earphone module is connected to a microprocessor.

[0011] Furthermore, the frame is provided with a wireless network communication module for communicating with external devices, and the wireless network communication module is connected to the microprocessor.

[0012] Furthermore, a charging slot is provided at the lower end of the frame, and the charging slot is detachably connected to a wireless charging plate.

[0013] A working method of smart glasses, comprising the following steps: The PPG signals on both sides of the user's face are obtained respectively, the pulse waveform is analyzed and the pulse conduction velocity is calculated based on the PPG signals on both sides of the user's face, and the blood pressure value is obtained through the calibration model and deep learning calculation, so that the projector can display the calculated blood pressure value on the lens.

[0014] Further, the pulse conduction velocity is calculated by measuring the pulse wave on one side of the user's face with a first micro-optical camera, and synchronously triggering the second micro-optical camera to measure the pulse wave on the other side of the user's face, thereby obtaining the pulse wave conduction time of the same pulse wave on both sides of the user's face and the distance between both sides of the user's face, wherein the pulse wave is synchronized with the pumping activity of the heart to generate a PPG signal; The pulse velocity is calculated based on the pulse wave transmission time and the distance between the two sides of the user's face.

[0015] Furthermore, the projector displays the calculated blood pressure value on the lens by using an optical waveguide lens to display the blood pressure value information on the optical waveguide lens through optical waveguide technology.

[0016] Furthermore, when the third micro optical camera detects abnormal food or high-calorie food, it sends voice through the bone conduction earphone module, and when abnormal data is continuously detected, it communicates with the outside through the wireless network communication module.

[0017] In general, the present invention has the following advantages: The present invention combines remote photoplethysmograph (rPPG) technology, optical waveguide technology, deep convolutional neural network technology and bone conduction technology. It can effectively detect blood pressure without being close to the user's skin, will not affect the user's overall comfort, is not easily disturbed by user activities and ambient light, has high detection results, can provide real-time blood pressure status information, and has additional functions such as intelligent voice assistant and emergency call, which can meet people's needs for timeliness and convenience of blood pressure measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the structure of the smart glasses of this embodiment.

[0019] Figure 2 FIG. 4 is a schematic structural diagram of the smart glasses of this embodiment from another perspective.

[0020] Figure 3 Schematic diagram of the working principle of the smart glasses of this embodiment.

[0021] In the figure: 1-lens, 2-frame, 3-first temple, 4-second temple, 5-first micro optical camera, 6-second micro optical camera, 7-third micro optical camera, 8-bone conduction earphone module, 9-first charging slot, 10-second charging slot, 11-first charging interface, 12-second charging interface, 13-charging indicator light, 14-type C interface, 15-nose pad, 16-projector, 18-microprocessor, 19-power switch, 20-power indicator light, 21-wireless charging plate. DETAILED DESCRIPTION

[0022] The present invention will be described in further detail below.

[0023] like Figure 1-Figure 3 As shown, a pair of smart glasses comprises a frame 2, on which a lens 1 is arranged, and the outer walls on both sides of the frame 2 are respectively connected with a first temple 3 and a second temple 4 by hinges. A bone conduction earphone module 8 is arranged on the outer side of the second temple 4. A first micro optical camera 5 and a second micro optical camera 6 are respectively fixed to the inner lower walls of both sides of the frame 2, two nose pads 15 are arranged on the inner side of the frame 2, and a third micro optical camera 7 is arranged between the two nose pads 15. A power switch 19 and a power indicator light 20 are arranged on the top of the frame 2.

[0024] Preferably, a mounting groove is formed on one side of the outer wall of the first temple 3, and a microprocessor 18 is built in the mounting groove. The first temple 3 or the second temple is provided with a projector 16. The frame 2 is provided with a wireless network communication module for communicating with an external device. The first micro optical camera 5, the second micro optical camera 6, the third micro optical camera, the projector 16, the bone conduction earphone module 8 and the wireless network communication module are electrically connected to the microprocessor 18 respectively.

[0025] The first micro optical camera 5 and the second micro optical camera 6 are used to collect PPG signals from both sides of the user's face and send them to the microprocessor 18; The microprocessor 18 is used to analyze the pulse waveform and calculate the pulse conduction velocity based on the PPG signals collected from both sides of the user's face, and obtain the blood pressure value through calibration model and deep learning calculation; The projector 16 is connected to the microprocessor 18 and arranged corresponding to the lens 1 , and is used to display the calculated blood pressure value on the lens 1 .

[0026] Furthermore, a power indicator is disposed on the upper wall of the microprocessor 18 , and the power indicator is electrically connected to the microprocessor 18 .

[0027] The wireless network communication module is connected to the remote assistance module through Bluetooth, wireless network and satellite signals. The first charging interface 11 and the second charging interface 12 on the upper side of the wireless charging plate 21 are respectively connected to the first charging slot 9 and the second charging slot 10 at the bottom of the frame 2. The low-power power management module powers the smart glasses through a high-energy ultra-light rechargeable battery and charges the rechargeable battery through the typeC interface 14. A charging indicator light 13 is provided above the typeC interface 14.

[0028] A working method of smart glasses, comprising the following steps: The PPG signals on both sides of the user's face are obtained respectively, the pulse waveform is analyzed and the pulse conduction velocity is calculated based on the PPG signals on both sides of the user's face, and the blood pressure value is calculated through the calibration model and deep learning, so that the projector 16 displays the calculated blood pressure value on the lens 1. The advantage of this method is that it can provide continuous and real-time blood pressure monitoring without the need for a traditional cuff-type blood pressure monitor.

[0029] The pulse conduction velocity is calculated by measuring the pulse wave on one side of the user's face with the first micro optical camera 5, and synchronously triggering the second micro optical camera 6 to measure the pulse wave on the other side of the user's face, thereby obtaining the pulse wave conduction time of the same pulse wave on both sides of the user's face and the distance between the two sides of the user's face, wherein the pulse wave is synchronized with the pumping activity of the heart to generate a PPG signal; The pulse wave velocity PWV is calculated based on the pulse wave transmission time and the distance between the two sides of the user's face. The formula used is: PWV = distance / time difference. Distance refers to the physical distance between the two points where the pulse wave propagation is measured. Usually, this distance is measured along the blood vessel path. It can be the straight-line distance between the arms, legs or other body parts. In this embodiment, it is the distance between the two sides of the user's face; the time difference is the pulse wave transmission time of the same pulse wave on the two sides of the user's face. Pulse Wave Velocity (PWV) is widely used to assess the elasticity and health of blood vessels, and as an indicator of cardiovascular disease risk.

[0030] In this embodiment, the first micro optical camera 5 and the second micro optical camera 6 collect the user's facial skin to obtain blood flow data, and then send these raw data to the photoelectric plethysmography rPPG sensor, CMOS sensor, analog-to-digital converter ADC, image processing unit IPU, digital signal processor DSP and microprocessor for signal analysis, and use the algorithm to analyze the extracted features to infer physiological parameters such as heart rate and blood flow velocity to estimate the blood pressure value. The analysis results are converted into digital signals and then sent to the data processing center of the microprocessor 18 through wireless communication modules such as Bluetooth, Wi-Fi, etc., and then the detailed data is displayed on the lens 1 through the projector 16.

[0031] The microprocessor 18 can also send the results to the bone conduction earphone module 8 to describe the current blood pressure situation to the user, and at the same time, when an early warning state is reached, timely communication and treatment can be obtained through the wireless network communication module.

[0032] The third micro optical camera 7 can capture and identify hand movements. The content presented on the lens 1 can be simply dragged and clicked using gestures, so that users can view detailed blood pressure monitoring information, including the current blood pressure range, blood pressure control rate, blood pressure day and night variability analysis, risk assessment prediction and other information; it can also capture and identify the user's daily diet, medication and living habits to remind the user to avoid eating abnormal or high-calorie foods, and monitor the surrounding environment, such as light intensity and temperature, and transmit the data to the bone conduction earphone module 8 for information reception and voice reminder playback. At the same time, the data report is transmitted to the remote terminal module through the wireless network information module to be shared with medical professionals for long-term tracking and evaluation as well as timely care by relevant contacts.

[0033] In this embodiment, the lens 1 is an optical waveguide lens. The microprocessor 18 analyzes and calculates the detection information of the first micro optical camera 5 and the second micro optical camera 6 to provide blood pressure data in a timely manner, and displays the data on the optical waveguide lens through optical waveguide technology for users to watch in real time. A detailed report will automatically transmit the user's blood pressure information to the background terminal and the connected mobile phone through the wireless network module. When abnormal data is found, an automatic alarm function will be used to contact the connected device, and voice communication will be carried out through the bone conduction earphone module 8.

[0034] The remote terminal module includes a doctor-side remote backend and a connected mobile phone. The data storage device analyzes blood pressure data through the machine learning algorithm and statistical model in the microprocessor 18, identifies the normal range of blood pressure, abnormal blood pressure warning, blood pressure control rate, blood pressure day and night variability analysis, risk assessment prediction, and detailed health report. By using the 5G network connection, the smart glasses can regularly receive machine learning algorithm updates from the backend server, including learning new categories and model optimization, and then download these updates to the data storage device.

[0035] The present invention has the following advantages: 1. Improve accuracy and conduct dynamic monitoring.

[0036] The present invention uses remote photoplethysmography (rPPG). rPPG technology uses computer vision and deep learning models to analyze subtle color changes in facial videos to achieve non-contact monitoring of physiological parameters such as heart rate. In daily activities, traditional photoplethysmography (PPG) devices (such as watches) are easily disturbed by activities and ambient lighting, and the signal quality deteriorates. At the same time, they need to be close to the skin, which affects the overall comfort of the user. The present invention uses rPPG technology to effectively solve problems such as activity interference and environmental changes, and improves accuracy. The use of non-contact measurement improves comfort while achieving efficient signal processing and analysis.

[0037] 2. Lens 1 displays data function.

[0038] The present invention uses optical waveguide micro-display technology to perform optical display, and displays blood pressure data in real time in the optical waveguide lens. Images are generated using miniaturized optical display elements, and display technology is integrated with sensors to achieve portability and lightness of the product, providing detailed information on blood pressure monitoring.

[0039] 3. Intelligent voice broadcast.

[0040] The present invention has a built-in voice assistant based on bone conduction technology, adopts advanced voice recognition and voice synthesis technology, supports user voice commands, provides prompts for risk factors (blood pressure), and provides voice reminders when blood pressure exceeds the normal range, and performs early warning functions. At the same time, compared with the basic voice prompt function of the prior art, the intelligent voice assistant of the present invention is not easily affected by the surrounding noise or disturbs the user or the people around, has privacy, and its appearance is comfortable to wear, and has more usage scenarios, which significantly improves the user experience and feedback quality, and increases user dependence.

[0041] 4. Transmission of health information.

[0042] The data storage device analyzes blood pressure data through the microprocessor 18 machine learning algorithm and statistical model, identifies the normal range of blood pressure, abnormal blood pressure warning, blood pressure control rate, blood pressure day and night variability analysis, risk assessment prediction, and detailed health report. By using the network connection, professional doctors and guardians can obtain the user's detailed data in a timely manner, and conduct full-time monitoring and timely intervention treatment.

[0043] 5. Help function.

[0044] When the algorithm finds data anomalies, a warning message appears on the lens 1, and the bone conduction earphone module 8 uses voice broadcast to remind you. At the same time, the wireless network communication module is used to connect to the remote assistance module to communicate and obtain supplements and assistance.

[0045] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A pair of smart glasses, characterized in that: The mirror frame comprises a lens, a first micro optical camera, a second micro optical camera, a microprocessor and a projector; The first micro optical camera and the second micro optical camera are respectively arranged on both sides of the frame and connected to the microprocessor, and are used to respectively collect PPG signals of both sides of the user's face and send them to the microprocessor; The microprocessor is used to analyze the pulse waveform and calculate the pulse conduction velocity based on the PPG signals collected from both sides of the user's face, and obtain the blood pressure value through calibration model and deep learning calculation; The projector is connected to the microprocessor and arranged corresponding to the lens, and is used for displaying the calculated blood pressure value on the lens.

2. The smart glasses according to claim 1, characterized in that: A third micro optical camera is arranged in the middle of the frame. The third micro optical camera is connected to the microprocessor and is used for photographing and identifying user gestures or external objects.

3. The smart glasses according to claim 2, characterized in that: Two nose pads are arranged on the inner side of the frame, and the third micro optical camera is arranged between the two nose pads.

4. The smart glasses according to claim 1, characterized in that: The frame is connected to a bone conduction earphone module, and the bone conduction earphone module is connected to a microprocessor.

5. The smart glasses according to claim 1, characterized in that: The mirror frame is provided with a wireless network communication module for communicating with external equipment, and the wireless network communication module is connected to the microprocessor.

6. The smart glasses according to claim 1, characterized in that: A charging slot is provided at the lower end of the frame, and a wireless charging plate is detachably connected to the charging slot.

7. A working method of smart glasses, characterized in that: The smart glasses according to any one of claims 1 to 6 include the following steps: The PPG signals on both sides of the user's face are obtained respectively, the pulse waveform is analyzed and the pulse conduction velocity is calculated based on the PPG signals on both sides of the user's face, and the blood pressure value is obtained through the calibration model and deep learning calculation, so that the projector can display the calculated blood pressure value on the lens.

8. The working method according to claim 7, characterized in that: The pulse velocity is calculated by measuring the pulse wave on one side of the user's face with a first micro-optical camera, and synchronously triggering the second micro-optical camera to measure the pulse wave on the other side of the user's face, thereby obtaining the pulse wave transmission time of the same pulse wave on both sides of the user's face and the distance between the two sides of the user's face, wherein the pulse wave is synchronized with the pumping activity of the heart to generate a PPG signal; The pulse velocity is calculated based on the pulse wave transmission time and the distance between the two sides of the user's face.

9. The working method according to claim 7, characterized in that: The projector displays the calculated blood pressure value on the lens by using an optical waveguide lens to display the blood pressure value information on the optical waveguide lens through optical waveguide technology.

10. The working method according to claim 7, characterized in that: When the third micro optical camera detects abnormal food or high-calorie food, it sends voice through the bone conduction earphone module. When abnormal data is continuously detected, it communicates with the outside through the wireless network communication module.

Images