An AI glasses
By dynamically adjusting image resolution, audio encoding, and video stream parameters within AI glasses using an edge processor, combined with local caching and multimodal triggering, the shortcomings of existing AI glasses in terms of recognition accuracy, bandwidth, and interaction methods are addressed, achieving a low-latency and efficient user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN XINZHONGXIN TECH CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-03
AI Technical Summary
Existing AI glasses fail to effectively balance recognition accuracy with bandwidth, voice quality, and network compatibility in image recognition, voice interaction, and real-time image transmission modes. They also lack local caching and multimodal interaction methods, resulting in latency and poor user experience.
The system uses an edge processor to dynamically adjust image resolution, audio encoding bitrate, and video stream parameters. Combined with local caching and multimodal triggering logic, it adaptively optimizes based on scene complexity, environmental noise, and network bandwidth to achieve low latency and efficient interaction.
It significantly improves the recognition accuracy, transmission smoothness, and interaction naturalness of AI glasses in complex environments, reduces latency and power consumption, and provides a more efficient user experience.
Smart Images
Figure CN122340320A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart wearable devices, and in particular to an AI glasses. Background Technology
[0002] With the rapid development of wearable devices and edge computing technology, AI glasses, as a new type of intelligent terminal, have shown broad application prospects in scenarios such as outdoor travel, industrial maintenance, and remote collaboration. Existing AI glasses typically integrate miniature cameras, microphones, wireless communication modules, and bone conduction speakers, enabling basic functions such as image recognition, voice interaction, and real-time image transmission. For example, when a user wears AI glasses to photograph an unknown plant, the device uploads the image to a cloud recognition server, returns the plant's name, and provides a voice announcement; or a user asks a question via voice, and a large cloud model analyzes the image and returns an audio answer. Furthermore, some AI glasses support direct Wi-Fi connection to mobile apps, enabling real-time transmission of first-person perspective images for easy recording and sharing.
[0003] However, existing AI glasses still have the following technical problems in practical applications: First, in image recognition mode, devices typically upload images at a fixed resolution, without considering the need to balance recognition accuracy and upload speed based on scene complexity. High-resolution images cause unnecessary bandwidth waste and latency in simple scenes (such as solid color backgrounds), while low-resolution images are prone to recognition errors in complex scenes (such as dense objects). Second, in voice interaction mode, the audio data volume is large, and the existing Opus encoding uses a fixed bit rate, failing to dynamically adjust encoding parameters according to the environmental signal-to-noise ratio, resulting in decreased voice quality in noisy environments or data redundancy in quiet environments. Third, in real-time image transmission mode, the video stream parameters (resolution, frame rate) between the mobile app and the device are fixed and cannot be dynamically adapted according to real-time network bandwidth. When Wi-Fi signal fluctuates, stuttering, screen tearing, or high latency are likely to occur, seriously affecting the user experience. Fourth, existing systems lack local caching and prefetching mechanisms for recent interaction results, requiring a complete round trip to the cloud for each request, resulting in significant auditory latency. Fifth, the user triggering method is singular (such as physical buttons or APP buttons), and it is impossible to use multimodal natural behaviors (such as gaze plus voice) to achieve more efficient interaction initiation.
[0004] Therefore, there is an urgent need in this field for an AI glasses system with dynamic adaptive capabilities, low latency, and high degree of natural interaction. This system should be able to adjust image resolution, audio encoding bit rate, and image transmission parameters in real time based on multi-dimensional parameters such as scene complexity, environmental noise, and network bandwidth. Combined with local caching and multimodal triggering logic, it can comprehensively improve the response speed, recognition accuracy, and user experience of AI glasses. Summary of the Invention
[0005] The purpose of this invention is to provide AI glasses to solve the problems existing in the prior art.
[0006] To achieve the above objectives, the present invention provides the following solution:
[0007] This invention provides an AI glasses system, comprising: an AI glasses body and a cloud server; The AI glasses and the cloud server transmit data via Wi-Fi wireless network. The AI glasses upload the collected image data or audio data to the cloud server, and the cloud server sends the recognition results or synthesized audio back to the AI glasses. The AI glasses are equipped with: a miniature camera module, a microphone array, an edge processor, a wireless communication module, a bone conduction speaker, a local cache unit, and a power management unit; The edge processor is electrically connected to the camera module, microphone array, wireless communication module, bone conduction speaker, and local cache unit, respectively, and is used to execute the following control methods: In the photo recognition mode, the image is captured and the scene information entropy H is calculated. The upload resolution is dynamically selected based on the information entropy H. In the voice dialogue mode, the audio is captured and the environmental signal-to-noise ratio SNR is estimated. The bit rate BR of the Opus encoding is dynamically adjusted based on the SNR. In the real-time image transmission mode, the current network bandwidth B is estimated. The resolution Res and frame rate FPS of the video stream are adjusted in conjunction with the bandwidth B. The local cache unit is used to store the most recent image recognition result and prefetched audio blocks. When a user repeatedly queries the same target or has a continuous conversation, the edge processor prioritizes reading data from the local cache unit to reduce the number of cloud requests.
[0008] Preferably, in the image recognition mode, the formula for calculating the information entropy H is: ; in, Gray values in grayscale images captured by a camera The probability of a pixel appearing; The edge processor selects the upload resolution based on H: If H <T low Using the first resolution; If H>T high Use the second resolution; Otherwise, use the third resolution; Among them, T low As a low-complexity threshold, T high For the high complexity threshold, T low =3.0, T high =6.0.
[0009] Preferably, the edge processor performs JPEG compression on the image before uploading it, and the compression quality factor Q is dynamically adjusted according to the information entropy H, calculated using the following formula: ; in, The value range is from 30 to 85. This represents the scene information entropy.
[0010] Preferably, in the voice dialogue mode, the edge processor performs Opus encoding on the audio collected by the microphone array, and the bit rate BR is adaptively adjusted according to the real-time signal-to-noise ratio SNR, calculated as follows: ; SNR is the ambient signal-to-noise ratio, measured in dB, which is estimated in real time by the edge processor from the input signal of the microphone array; BR ranges from 8 kbps to 32 kbps.
[0011] Preferably, in the real-time image transmission mode, the edge processor and the mobile APP establish a connection through the same Wi-Fi network. The edge processor measures the network throughput in real time as the current bandwidth estimate B, in Mbps, and adjusts the video stream parameters accordingly based on B. The formula for calculating frame rate (FPS) is: ; Resolution Res is determined by the following piecewise function: ; The edge processor encodes the images captured by the camera according to the adjusted resolution and frame rate, and pushes them to the mobile app via the WebSocket protocol.
[0012] Preferably, the local caching unit is also used to prefetch audio playback blocks: when the cloud server returns the synthesized audio of the voice dialogue, the edge processor requests and caches the next 2 seconds of audio data blocks at once, and prefetches the next block while playing the current block to achieve low-latency continuous playback; when the user initiates the recognition request for the same image target again, the edge processor first calculates the structural similarity between the current image and the cached image. If the structural similarity is greater than the preset threshold of 0.85, the recognition result in the cache is directly read and played through the bone conduction speaker without being uploaded to the cloud.
[0013] Preferably, the edge processor is further configured with dual-modal triggering logic: continuously acquiring user eye images through the camera module, calculating the gaze point coordinates using the pupil-corneal reflex method, and initiating voice wake-up monitoring when the gaze point falls on the center area of the screen for more than 1.5 seconds; if the microphone array acquires the preset keyword "What is this?" within the following second, it automatically enters the image recognition mode; if "Hello" or "Hi Glass" is acquired, it automatically enters the voice dialogue mode; the activation and deactivation of this dual-modal triggering logic are controlled by the low-power standby state of the power management unit.
[0014] Preferably, the wireless communication module adopts dual-mode transmission of Wi-Fi 6 and Bluetooth 5.3. The Wi-Fi 6 transmission channel uses the WPA3 encryption protocol to encrypt the interactive information, and the Bluetooth 5.3 transmission channel uses the AES-128 encryption algorithm to encrypt the collected sensor data. The encryption key is randomly generated by the edge processor every 30 minutes and the key is exchanged with the mobile APP or cloud server through asymmetric encryption.
[0015] Preferably, the edge processor is an ESP32-S3 or equivalent chip, which integrates an image signal processor and an audio encoder. The local cache unit is a Flash memory with an SPI interface and a capacity of at least 16MB, used to store the most recent 5 image recognition records and the most recent 3 prefetched audio segments. After the power management unit has been idle for more than 10 seconds, it puts the camera module, microphone array and wireless communication module into a sleep state, and only keeps the low-power voice wake-up module of the edge processor running. The overall standby power consumption of the system is less than 5mW.
[0016] Preferably, the cloud server includes an image recognition engine and a large speech model engine; the image recognition engine uses a convolutional neural network model to output recognition labels and corresponding text descriptions; the large speech model engine includes an automatic speech recognition module, a large language model module, and a text-to-speech module, wherein the automatic speech recognition module converts Opus-encoded audio into text, the large language model module generates response text based on the text, and the text-to-speech module synthesizes the response text into an audio stream in Opus or MP3 format, and sends it to the AI glasses via HTTP chunked transmission.
[0017] The present invention achieves the following beneficial technical effects compared to the prior art: This invention provides an AI glasses system featuring dynamic adaptation and low-latency collaborative interaction. The image acquisition module dynamically selects the upload resolution based on scene information entropy, significantly reducing the amount of uploaded data and response time while ensuring recognition accuracy. The speech encoding module adaptively adjusts the Opus bit rate based on real-time signal-to-noise ratio, improving speech clarity in noisy environments and reducing transmission load in quiet environments. The real-time image transmission module adjusts resolution and frame rate in conjunction with bandwidth estimation to ensure smooth video streaming under different Wi-Fi signal strengths. The local caching unit prefetches recent interaction results and audio blocks, reducing cloud round trips and lowering auditory latency. Dual-modal triggering logic fuses user gaze duration and speech keywords to achieve natural and seamless interaction initiation. This system effectively solves the technical bottlenecks of existing AI glasses in image uploading, speech transmission, image transmission smoothness, and natural interaction, providing a more efficient and intelligent solution for applications such as outdoor recording, encyclopedia queries, and voice companionship. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a diagram illustrating the architecture of AI glasses provided by the present invention. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] The purpose of this invention is to provide an AI glasses solution to the problems existing in the prior art.
[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0023] Example 1: This invention provides AI glasses, comprising an AI glasses unit and a cloud server. The AI glasses unit and the cloud server transmit data via a Wi-Fi wireless network. The AI glasses unit uploads collected image or audio data to the cloud server, and the cloud server sends the recognition results or synthesized audio back to the AI glasses unit. The AI glasses unit is specifically configured with a miniature camera module, a microphone array, an edge processor, a wireless communication module, a bone conduction speaker, a local cache unit, and a power management unit. The edge processor is electrically connected to the camera module, microphone array, wireless communication module, bone conduction speaker, and local cache unit, and is used to execute various adaptive control methods. The local cache unit stores the most recent image recognition result and pre-fetched audio blocks. When the user repeatedly queries the same target or engages in continuous dialogue, the edge processor prioritizes reading data from the local cache unit, thereby reducing the number of cloud requests and lowering interaction latency.
[0024] In image recognition mode, after the user wears AI glasses and triggers image acquisition, the edge processor first obtains a YUV format image from the camera module and converts it into a grayscale image. To balance recognition accuracy and upload speed, the edge processor calculates the information entropy H of the grayscale image. Information entropy is used to quantify the complexity of the scene. The formula for calculating information entropy H is: ; in, Gray values in grayscale images captured by a camera (Value range 0 to 255) The probability of a pixel appearing. Represents the logarithm to base 2. A larger information entropy H indicates a more complex image texture and richer details; a smaller H value indicates a flatter image (e.g., a solid color background or sky). Based on the size of H, the edge processor dynamically selects the upload resolution: when H... <T low When H > T, it indicates a simple scene, using the first resolution of 320×240; when H > T high When T indicates a complex scene, a second resolution of 640×480 is used; when T... low ≤H≤T high At that time, a third resolution of 480×360 is used. In this embodiment, the low complexity threshold T low The value is 3.0, and the high complexity threshold T is... high The value is 6.0, and the threshold was determined through statistical analysis of a large number of outdoor and indoor scene images. This dynamic resolution selection strategy significantly reduces the amount of data uploaded in simple scenes, while preserving sufficient detail to ensure accurate cloud-based recognition in complex scenes.
[0025] Before uploading the image, the edge processor also performs JPEG compression to further reduce the transmission size. The compression quality factor Q is not a fixed value, but is dynamically adjusted according to the information entropy H, and its calculation formula is as follows: ; in, The value range is from 30 to 85. Let H be the scene information entropy. When the information entropy H is close to 5 (medium complexity), Q is close to 85, and the image quality is the highest. When H deviates from 5, Q decreases accordingly, thus compressing the file size as much as possible while ensuring the minimum quality required for recognition. For example, in a flat scene with H=3, Q=83 is calculated; while in a highly textured scene with H=8, Q=80.5 is rounded down to 80. The JPEG image after the above adaptive compression is uploaded to the image recognition engine on the cloud server via a wireless communication module. This engine uses a convolutional neural network model to output recognition labels and corresponding text descriptions. The cloud returns the results to the AI glasses, and the edge processor parses them and then broadcasts them to the AI glasses via a bone conduction speaker.
[0026] In voice dialogue mode, users activate the interaction via voice wake-up or button presses. A microphone array (in this embodiment, a circular four-microphone array supporting beamforming noise reduction) collects user voice and ambient audio. The edge processor first estimates the signal-to-noise ratio (SNR) of the current environment, expressed in dB. The estimation method can be the energy ratio of frames before and after voice activity detection. To minimize the amount of data uploaded while ensuring voice clarity, the edge processor performs Opus encoding on the audio and adaptively adjusts the encoding bit rate (BR) in real time based on the SNR. The calculation formula is as follows: ; SNR, or Environmental Signal-to-Noise Ratio, is measured in dB and is estimated in real-time by the edge processor from the microphone array input signal. BR ranges from 8 kbps to 32 kbps. This linear mapping ensures that a lower SNR (noisier environment) results in a lower BR value, reducing the amount of data uploaded and mitigating network congestion risk; conversely, a higher SNR (quieter environment) results in a higher BR value, preserving more speech details and improving ASR recognition accuracy. For example, when SNR = 8 dB (noisy environment), BR = 8 kbps; when SNR = 24 dB (quiet environment), BR = 12 kbps; and when SNR = 100 dB (extremely quiet environment), the upper limit of BR is 32 kbps. The encoded Opus frames are uploaded to the speech large model engine on the cloud server via the wireless communication module. This engine includes an Automatic Speech Recognition (ASR) module, a Large Language Model (LLM) module, and a Text-to-Speech (TTS) module. The ASR module converts Opus audio into text, the LLM module generates response text based on the text, and the TTS module synthesizes the response text into an audio stream in Opus or MP3 format, which is then sent to the AI glasses via HTTP chunked transmission.
[0027] In real-time image transmission mode, users establish a connection with the AI glasses via a mobile app. When the mobile app and AI glasses are on the same Wi-Fi network, the edge processor of the AI glasses initiates the mDNS service for automatic device discovery. After the connection is established, the edge processor measures the network throughput in real time as a current bandwidth estimate B (in Mbps), and adjusts the video stream's resolution Res and frame rate FPS accordingly to adapt to network fluctuations. The formula for calculating frame rate FPS is: ; The FPS was ultimately limited to between 10 fps and 30 fps. Bandwidth B has a linear relationship with the frame rate: when B = 2 Mbps, FPS = 10; when B = 6 Mbps, FPS = 30. Resolution Res is determined by the following piecewise function: ; When the bandwidth B is greater than 5 Mbps, a standard definition resolution of 640×480 is used; when the bandwidth is between 2 Mbps and 5 Mbps, a WVGA resolution of 480×360 is used; and when the bandwidth is less than 2 Mbps, a QVGA resolution of 320×240 is used. The edge processor encodes the images captured by the camera using MJPEG or H.264 according to the adjusted resolution and frame rate, and pushes them to the mobile app via the WebSocket protocol. This linkage mechanism effectively avoids screen stuttering or glitches caused by network jitter, improving the real-time viewing experience.
[0028] To further reduce interaction latency, this invention incorporates a local caching unit within the AI glasses. This caching unit not only stores the most recent image recognition result (including image features and text tags returned from the cloud) but also supports an audio prefetching mechanism. Specifically, when the cloud server returns synthesized audio of a voice conversation, the edge processor requests and caches the subsequent 2-second audio data block at once, prefetching the next block while playing the current one, thus achieving low-latency continuous playback and avoiding network round-trip waiting caused by requesting each block individually. When the user initiates another recognition request for the same image target (e.g., repeatedly photographing the cover of the same book), the edge processor first calculates the structural similarity (SSIM) between the currently captured image and the cached image. If the SSIM is greater than a preset threshold of 0.85, it is determined to be the same target, and the edge processor directly reads the recognition result from the local cache and plays it through the bone conduction speaker, without repeatedly uploading it to the cloud. This mechanism can reduce the response time from seconds to milliseconds when users frequently query fixed objects (such as word cards in a learning scenario or exhibits in a museum).
[0029] To enhance the naturalness and convenience of interaction, this invention also incorporates dual-modal triggering logic within the edge processor. This logic continuously captures images of the user's eyes using the camera module and calculates the gaze coordinates in real time via the pupil-corneal reflex method. When the edge processor detects that the user's gaze falls on the central area of the AI glasses' display screen (e.g., a circular area with the screen center as the center and a radius of 1 / 4 of the screen width) for more than 1.5 seconds, the system determines that the user has a clear intention to interact and immediately initiates voice wake-up monitoring. Within the following second, if the microphone array detects the preset keyword "What is this?", it automatically enters the image recognition mode; if it detects "Hello" or "Hi Glass", it automatically enters the voice dialogue mode. If no valid keyword is detected within 1 second, the system returns to standby mode. The activation and deactivation of this dual-modal triggering logic are controlled by the low-power standby state of the power management unit—when the user is not wearing glasses or has not operated for a long time, the relevant sensor modules enter sleep mode, retaining only the low-power voice wake-up module to reduce energy consumption.
[0030] Regarding data transmission security, the wireless communication module of this invention employs dual-mode transmission of Wi-Fi 6 and Bluetooth 5.3. The Wi-Fi 6 transmission channel uses the WPA3 encryption protocol to encrypt interactive information (such as images, audio, and recognition results). The WPA3 protocol replaces the traditional pre-shared key method with Synchronous Peer Authentication (SAE), effectively preventing dictionary attacks and offline cracking. The Bluetooth 5.3 transmission channel uses the AES-128 encryption algorithm to encrypt collected sensor data (such as accelerometer and gyroscope information). The encryption key is randomly generated by the edge processor every 30 minutes and exchanged with the mobile app or cloud server using an asymmetric encryption method (such as RSA-2048) to ensure the security of the key transmission process. This dual encryption mechanism effectively prevents man-in-the-middle attacks and data leakage, protecting user privacy.
[0031] In a preferred embodiment of the present invention, the edge processor uses the Espressif ESP32-S3 chip, which integrates an image signal processor (ISP) and an audio encoder, enabling efficient image preprocessing and Opus encoding tasks. The local cache unit is a 16MB Flash memory with an SPI interface, used to store the five most recent image recognition records and the three most recent prefetched audio segments (each segment is 2 seconds long, approximately 2-8KB after Opus encoding). After more than 10 seconds of inactivity, the power management unit puts the camera module, microphone array, and wireless communication module into sleep mode, retaining only the edge processor's low-power voice wake-up module (e.g., using the ESP32-S3's ULP coprocessor). Actual measurements show that the overall system standby power consumption is less than 5mW, meeting the long-term battery life requirements of wearable devices.
[0032] The following example, using a specific outdoor travel application scenario, illustrates the complete workflow of this invention. A user wearing AI glasses is hiking and sees an unfamiliar wildflower. The user focuses their gaze on the center area of the glasses screen for 1.5 seconds and then says, "What is this?" The edge processor detects that the bimodal trigger condition is met and automatically enters the image recognition mode. The camera captures an image of the wildflower, and the edge processor calculates the information entropy H=4.2. Since H is between 3.0 and 6.0, a resolution of 480×360 is selected, and Q=84 is calculated according to the formula. After JPEG compression, the image size is approximately 45KB and uploaded to the cloud via Wi-Fi. The cloud image recognition engine returns the text "Azalea, Ericaceae, flowering period April-June," which the edge processor broadcasts through a bone conduction speaker. Simultaneously, the recognition result and image features are stored in the local cache. The user then continues hiking and encounters another plant with a similar appearance, triggering image recognition again. The edge processor calculates the SSIM of the current image and the cached azalea image to be 0.91 (greater than 0.85), determining them to be the same target. It directly broadcasts the cached result with a response time of only 0.1 seconds, eliminating the need for repeated uploads. The user then wants to share the scenery with companions and initiates a real-time image transmission request via the mobile app. At this time, the Wi-Fi signal is weak, and the edge processor measures a bandwidth of B = 1.8 Mbps. Based on the formula, it adjusts the FPS to min(30, max(10, 9)) = 10fps, and the resolution Res is set to 320×240, maintaining a relatively smooth image transmission. When the user moves to a location with a better signal (B = 6Mbps), the system automatically increases the FPS to 30 and the resolution to 640×480, resulting in a clearer and smoother image on the user's phone. Throughout the process, all data transmission is double-encrypted using WPA3 and AES-128 to ensure the user's location and voice privacy are not compromised. When the user removes their glasses and remains still for more than 10 seconds, the system automatically enters a low-power standby mode, with a standby current of less than 1mA.
[0033] In summary, the AI glasses provided by this invention significantly improve the response speed, recognition accuracy, transmission smoothness, and interaction naturalness of AI glasses in diverse environments through information entropy-driven resolution and JPEG quality adaptation, signal-to-noise ratio-driven Opus bit rate adaptation, bandwidth-driven image transmission resolution / frame rate linkage, local caching and prefetching mechanisms, and dual-modal natural interaction triggering. At the same time, it takes into account data security and low power consumption design, and has good prospects for industrial application.
[0034] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0035] It should be noted that the components mentioned in the above embodiments are all general standard parts or components known to those skilled in the art. Their structures and principles can be learned by those skilled in the art through technical manuals or conventional experimental methods.
[0036] This invention has illustrated its principles and implementation methods using specific examples. The descriptions of these embodiments are merely illustrative of the method and its core ideas; furthermore, those skilled in the art will recognize that modifications may be made to the specific implementation methods and application scope based on the principles of this invention. Therefore, the content of this specification should not be construed as limiting the invention.
Claims
1. An AI glasses, characterized in that, include: AI glasses and cloud server; The AI glasses and the cloud server transmit data via Wi-Fi wireless network. The AI glasses upload the collected image data or audio data to the cloud server, and the cloud server sends the recognition results or synthesized audio back to the AI glasses. The AI glasses are equipped with: a miniature camera module, a microphone array, an edge processor, a wireless communication module, a bone conduction speaker, a local cache unit, and a power management unit; The edge processor is electrically connected to the camera module, microphone array, wireless communication module, bone conduction speaker, and local cache unit, respectively, and is used to execute the following control methods: In the photo recognition mode, the image is captured and the scene information entropy H is calculated. The upload resolution is dynamically selected based on the information entropy H. In the voice dialogue mode, the audio is captured and the environmental signal-to-noise ratio SNR is estimated. The bit rate BR of the Opus encoding is dynamically adjusted based on the SNR. In the real-time image transmission mode, the current network bandwidth B is estimated. The resolution Res and frame rate FPS of the video stream are adjusted in conjunction with the bandwidth B. The local cache unit is used to store the most recent image recognition result and prefetched audio blocks. When a user repeatedly queries the same target or has a continuous conversation, the edge processor prioritizes reading data from the local cache unit to reduce the number of cloud requests.
2. The AI glasses according to claim 1, characterized in that, In the image recognition mode, the formula for calculating information entropy H is: ； in, Gray values in grayscale images captured by a camera The probability of a pixel appearing; The edge processor selects the upload resolution based on H: If H <T low Using the first resolution; If H>T high Use the second resolution; Otherwise, use the third resolution; Among them, T low As a low-complexity threshold, T high For high complexity threshold, T low =3.0, T high =6.
0.
3. The AI glasses according to claim 2, characterized in that, The edge processor performs JPEG compression on the image before uploading it. The compression quality factor Q is dynamically adjusted based on the information entropy H, and the calculation formula is as follows: ； in, The value range is from 30 to 85. This represents the scene information entropy.
4. The AI glasses according to claim 1, characterized in that, In the voice dialogue mode, the edge processor performs Opus encoding on the audio collected by the microphone array, and the bit rate BR is adaptively adjusted according to the real-time signal-to-noise ratio SNR. The calculation formula is as follows: ； SNR is the ambient signal-to-noise ratio, measured in dB, which is estimated in real time by the edge processor from the input signal of the microphone array; BR ranges from 8 kbps to 32 kbps.
5. The AI glasses according to claim 1, characterized in that, In the real-time image transmission mode, the edge processor and the mobile app establish a connection through the same Wi-Fi network. The edge processor measures the network throughput in real time as the current bandwidth estimate B, in Mbps, and adjusts the video stream parameters accordingly based on B. The formula for calculating frame rate (FPS) is: ； Resolution Res is determined by the following piecewise function: ； The edge processor encodes the images captured by the camera according to the adjusted resolution and frame rate, and pushes them to the mobile app via the WebSocket protocol.
6. The AI glasses according to claim 1, characterized in that, The local caching unit is also used to prefetch audio playback blocks: when the cloud server returns the synthesized audio of the voice dialogue, the edge processor requests and caches the next 2 seconds of audio data blocks at once, and prefetches the next block while playing the current block to achieve low-latency continuous playback; when the user initiates the recognition request for the same image target again, the edge processor first calculates the structural similarity between the current image and the cached image. If the structural similarity is greater than the preset threshold of 0.85, the recognition result in the cache is directly read and played through the bone conduction speaker without being uploaded to the cloud.
7. The AI glasses according to claim 1, characterized in that, The edge processor is also equipped with dual-modal triggering logic: continuously acquiring user eye images through the camera module, calculating the gaze coordinates using the pupil-corneal reflex method, and initiating voice wake-up monitoring when the gaze falls on the center area of the screen for more than 1.5 seconds; if the microphone array acquires the preset keyword "What is this?" within the next second, it automatically enters the image recognition mode; if it acquires "Hello" or "HiGlass", it automatically enters the voice dialogue mode; the activation and deactivation of this dual-modal triggering logic is controlled by the low-power standby state of the power management unit.
8. The AI glasses according to claim 1, characterized in that, The wireless communication module adopts dual-mode transmission of Wi-Fi 6 and Bluetooth 5.
3. The Wi-Fi 6 transmission channel uses the WPA3 encryption protocol to encrypt the interactive information, while the Bluetooth 5.3 transmission channel uses the AES-128 encryption algorithm to encrypt the collected sensor data. The encryption key is randomly generated by the edge processor every 30 minutes and the key is exchanged with the mobile APP or cloud server through asymmetric encryption.
9. The AI glasses according to claim 1, characterized in that, The edge processor is an ESP32-S3 or equivalent chip, which integrates an image signal processor and an audio encoder. The local cache unit is a Flash memory with an SPI interface and a capacity of at least 16MB, used to store the most recent 5 image recognition records and the most recent 3 prefetched audio segments. After no operation for more than 10 seconds, the power management unit puts the camera module, microphone array and wireless communication module into sleep mode, and only keeps the low-power voice wake-up module of the edge processor running. The overall standby power consumption of the system is less than 5mW.
10. The AI glasses according to claim 1, characterized in that, The cloud server includes an image recognition engine and a large speech model engine. The image recognition engine uses a convolutional neural network model to output recognition labels and corresponding text descriptions. The large speech model engine includes an automatic speech recognition module, a large language model module, and a text-to-speech module. The automatic speech recognition module converts Opus-encoded audio into text, the large language model module generates response text based on the text, and the text-to-speech module synthesizes the response text into an audio stream in Opus or MP3 format and sends it to the AI glasses via HTTP chunked transmission.