Dynamic interaction method and system of intelligent glasses

Abstract

The invention discloses a dynamic interaction method and system for intelligent glasses, and the method specifically comprises the steps: dynamically adjusting a temperature alarm threshold value according to the head movement frequency, and triggering a hierarchical frequency reduction strategy when detecting that the local temperature exceeds a preset temperature threshold value; according to a multi-spectral imaging unit of the intelligent glasses, analyzing reflection attenuation characteristics of eyelids to near-infrared light and ultrasonic echo time delay changes, generating a composite blink characteristic vector, and constructing a blink mode classifier; vehicle data and environment light sensor data built in the intelligent glasses are fused, a driving scene three-dimensional feature space is constructed, and an online incremental learning algorithm is utilized to dynamically generate an interaction strategy set adaptive to the scene; and based on the interaction strategy set, generating spatialization warning audio through a bone conduction unit of the intelligent glasses, and projecting dynamic focal plane AR prompt information on a waveguide display of the intelligent glasses. The problem of heating of the intelligent glasses is effectively solved, and the interaction experience and safety in the driving process are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a dynamic interaction method and system for smart glasses. Background Art

[0002] With the rapid development of smart glasses technology, its application potential in the driving scenario has become increasingly prominent. However, although the application prospect of smart glasses in the driving scenario is broad, its existing interaction methods have to a certain extent restricted its effective use during driving. Currently, the interaction methods of most smart glasses mainly rely on manual operations or voice commands. Manually operating smart glasses requires the driver to be distracted to operate the device during driving, which not only distracts the driver's attention and increases driving risks, but also in high-speed driving or complex road conditions, the driver needs to concentrate on the road conditions and vehicle control, and at this time manual operation is obviously impractical.

[0003] Although the voice command interaction method has to a certain extent solved the problem of manual operation, enabling the driver to control the smart glasses through voice commands, in a noisy driving environment, the accuracy and reliability of voice recognition are often seriously affected. For example, the conversation sounds of passengers in the car, the noises outside the car, and the working noises of the smart glasses themselves may all interfere with the voice recognition effect, resulting in incorrect or unrecognized commands, thereby affecting the accuracy and efficiency of the interaction.

[0004] In addition to the limitations of the interaction method, the problem that smart glasses are prone to heat during use cannot be ignored. In a high-temperature cockpit, since smart glasses need to continuously operate to provide various functions, its internal components such as processors, displays, etc. will generate a large amount of heat. If these heats cannot be dissipated in time, it will cause the temperature of the smart glasses to rise, thereby affecting its performance and stability. For example, high temperature may cause problems such as processor downclocking, reduced display brightness or color distortion, thus affecting the normal use of smart glasses.

[0005] More importantly, the heat generation of smart glasses may also have a negative impact on the driver's comfort. Wearing a hot smart glasses for a long time will make the driver feel uncomfortable, and may even cause problems such as eye fatigue or headache, further affecting driving safety.

[0006] However, most of the existing smart glasses lack effective temperature monitoring and regulation mechanisms. They cannot dynamically adjust interaction strategies and parameters according to the working state of the glasses and user behavior to cope with the impact of heat generation on the interaction effect. For example, when the temperature of the smart glasses is too high, existing devices often cannot timely reduce parameters such as the frequency or brightness of the processor to reduce the heat generation, resulting in a decline in the interaction effect or even device damage.

[0007] In summary, the existing interaction methods of smart glasses in the driving scenario have limitations, and lack effective temperature monitoring and regulation mechanisms. These problems not only limit the effective application of smart glasses in the driving scenario, but also may pose potential threats to driving safety. Summary of the Invention

[0008] The purpose of the present invention is to provide a dynamic interaction method and system for smart glasses, which meet the special requirements in the driving scenario, effectively solve the heat generation problem of smart glasses, and improve the interaction experience and safety during driving, so as to solve at least one of the above-mentioned prior art problems.

[0009] In the first aspect, the present invention provides a dynamic interaction method for smart glasses, and the method specifically includes: Real-time collect the temperature distribution data of the smart glasses body and the acceleration data of the user's head movement, and establish a dynamic temperature threshold model according to the temperature distribution data and the acceleration data; Based on the dynamic temperature threshold model, dynamically adjust the temperature alarm threshold according to the head movement frequency. When it is detected that the local temperature exceeds the preset temperature threshold, trigger a hierarchical frequency reduction strategy; Analyze the reflection attenuation characteristics of the eyelid to near-infrared light and the change of ultrasonic echo delay according to the multispectral imaging unit of the smart glasses, generate a composite blink feature vector, and construct a blink pattern classifier in combination with a convolutional pulse neural network. The blink pattern classifier is used to distinguish between regular physiological blinks and command blinks; Real-time obtain the vehicle driving speed, steering wheel angle and GPS positioning information to form vehicle data, fuse the vehicle data and the ambient light sensor data built in the smart glasses, construct a three-dimensional feature space of the driving scenario, and dynamically generate an interaction strategy set adapted to the scenario by using an online incremental learning algorithm; Based on the interaction strategy set, generate a spatial warning audio through the bone conduction unit of the smart glasses, and project dynamic focal plane AR prompt information on the waveguide display of the smart glasses at the same time.

[0010] In the second aspect, the present invention provides a dynamic interaction system for smart glasses, and the system specifically includes: The first dynamic interaction module is used to real-time collect the temperature distribution data of the smart glasses body and the acceleration data of the user's head movement, and establish a dynamic temperature threshold model according to the temperature distribution data and the acceleration data; The second dynamic interaction module is used to dynamically adjust the temperature alarm threshold according to the head movement frequency based on the dynamic temperature threshold model. When it is detected that the local temperature exceeds the preset temperature threshold, trigger a hierarchical frequency reduction strategy; The third dynamic interaction module is used to analyze the reflection attenuation characteristics of the eyelid to near-infrared light and the change of ultrasonic echo delay according to the multispectral imaging unit of the smart glasses, generate a composite blink feature vector, and construct a blink pattern classifier in combination with a convolutional pulse neural network. The blink pattern classifier is used to distinguish between regular physiological blinks and command blinks; The fourth dynamic interaction module is used to obtain the vehicle driving speed, steering wheel angle and GPS positioning information in real time, form vehicle data, fuse the vehicle data with the ambient light sensor data built into the smart glasses, construct a three-dimensional feature space of the driving scene, and dynamically generate a set of interaction strategies adapted to the scene by using an online incremental learning algorithm; The fifth dynamic interaction module is used to generate a spatialized warning audio through the bone conduction unit of the smart glasses based on the set of interaction strategies, and at the same time project dynamic focal plane AR prompt information on the waveguide display of the smart glasses.

[0011] In a third aspect, the present invention provides a computer device, including: a memory, a processor, and a computer program stored on the memory. When the computer program is executed on the processor, it implements the dynamic interaction method of the smart glasses as described in any one of the above methods.

[0012] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it implements the dynamic interaction method of the smart glasses as described in any one of the above methods.

[0013] Compared with the prior art, the present invention has at least one of the following technical effects: 1. The present invention meets the special requirements in the driving scene, effectively solves the heat generation problem of the smart glasses, and improves the interaction experience and safety during driving.

[0014] 2. The present invention reduces the manual operations of the driver during driving by introducing natural interaction methods such as blink control, reduces the safety hazards caused by operation distraction, and at the same time provides a more natural and intuitive interaction experience.

[0015] 3. The present invention dynamically adjusts the interaction strategies and parameters, such as triggering a hierarchical frequency reduction strategy, etc., by monitoring the working state of the glasses in real time to cope with the impact of glasses heat generation on the interaction effect, and ensures the stability and reliability of the interaction.

[0016] 4. The present invention constructs a three-dimensional feature space of the driving scene and dynamically generates a set of interaction strategies adapted to the scene by using an online incremental learning algorithm, providing a more personalized and intelligent driving assistance experience for the driver.

[0017] 5. The present invention dynamically adjusts the interaction strategy by identifying changes in the driving scenario, enabling the smart glasses to better adapt to the special requirements in the driving scenario and enhancing the convenience and safety of driving.

[0018] 6. Through the dynamic temperature threshold model, the smart glasses can adaptively adjust the temperature alarm threshold, effectively coping with the heating problem in different usage scenarios and enhancing the device stability and user comfort.

[0019] 7. By dynamically adjusting the temperature alarm threshold and triggering the hierarchical frequency reduction strategy, the smart glasses can effectively reduce the heat generation while ensuring performance, extending the service life of the device.

[0020] 8. By combining multi-spectral imaging and ultrasonic technology with convolutional pulse neural network, the smart glasses can accurately identify the user's blinking action, realizing natural and intuitive interactive control.

[0021] 9. By fusing multi-source data to construct a three-dimensional feature space of the driving scenario and using the online incremental learning algorithm, the smart glasses can generate real-time interaction strategies adapted to different driving scenarios, enhancing the driving safety and convenience.

[0022] 10. Through the online incremental learning algorithm, the smart glasses can continuously learn and optimize the interaction strategy, adapt to the changing driving environment, and provide more personalized driving assistance.

[0023] 11. By using the kernel density estimation algorithm with a forgetting factor, the accuracy and robustness of the clustering analysis are improved, enabling the smart glasses to more precisely select the optimal interaction strategy and enhancing the user experience.

[0024] 12. By combining bone conduction audio and AR prompt information, the smart glasses can provide warning and navigation information to the driver in a non-intrusive manner, improving the driving safety and the intuitiveness of information acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 is a flowchart of a dynamic interaction method for a smart glasses provided by an embodiment of the present invention; Figure 2 is a structural diagram of a dynamic interaction system for a smart glasses provided by an embodiment of the present invention; Figure 3 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0027] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0028] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0029] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] As used in the specification of the present application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" depending on the context.

[0031] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0032] The reference to "an embodiment" or "some embodiments" etc. described in the specification of the present application means that specific features, structures, or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Thus, statements such as "in an embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0033] In the embodiment of the present application, the execution subject of the process includes a terminal device, which includes but is not limited to: a server, a computer, a smart phone, a tablet computer, and other devices capable of executing the method disclosed in the present application. Figure 1 A schematic flow chart of a dynamic interaction method of smart glasses disclosed in an embodiment of the present invention is shown, and is described in detail as follows: S101, collecting temperature distribution data of the smart glasses body and acceleration data of the user's head movement in real time, and establishing a dynamic temperature threshold model according to the temperature distribution data and acceleration data.

[0034] In this embodiment, with the continuous development of smart glasses technology, its application potential in driving scenarios has received increasing attention. However, existing smart glasses have problems such as limited interaction methods in driving scenarios, heat affecting performance and comfort. In particular, when smart glasses are running in a high temperature environment, if the heat generated by its internal components cannot be dissipated in time, it will seriously affect its performance and stability, and may even pose a threat to driving safety. Therefore, this embodiment provides a method for establishing a dynamic temperature threshold model based on the temperature distribution data of the smart glasses body and the acceleration data of the user's head movement.

[0035] Specifically, high-precision temperature sensors are placed in key heat-generating areas of smart glasses (such as processors, display screens, etc.) to collect temperature data in these areas in real time. The temperature data collected by the temperature sensor is transmitted to the main control unit of the smart glasses in real time through a built-in wireless communication module (such as Bluetooth, Wi-Fi, etc.). The main control unit pre-processes the received temperature data, including filtering, denoising, and other operations, to improve the reliability of the data.

[0036] A high-precision, low-power three-axis accelerometer is selected and installed on the frame of the smart glasses to collect acceleration data of the user's head movement in real time. The accelerometer should have the characteristics of wide measurement range, high resolution and fast response to accurately capture the slight movement of the user's head. The acceleration data collected by the accelerometer is also transmitted to the main control unit through the wireless communication module. The main control unit processes the acceleration data in real time, including motion trajectory analysis, motion frequency calculation, etc., to obtain the key characteristic parameters of the user's head movement.

[0037] Fuse the collected temperature distribution data and head movement acceleration data to form a multi-dimensional data set containing temperature and movement characteristics. Data fusion can adopt methods such as feature fusion and decision fusion to improve the comprehensive utilization efficiency of data. Use machine learning algorithms (such as support vector machines, neural networks, etc.) to train the fused multi-dimensional data set to establish a dynamic temperature threshold model. During the model training process, factors such as different driving scenarios and different user habits that affect the temperature threshold should be fully considered to ensure the accuracy and generalization ability of the model. Verify and optimize the established dynamic temperature threshold model through actual driving tests. During the test, record the temperature changes of the smart glasses and the head movement of the user in different driving scenarios, and compare and analyze them with the model prediction results. Iteratively optimize the model according to the comparison and analysis results to improve the prediction accuracy and stability of the model.

[0038] Based on the established dynamic temperature threshold model, dynamically adjust the temperature alarm threshold according to parameters such as the head movement frequency of the user. When it is detected that the local temperature of the smart glasses exceeds the preset temperature threshold, trigger the corresponding alarm mechanism (such as sound prompt, vibration reminder, etc.) to remind the user to take cooling measures in time. Dynamically adjust the interaction strategy of the smart glasses according to the temperature alarm threshold and driving scenario characteristics. For example, reduce the processor frequency of the smart glasses, adjust parameters such as the display screen brightness in a high-temperature environment to reduce the heat generation; in a complex driving scenario, give priority to ensuring the normal operation of key functions such as navigation assistance and safety warning.

[0039] By implementing this embodiment, the smart glasses can real-time monitor its body temperature distribution and the head movement acceleration data of the user, and accordingly establish a dynamic temperature threshold model. This model can dynamically adjust the temperature alarm threshold and interaction strategy according to different driving scenarios and user habits, effectively avoiding the problems of performance degradation and comfort reduction caused by the operation of the smart glasses in a high-temperature environment. At the same time, this embodiment also improves the safety and reliability of the smart glasses in the driving scenario, providing a more convenient and intelligent driving experience for users.

[0040] S102, based on the dynamic temperature threshold model, dynamically adjust the temperature alarm threshold according to the head movement frequency. When it is detected that the local temperature exceeds the preset temperature threshold, trigger a hierarchical frequency reduction strategy.

[0041] In this embodiment, in the driving scenario, the smart glasses are an important device for assisted driving, and its performance stability and user experience are crucial. However, since the smart glasses generate heat during operation, if heat cannot be dissipated in a timely and effective manner, it may lead to a decline in device performance, a deterioration of the user experience, and even affect driving safety. Therefore, how to dynamically adjust the temperature alarm threshold according to the actual operating state of the smart glasses and user behavior, and take frequency reduction measures when necessary to reduce the device heat generation, has become an urgent problem to be solved.

[0042] Specifically, using the trained dynamic temperature threshold model, the temperature alarm threshold is dynamically adjusted according to the real-time monitored head movement frequency of the user. For example, when the head movement frequency of the user is high (such as frequent head shaking, nodding, etc.), it indicates that the user may be in a highly tense or focused state. At this time, the temperature alarm threshold should be appropriately increased to reduce the interference caused by temperature alarms; on the contrary, when the head movement frequency of the user is low, the temperature alarm threshold can be appropriately decreased to detect and handle potential overheating problems earlier.

[0043] During the operation of the smart glasses, the temperature data of each key area is continuously monitored. When it is detected that the local temperature exceeds the currently set temperature alarm threshold, the temperature alarm mechanism is immediately triggered, and the user is notified by means such as sound prompts and vibration reminders.

[0044] According to the hardware characteristics and heat dissipation performance of the smart glasses, a hierarchical frequency reduction strategy is formulated. This strategy should include multiple frequency reduction levels, and each level corresponds to different frequency reduction amplitudes and durations. For example, when the temperature exceeds the first-level alarm threshold, the processor frequency can be reduced by 10%, and the temperature change is continuously observed; if the temperature continues to rise and exceeds the second-level alarm threshold, the processor frequency is further reduced by 20%, and other heat dissipation measures (such as increasing the fan speed, etc.) are taken.

[0045] After the temperature alarm is triggered, the smart glasses automatically execute corresponding frequency reduction measures according to the current temperature situation and the hierarchical frequency reduction strategy. At the same time, the temperature change is continuously monitored and fed back to the user. If the temperature is effectively controlled and gradually reduced to the safe range, the processor frequency is gradually restored to the normal level; if the temperature continues to rise or cannot be effectively controlled, more advanced frequency reduction measures are considered or the user is prompted to suspend using the smart glasses and seek professional help.

[0046] In this embodiment, the smart glasses can dynamically adjust the temperature alarm threshold according to the user's head movement frequency, and trigger a hierarchical frequency reduction strategy when it is detected that the local temperature exceeds the preset temperature threshold. It effectively avoids the problems of performance degradation and poor user experience caused by overheating of the smart glasses in the driving scenario, and improves the safety and reliability of the device. At the same time, through the dynamic adjustment of the temperature alarm threshold and the hierarchical frequency reduction strategy, the precise control and effective management of the heat generation problem of the smart glasses are realized.

[0047] S103, analyze the reflection attenuation characteristics of the eyelid to near-infrared light and the ultrasonic echo time delay change according to the multispectral imaging unit of the smart glasses, generate a composite blink feature vector, and construct a blink pattern classifier in combination with a convolutional pulse neural network. The blink pattern classifier is used to distinguish between regular physiological blinks and command blinks.

[0048] In this embodiment, with the increasingly widespread application of smart glasses in driving scenarios, how to achieve efficient and accurate interaction has become an urgent problem to be solved. Existing interaction methods, such as manual operation and voice commands, have problems such as distracting the driver's attention and low recognition rate in noisy environments. In addition, smart glasses are prone to heat during use, which affects performance and comfort, and there is a lack of effective temperature monitoring and control mechanisms. Therefore, developing a new type of interaction method that can not only reduce the driver's operation burden but also improve the accuracy and efficiency of interaction is of great significance for enhancing the application value of smart glasses in driving scenarios. The blinking action, as one of the natural physiological reactions of the human body, not only includes regular physiological blinking but may also include directive blinking actions consciously issued by the user. By accurately distinguishing these two blinking modes, a new and natural interaction method can be provided for smart glasses. However, existing technologies are mostly based on single sensors or simple feature extraction methods, making it difficult to effectively distinguish between regular physiological blinking and directive blinking actions.

[0049] Specifically, a multi-spectral imaging unit is integrated on the smart glasses frame. This unit includes cameras with multiple different bands (such as visible light, near-infrared light, etc.) for capturing the reflection images of the user's eyelids under multi-spectral conditions. In particular, a near-infrared light camera is used to capture the reflection attenuation characteristics of the eyelids to near-infrared light because near-infrared light can penetrate the eyelid epidermis, reflect the minute changes in the internal tissues of the eyelids, and help distinguish between physiological blinking and directive blinking.

[0050] An ultrasonic sensor is installed on the smart glasses frame to emit ultrasonic waves and receive the echo signals reflected by the eyelids. By analyzing the time-delay changes of the ultrasonic echoes, dynamic information such as the thickness and shape of the eyelids can be obtained to further assist in distinguishing between physiological blinking and directive blinking.

[0051] From the images obtained by the multi-spectral imaging unit, the reflection attenuation characteristics of the eyelids to near-infrared light are extracted. For example, features such as the brightness, contrast, and texture of the eyelids under near-infrared light can be analyzed, and these features can reflect the movement state of the eyelid muscles and the physiological changes inside the eyelids.

[0052] From the echo signals received by the ultrasonic sensor, features such as echo time-delay and amplitude are extracted, and these features can reflect dynamic information such as the thickness and shape of the eyelids and the movement speed and acceleration of the eyelid muscles.

[0053] The multi-spectral features and ultrasonic features are fused to generate a composite blinking feature vector. This feature vector contains the static and dynamic information of the eyelids and can comprehensively reflect the characteristics of the blinking action.

[0054] Design a Convolutional Spiking Neural Network (CSNN) for processing composite blink feature vectors. CSNN combines the feature extraction ability of Convolutional Neural Network (CNN) and the temporal processing ability of Spiking Neural Network (SNN), and can effectively process blink data with temporal features. Collect a large amount of physiological blink and command blink data, label the corresponding tags, and use them to train the CSNN model. During the training process, a supervised learning algorithm is adopted to optimize the model parameters by minimizing the error between the predicted label and the true label. Use an independent test data set to verify the trained CSNN model and evaluate its classification accuracy and generalization ability. According to the verification results, optimize and adjust the model, such as adjusting the network structure, adding regularization terms, etc., to improve the performance of the model.

[0055] During the operation of the smart glasses, collect the user's multi-spectral images and ultrasonic echo signals in real time, and generate composite blink feature vectors. Input the feature vectors into the trained CSNN model for real-time blink pattern classification. According to the blink pattern classification results, generate corresponding interaction instructions. For example, when a command blink action is detected, the smart glasses can be triggered to execute specific operations, such as switching the navigation interface, answering the phone, etc. Continuously optimize and adjust the blink pattern classifier according to the user's feedback and interaction effects. For example, collect the user's satisfaction evaluation of the interaction instructions and use this evaluation data to improve the classification accuracy and interaction effects of the model.

[0056] In this embodiment, the smart glasses can accurately distinguish between regular physiological blinks and command blink actions, providing a new and natural interaction method for drivers. It not only improves the interaction performance and safety of the smart glasses in the driving scenario, but also reduces the driver's operation burden and attention distraction problems. At the same time, it also has good generalization ability and adaptability, and can meet the interaction needs of different users and different driving scenarios.

[0057] S104, obtain the vehicle driving speed, steering wheel angle and GPS positioning information in real time to form vehicle data, fuse the vehicle data with the ambient light sensor data built in the smart glasses, construct a three-dimensional feature space of the driving scenario, and dynamically generate a set of interaction strategies adapted to the scenario using an online incremental learning algorithm.

[0058] In this embodiment, as the application of smart glasses in driving scenarios becomes increasingly widespread, how to dynamically adjust the interaction strategy according to different driving environments has become the key to enhancing user experience and driving safety. Existing smart glasses interaction strategies are mostly based on preset rules or simple environmental perception, making it difficult to adapt to complex and changing driving scenarios. For example, in different scenarios such as highways, urban roads, and tunnels, there are significant differences in drivers' needs and preferences for interaction information. Therefore, developing a method that can real-time sense the driving environment and dynamically generate an adapted interaction strategy is of great significance for improving the practicality and safety of smart glasses.

[0059] Specifically, through the in-vehicle OBD (On-Board Diagnostics) interface or CAN (Controller Area Network) bus, key data such as vehicle driving speed and steering wheel angle are obtained in real time. At the same time, the GPS module is used to obtain the real-time positioning information of the vehicle. These data together constitute the vehicle data set, which is used to describe the driving state and position information of the vehicle.

[0060] A high-precision ambient light sensor is integrated on the smart glasses frame to real-time sense parameters such as the illumination intensity and color temperature of the surrounding environment. The ambient light sensor data can reflect the lighting conditions of the driving scenario, providing an important basis for the subsequent construction of the feature space.

[0061] The collected vehicle data and ambient light sensor data are preprocessed, including operations such as data cleaning, denoising, and normalization. The preprocessed data will be used for subsequent feature extraction and fusion.

[0062] Key features are extracted from the preprocessed vehicle data, such as driving speed, acceleration, and the change rate of the steering wheel angle. At the same time, features such as illumination intensity and color temperature are extracted from the ambient light sensor data. These features will together constitute the feature vector of the driving scenario. The extracted feature vector is mapped into a three-dimensional space to construct the three-dimensional feature space of the driving scenario. Among them, the X-axis, Y-axis, and Z-axis can respectively represent feature dimensions such as driving speed, the change rate of the steering wheel angle (or acceleration), and illumination intensity (or color temperature). By constructing the three-dimensional feature space, the diversity and complexity of the driving scenario can be intuitively described.

[0063] In the offline stage, an initial classification or regression model is trained using historical driving data and annotated interaction strategies. This model can predict the appropriate interaction strategy based on the input feature vector. The initial model can employ traditional machine learning algorithms (such as support vector machines, decision trees, etc.) or deep learning algorithms (such as multi-layer perceptrons, convolutional neural networks, etc.). During the operation of the smart glasses, new vehicle data and ambient light sensor data are collected in real time, and the corresponding feature vectors are extracted. These new feature vectors are input into the initial model for online incremental learning. The online incremental learning algorithm can dynamically adjust the model parameters according to the new data, enabling the model to adapt to the changing driving scenarios. Preferably, an online learning algorithm based on gradient descent, such as stochastic gradient descent (SGD) or adaptive moment estimation (Adam), etc., can be used. These algorithms can fine-tune the model parameters each time new data is received, making the model gradually approach the true data distribution. To ensure the effectiveness and stability of the online incremental learning algorithm, the model needs to be evaluated regularly. The evaluation metrics can include classification accuracy, regression error, etc. If the model performance deteriorates or no longer meets the requirements, the model needs to be retrained or the parameters of the online incremental learning algorithm need to be adjusted.

[0064] According to the model prediction results output by the online incremental learning algorithm, the feature vectors are mapped to the corresponding interaction strategies. The interaction strategies can include adjustments to the display content (such as brightness, contrast, display area, etc.), the type of prompt information (such as voice prompts, visual prompts, etc.), and the switching of interaction methods (such as gesture control, voice control, etc.). To cope with the diversity and uncertainty in different driving scenarios, an interaction strategy set can be generated. This set contains multiple possible interaction strategies, and each strategy corresponds to a specific range of feature vectors or type of driving scenario. In practical applications, the most suitable interaction strategy can be selected from the strategy set according to the current feature vector and driving scenario type for execution. The generated interaction strategy is executed through the display unit, bone conduction unit, etc. of the smart glasses, and the feedback information of the user is collected in real time. The feedback information can include evaluations of the user's satisfaction with the interaction strategy, operation convenience, etc. According to the feedback information, the interaction strategy can be further optimized and adjusted to improve the user experience and driving safety.

[0065] In this embodiment, the smart glasses can real-time sense the driving environment and dynamically generate the appropriate interaction strategies. It not only improves the interaction adaptability and user experience of the smart glasses in the driving scenario, but also enhances driving safety. At the same time, it also has good scalability and flexibility, and can adapt to the interaction requirements of different vehicle models, different driving habits, and different driving scenarios.

[0066] S105. Based on the interaction strategy set, generate spatialized warning audio through the bone conduction unit of the smart glasses, and at the same time project dynamic focal plane AR prompt information on the waveguide display of the smart glasses.

[0067] In this embodiment, during driving, the driver needs to constantly pay attention to various aspects of information such as road conditions, traffic signals, and vehicle status. Traditional driving assistance systems usually provide warning information through visual or auditory means, but these methods may have problems such as information overload and interference with driving attention in complex and changeable driving environments. As an emerging driving assistance device, smart glasses have bone conduction audio output and AR (augmented reality) display functions, and can provide a more natural, intuitive, and non-interfering interaction experience for users. However, how to dynamically generate adapted warning audio and AR prompt information according to driving scenarios has become the key to improving the practicality of smart glasses.

[0068] Specifically, the strategy set includes various warning audio modes (such as emergency braking prompt sounds, lane departure warning sounds, etc.) and AR prompt information types (such as navigation arrows, speed limit icons, danger warning symbols, etc.). Each strategy defines a trigger condition, an execution method, and a priority.

[0069] For the generated warning audio strategy, spatial audio coding technology is adopted to encode the audio signal according to parameters such as sound source position, distance, and direction. By simulating the sound propagation characteristics in the real environment, the driver can perceive the spatial position information of the sound through the bone conduction unit, enhancing the realism and immersion of the warning effect.

[0070] Dynamically adjust the volume and frequency of the warning audio according to the vehicle speed, ambient noise level, and driver attention state. For example, when driving at high speed or in a noisy environment, appropriately increase the volume and frequency to ensure that the driver can clearly hear the warning information; when the driver's attention is distracted, adopt more penetrating audio characteristics to attract the driver's attention.

[0071] Output the encoded spatialized warning audio to the driver's ear through the bone conduction unit of the smart glasses. At the same time, collect the driver's feedback information on the warning audio (such as click to confirm, ignore, etc.) for optimizing subsequent audio generation strategies.

[0072] Utilize the waveguide display of the smart glasses and the built-in eye tracking technology to real-time locate the driver's fixation point and calibrate the focal plane position of the waveguide display. By precisely controlling the refraction and reflection paths of light, ensure that the AR prompt information can be accurately projected in front of the driver's line of sight, forming a visual effect that seamlessly integrates with the real environment.

[0073] According to the generated AR prompt information strategy, combined with the three-dimensional model data of the driving scenario, AR content with attributes such as spatial position, size, and color is generated. An efficient graphics rendering algorithm (such as ray tracing, rasterization, etc.) is used to perform real-time rendering of the AR content to ensure the clarity and fluency of the prompt information.

[0074] The rendered AR content is projected in front of the driver's line of sight through a waveguide display, and the display position and content of the AR prompt information are dynamically adjusted according to the driver's fixation point and the vehicle's driving state. For example, during navigation, as the vehicle moves, the direction and distance information of the navigation arrow are dynamically updated; when a potential risk is detected, a danger warning symbol is immediately displayed, accompanied by a spatialized warning audio prompt.

[0075] Support the driver to interact with the AR prompt information through gesture control, voice control, etc. For example, the driver can wave a hand to close unnecessary prompt information, or query detailed information through voice commands. At the same time, collect the driver's feedback on the AR prompt information for optimizing subsequent content generation and display strategies.

[0076] In this embodiment, the smart glasses can dynamically generate adapted spatialized warning audio and dynamic focal plane AR prompt information according to the driving scenario, providing a more natural, intuitive and non-interfering interaction experience for the driver. It not only improves the safety and information interaction efficiency during driving, but also enhances the driver's perception ability and response ability to the driving environment. At the same time, it also has good scalability and flexibility, and can adapt to the interaction requirements under different vehicle models, different driving habits and different driving scenarios.

[0077] In some embodiments, in the above step S101, the real-time acquisition of the temperature distribution data of the smart glasses body and the acceleration data of the user's head movement, and the establishment of a dynamic temperature threshold model according to the temperature distribution data and the acceleration data specifically include: The temperature distribution data is obtained in real time through a thermopile sensor array distributed on the key heat conduction paths of the smart glasses body; The acceleration data of the user's head movement is continuously collected by using the three-axis MEMS accelerometer of the smart glasses; Based on the acceleration data, the root mean square value of the motion intensity is calculated through a sliding time window, and the root mean square value of the motion intensity is input into a preset temperature threshold function to dynamically generate a first temperature warning threshold that adapts to the head movement state; According to the first dynamic temperature threshold, weighted statistics are performed on the temperature distribution data to obtain the over-temperature node distribution of the thermopile sensor array, and a dynamic temperature threshold model is formed.

[0078] In this embodiment, the smart glasses need to continuously operate in the driving scenario to provide functions such as navigation and AR display. However, its high-density electronic components are prone to local overheating. Existing technologies mostly use fixed temperature thresholds for monitoring, but they cannot adapt to the differences in heat dissipation conditions caused by the user's motion state (such as running and cycling) or environmental temperature changes. For example, the increased wind speed on the head during high-speed movement can improve the heat dissipation efficiency, while the heat dissipation conditions deteriorate when stationary, and fixed thresholds are prone to false alarms or missed alarms. In addition, traditional methods do not consider the spatial heterogeneity of temperature distribution (such as the temperature difference between the frame and the temple), resulting in the inability to effectively identify local overheating.

[0079] Specifically, deploy a high-precision thermopile sensor array on the key heat conduction paths of the smart glasses body (the areas where the frame, nose pad, temple contact the skin). Each sensor has an accuracy of ±0.1°C. The sensors are connected to the main control chip through the I²C bus to ensure real-time data transmission. Filter the collected raw temperature data (such as Kalman filtering) to eliminate sensor noise and transient fluctuations, ensuring the stability of the temperature distribution data.

[0080] Integrate a high-precision three-axis MEMS accelerometer in the smart glasses frame. The sampling rate is set to 100Hz, and it can detect an acceleration range of ±16g. Synchronize the data acquisition of the accelerometer and the thermopile sensor through timestamps, and use the hexahedron calibration method to correct the zero bias and sensitivity of the accelerometer to ensure data accuracy.

[0081] Set the time window length to 2 seconds, and slide with a step of 100ms to calculate the root mean square value (RMS) of the acceleration data to characterize the intensity of the user's head movement. Map the RMS value to a preset grading interval (such as low intensity [0-1]g, medium intensity [1-3]g, high intensity [3-16]g), and each interval corresponds to a different temperature threshold correction coefficient. For example, the heat dissipation conditions improve during high-intensity exercise, and the temperature threshold can be appropriately increased.

[0082] Preset the basic temperature threshold T 0 , and dynamically adjust the threshold according to the motion intensity grading: T = T 0 +ΔT·f(RMS), where T represents the first dynamic temperature threshold, ΔT represents the correction coefficient (such as +2°C for low intensity, +1°C for medium intensity, +0°C for high intensity), and f(RMS) represents the non-linear mapping function based on the RMS value (such as linear / exponential relationship). Recalculate the threshold every 2 seconds according to the latest RMS value and synchronize it to the thermopile sensor array through the I²C bus.

[0083] For the temperature data of each node in the thermopile sensor array, weighted statistics are performed according to its spatial position weights (such as a weight of 0.8 for the frame area and a weight of 0.5 for the temple area). Judgment condition: If the temperature of a certain node exceeds the dynamic threshold, it is marked as an over-temperature node. Generate a distribution map of over-temperature nodes, and mark the comparison results of the temperature values of each node with the threshold. The model continuously outputs the positions, temperature values, and dynamic thresholds of over-temperature nodes for subsequent thermal management strategy calls.

[0084] In this embodiment, the dynamic temperature threshold model can adjust the threshold in real time according to the user's motion state, reducing the false alarm rate. For example, when riding at high speed, the threshold is automatically increased to 47 °C to avoid false alarms caused by improved heat dissipation. Through weighted statistics and spatial distribution analysis, the model can accurately identify locally overheated nodes. For example, an over-temperature of 48 °C is detected on the left side of the frame, while the temperatures in other areas are normal. The dynamic threshold adjustment avoids frequent alarms caused by a fixed threshold, enhancing the user's trust in the smart glasses. By promptly identifying over-temperature nodes, a frequency reduction strategy or a heat dissipation mechanism can be triggered to prevent device damage or performance degradation and ensure driving safety.

[0085] In some embodiments, in the above step S102, based on the dynamic temperature threshold model, the temperature alarm threshold is dynamically adjusted according to the head movement frequency. When it is detected that the local temperature exceeds the preset temperature threshold, a hierarchical frequency reduction strategy is triggered, which specifically includes: Real-time collect the acceleration, angular velocity, and azimuth data of head movement, and calculate the time-frequency characteristics of the acceleration vector according to the acceleration, angular velocity, and azimuth data; Based on the video features, extract the proportion of the main frequency band energy of head movement, and input the proportion of the main frequency band energy into an exponential decay function to dynamically calculate the second temperature alarm threshold; Input the second temperature alarm threshold into the dynamic temperature threshold model, detect the difference between the local temperature of each area of the smart glasses body and the second temperature alarm threshold, and obtain the over-temperature level detection data; Based on the over-temperature level detection data, calculate the current frequency reduction level according to the preset hierarchical step size, and generate a hierarchical frequency reduction control instruction.

[0086] In this embodiment, the smart glasses need to continuously operate in the driving scenario to provide functions such as navigation and AR display, but its high-density electronic components are prone to local overheating. Existing technologies mostly use a fixed temperature threshold for monitoring, but they cannot adapt to the differences in heat dissipation conditions caused by the user's motion state (such as high-speed driving and sharp turns) or environmental temperature changes. For example, when the head swings rapidly, the wind speed increases, which can improve the heat dissipation efficiency, while when it is stationary, the heat dissipation condition deteriorates, and the fixed threshold is prone to false alarms or missed alarms. In addition, traditional methods do not consider the dynamic impact of head movement on heat dissipation, resulting in a lag in the response of the overheat protection mechanism.

[0087] Specifically, a high-precision three-axis accelerometer, a three-axis gyroscope, and an orientation sensor (such as an electronic compass) are integrated into the smart glasses frame, and the sampling rate is set to 200 Hz to ensure that the subtle movements of the user's head can be captured. The accelerometer and gyroscope data are fused to calculate the acceleration vector of the head movement. The short-time Fourier transform (STFT) is used to perform time-frequency analysis on the acceleration vector, and the energy ratio of the main frequency band (such as 0.5 - 5 Hz) is extracted. The energy ratio of the main frequency band is normalized to the interval [0, 1] and used as the input parameter for subsequent temperature threshold adjustment.

[0088] Preset a basic temperature threshold T 1 (such as 45 °C) as the static monitoring benchmark, and input the energy ratio E of the main frequency band into the exponential decay function: T dynamic = T 1 - T max · e -aE , where T dynamic represents the second temperature warning threshold, T max represents the maximum adjustment amplitude, and a represents the decay coefficient, which is used to control the influence degree of the energy ratio on the second temperature warning threshold. The dynamic threshold is recalculated every 1 second according to the latest energy ratio and synchronized to the thermal management module through the I²C bus.

[0089] A thermopile sensor array is deployed on the key heat conduction paths of the smart glasses body (such as the frame, nose pad, and temple) to obtain the temperature data of each area in real time. The difference between the temperature of each area and the dynamic threshold is calculated. If the difference exceeds the preset threshold (such as 2 °C), it is marked as an over-temperature area. The over-temperature level is divided according to the over-temperature degree (such as mild over-temperature [2 - 4 °C], moderate over-temperature [4 - 6 °C], severe over-temperature [> 6 °C]). A frequency reduction step size is set for each level (such as a 10% frequency reduction for mild over-temperature and a 30% frequency reduction for severe over-temperature). A control instruction including the frequency reduction level, target frequency, and duration is generated. For example, when severe over-temperature is detected in the frame area, the instruction "Reduce the frequency of the frame area by 30% for 10 minutes" is generated. The frequency reduction instruction is executed through the main control chip of the smart glasses to dynamically adjust the operating frequencies of high-power components such as the processor and display screen.

[0090] In this embodiment, the dynamic temperature warning threshold can be adjusted in real time according to the user's head movement state, reducing the false alarm rate. For example, when driving at high speed, the threshold is automatically increased to 47 °C to avoid false alarms caused by improved heat dissipation. Through multi-area temperature detection and over-temperature level division, the local over-temperature area and over-temperature degree can be accurately identified. For example, severe over-temperature is detected on the left side of the frame while the temperatures of other areas are normal. The hierarchical frequency reduction strategy can dynamically adjust the device power consumption according to the over-temperature degree, extend the device battery life, and improve the user experience. By promptly identifying the over-temperature area and triggering the frequency reduction strategy, device damage or performance degradation can be prevented, ensuring driving safety.

[0091] In some embodiments, in the above step S103, analyzing the reflection attenuation characteristics of the eyelid to near-infrared light and the change in ultrasonic echo time delay according to the multispectral imaging unit of the smart glasses, generating a composite blink feature vector, and constructing a blink pattern classifier in combination with a convolutional spiking neural network specifically includes: Based on the multispectral imaging unit of the smart glasses, alternately trigger the emission of near-infrared laser pulses and ultrasonic envelopes, and synchronously receive the reflected optical signal and the scattered acoustic signal in the eyelid area; Calculate the reflectance attenuation curve during eyelid closure according to the reflected optical signal, and extract the eye socket deformation time delay feature according to the scattered acoustic signal; Perform tensor splicing on the time-domain derivative of the reflectance attenuation curve and the spatial gradient of the eye socket deformation time delay feature to generate a composite blink feature vector with spatio-temporal correlation; Input the composite blink feature vector into a pre-trained convolutional spiking neural network, extract local time patterns through the convolutional sub-network, capture global spatio-temporal correlations through the spiking sub-network, and combine attention weights to fuse local time patterns and global spatio-temporal correlations to form a blink pattern classifier.

[0092] In this embodiment, in a driving scenario, the interaction method of the smart glasses needs to balance safety and convenience. Traditional manual operations and voice commands have the following defects: 1. Manual operations: The driver needs to be distracted to operate the device, increasing the driving risk; 2. Voice commands: They are easily interfered by environmental noise and have a low recognition accuracy. To solve the above problems, some studies have tried to achieve interaction through eye tracking technology, but existing methods mostly rely on a single signal source (such as optical or electrophysiological signals), and are easily affected by factors such as light and head posture, resulting in a high false trigger rate. For example, it is difficult to effectively distinguish the characteristics between conventional physiological blinks (such as fatigue and dryness) and command blinks (such as confirmation and switching).

[0093] Specifically, the smart glasses integrate a near-infrared laser emitter (wavelength 850 nm) and a high-speed CMOS image sensor with a frame rate ≥ 120 fps; a phased array ultrasonic probe is adopted with a frequency range of 1 - 5 MHz, and the emission angle can be dynamically adjusted; the near-infrared laser pulse and the ultrasonic envelope are triggered synchronously through an FPGA, and the time delay error ≤ 10 μs. The near-infrared laser pulse (pulse width 10 ns) and the ultrasonic envelope (pulse width 50 μs) are alternately triggered at a frequency of 10 Hz; the reflected optical signal (integration time 20 μs) and the scattered acoustic signal (sampling rate 10 MHz) in the eyelid area are synchronously received; the optical signal is spatially filtered to extract the light intensity distribution in the eyelid area; the time delay of the acoustic signal is estimated to calculate the eye socket deformation characteristics. The reflected optical signal is normalized, and the temporal change of the reflectivity during eyelid closure is calculated; the time derivative of the reflectivity decay curve is calculated through a sliding window (window length 50 ms, step size 10 ms) to extract the blinking dynamic characteristics. Based on the time delay information of the ultrasonic echo, a three-dimensional deformation model of the eye socket area is constructed; the time gradient of the deformation characteristics (such as deformation rate, acceleration) is extracted to form an acoustic feature vector. The time derivative of the optical signal (dimension: time step × feature channel) and the spatial gradient of the acoustic signal (dimension: spatial point × feature channel) are tensor concatenated to generate a composite blinking feature vector with a dimension of t × S × F, where t is the time step, S is the number of spatial points, and F is the number of feature channels.

[0094] The convolutional spiking neural network includes a convolutional sub-network, a spiking sub-network, a feature fusion layer, and a classification layer. Among them, the convolutional sub-network includes an input layer: receiving the composite blinking feature vector; a convolutional layer: using a 3×3 convolutional kernel, with 64 channels, and the activation function is ReLU; a pooling layer: max pooling, with a step size of 2; the output: a local time pattern feature map (dimension: y / 2 × S / 2 × 64). The spiking sub-network includes an input layer: receiving the output feature map of the convolutional sub-network; a spiking neuron layer: using the LIF (Leaky Integrate-and-Fire) model, with a time constant τ = 10 ms; a pooling layer: spiking density pooling in the time dimension, outputting the global spatio-temporal correlation feature (dimension: 1 × 1 × 64). The feature fusion layer performs weighted summation on the local time pattern of the convolutional sub-network and the global spatio-temporal correlation feature of the spiking sub-network, and the weights are normalized through the Softmax function; the classification layer includes a fully connected layer + a Softmax output layer, outputting the probability distributions of normal physiological blinking and command blinking.

[0095] Collect the blink data of 100 drivers (including light changes and head pose disturbances), label the blink types, select the cross-entropy loss function for training, use the Adam optimizer for optimization, set the learning rate to 0.001, set the batch size to 32, adopt transfer learning, and fine-tune the parameters of the convolutional sub-network based on a pre-trained convolutional neural network (such as ResNet-18).

[0096] In this embodiment, the driver can control the smart glasses through natural blinking actions without manual operation or voice commands, significantly reducing the risk of driving distraction.

[0097] In some embodiments, in the above step S104, the fusion of the vehicle data and the ambient light sensor data built into the smart glasses to construct a three-dimensional feature space of the driving scene and the use of an online incremental learning algorithm to dynamically generate a set of interaction strategies adapted to the scene specifically include: Through the timestamp alignment algorithm, match the vehicle data with the ambient light sensor data built into the smart glasses in the time domain to generate a synchronous data stream; Based on the synchronous data stream, calculate the dynamic complexity, environmental interference degree, and road topology eigenvalue respectively, and construct a three-dimensional driving scene feature space through orthogonal projection. The three-dimensional driving scene feature space includes multiple driving scene feature vectors; Based on the three-dimensional driving scene feature space, use an online incremental learning algorithm to dynamically generate a set of interaction strategies adapted to the scene.

[0098] In this embodiment, in the driving scene, the interaction strategy of the smart glasses needs to be dynamically adjusted according to the real-time environment. However, the prior art has the following defects: the vehicle data (such as vehicle speed, steering wheel angle) and the environmental perception data (such as light intensity) are not effectively fused, resulting in the lack of scene adaptability of the interaction strategy; most of the interaction strategies are preset rules and cannot be adjusted in real time according to the dynamic changes of the driving scene (such as sharp turns, tunnel driving); traditional machine learning methods need to be trained offline and cannot adapt to new scenes online, resulting in a lag in interaction response.

[0099] Specifically, obtain the vehicle speed, steering wheel angle, and GPS positioning information in real time through the OBD-II interface with a sampling frequency of 10 Hz; collect the ambient light intensity and color temperature through the photodiode array built into the smart glasses with a sampling frequency of 50 Hz; adopt a time synchronization mechanism based on the NTP protocol to ensure that the time error between the vehicle data and the ambient light sensor data is ≤ 10 ms; use an interpolation algorithm (such as cubic spline interpolation) to complement the low-frequency data (such as GPS positioning) to generate a synchronous data stream with a unified frequency (100 Hz).

[0100] Calculate the acceleration change rate of vehicle data (such as steering wheel angle acceleration) to quantify the dynamic complexity of driving operations; if the steering wheel angle acceleration exceeds a threshold (such as 5° / s²), it is determined as a high-complexity scenario.

[0101] Calculate the light intensity change rate (such as the amount of light intensity change per second) based on the ambient light sensor data to quantify the degree of environmental interference; if the light intensity change rate exceeds a threshold (such as 200 lux / s), it is determined as a high-interference scenario.

[0102] Match the GPS positioning information with the electronic map to extract features such as road type (such as highway, tunnel, urban road) and radius of curvature.

[0103] Encode the dynamic complexity, environmental interference degree, and road topology feature values into feature components respectively to generate a three-dimensional driving scenario feature vector. Use principal component analysis (PCA) to reduce the dimension of the feature vector, retain the first three principal components, and construct a three-dimensional driving scenario feature space. Use the online support vector machine (Online SVM) as an incremental learning algorithm to support dynamic update of model parameters. Train the initial model with preset scenario data (such as highway, tunnel, urban road) to generate an initial set of interaction strategies. For example, the initial strategies include "automatically dim the AR display brightness in the highway scenario" and "increase the voice prompt volume in the tunnel scenario". When a new scenario feature vector enters the feature space, the Online SVM automatically adjusts the classification boundary to generate an interaction strategy for the new scenario. For example, if a "high complexity + high interference degree" scenario is detected, a strategy of "switch to the voice interaction mode" is automatically generated. According to the scenario classification results in the feature space, a set of interaction strategies adapted to the scenario is dynamically generated. For example, the set of strategies includes "AR display mode switch", "voice prompt volume adjustment", "interaction frequency limit", etc.

[0104] In this embodiment, the interaction strategy can be dynamically adjusted according to the driving scenario to adapt to complex road conditions (such as sharp turns, tunnels, strong light environments). Automatically match the interaction strategy of the scenario (such as reducing the AR display brightness at night) to improve comfort. Through environmental perception data fusion, reduce driver distraction and reduce driving risks.

[0105] Furthermore, based on the three-dimensional driving scenario feature space, using the online incremental learning algorithm to dynamically generate a set of interaction strategies adapted to the scenario specifically includes: Adopt a kernel density estimation algorithm with a forgetting factor to perform online clustering analysis on continuously input driving scenario feature vectors and dynamically update the center coordinates of the strategy clusters; According to the Mahalanobis distance between the current driving scenario feature vector and the center coordinates of each strategy cluster, select the optimal interaction strategy to obtain a set of interaction strategies adapted to the scenario.

[0106] In this embodiment, in a driving scenario, the smart glasses need to dynamically adjust the interaction strategy according to the real-time environment. However, the existing technologies have the following defects: the preset interaction strategies cannot adapt to complex and changeable driving scenarios (such as sharp turns, tunnels, strong light environments); traditional clustering algorithms (such as K-Means) need to be calculated offline and cannot adapt to new scenarios online, resulting in high resource occupancy; there is a lack of accurate matching of scene features, leading to inaccurate selection of interaction strategies and affecting driving safety.

[0107] Specifically, kernel density estimation (KDE) is used to perform online clustering analysis on continuously input driving scenario feature vectors; a forgetting factor λ (such as λ = 0.95) is introduced to make the influence of old data decay over time, ensuring that the clustering results reflect the current scene features in real time. For example, set the initial set of coordinates of the policy cluster centers, where each cluster corresponds to an interaction strategy; receive the driving scenario feature vectors in real time, calculate the kernel density values with respect to the centers of each cluster, and update the density distribution of the clusters; apply the forgetting factor λ to the old data to reduce its influence on density estimation; dynamically adjust the coordinates of the cluster centers according to the updated density distribution.

[0108] When a new feature vector is input, the algorithm automatically adjusts the coordinates of the cluster centers to ensure that the cluster centers always reflect the typical features of the current driving scenario. For example, if a "high complexity + high interference" scenario is continuously detected, the cluster centers will shift towards this area to generate corresponding interaction strategies for high-risk scenarios.

[0109] Calculate the Mahalanobis distance between the current driving scenario feature vector and the coordinates of the centers of each policy cluster. The formula is: , where represents the Mahalanobis distance between the current driving scenario feature vector and the coordinates of the center of the i-th policy cluster, represents the current driving scenario feature vector and \({F}_{t}=\left [ {{{{C}_{d}}_{}}_{},{E}_{d},{T}_{r}} \right ] , respectively represent the dynamic complexity, environmental interference degree, and road topology eigenvalue, represents the covariance matrix of the current driving scenario feature vector, which is used to eliminate the dimensional differences between different feature dimensions, represents the coordinates of the center of the i-th policy cluster.

[0110] Select the policy cluster with the smallest Mahalanobis distance and obtain the corresponding interaction strategy. For example, if the Mahalanobis distance between the current driving scenario feature vector and the center of the "tunnel scenario" cluster is the smallest, then select the "enhance voice prompt volume" strategy.

[0111] Dynamically combine multiple strategies according to the complexity and risk level of the current driving scenario to generate a set of interaction strategies adapted to the scenario. For example, in the "night + tunnel + sharp turn" scenario, the strategy set may include "dimming the AR display brightness", "increasing the voice prompt volume", and "limiting the interaction frequency".

[0112] In another possible implementation, when the dynamic complexity exceeds the preset dynamic complexity threshold, disable the AR entertainment interface and enhance the navigation warning; when the environmental interference degree is lower than the preset environmental interference degree threshold, automatically increase the brightness and contrast of the display screen; when the road topology feature value enters the curve warning area, project a dynamic guiding line 200 meters in advance.

[0113] In this embodiment, the interaction strategy that automatically matches the scenario (such as reducing the AR display brightness at night) improves comfort. Through precise scenario matching, driver distraction is reduced and driving risk is lowered.

[0114] In some embodiments, in the above step S105, generating a spatialized warning audio through the bone conduction unit of the smart glasses and simultaneously projecting dynamic focal plane AR prompt information on the waveguide display of the smart glasses specifically includes: Based on the obstacle azimuth, relative speed, and collision time data, calculate a three-dimensional space threat level distribution map; According to the threat level distribution map, match the corresponding sound field filtering parameters from the pre-stored head-related transfer function library, and generate an azimuth-directional warning audio baseband signal through a binaural acoustic algorithm; Decompose the warning audio baseband signal into the driving components of each oscillator of the bone conduction array of the smart glasses. The driving components include high-frequency components and low-frequency components. The high-frequency components are allocated to the oscillators in the temporal bone region, and the low-frequency components are allocated to the oscillators in the occipital bone region; According to the real-time vehicle speed and steering wheel angle, dynamically adjust the transparency and projection position of the AR prompt information on the waveguide display of the smart glasses. The transparency is negatively correlated with the vehicle speed, and the projection position undergoes an affine transformation with the steering wheel angle to keep the visual focus stable.

[0115] In this embodiment, in the application of the existing smart glasses in the driving scenario, the traditional warning methods (such as beepers and AR prompts at fixed positions) have the following defects: 1. The warning information lacks a sense of space: it cannot intuitively indicate the obstacle azimuth, and the driver needs to make additional judgments; 2. The AR prompt information is fixed: the influence of vehicle speed and steering is not considered, which may cause the driver's visual focus to shift; 3. The interaction strategy is single: it does not dynamically adjust in combination with the real-time driving scenario, resulting in information overload or insufficiency.

[0116] Specifically, obtain data on the obstacle's azimuth (latitude and longitude coordinates), relative speed (km / h), and time to collision (TTC, in seconds); convert the obstacle's azimuth from the vehicle coordinate system to the head coordinate system of the smart glasses (with the center of the driver's head as the origin); calculate the threat level. For example, high threat: TTC ≤ 2 seconds, or relative speed ≥ 50 km / h; medium threat: 2 seconds < TTC ≤ 5 seconds, or relative speed 30 km / h ≤ v < 50 km / h; low threat: TTC > 5 seconds, or relative speed < 30 km / h.

[0117] Divide the 180° field of view in front of the driver into 10°×10° grids. According to the obstacle coordinates, mark the threat levels of the corresponding grids as high, medium, and low to generate a three-dimensional space threat level distribution map. Pre-store the HRTF parameters for different azimuths (from 0° to 180°, with a step of 10°). Each parameter contains frequency response and phase delay information. According to the obstacle's azimuth, extract the corresponding parameters from the HRTF library to generate a sound field filter.

[0118] Design the baseband signal. For example, for high threat: generate an alarm sound from 800 Hz to 1200 Hz, lasting 0.5 seconds, with an interval of 0.2 seconds; for medium threat: generate a prompt tone from 400 Hz to 800 Hz, lasting 1 second, with an interval of 0.5 seconds; for low threat: generate a soft prompt tone from 200 Hz to 400 Hz, lasting 2 seconds, with an interval of 1 second. Process the baseband signal through the sound field filter to generate a warning audio signal with azimuth directivity.

[0119] The bone conduction array of the smart glasses contains 4 oscillators, located in the temporal bone region (2) and the occipital bone region (2) respectively. High-frequency components (above 1000 Hz) are assigned to the oscillators in the temporal bone region, taking advantage of the high-frequency conduction characteristics of the temporal bone, and low-frequency components (below 1000 Hz) are assigned to the oscillators in the occipital bone region, taking advantage of the low-frequency conduction characteristics of the occipital bone. According to the obstacle's azimuth, adjust the phase and amplitude of each oscillator to achieve sound source localization.

[0120] Design AR prompt information, the information types include obstacle contours, relative speed, collision time, navigation arrows, etc. In the initial parameters, the transparency is default set to 50%, and the projection position is fixed 2 meters in front of the driver. In high-speed scenarios (vehicle speed ≥ 80 km / h), the transparency is reduced to 30% to reduce visual interference; in low-speed scenarios (vehicle speed < 40 km / h), the transparency is increased to 70% to enhance information readability. In the case of small-angle steering (steering angle ≤ 15°), the AR prompt information remains in place; in the case of large-angle steering (steering angle > 15°), according to the size of the steering angle, the projection position is adjusted through affine transformation to keep the field of view focus stable. According to the steering wheel angle θ, an affine transformation matrix T(θ) is generated; the original coordinates (x, y) of the AR prompt information are converted to new coordinates (x', y') through T(θ); the projection position is updated every 50 milliseconds to ensure that the prompt information is synchronized with the driver's line of sight.

[0121] In this embodiment, the driver can intuitively perceive the obstacle orientation through bone conduction audio without additional judgment. The AR prompt information dynamically adjusts the transparency and position according to the vehicle speed and steering, reduces visual interference, and keeps the field of view focus stable. Combining with the real-time driving scenario, it provides differentiated warning and prompt information to improve driving safety and user experience.

[0122] Through the synergistic effect of the three-dimensional space threat level distribution map, spatialized warning audio and dynamic focal plane AR prompt information, this embodiment effectively solves the interaction limitations of existing smart glasses in driving scenarios and provides technical support for the development of intelligent driving assistance systems.

[0123] Refer to Figure 2 , an embodiment of the present invention provides a dynamic interaction system 2 of a smart glasses, and the system 2 specifically includes: The first dynamic interaction module 201 is used to collect the temperature distribution data of the smart glasses body and the acceleration data of the user's head movement in real time, and establish a dynamic temperature threshold model according to the temperature distribution data and the acceleration data; The second dynamic interaction module 202 is used to dynamically adjust the temperature alarm threshold based on the dynamic temperature threshold model according to the head movement frequency, and trigger a hierarchical frequency reduction strategy when it detects that the local temperature exceeds the preset temperature threshold; The third dynamic interaction module 203 is used to analyze the reflection attenuation characteristics of the eyelid to near-infrared light and the ultrasonic echo time delay change according to the multispectral imaging unit of the smart glasses, generate a composite blink feature vector, and construct a blink pattern classifier in combination with a convolutional pulse neural network. The blink pattern classifier is used to distinguish between regular physiological blinks and command blinks; The fourth dynamic interaction module 204 is used to obtain the vehicle driving speed, steering wheel angle, and GPS positioning information in real time, form vehicle data, fuse the vehicle data with the ambient light sensor data built in the smart glasses, construct a three-dimensional feature space of the driving scene, and dynamically generate a set of interaction strategies adapted to the scene by using an online incremental learning algorithm; The fifth dynamic interaction module 205 is used to generate spatial warning audio through the bone conduction unit of the smart glasses based on the set of interaction strategies, and at the same time project dynamic focal plane AR prompt information on the waveguide display of the smart glasses.

[0124] It can be understood that, as Figure 1 shown, the content in the embodiment of the dynamic interaction method of the smart glasses is applicable to the embodiment of the dynamic interaction system of this smart glasses. The functions specifically implemented by the embodiment of the dynamic interaction system of this smart glasses are the same as those in the embodiment of the dynamic interaction method of the smart glasses as Figure 1 shown, and the beneficial effects achieved are also the same as those achieved in the embodiment of the dynamic interaction method of the smart glasses as Figure 1 shown.

[0125] It should be noted that, regarding the information interaction, execution process, etc. between the above systems, since they are based on the same concept as the method embodiment of the present invention, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details will not be elaborated here.

[0126] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, and details will not be elaborated here.

[0127] Referring to Figure 3 , an embodiment of the present invention also provides a computer device 3, including: a memory 302, a processor 301, and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, the dynamic interaction method of the smart glasses as described in any one of the above methods is implemented.

[0128] The computer device 3 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art can understand that Figure 3 These are merely examples of the computer device 3 and do not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or combine certain components, or have different components. For example, it may also include input / output devices, network access devices, etc.

[0129] The so-called processor 301 may be a central processing unit (CPU), and the processor 301 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0130] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3. Further, the memory 302 may also include both the internal storage unit and the external storage device of the computer device 3. The memory 302 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 302 may also be used to temporarily store data that has been output or will be output.

[0131] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it implements the dynamic interaction method of the smart glasses as described in any one of the above methods.

[0132] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the method of the above embodiment in the present application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0133] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this document can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0135] In the embodiments disclosed in the present application, it should be understood that the disclosed device / terminal device and method can be implemented in other ways. For example, the device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0136] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. A dynamic interaction method for smart glasses, characterized in that: The method specifically comprises: Collect the temperature distribution data of the smart glasses and the acceleration data of the user's head movement in real time, and establish a dynamic temperature threshold model based on the temperature distribution data and acceleration data; Based on the dynamic temperature threshold model, the temperature alarm threshold is dynamically adjusted according to the head movement frequency. When the local temperature is detected to exceed the preset temperature threshold, the hierarchical frequency reduction strategy is triggered; The multispectral imaging unit of the smart glasses analyzes the reflection attenuation characteristics of the eyelid to near-infrared light and the ultrasonic echo delay change to generate a composite blink feature vector, and combines the convolutional spike neural network to construct a blink pattern classifier, which is used to distinguish between regular physiological blinking and commanded blinking actions; Real-time acquisition of vehicle speed, steering wheel angle and GPS positioning information to form vehicle data, which is then integrated with the ambient light sensor data built into the smart glasses to construct a three-dimensional feature space for the driving scene, and an online incremental learning algorithm is used to dynamically generate a set of scene-adaptive interaction strategies. Based on the set of interaction strategies, spatialized warning audio is generated through the bone conduction unit of the smart glasses, and dynamic focal plane AR prompt information is projected on the waveguide display of the smart glasses.

2. The method according to claim 1, characterized in that The real-time collection of temperature distribution data of the smart glasses body and acceleration data of the user's head movement, and the establishment of a dynamic temperature threshold model according to the temperature distribution data and the acceleration data, specifically includes: The temperature distribution data is obtained in real time through the thermopile sensor array distributed on the key heat conduction path of the smart glasses body; The three-axis MEMS accelerometer of smart glasses is used to continuously collect acceleration data of the user's head movement; Based on the acceleration data, the root mean square value of the motion intensity is calculated through a sliding time window, and the root mean square value of the motion intensity is input into a preset temperature threshold function to dynamically generate a first temperature alarm threshold that is adaptive to the head motion state; The temperature distribution data is weightedly counted according to the first dynamic temperature threshold to obtain the distribution of over-temperature nodes of the thermopile sensor array and form a dynamic temperature threshold model.

3. The method according to claim 1, characterized in that The temperature alarm threshold is dynamically adjusted based on the dynamic temperature threshold model according to the head movement frequency. When the local temperature is detected to exceed the preset temperature threshold, the hierarchical frequency reduction strategy is triggered, which specifically includes: Collect the acceleration, angular velocity and orientation data of the head movement in real time, and calculate the time-frequency characteristics of the acceleration vector based on the acceleration, angular velocity and orientation data; Extracting the main frequency band energy ratio of head movement based on video features, and inputting the main frequency band energy ratio into an exponential decay function to dynamically calculate a second temperature alarm threshold; Inputting the second temperature alarm threshold into the dynamic temperature threshold model, detecting the difference between the local temperature of each area of ​​the smart glasses body and the second temperature alarm threshold, and obtaining over-temperature level detection data; Based on the over-temperature level detection data, the current frequency reduction level is calculated according to the preset grading step size, and a graded frequency reduction control instruction is generated.

4. The method according to claim 1, characterized in that: The method analyzes the reflection attenuation characteristics of near-infrared light and the ultrasonic echo delay change of the eyelids according to the multispectral imaging unit of the smart glasses, generates a composite blink feature vector, and constructs a blink pattern classifier in combination with a convolutional pulse neural network, specifically including: A multispectral imaging unit based on smart glasses alternately triggers the emission of near-infrared laser pulses and ultrasonic envelopes, and synchronously receives reflected optical signals and scattered acoustic signals from the eyelid area; The reflectivity attenuation curve during eyelid closure is calculated based on the reflected optical signal, and the time delay characteristics of eye socket deformation are extracted based on the scattered acoustic signal. The time-domain derivative of the reflectivity attenuation curve and the spatial gradient of the orbital deformation delay feature are tensor-joined to generate a composite blink feature vector with temporal and spatial correlation. The composite blink feature vector is input into a pre-trained convolutional spiking neural network, the local temporal pattern is extracted by the convolutional sub-network, the global spatiotemporal correlation is captured by the spiking sub-network, and the local temporal pattern and the global spatiotemporal correlation are fused with the attention weight to form a blink pattern classifier.

5. The method according to claim 1, characterized in that The vehicle data and the ambient light sensor data built into the smart glasses are integrated to construct a three-dimensional feature space of the driving scene, and an online incremental learning algorithm is used to dynamically generate a set of scene-adaptive interaction strategies, specifically including: Through the timestamp alignment algorithm, the vehicle data is matched with the ambient light sensor data built into the smart glasses in the time domain to generate a synchronized data stream; Based on the synchronous data stream, dynamic complexity, environmental interference and road topology characteristic values ​​are calculated respectively, and a three-dimensional driving scene feature space is constructed through orthogonal projection, wherein the three-dimensional driving scene feature space includes a plurality of driving scene feature vectors; Based on the three-dimensional driving scene feature space, an online incremental learning algorithm is used to dynamically generate a set of scene-adaptive interaction strategies.

6. The method according to claim 5, characterized in that The interactive strategy set for scene adaptation is dynamically generated by using an online incremental learning algorithm based on the three-dimensional driving scene feature space, specifically including: A kernel density estimation algorithm with a forgetting factor is used to perform online clustering analysis on continuously input driving scene feature vectors and dynamically update the center coordinates of the strategy cluster. According to the Mahalanobis distance between the feature vector of the current driving scene and the center coordinates of each strategy cluster, the optimal interaction strategy is selected to obtain a scene-adaptive interaction strategy set.

7. The method according to any one of claims 1 to 6, characterized in that The method of generating spatialized warning audio through the bone conduction unit of the smart glasses and projecting dynamic focal plane AR prompt information on the waveguide display of the smart glasses specifically includes: Calculate the three-dimensional threat level distribution map based on obstacle position, relative speed and collision time data; According to the threat level distribution map, the corresponding sound field filter parameters are matched from the pre-stored head-related transfer function library, and a warning audio baseband signal with azimuth directivity is generated through a binaural acoustic algorithm; Decomposing the warning audio baseband signal into driving components of each vibrator of the bone conduction array of the smart glasses, wherein the driving components include high-frequency components and low-frequency components, wherein the high-frequency components are allocated to the vibrators in the temporal bone region, and the low-frequency components are allocated to the vibrators in the occipital bone region; According to the real-time vehicle speed and steering wheel angle, the transparency and projection position of the AR prompt information on the waveguide display of the smart glasses are dynamically adjusted. The transparency is negatively correlated with the vehicle speed, and the projection position is affine transformed with the steering wheel angle to keep the field of view focus stable.

8. A dynamic interaction system for smart glasses, characterized in that: The system specifically comprises: The first dynamic interaction module is used to collect temperature distribution data of the smart glasses body and acceleration data of the user's head movement in real time, and establish a dynamic temperature threshold model according to the temperature distribution data and acceleration data; The second dynamic interaction module is used to dynamically adjust the temperature alarm threshold according to the head movement frequency based on the dynamic temperature threshold model, and trigger the graded frequency reduction strategy when it is detected that the local temperature exceeds the preset temperature threshold; The third dynamic interaction module is used to analyze the reflection attenuation characteristics of the eyelid to the near-infrared light and the ultrasonic echo delay change according to the multi-spectral imaging unit of the smart glasses, generate a composite blink feature vector, and build a blink pattern classifier in combination with a convolutional pulse neural network, wherein the blink pattern classifier is used to distinguish between conventional physiological blinking and commanded blinking actions; The fourth dynamic interaction module is used to obtain the vehicle speed, steering wheel angle and GPS positioning information in real time to form vehicle data, fuse the vehicle data with the ambient light sensor data built into the smart glasses, construct a three-dimensional feature space of the driving scene, and use the online incremental learning algorithm to dynamically generate a scene-adaptive interaction strategy set; The fifth dynamic interaction module is used to generate spatialized warning audio through the bone conduction unit of the smart glasses based on the interaction strategy set, and at the same time project dynamic focal plane AR prompt information on the waveguide display of the smart glasses.

9. A computer device, characterized in that: include: A memory and a processor and a computer program stored in the memory, when the computer program is executed on the processor, implements the dynamic interaction method of the smart glasses as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the dynamic interaction method of the smart glasses as described in any one of claims 1 to 7 is implemented.

Images