A method and AI glasses for automatically changing the frame color according to mood and weather

Abstract

The present application provides a method and an AI glasses for automatically changing the frame color according to the mood and weather, which relates to the technical field of light source control. The method includes: collecting in real time the language information of the people around the wearer and the current weather information; after preprocessing, converting to obtain the audio feature and the weather feature in vector form; inputting the audio feature and the weather feature into a pre-trained neural network model, and outputting the color information corresponding to the current emotional state of the wearer in the HSV color space; converting the color information into the color value corresponding to the RGB color space, and then generating an overall bead control signal according to the actual layout of the multi-color LED strip beads on the glasses frame, controlling the multi-color LED strip beads, so that the color corresponding to the emotional state is displayed on the glasses frame. Implementing this method can combine the climate conditions of the surrounding environment, enable the frame color to dynamically reflect the emotional changes of the wearer in real time, and provide a more personalized and contextual visual experience for users.

Description

Technical Field

[0001] The present application relates to the technical field of light source control, and particularly relates to a method for automatically changing the frame color according to mood and weather and an AI glasses. Background Art

[0002] With the rapid development of intelligent wearable devices, as an emerging human-computer interaction device, intelligent AI glasses have gradually received extensive attention from the market and the emphasis of researchers. Intelligent glasses can not only provide traditional visual assistance functions, but also interact with users at a deeper level by sensing the user's emotional state, environmental information, etc. For example, expressing or giving feedback to the user's emotions by automatically changing the display color, light effect, etc.

[0003] In related technologies, the electroencephalogram (EEG) signals of the wearer are mainly collected by an EEG acquisition module and transmitted to an emotion assessment module for feature extraction and emotion classification. The emotion classification result is used to control the color change of an electrochromic component (such as a glasses frame). At the same time, music corresponding to the emotion can also be generated according to the EEG signals to further adjust the emotional state of the wearer.

[0004] However, it is difficult to accurately meet the personalized needs of the wearer by adjusting the color of the glasses frame according to the emotional state reflected by the EEG signals. For example, the anxiety emotion generated by heavy rain weather may show a high degree of similarity in EEG features to the anxiety emotion caused by social conflicts, resulting in the same color feedback for the same emotion category in different environmental scenarios. Summary of the Invention

[0005] The present application provides a method for automatically changing the frame color according to mood and weather and an AI glasses to address the problem that related technologies are difficult to distinguish similar emotions in different environmental scenarios, resulting in a single frame color feedback.

[0006] In a first aspect, the present application provides a method for automatically changing the frame color according to mood and weather, which is applied to AI glasses. The method includes:

[0007] Collecting the surrounding people's communication language information of the wearer and the current weather information in real time;

[0008] Preprocessing the people's communication language information and the current weather information respectively, and converting them into audio features and weather features in vector form;

[0009] Inputting the audio features and weather features into a pre-trained neural network model, and outputting the current emotional state of the wearer. The emotional state includes an emotion label and an emotion intensity feature, and the emotion intensity feature includes three feature values: pleasantness, arousal, and dominance;

[0010] Convert the emotional state into color information corresponding to the HSV color space through a preset mapping function. The color information includes the values of three channels: hue, saturation, and brightness. Different emotional states correspond to different color information.

[0011] Convert the color information into color values corresponding to the RGB color space, and then generate an overall lamp bead control signal according to the actual layout of the multi-color LED strip lamp beads on the glasses frame.

[0012] Control the multi-color LED strip lamp beads according to the lamp bead control signal, so that the multi-color LED strip lamp beads display colors corresponding to the emotional state on the glasses frame.

[0013] Through the above embodiments, the AI glasses can collect the language information of the people around the wearer in real time, combine the current weather information, and determine the current emotional state of the wearer through the pre-trained neural network model. And according to the mapping relationship between the emotional state and the HSV color space, output color information including hue, saturation, and brightness, and then display the color information on the glasses frame through the multi-color LED lamp beads. This method can combine the climate conditions of the surrounding environment, enable the glasses frame to dynamically reflect the emotional changes of the wearer in real time, and provide a more personalized and contextual visual experience for users.

[0014] In some embodiments, before the step of inputting the audio feature and the weather feature into the pre-trained neural network model and outputting the current emotional state of the wearer, it further includes:

[0015] Collect language communication audio data and corresponding environmental weather data in multiple scenarios.

[0016] Perform emotional state annotation on the language communication audio data and the environmental weather data to obtain a training data set. The language communication audio data and the corresponding environmental weather data are the input data of the model, and the emotional state annotation is the output verification data of the model.

[0017] Use the training data set to train the neural network model to obtain a pre-trained neural network model. The neural network model is constructed based on a hybrid of a convolutional neural network and a recurrent neural network.

[0018] Through the above embodiments, the AI glasses collect and annotate the language communication audio data and the corresponding environmental weather data in multiple scenarios for training the neural network model, construct the mapping relationship between the audio data and the emotional state of the wearer under different weather data, so that the AI glasses can more accurately identify the emotional state of the wearer and adjust the color of the glasses frame according to these emotional states.

[0019] In some embodiments, the step of inputting the audio features and weather features into a pre-trained neural network model and outputting the current emotional state of the wearer specifically includes:

[0020] Convert the audio features within a preset time period into Mel spectrogram form to obtain a Mel spectrogram matrix;

[0021] Use a convolutional neural network to extract features from the Mel spectrogram matrix, and input the extracted features into a recurrent neural network to obtain the time series features of the language communication audio data;

[0022] Map the weather features corresponding to the time series features to the same dimension as the time series features through a fully connected layer and perform feature fusion to obtain a comprehensive feature vector;

[0023] Determine the current emotional state of the wearer based on the comprehensive feature vector.

[0024] Through the above embodiments, the AI glasses convert the audio data into Mel spectrogram form and extract features, combine the weather features, and obtain a comprehensive feature vector through feature fusion. This method makes the emotional judgment more refined and comprehensive, improves the accuracy and response speed of emotional recognition, and further makes the change of the glasses color more in line with the actual emotional state.

[0025] In some embodiments, the step of converting the emotional state into color information corresponding to the HSV color space through a preset mapping function specifically includes:

[0026] Determine the hue of the color information based on the classification index of the emotional label and the total number of emotional categories;

[0027] Obtain the saturation of the color information by mapping the arousal level;

[0028] Calculate the brightness of the color information based on the arousal level and dominance.

[0029] Through the above embodiments, the AI glasses establish the correspondence between different dimensions (emotional label, arousal level, and dominance) of the emotional state and the hue, saturation, and brightness in the HSV color space. This detailed color adjustment mechanism allows the glasses to display richer and more accurate color changes, and can more finely reflect the emotional changes of the wearer.

[0030] In some embodiments, before the step of collecting the language information of the people around the wearer and the current weather information in real time, it further includes:

[0031] Determine the average weather change period based on the historical weather information of the location where the wearer is located;

[0032] Obtain the current weather information from the wearer's mobile terminal according to the average weather change cycle, and the mobile terminal is wirelessly connected to the AI glasses.

[0033] Through the above embodiments, the AI glasses obtain the current weather information from the wearer's mobile terminal, and determine the data request cycle for reading the weather information from the mobile terminal by using the variation law of the historical weather information. This method can timely monitor the change of the current weather information on the premise of reducing data transmission, which is convenient for subsequent color adjustment of the frame.

[0034] In some embodiments, after the step of collecting the surrounding people's communication language information and the current weather information of the wearer in real time, it further includes:

[0035] Capture the bone vibration signal during the wearer's communication through the bone conduction sensor;

[0036] Convert the bone vibration signal into speech content through a preset algorithm;

[0037] Calibrate the people's communication language information according to the speech content.

[0038] Through the above embodiments, the AI glasses capture the bone vibration signal during communication through the bone conduction sensor, convert these signals into speech content and calibrate the audio data received by the microphone. This technology can improve the accuracy of language information collection. Especially in a noisy environment, the bone conduction technology can effectively reduce the interference of background noise, making the emotion analysis more accurate.

[0039] In some embodiments, before the step of generating the overall bead control signal according to the actual layout of the multi-color LED strip beads on the glasses frame, it further includes:

[0040] Receive the personalized area set by the wearer on the mobile terminal;

[0041] Determine the multi-color LED strip beads that can be controlled in the actual layout according to the personalized area.

[0042] Through the above embodiments, the AI glasses receive the personalized area set by the wearer before generating the bead control signal, and adjust the actual layout control of the LED beads according to these settings. This allows the user to customize the color area displayed by the glasses according to personal preferences, increasing the personalization and practicality of the frame color display. The user can select a specific color display area, making the color change of the glasses not only reflect the emotional state, but also meet personal aesthetic and situational needs.

[0043] In a second aspect, the present application provides an AI glasses, and the AI glasses include: one or more processors and a memory;

[0044] The memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions. The one or more processors call the computer instructions so that the AI glasses can implement a method for automatically changing the frame color according to mood and weather provided in the above embodiments, which will not be elaborated here.

[0045] In a third aspect, the present application provides a computer-readable storage medium, including instructions, which when running on the AI glasses, enable the AI glasses to implement a method for automatically changing the frame color according to mood and weather provided in the above embodiments, which will not be elaborated here.

[0046] In a fourth aspect, the present application provides a computer program product, which when running on the AI glasses, enables the AI glasses to implement a method for automatically changing the frame color according to mood and weather provided in the above embodiments, which will not be elaborated here.

[0047] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0048] 1. By collecting and analyzing the surrounding people's communication language information and the current weather information of the wearer in real time, and then identifying the emotional state of the wearer through a pre-trained neural network model, and automatically adjusting the color of the glasses frame according to the mapping relationship between these emotions and specific colors, the AI glasses not only reflect the emotional changes of the user, but also consider environmental factors, providing a more dynamic and personalized visual experience, and enhancing the naturalness and intuitiveness of human-computer interaction.

[0049] 2. Combining convolutional neural networks and recurrent neural networks for deep feature extraction and fusion of audio and weather data. This not only improves the accuracy of emotion recognition, but also enables the AI glasses to more accurately reflect the emotions of the wearer in complex environments, providing more refined emotional responses and color adjustments for users.

[0050] 3. By using the change law of historical weather information to determine the acquisition period of weather data and combining with the personalized area set by the user to adjust the layout of the light beads, not only the personalization and practicality of the AI glasses are enhanced, but also through precise adjustment of the control and output of the light beads, the device can more flexibly adapt to the needs of users and environmental changes, enhancing the functionality and attractiveness of the smart glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flowchart of a method for automatically changing the frame color according to mood and weather in an embodiment of the present application;

[0052] Figure 2 It is another flowchart of a method for automatically changing the frame color according to mood and weather in an embodiment of the present application;

[0053] Figure 3 It is a schematic structural diagram of an entity device of an AI glasses in an embodiment of the present application. Detailed implementation manners

[0054] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed items.

[0055] Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0056] For the convenience of understanding, the method provided in this embodiment will be described in terms of a process below. Please refer to Figure 1 , which is a flowchart of a method for automatically changing the frame color according to mood and weather in an embodiment of the present application.

[0057] S101. Real-time collect the language information of the people around the wearer and the current weather information.

[0058] The AI glasses collect the surrounding voice information (language information of people's communication) in real time by integrating a micro-microphone array at positions such as the temple. Among them, the microphone array is composed of multiple directional microphones. The beamforming technology can effectively isolate background noise and accurately capture the voice of the converser. During the voice collection process, the microphone array dynamically adjusts the sound pickup direction through an adaptive beamforming algorithm, and focuses on capturing the voice signals (analog sound signals) within 120 degrees in front of the wearer. Then, the collected analog sound signals are converted into digital audio data through a built-in analog-to-digital converter (ADC), and are digitized according to a certain sampling frequency (such as the commonly used 44.1 kHz or 48 kHz) and quantization accuracy (such as 16 bits or 24 bits) to ensure the integrity and accuracy of the audio information for subsequent analysis.

[0059] The current weather information can establish a wireless connection with the weather application on the wearer's mobile terminal (such as a mobile phone) through the low-power Bluetooth (BLE) module or Wi-Fi module built into the AI glasses, and obtain real-time weather information from the mobile phone weather APP in real time. Among them, the obtained weather information includes, but is not limited to, detailed weather parameters such as weather type (sunny, cloudy, rainy, snowy, etc.), temperature value, humidity value, air quality index, etc.

[0060] Optionally, the above weather information can also be obtained by installing sensors in the AI glasses. In a specific embodiment, a miniaturized meteorological sensor can be integrated into the frame or other parts of the AI glasses, such as a temperature sensor (using high-precision thermistor and other sensing elements, which can accurately measure the ambient temperature, and the error range is controlled within ±0.5°C), a humidity sensor (based on capacitive or resistive humidity sensing principle, ensuring that the humidity measurement accuracy reaches within ±3% of the relative humidity), a barometric pressure sensor (used to assist in judging the weather change trend, etc., and the measurement accuracy can reach ±1 hPa), etc., to directly obtain the meteorological data of the local environment as a supplement or backup acquisition method for the weather information to cope with situations such as wireless connection failure, which is not limited here.

[0061] S102: Preprocess the personnel communication language information and the current weather information respectively, and convert them into audio features and weather features in vector form.

[0062] The AI glasses preprocess the collected personnel communication language information, including but not limited to noise reduction, framing, and windowing processing, etc. Optionally, spectral subtraction is used to suppress background noise, and then the continuous speech signal is segmented into 25-ms short-time analysis frames with an overlap of 15 ms between adjacent frames. A Hamming window is applied to each frame signal to reduce spectral leakage. Then, multi-dimensional acoustic features are extracted, including fundamental frequency, formants, Mel-frequency cepstral coefficients (MFCC), etc. For each frame of speech, 13-dimensional MFCC features are extracted, and first-order and second-order difference coefficients are calculated to obtain a 39-dimensional feature vector. At the same time, prosodic features such as pitch, energy, and zero-crossing rate are extracted. These features are organized into a feature matrix, with each column corresponding to a time frame and the number of rows equal to the feature dimension.

[0063] The preprocessing of the current weather information includes numerical normalization and feature encoding. Continuous values such as temperature and humidity are mapped to the [0, 1] interval through min-max normalization. The weather type (sunny, cloudy, rainy, etc.) is converted into vector form using one-hot encoding. Finally, a feature vector containing all weather parameters is obtained.

[0064] For example, when processing the sentence "The weather is really nice today", the AI glasses convert it into a feature sequence. Each frame in the speech feature matrix contains 39-dimensional acoustic features, which reflect information such as the speaker's timbre and intonation. The weather data "sunny, 25°C" is converted into a feature vector of a unified scale for subsequent neural network processing.

[0065] S103. Input the audio features and weather features into a pre-trained neural network model to determine the current emotional state of the wearer, and output the corresponding color information in the HSV color space according to the mapping relationship between the emotional state and HSV.

[0066] The AI glasses use a neural network model constructed by mixing a convolutional neural network and a recurrent neural network for multi-modal emotion analysis. This model adopts a two-stream architecture to process speech features and weather features respectively. Specifically, the audio features are used for local feature extraction through multiple layers of CNNs, and then fed into a BiLSTM network to capture temporal dependencies. After the weather features are dimension-reduced through a fully connected layer, they are fused with the speech features at the feature level. The fused features pass through an attention mechanism and a fully connected layer, and finally the emotion state prediction result is output. Among them, the emotion state includes discrete emotion labels (such as happy, sad, angry, etc.) and continuous emotion intensity values (pleasantness, arousal, and dominance). The above neural network model is pre-trained on a large-scale emotional speech dataset, ensuring accurate emotion recognition ability.

[0067] Next, the mapping from the emotion state to the HSV color space can be determined by the color-emotion correspondence relationship set by relevant technicians. For example, setting the mapping relationship of the hue (H) from the emotion label, optionally, happy corresponds to yellow (60°) in the hue, and sad corresponds to blue (240°) in the hue; setting the saturation (S) to be positively correlated with the arousal (represented by the symbol "A") to reflect the emotion intensity; the brightness (V) is jointly determined by the arousal and dominance (represented by the symbol "D"), etc.

[0068] For example, in a specific embodiment, when detecting that the user says a sentence like "The weather is really nice today", combined with the sunny weather information, the model may output the emotion label "happy", and at the same time give relatively high pleasantness and arousal values. These emotion parameters are mapped to HSV values, such as H = 60° (yellow), S = 0.8 (relatively saturated), V = 0.9 (bright), reflecting the current positive emotional state of the wearer.

[0069] S104. Convert the color information into the corresponding color values in the RGB color space, and generate an overall bead control signal according to the actual layout of the multi-color LED strip beads on the glasses frame.

[0070] The AI glasses convert the values in the HSV color space into RGB values that can be directly used by the LED beads. The specific conversion process can be based on the standard HSV-to-RGB conversion algorithm, and the optical characteristics of the LED beads are also considered. The range of the converted RGB values is 0 - 255, corresponding to the brightness levels of the red, green, and blue channels respectively.

[0071] Optionally, the frame of the AI glasses can adopt a flexible PCB design, integrating multiple RGB LED beads to form a continuous light-emitting band. Each LED bead can independently control the brightness values of its RGB three channels. To achieve a smooth gradient effect, linear interpolation algorithm can be used for the color transition between adjacent beads. And adjacent beads can be electrically connected through a flexible circuit board (FPC) or extremely thin wires to ensure the stability of signal transmission, and considering the usage characteristics such as folding and bending of the glasses, ensure that the circuit will not be easily damaged.

[0072] Optionally, the generation of the control signal can also be determined according to the physical layout of the LEDs. For example, when the AI glasses frame is divided into two independent control areas on the left and right, a corresponding PWM control signal sequence is generated for each area. The control signal of each LED bead includes the duty cycle ratios of the RGB three channels, and the target color display is achieved through precise timing control.

[0073] It should be noted that in the design of the AI glasses frame, the bead layout can be carried out according to the aesthetic and visibility principles. Specifically, it can be evenly arranged along the entire frame to form a coherent light band effect for color display; or it can be appropriately densified in key areas such as both sides of the frame to highlight the display effect, while avoiding problems such as excessive power consumption and heat generation caused by overly dense beads. There is no limitation here.

[0074] For example, in a specific embodiment, when the HSV value is (60°, 0.8, 0.9), the converted RGB value is approximately (230, 230, 0). Subsequently, a corresponding PWM control sequence is generated to ensure that each LED bead can accurately display this bright yellow color. The gradient effect can be achieved by interpolating between adjacent beads, making the entire glasses frame present a uniform and smooth color transition.

[0075] S105. Control the multi-color LED strip beads according to the bead control signal to make the color corresponding to the emotional state displayed on the glasses frame.

[0076] Specifically, according to the PWM control signal generated in step S104, the AI glasses control each LED to adjust its own emission color and brightness in real time, presenting an overall color effect corresponding to the emotional state of the wearer. For example, when the artificial intelligence emotion inference module determines that the wearer is in a happy emotional state, it may display a bright warm color phase (such as orange with an H value close to about 30°, and high S and V values indicating high saturation and lightness); while when it is judged to be in a sad mood, it displays a dull cold color phase (such as dark blue with an H value close to about 240°, and relatively low S and V values). Through this intuitive color change, a clear and intuitive visual presentation of emotional feedback is provided for the wearer.

[0077] Optionally, a smooth transition algorithm can be adopted during the color display process to avoid visual discomfort caused by sudden color changes. When it is detected that the emotional state changes, the LED color can be adjusted within a time period of 100 - 300 ms. The intermediate color values during the transition period can be calculated through an interpolation algorithm to ensure the continuity of color changes. In addition, the ambient light intensity can be monitored in real time, and the LED brightness can be automatically adjusted to maintain the best display effect under different lighting conditions.

[0078] It can be understood that the AI glasses can use a high-performance LED driver chip to execute color display control. Optionally, the driver chip can support 16-bit PWM precision to achieve delicate color gradations. It can receive control signals through the I2C interface, and the built-in timer module precisely controls the switching timing of each LED channel. In addition, to ensure the stability of the display effect, a constant current output control and temperature compensation function can also be added to the driver chip, which is not limited here.

[0079] In the above embodiment, the AI glasses can collect the language information of the people around the wearer in real time, and combine the current weather information, and process it through a pre-trained neural network model to determine the current emotional state of the wearer. And according to the mapping relationship between the emotional state and the HSV color space, color information including hue, saturation, and brightness is output, and then the color information is displayed on the glasses frame through multi-color LED beads. This method can dynamically reflect the emotional changes of the wearer in real time, and combine the climate conditions of the surrounding environment to provide a more personalized and contextual visual experience.

[0080] The following further describes the more specific process of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of a method for automatically changing the frame color according to mood and weather in the embodiment of the present application.

[0081] S201. Determine the average weather change cycle based on the historical weather information of the location where the wearer is located.

[0082] The AI glasses can obtain the historical weather data of the area where the wearer is located in the past period (such as one year) through the weather application interface of the mobile terminal. These data contain the time series of meteorological parameters such as daily temperature, humidity, air pressure, precipitation, etc. The AI glasses use time series analysis methods to preprocess the historical weather data, including missing value filling, outlier detection and processing, etc. Then, the fast Fourier transform (FFT) is used to perform spectral analysis on the processed time series to identify the main periodic change patterns. By analyzing the main peaks in the spectrum, the basic period of weather change is determined.

[0083] In the specific implementation process, the AI glasses may find that the weather in a certain area changes significantly in about 4 hours. For example, in coastal cities, affected by sea-land breezes, the temperature and humidity usually change significantly around 6 am, 12 noon, 4 pm and 8 pm. The AI glasses record this change rule as the basis for the time interval of subsequent real-time weather information acquisition. In addition, the AI glasses can also be affected by seasonal changes and adjust the sampling period in different seasons. For example, more frequent sampling may be required in summer to capture rapid weather changes, etc., which is not limited here.

[0084] S202. Obtain the current weather information from the wearer's mobile terminal according to the average weather change period.

[0085] Based on the average weather change period determined in step S201, the AI glasses establish an intelligent data acquisition mechanism. It establishes a secure wireless connection with the wearer's mobile terminal through a low-power Bluetooth (BLE) or Wi-Fi module. The weather application running on the mobile terminal will update the local weather data in real time through the API. The AI glasses send data requests to the mobile terminal regularly according to the preset weather change period. Each request will obtain a set of complete weather parameters, including environmental indicators such as weather type, temperature, humidity, air pressure, precipitation, ultraviolet intensity, PM2.5 concentration, etc.

[0086] Furthermore, in order to optimize data transmission and energy consumption, the AI glasses can also adopt an adaptive data acquisition strategy on the basis of regularly obtaining weather information. Specifically, when a sudden weather change (such as sudden rainfall) is detected, the sampling frequency will be temporarily increased; during stable weather periods, the sampling frequency will be appropriately reduced. For example, under clear and cloudless weather conditions, a sampling frequency of once every 4 hours may be adopted; while in seasons with variable weather, it may be adjusted to once every hour. At the same time, through technologies such as data compression and differential coding, the amount of data transmitted each time is reduced, and the transmission efficiency is improved.

[0087] In the above embodiment, the AI glasses obtain the current weather information from the wearer's mobile terminal, and determine the data request period for reading the weather information from the mobile terminal by using the variation law of historical weather information. This method can timely monitor the change of the current weather information on the premise of reducing data transmission, which is convenient for subsequent color adjustment of the frame.

[0088] S203. Capture the bone vibration signal during the wearer's communication through the bone conduction sensor, and convert it into speech content through a preset algorithm.

[0089] The AI glasses can integrate highly sensitive bone conduction sensors at positions such as the temple that contact the ears. These sensors can capture the mechanical vibration signals transmitted through the skull when the wearer speaks. Specifically, when the wearer speaks, the sound waves generated by the vibration of the vocal cords are transmitted to the sensors through the skull, and the sensors convert these mechanical vibrations into electrical signals. These original vibration signals are preliminarily amplified by a preamplifier, and then converted into digital signals through a high-precision analog-to-digital converter (ADC).

[0090] Furthermore, the AI glasses perform noise reduction on the digital signals to remove the interference of environmental vibrations and physiological noises (such as chewing, breathing, etc.). Then, acoustic features, including feature parameters such as fundamental frequency and formant, are extracted through a convolutional neural network. The long short-term memory network (LSTM) is combined to perform temporal modeling on these features, and finally the bone conduction signal is converted into the corresponding speech content. For example, when the wearer says "The weather is nice today", the processing result of the bone conduction signal can accurately restore the content of this sentence, providing an important reference for subsequent speech information calibration.

[0091] S204. Calibrate the personnel communication language information captured by the microphone according to the speech content.

[0092] The AI glasses use the speech content parsed from the bone conduction signal as a reference to calibrate and enhance the environmental speech signal collected by the microphone array in step S101. Specifically, through the time alignment algorithm, the bone conduction speech and the speech collected by the microphone are accurately synchronized in the time dimension. Then, an adaptive filter is used to suppress the background noise in the microphone signal, and the clear speech component is extracted for comparison and calibration. This dual verification mechanism significantly improves the accuracy of speech recognition and provides more reliable input data for subsequent sentiment analysis.

[0093] In the above embodiment, the AI glasses capture the bone vibration signal during communication through the bone conduction sensor, convert these signals into speech content, and calibrate the audio data received by the microphone. This technology can improve the accuracy of language information collection. Especially in a noisy environment, the interference of background noise can be effectively reduced through bone conduction technology, thus making the sentiment analysis more accurate.

[0094] S205. Preprocess the language information of personnel communication and the current weather information respectively, and convert them to obtain audio features and weather features in vector form.

[0095] This step is the same as step S102 and will not be elaborated here.

[0096] S206. Convert the audio features within a preset time period into Mel spectrogram form to obtain a Mel spectrogram matrix.

[0097] The AI glasses will collect audio signals from the environment, which usually include voice communication content. To extract emotion-related features, converting these audio signals into Mel spectrogram form is an effective method. Specifically, the Mel spectrogram can better capture the features of speech by simulating the non-linear perception of frequency by the human ear. The transformation formula of the Mel spectrogram is:

[0098] ;

[0099] where X is the frequency-domain signal obtained by performing short-time Fourier transform on the time-domain signal, f is the complex spectral value of the f-th frequency component, t is the t-th time frame, H represents the distance between frames, and ϵ is a positive constant.

[0100] In specific implementation, the AI glasses first capture voice signals through the built-in microphone array, and then optimize the signal quality through preprocessing (such as noise reduction, normalization, etc.). Subsequently, these processed audio data are frame-processed, usually 20 - 30 milliseconds per frame, and a window function (such as Hamming window) is applied to each frame to reduce boundary effects. Then, the fast Fourier transform (FFT) is used for each frame to obtain the spectrum, and the spectrum is converted into a Mel spectrogram through a Mel filter bank. Finally, these Mel spectrogram data are organized into matrix form as the input for the subsequent deep learning model.

[0101] S207. Use a convolutional neural network to extract features from the Mel spectrogram matrix, and input the extracted features into a recurrent neural network to obtain the time series features of the language communication audio data.

[0102] Use a convolutional neural network (CNN) to extract features from the Mel spectrogram matrix. Among them, CNN performs well in processing image and audio data, and can effectively capture local features and maintain the spatial hierarchical relationship of features. In the AI glasses, the preprocessed Mel spectrogram matrix can be regarded as an input similar to an image, and key features are extracted through multiple convolutional layers and pooling layers.

[0103] Among them, the mathematical formula for convolutional feature extraction is:

[0104] ;

[0105] Among them, is the value of the -th channel at position (t, f) in the feature map. C is the number of input channels, is the output channel, W is the weight of the convolutional kernel, is the bias term of the -th channel of the output feature map, and represent the height and width of the convolutional kernel respectively;

[0106] After feature extraction, these key features are fed into a recurrent neural network (RNN), and specifically, a long short-term memory network (LSTM) (or gated recurrent unit (GRU)) can be used to process them. Among them, LSTM is suitable for processing time series data and can learn long-term dependencies in the data. Through the processing of LSTM, the AI glasses can understand and extract time-related dynamic changes (i.e., time series features) from continuous Mel spectrogram features, thereby effectively capturing the emotional state transitions of the wearer.

[0107] S208. Map the weather features corresponding to the time series features through a fully connected layer to the same dimension as the time series features and perform feature fusion to obtain a comprehensive feature vector, and then determine the current emotional state of the wearer.

[0108] The AI glasses not only capture the wearer's speech information but also synchronously obtain the corresponding external weather information when collecting the wearer's speech information. The weather type information in the weather information is represented by one-hot encoding, and the numerical information such as temperature and humidity is normalized and then concatenated with the one-hot encoding representation of the weather type information into a vector form to obtain the weather features. In this step, the AI glasses fuse the weather features (such as temperature, humidity, weather conditions, etc.) with the time series features of the speech. Specifically, the weather features are processed through a fully connected layer and mapped to the same dimension as the speech features. The mapping process is:

[0109] ;

[0110] Among them, is the dimension projection vector, is the bias.

[0111] Then, the two are subjected to feature fusion, enabling the AI glasses to comprehensively consider the speech information and weather information, generate a comprehensive feature vector representing the joint characteristics of the audio data and weather data, and further infer the emotional state based on this comprehensive feature vector using a fully connected layer, and output the current emotional state of the wearer. Among them, the output emotional state includes discrete emotional labels (such as happy, sad, angry, etc.) and continuous emotional intensity values (pleasure P, arousal A, and dominance D).

[0112] In the above embodiment, the AI glasses convert the audio data into the Mel spectrogram form, extract features, combine the weather features, and obtain the comprehensive feature vector through feature fusion. This method makes the emotion judgment more refined and comprehensive, improves the accuracy and response speed of emotion recognition, and further makes the change of the glasses color more in line with the actual emotion state.

[0113] S209. Determine the hue, saturation, and brightness of the color information respectively according to the emotion label, arousal, and dominance in the emotion state.

[0114] Specifically, the mapping relationship between the emotion state and HSV is as follows:

[0115] ;

[0116] where H represents the hue, i is the index of the emotion label in all emotion categories, and N is the total number of emotion categories;

[0117] Optionally, the hue H can also be determined according to the pleasure degree P, that is , where H represents the hue, P represents the pleasure degree, and when P = -1, the hue is , and when P = 1, the hue is ;

[0118] S = A, where S represents the saturation and A represents the arousal;

[0119] V = 0.5×(A + D) + 0.5, where V represents the brightness, A represents the arousal, and D represents the dominance.

[0120] In the above embodiment, the AI glasses establish the correspondence between different dimensions (emotion label, arousal, and dominance) of the emotion state and the hue, saturation, and brightness in the HSV color space. This detailed color adjustment mechanism allows the glasses to display richer and more accurate color changes and can more finely reflect the emotional changes of the wearer.

[0121] S210. After converting the color information into the color values corresponding to the RGB color space, receive the personalized area set by the wearer on the mobile terminal.

[0122] After the AI glasses complete the color space conversion from HSV to RGB, an interactive interface can be provided through a mobile terminal application, allowing the wearer to customize the display area on the glasses frame. Optionally, the interactive interface uses 3D modeling technology to display a three-dimensional model of the glasses frame on the mobile phone screen, and the wearer can select and adjust the area where the color is to be displayed through touch operations. Or the mobile terminal application divides the glasses frame into multiple independently controllable display partitions, such as the left and right sides of the temple, the upper and lower edges of the frame, etc. The wearer can select a specific area by swiping and enable or disable the display function of certain areas through a switch control.

[0123] For example, when the wearer wants to use the AI glasses in a business occasion, they may only want to display color changes on the outer side of the temple to avoid being too conspicuous. Through the settings interface of the mobile terminal, the wearer can only enable the display area on the outer side of the temple and can further adjust the display range and boundary of this area. These settings will be synchronized to the AI glasses in real time and recorded and executed by the control system of the glasses. At the same time, the mobile terminal also provides preset scene modes, such as "business mode", "leisure mode", etc., and each mode corresponds to a different display area configuration, which is convenient for the wearer to quickly switch the usage scenario.

[0124] S211. Determine the multi-color LED strip lamp beads that can be controlled in the actual layout according to the personalized area, and generate corresponding lamp bead control signals.

[0125] The AI glasses selectively control the actual LED layout according to the personalized area settings received in step S210. The LED strip lamp beads on the glasses frame adopt a modular design, and each module contains multiple independently addressable RGB LED lamp beads. The AI glasses map the coordinates of the personalized area to the actual LED layout coordinate system to determine the set of LED lamp beads to be controlled. Then, according to the physical positions and circuit connection relationships of these LEDs, a corresponding PWM control signal sequence is generated.

[0126] It should be noted that the lamp bead control system of the AI glasses can adopt a master-slave structure, and the master controller communicates with multiple LED driver chips through the SPI or I2C bus. Each driver chip is responsible for controlling the LED lamp beads in a specific area and can precisely adjust the brightness values of the RGB three channels of each LED. In order to achieve a smooth color gradient effect, the control system can also use a linear interpolation algorithm to calculate the transition color values between adjacent LEDs. For example, when the wearer selects to display color only on the right temple, the control system will identify the LED module corresponding to this area and generate PWM signals that only control these LEDs, while keeping the LEDs in other areas turned off.

[0127] In the above embodiments, the AI glasses receive the personalized area set by the wearer before generating the bead control signal, and adjust the actual layout control of the LED beads according to these settings. This allows users to customize the color area displayed on the glasses according to their personal preferences, increasing the personalization and practicality of the frame color display. Users can select specific color display areas, so that the color change of the glasses not only reflects the emotional state, but also meets personal aesthetic and situational needs.

[0128] S212. Collect language communication audio data and corresponding environmental weather data in multiple scenarios.

[0129] Among them, the collection methods of the language communication audio data and the corresponding environmental weather data are the same as those in step S101, and will not be elaborated here.

[0130] S213. Perform emotional state annotation on the language communication audio data and the environmental weather data to obtain a training data set.

[0131] The AI glasses can adopt a multi-level annotation strategy to perform emotional state annotation on the collected language communication audio data and environmental weather data. Optionally, relevant technical experts annotate the audio data and the corresponding environmental weather data, and the annotation content includes basic emotional categories (such as happiness, sadness, anger, etc.) and emotional intensity values (pleasantness, arousal, and dominance). The annotation process uses a standardized evaluation scale, such as the Geneva Emotion Wheel or the SAM scale, to ensure the consistency and reliability of the annotation.

[0132] S214. Use the training data set to train the neural network model to obtain a pre-trained neural network model.

[0133] The neural network model adopts a multi-stream structure, including an audio processing branch and a weather feature processing branch. Among them, the audio processing branch uses a CNN-LSTM hybrid architecture, where the CNN is responsible for extracting the local features of the audio, and the LSTM captures the temporal dependencies; the weather feature branch performs feature transformation through a multi-layer perceptron and then fuses with the audio features. The training process of the model can adopt the batch gradient descent method, use the Adam optimizer to update the parameters, and the learning rate is dynamically adjusted using the cosine annealing strategy.

[0134] During the training process, the model simultaneously optimizes the sentiment classification loss and the sentiment intensity regression loss. To address the problem of data imbalance, a weighted cross-entropy loss function is adopted, assigning higher weights to minority-class samples. At the same time, data augmentation techniques, such as adding Gaussian noise and time stretching, are used to enhance the generalization ability of the model. The validation set is used during the training process to monitor the model performance and implement an early stopping strategy to avoid overfitting. For example, when processing "complaints on a rainy day", the model can accurately identify the negative emotions therein and take into account the impact of the rainy weather on emotions to generate more accurate sentiment state prediction results.

[0135] In the above embodiment, the AI glasses collect and label the language communication audio data and the corresponding environmental weather data in various scenarios for training the neural network model, establishing the mapping relationship between the audio data and the wearer's sentiment state under different weather data, enabling the AI glasses to more accurately identify the wearer's sentiment state and adjust the color of the glasses frame according to these sentiment states.

[0136] The AI glasses of the embodiments of the present invention are electronic devices. Figure 3 The schematic diagram of the architecture of the electronic device suitable for implementing the embodiments of the present invention is shown.

[0137] It should be noted that Figure 3 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0138] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions (computer programs), or the relevant hardware can be controlled by instructions (computer programs). The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor. Among them, multiple instructions are stored in the storage medium, and the instructions can be loaded by the processor to execute any step of the method provided by the embodiments of the present invention.

[0139] Specifically, the storage medium and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more signal lines. The storage medium stores computer-executable instructions for implementing the data access control method, including at least one software function module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium. The storage medium can be, but is not limited to, a random access memory (Random Access Memory, abbreviated as RAM), a read-only memory (Read Only Memory, abbreviated as ROM), a programmable read-only memory (Programmable Read-Only Memory, abbreviated as PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory, abbreviated as EPROM), an electrically erasable programmable read-only memory (Electric Erasable Programmable Read-Only Memory, abbreviated as EEPROM), etc. Among them, the storage medium is used to store programs, and the processor executes the programs after receiving the execution instructions.

[0140] Further, the software programs and modules in the above storage medium may further include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide a running environment for other software components. The processor can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc., which can implement or execute the various methods, steps, and logic flow block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0141] Since the instructions stored in the storage medium can execute the steps in any method provided by the embodiments of the present invention, the beneficial effects of any method provided by the embodiments of the present invention can be achieved. For details, please refer to the previous embodiments and will not be repeated here.

[0142] As described above, it is only the preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for automatically changing the frame color according to mood and weather, applied to AI glasses, characterized in that, The method includes: Collecting in real time the personnel communication language information around the wearer and the current weather information; Preprocessing the personnel communication language information and the current weather information respectively, and converting them to obtain audio features and weather features in vector form; Collecting language communication audio data and corresponding environmental weather data in multiple scenarios; Performing emotional state annotation on the language communication audio data and the environmental weather data to obtain a training data set, where the language communication audio data and the corresponding environmental weather data are input data of a neural network model, and the emotional state annotation is output verification data of the neural network model; Training a neural network model using the training data set to obtain a pre-trained neural network model, where the neural network model is constructed by mixing a convolutional neural network and a recurrent neural network; Converting the audio features within a preset time period into Mel spectrogram form to obtain a Mel spectrogram matrix; Using a convolutional neural network to extract features from the Mel spectrogram matrix, and inputting the extracted features into a recurrent neural network to obtain time series features of the language communication audio data; Mapping the weather features corresponding to the time series features to the same dimension as the time series features through a fully connected layer and performing feature fusion to obtain a comprehensive feature vector; Determining the current emotional state of the wearer according to the comprehensive feature vector; the emotional state includes an emotional label and emotional intensity features, and the emotional intensity features include three feature values: pleasantness, arousal, and dominance; Determining the hue of the color information corresponding to the HSV color space according to the classification index of the emotional label and the total number of emotional categories; Mapping the saturation of the color information according to the arousal; Calculating the brightness of the color information according to the arousal and dominance; the color information includes values of three channels: hue, saturation, and brightness, and different emotional states correspond to different color information; Converting the color information into color values corresponding to the RGB color space, and then generating an overall bead control signal according to the actual layout of multi-color LED strip beads on the glasses frame; Controlling the multi-color LED strip beads according to the bead control signal, so that the multi-color LED strip beads display colors corresponding to the emotional state on the glasses frame.

2. The method according to claim 1, wherein Before the step of collecting in real time the personnel communication language information around the wearer and the current weather information, it further includes: Determining the average weather change period according to the historical weather information of the location where the wearer is; Obtaining the current weather information from the wearer's mobile terminal according to the average weather change period, where the mobile terminal is wirelessly connected to the AI glasses.

3. The method according to claim 1, wherein After the step of collecting in real time the personnel communication language information around the wearer and the current weather information, it further includes: Capturing the bone vibration signal during the wearer's communication process through a bone conduction sensor; Converting the bone vibration signal into speech content through a preset algorithm; Calibrating the personnel communication language information according to the speech content.

4. The method according to claim 1, wherein Before the step of generating an overall bead control signal according to the actual layout of the multi-color LED strip beads on the glasses frame, the method further includes: Receiving a personalized area set by the wearer on the mobile terminal; Determining the multi-color LED strip beads that can be controlled in the actual layout according to the personalized area.

5. An AI glasses, characterized in that, The AI glasses include: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code. The computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the AI glasses to execute the method according to any one of claims 1-4.

6. A computer-readable storage medium, comprising instructions, characterized in that, When the instructions run on the AI glasses, the AI glasses are caused to execute the method according to any one of claims 1-4.

7. A computer program product, characterized in that, When the computer program product runs on the AI glasses, the AI glasses are caused to execute the method according to any one of claims 1-4.

Images