Interactive method and system for in-cabin makeup function

Abstract

The present invention relates to the field of lighting control and specifically discloses a method and system for interacting with an in-cabin makeup function. The method comprises: S1: collecting ambient and user information in real time using a light sensor and a temperature sensor; simultaneously, obtaining user motion information using an infrared array sensor; S2: filtering makeup motion interference in motion information using a pattern recognition algorithm in combination with a preset makeup motion signal model; retaining only motion signals with significant differences in characteristics from the makeup motion signal and proceeding to S3; and S3: analyzing the processed motion signal using a microprocessor. The technical solution of the present invention can effectively resolve the problem of misidentification, balance the contradiction between interaction distance and user experience, and provide users with a stable and efficient in-car makeup interaction experience.

Description

Technical Field

[0001] The present invention relates to the field of lighting control, and in particular to an interactive method for a makeup function in a cabin. Background Art

[0002] As modern vehicles become more intelligent, in-cabin makeup applications are becoming a growing concern for many users. To enhance the convenience of in-car makeup application, contactless interactive technology has been introduced into cabin makeup mirror systems, promising a more hygienic and intelligent experience. However, in real-world applications, this technology presents numerous challenges that are difficult to ignore.

[0003] When applying makeup in the car, users frequently move their hands to apply, touch up, and perform other makeup actions. During this process, existing contactless command recognition systems often misinterpret these normal makeup movements as operational commands. For example, when a user holds cosmetics close to a makeup mirror while applying, the system might mistakenly interpret the user's instructions as adjusting the light brightness or switching makeup modes. This can cause the mirror to function erratically, seriously disrupting the user's makeup process.

[0004] Contactless command input inherently suffers from a distance constraint. If the user interacts too closely with the makeup mirror, while this can reduce the probability of misidentification to a certain extent, it significantly limits the user's operating space, making it difficult for them to comfortably apply makeup and significantly reducing the user experience. Conversely, if the interaction distance is too far, while the user's operating space is guaranteed, the complexity of the cabin environment means infrared signals are susceptible to reflections and obstructions from objects inside the vehicle, such as seats and dashboards, and electromagnetic interference from other electronic devices, significantly reducing signal accuracy and leading to frequent misidentification.

[0005] It's particularly important to note that when using the in-cabin makeup function, users often move very close to the makeup mirror to more clearly observe facial details, further exacerbating misidentification. At close range, the signals generated by the user's makeup movements are more easily captured and misinterpreted by the system. Furthermore, even minor environmental interference can significantly impact the accuracy of the signal, posing a significant challenge to the stability and reliability of contactless interaction technology.

[0006] The current contactless interactive technology for in-cabin makeup functions has obvious defects in distinguishing between user makeup actions and operating instructions, and controlling the interaction distance. An innovative interaction method is urgently needed to effectively solve the problem of misrecognition, balance the contradiction between interaction distance and user experience, and provide users with a stable and efficient in-car makeup interaction experience. Summary of the Invention

[0007] The present invention provides an interactive method for an in-cabin makeup function, which can effectively solve the problem of misrecognition, balance the contradiction between interaction distance and user experience, and provide users with a stable and efficient in-car makeup interactive experience.

[0008] In order to solve the above technical problems, this application provides the following technical solutions:

[0009] An interactive method for an in-cabin makeup function, comprising:

[0010] S1 Environmental and User Information Collection Step: Ambient light intensity information and cabin temperature information are collected in real time through light sensors and temperature sensors. At the same time, user motion information is acquired through infrared array sensors.

[0011] S2 makeup action interference filtering step: Combined with the preset makeup action signal model, the pattern recognition algorithm is used to filter out the interference signals generated by the makeup action in the action information; only the action signals with large differences in makeup action signal characteristics are retained and enter S3;

[0012] S3 Command Judgment and Confirmation: The processed motion signal is analyzed by the microprocessor. Based on the intensity, frequency of change, and duration characteristics of the motion signal, combined with a preset operation instruction rule library, if the motion signal characteristics completely match the preset operation instruction and are within a reasonable operating distance range, the operating distance is determined by the infrared ranging algorithm, then the signal is confirmed to be a valid operation instruction and the process proceeds to S4. If there are multiple matches between the motion signal and the preset operation instruction, the process returns to S2 and counts the returned motions. After the count exceeds 2, the multiple matches are combined with the ambient light intensity information and the cabin temperature information, and according to the preset rationality judgment model, the one with a reasonable match is selected as the valid operation instruction. If there is no reasonable match, the process returns to S1 and notifies the user.

[0013] S4 function execution and feedback: According to the confirmed operation instructions, perform the corresponding makeup function operation and adjust the brightness and color temperature of the makeup mirror.

[0014] Furthermore, in said S2, when constructing the makeup action signal model, infrared sensor signal data generated by hand movements during normal makeup is collected in advance; the action information collected in real time is compared with the makeup action signal model; if the amplitude change of the signal is within the common amplitude range of makeup actions, and the frequency distribution is similar to the frequency characteristics of the makeup action signal, and the duration is consistent with the general duration of the makeup action, then the signal is determined to be an interference signal generated by the makeup action and is filtered.

[0015] Furthermore, when the pattern recognition algorithm is used for comparison in S2, the dynamic time warping algorithm is used to calculate the real-time action signal sequence. Model template sequence with preset makeup action signal The similarity is calculated as follows:

[0016]

[0017] in, for and The optimal alignment path between ,and express No. Points and No. points are aligned; when and and When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the dynamic time warping threshold, 、 is the amplitude change ratio threshold, 、 is the frequency peak ratio threshold.

[0018] Furthermore, the makeup action signal model constructed in S2 uses a Gaussian mixture model for probability density estimation, and the model parameters are obtained by iterative optimization of the expectation maximization algorithm; for the action signal feature vector collected in real time , the probability that it belongs to the makeup action interference signal is:

[0019]

[0020] in, is the number of mixed ingredients, For the The weight of the components, For the Gaussian distribution with mean , the covariance matrix is ; when When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the preset probability threshold; the parameter update formula of the Gaussian mixture model is:

[0021]

[0022]

[0023]

[0024] in, is a latent variable, representing the data point Belong to The posterior probability of each component is calculated as follows:

[0025]

[0026] Furthermore, when the microprocessor in S3 analyzes the processed action signal, it uses a Bayesian decision model with multi-feature fusion to make instruction judgments. The calculation formula is as follows:

[0027]

[0028] in, is a set of preset operation instructions. is the fused feature vector, which includes the intensity, change frequency, duration of the action signal and the operation distance determined by the infrared ranging algorithm; conditional probability Estimation is performed using the Gaussian kernel function:

[0029]

[0030] in, For instructions The number of training samples, For instructions No. training samples, For instructions The covariance matrix of Dynamic adjustment based on the historical frequency of operation instructions:

[0031]

[0032] in, is the smoothing parameter, For instructions The number of historical executions of and When the signal is confirmed as a command Valid operation instructions, where:

[0033] is the Bayesian decision threshold

[0034] For instructions Optimal operating distance

[0035] For instructions Standard deviation of operating distance

[0036] is the distance tolerance coefficient, and its value range is .

[0037] Furthermore, the rationality judgment model of the multiple matching items adopts a hierarchical decision tree structure, combining environmental context and user historical behavior to perform multi-dimensional scoring. The specific steps are as follows:

[0038] Step 1: Build a scene containing ambient light intensity and cabin temperature The environmental feature vector , and normalize it:

[0039]

[0040] in, and are the historical observation ranges of light intensity and temperature, respectively.

[0041] Step 2: Build instructions based on user historical operation data In the environment Preference probability distribution under , estimated by Gaussian process regression:

[0042]

[0043] in, is the mean function, is the covariance function, through historical data Trained, Indicates instructions In the environment The frequency of occurrence of the following.

[0044] Step 3: Calculate each candidate instruction Environmental adaptability score:

[0045]

[0046] in:

[0047] For instructions The optimal environment feature vector

[0048] is the environmental sensitivity diagonal matrix, which is dynamically updated by the following formula:

[0049]

[0050] is the learning rate, the value range is , Indicates that the diagonal elements of the matrix are taken to form a diagonal matrix, ensuring Always a positive definite diagonal matrix.

[0051] Step 4: Comprehensive Bayesian Probability , historical preferences and environmental adaptability , calculate the final rationality score:

[0052]

[0053] in, is the weight coefficient, which is dynamically adjusted by the following formula:

[0054]

[0055] is the learning rate, is the loss function based on historical decision accuracy.

[0056] Step 5: Select the instruction with the highest final score as the valid operation instruction:

[0057]

[0058] when When , return to S1 and prompt the user, where is the rationality threshold.

[0059] Furthermore, it also includes S5, collecting the user's performance after adjusting the brightness and color temperature of the makeup mirror. If the user's performance is not in line with expectations, the operation of adjusting the brightness and color temperature of the makeup mirror is returned; if the user's performance is in line with expectations, the process ends.

[0060] Furthermore, the user performance is collected through a cockpit microphone array and an infrared array sensor.

[0061] Furthermore, the evaluation of user performance in S5 adopts a multimodal emotion recognition model, specifically including:

[0062] Step 1: Synchronously collect user voice signals and action posture sequences :

[0063] Sound signal: collected by cockpit microphone array, sampling rate

[0064] Action posture: acquired through the infrared array sensor in S1, with a frame rate of

[0065] Step 2: Construct a sound vector containing prosodic, acoustic, and semantic features:

[0066]

[0067] in:

[0068] Prosodic feature: fundamental frequency change rate , standard deviation of tone length :

[0069]

[0070] Acoustic features: Dynamic changes in Mel-frequency cepstral coefficients:

[0071]

[0072] Semantic features: Tone intensity index obtained by matching sentiment dictionaries :

[0073]

[0074] Step 3: Calculate the motion change rate of key joints using infrared array sensors:

[0075]

[0076] in:

[0077] For the The joints are Three-dimensional coordinates of the moment

[0078] is the time window, value

[0079] represents the Euclidean norm

[0080] Step 4: Build a Bayesian fusion model to calculate the probability of user dissatisfaction:

[0081]

[0082] Among them, the likelihood function is expressed by the Gaussian mixture model:

[0083]

[0084] is the fused feature vector.

[0085] Step 5: Adaptive adjustment of decision threshold Dynamically adjust the unsatisfactory threshold based on historical interaction data :

[0086]

[0087] in:

[0088] For the The actual dissatisfaction label of historical interactions

[0089] is the historical data window size, taking the most recent 50 interactions

[0090] is the learning rate, with a value of 0.05

[0091] Step 6: When the following conditions are met at the same time, the user is deemed dissatisfied and a return is performed:

[0092] The dissatisfaction probability output by the multimodal sentiment fusion model exceeds the threshold:

[0093]

[0094] If the motion change rate exceeds the preset threshold:

[0095]

[0096] Then return to the previous step with the first adjustment step:

[0097]

[0098] If the motion change rate does not exceed the threshold:

[0099]

[0100] Then return to the previous step with the second adjustment step and start the voice soothing module:

[0101]

[0102] in, , , , is the action change rate threshold, the value range is .

[0103] The principle and beneficial effects of this solution are as follows: Data from infrared array sensors is converted into characteristic parameters such as hand motion coordinate trajectory and signal intensity change rate. Based on pre-collected data from normal makeup movements (such as brush application and powder pressing), an interference signal feature space is constructed, encompassing movement amplitude, frequency, and duration. Real-time motion signals are then compared against this feature space in multiple dimensions. If the signal characteristics fall within the pre-defined makeup movement feature range, it is identified as an interference signal and filtered out. Compared to first identifying the instruction and then determining interference, this solution identifies interference first, effectively resolving the pre-existing interference problem, reducing the computational complexity of the judgment logic, and shortening response time.

[0104] The microprocessor extracts characteristic values ​​from the filtered action signal, based on intensity, frequency, and duration, and matches them against a pre-set action command rule base (e.g., "brighten the lights" corresponds to intensity > 1.5V, frequency < 3Hz, and duration 1-2 seconds). Using an infrared ranging algorithm (such as triangulation), the microprocessor calculates the operating distance between the user and the mirror. The command is validated only when the signal characteristics match and the distance is within a reasonable range (e.g., 30-50cm), preventing false triggering of makeup actions at close range.

[0105] When multiple possible instructions appear, the system uses a counting mechanism (such as triggering after two consecutive failed matches) combined with environmental information (lighting, temperature) to filter the most likely instructions based on a preset rationality judgment model (such as a rule engine or heuristic algorithm), thereby reducing misjudgments (or non-recognition) in ambiguous scenarios and improving user experience.

[0106] After confirming a valid command, the microprocessor adjusts the mirror's LED brightness (0-1000 lux) and color temperature (2700K-6500K) via a hardware interface (e.g., PWM signal), providing real-time feedback. If no reasonable match exists, the system returns to its initial state and prompts the user to re-enter the input through flashing lights, voice prompts, and other methods, ensuring the robustness of the interaction process.

[0107] This invention effectively filters interference signals generated by normal hand movements (such as applying mascara) through pre-set makeup motion feature comparisons, reducing the false trigger rate of contactless commands. By combining three-dimensional feature matching based on motion intensity, frequency, and duration, it improves accuracy compared to traditional single-threshold judgment methods (e.g., relying solely on signal strength). It performs particularly well in complex lighting conditions (such as backlighting and tunnels) and in multi-person cabin environments. A distance constraint mechanism further eliminates long-range interference (such as accidental touches by passengers).

[0108] This invention relies solely on low-cost hardware, such as infrared arrays and light / temperature sensors, eliminating the need for additional cameras or depth sensors, thus reducing hardware costs. In summary, this invention effectively addresses the issue of misidentification, balances interaction distance with user experience, and provides users with a stable and efficient in-car makeup interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 The present invention is a flowchart of an embodiment of an interactive method for an in-cabin makeup function. DETAILED DESCRIPTION

[0110] The following is further described in detail through specific implementation methods:

[0111] An interactive method for in-cabin makeup functions (e.g. Figure 1 shown), including:

[0112] S1 Environmental and User Information Collection Step: Ambient light intensity information and cabin temperature information are collected in real time through light sensors and temperature sensors. At the same time, user motion information is acquired through infrared array sensors.

[0113] S2 makeup action interference filtering step: Combined with the preset makeup action signal model, the pattern recognition algorithm is used to filter out the interference signals caused by the makeup action in the action information; only the action signals with large differences from the makeup action signal characteristics are retained and enter S3; in S2, when constructing the makeup action signal model, the infrared sensor signal data generated by the hand movements during the normal makeup process are collected in advance; the real-time collected action information is compared with the makeup action signal model; if the amplitude change of the signal is within the common amplitude range of the makeup action, and the frequency distribution is similar to the frequency characteristics of the makeup action signal, and the duration is consistent with the general duration of the makeup action, then the signal is determined to be an interference signal generated by the makeup action and is filtered.

[0114] When using pattern recognition algorithm for comparison in S2, dynamic time warping algorithm is used to calculate the real-time action signal sequence Model template sequence with preset makeup action signal The similarity is calculated as follows:

[0115]

[0116] in, for and The optimal alignment path between ,and express No. Points and No. points are aligned; when and and When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the dynamic time warping threshold, 、 is the amplitude change ratio threshold, 、 is the frequency peak ratio threshold.

[0117] The makeup action signal model constructed in S2 uses Gaussian mixture model for probability density estimation, and the model parameters are iteratively optimized by expectation maximization algorithm; for the action signal feature vector collected in real time, , the probability that it belongs to the makeup action interference signal is:

[0118]

[0119] in, is the number of mixed ingredients, For the The weight of the components, For the Gaussian distribution with mean , the covariance matrix is ; when When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the preset probability threshold; the parameter update formula of the Gaussian mixture model is:

[0120]

[0121]

[0122]

[0123] in, is a latent variable, representing the data point Belong to The posterior probability of each component is calculated as follows:

[0124]

[0125] S3 Command Judgment and Confirmation: The processed motion signal is analyzed by the microprocessor. Based on the intensity, frequency of change, and duration characteristics of the motion signal, combined with a preset operation instruction rule library, if the motion signal characteristics completely match the preset operation instruction and are within a reasonable operating distance range, the operating distance is determined by the infrared ranging algorithm, then the signal is confirmed to be a valid operation instruction and the process proceeds to S4. If there are multiple matches between the motion signal and the preset operation instruction, the process returns to S2 and counts the returned motions. After the count exceeds 2, the multiple matches are combined with the ambient light intensity information and the cabin temperature information, and according to the preset rationality judgment model, the one with a reasonable match is selected as the valid operation instruction. If there is no reasonable match, the process returns to S1 and notifies the user.

[0126] When the microprocessor in S3 analyzes the processed action signal, it uses a Bayesian decision model with multi-feature fusion to make instruction judgments. The calculation formula is as follows:

[0127]

[0128] in, is a set of preset operation instructions. is the fused feature vector, which includes the intensity, change frequency, duration of the action signal and the operation distance determined by the infrared ranging algorithm; conditional probability Estimation is performed using the Gaussian kernel function:

[0129]

[0130] in, For instructions The number of training samples, For instructions No. training samples, For instructions The covariance matrix of Dynamic adjustment based on the historical frequency of operation instructions:

[0131]

[0132] in, is the smoothing parameter, For instructions The number of historical executions of and When the signal is confirmed as a command Valid operation instructions, where:

[0133] is the Bayesian decision threshold

[0134] For instructions Optimal operating distance

[0135] For instructions Standard deviation of operating distance

[0136] is the distance tolerance coefficient, and its value range is .

[0137] The rationality judgment model for the multiple matching items adopts a hierarchical decision tree structure and combines environmental context and user historical behavior to perform multi-dimensional scoring. The specific steps are as follows:

[0138] Step 1: Build a scene containing ambient light intensity and cabin temperature The environmental feature vector , and normalize it:

[0139]

[0140] in, and are the historical observation ranges of light intensity and temperature, respectively.

[0141] Step 2: Build instructions based on user historical operation data In the environment Preference probability distribution under , estimated by Gaussian process regression:

[0142]

[0143] in, is the mean function, is the covariance function, through historical data Trained, Indicates instructions In the environment The frequency of occurrence of the following.

[0144] Step 3: Calculate each candidate instruction Environmental adaptability score:

[0145]

[0146] in:

[0147] For instructions The optimal environment feature vector

[0148] is the environmental sensitivity diagonal matrix, which is dynamically updated by the following formula:

[0149]

[0150] is the learning rate, the value range is , Indicates that the diagonal elements of the matrix are taken to form a diagonal matrix, ensuring Always a positive definite diagonal matrix.

[0151] Step 4: Comprehensive Bayesian Probability , historical preferences and environmental adaptability , calculate the final rationality score:

[0152]

[0153] in, is the weight coefficient, which is dynamically adjusted by the following formula:

[0154]

[0155] is the learning rate, is the loss function based on historical decision accuracy.

[0156] Step 5: Select the instruction with the highest final score as the valid operation instruction:

[0157]

[0158] when When , return to S1 and prompt the user, where is the rationality threshold.

[0159] S4 function execution and feedback: according to the confirmed operation instructions, the corresponding makeup function operation is executed and the brightness and color temperature of the makeup mirror light are adjusted;

[0160] S5, collecting the user's performance after adjusting the brightness and color temperature of the makeup mirror. If the user's performance does not meet expectations, the operation of adjusting the brightness and color temperature of the makeup mirror is returned; if the user's performance meets expectations, the process ends.

[0161] The evaluation of user performance in S5 uses a multimodal emotion recognition model, which includes:

[0162] Step 1: Synchronously collect user voice signals and action posture sequences :

[0163] Sound signal: collected by cockpit microphone array, sampling rate

[0164] Action posture: acquired through the infrared array sensor in S1, with a frame rate of

[0165] Step 2: Construct a sound vector containing prosodic, acoustic, and semantic features:

[0166]

[0167] in:

[0168] Prosodic feature: fundamental frequency change rate , standard deviation of tone length :

[0169]

[0170] Acoustic features: Dynamic changes in Mel-frequency cepstral coefficients:

[0171]

[0172] Semantic features: Tone intensity index obtained by matching sentiment dictionaries :

[0173]

[0174] Step 3: Calculate the motion change rate of key joints using infrared array sensors:

[0175]

[0176] in:

[0177] For the The joints are Three-dimensional coordinates of the moment

[0178] is the time window, value

[0179] represents the Euclidean norm

[0180] Step 4: Build a Bayesian fusion model to calculate the probability of user dissatisfaction:

[0181]

[0182] Among them, the likelihood function is expressed by the Gaussian mixture model:

[0183]

[0184] is the fused feature vector.

[0185] Step 5: Adaptive adjustment of decision threshold Dynamically adjust the unsatisfactory threshold based on historical interaction data :

[0186]

[0187] in:

[0188] For the The actual dissatisfaction label of historical interactions

[0189] is the historical data window size, taking the most recent 50 interactions

[0190] is the learning rate, with a value of 0.05

[0191] Step 6: When the following conditions are met at the same time, the user is deemed dissatisfied and a return is performed:

[0192] The dissatisfaction probability output by the multimodal sentiment fusion model exceeds the threshold:

[0193]

[0194] If the motion change rate exceeds the preset threshold:

[0195]

[0196] Then return to the previous step with the first adjustment step:

[0197]

[0198] If the motion change rate does not exceed the threshold:

[0199]

[0200] Then return to the previous step with the second adjustment step and start the voice soothing module:

[0201]

[0202] in, , , , is the action change rate threshold, the value range is .

[0203] In specific usage: The light sensor uses a photoresistor (such as the GL5528) to collect ambient light intensity in real time, with a detection range of 100-10,000 lux and an accuracy of ±5%. The temperature sensor uses a digital temperature sensor (such as the DS18B20) to collect cabin temperature, with a range of -20°C to 85°C and an accuracy of ±0.5°C. The infrared array sensor deploys an 8x8 infrared transmitter-receiver pair (such as the Sharp GP2Y0A21YK0F), covering a 50cm x 50cm area in front of the vanity mirror, detecting the two-dimensional coordinates and signal strength of hand movements.

[0204] Light and temperature data are updated 10 times per second, and infrared motion data is collected 30 frames per second to form a motion signal sequence. , where each sample Including infrared signal strength, coordinate offset and other features. When the user enters the car and turns on the makeup mirror, the sensor is automatically started to collect environmental data (such as ambient light intensity , temperature ℃ ), while capturing hand movements (such as moving mascara horizontally) in real time.

[0205] 1,000 sets of typical makeup action data are collected in advance, including:

[0206] Brush application: amplitude voltage fluctuation 0.5-1.2V, frequency 8-15Hz, duration 1-3 seconds;

[0207] Press with the powder puff: the amplitude voltage fluctuation is 0.7-1.8V, the frequency is 4-8Hz, and the duration is 1-4 seconds.

[0208] Preset template sequence For the brush action feature, set , amplitude ratio threshold 、 , frequency threshold 、 .

[0209] set up mixture components, initial mean (corresponding to amplitude, frequency, duration), iterative optimization through EM algorithm, preset probability threshold . Real-time action signal If the DTW distance is 0.6 (<0.8), the amplitude change ratio is 1.05 (between 0.9-1.1), and the frequency peak ratio is 1.1 (between 0.8-1.2), it is determined to be makeup action interference and is filtered out; if the signal characteristics exceed the threshold range (such as frequency 25Hz), it is retained as a potential operation instruction.

[0210] Preset commands include .

[0211] The eigenvector is For example, the typical characteristics of the "turn up the lights" command are: intensity 1.5V, frequency 3Hz, duration 1.2 seconds, distance 35cm (optimal distance , standard deviation , distance tolerance factor ).

[0212] The probability calculation is to estimate the conditional probability through the Gaussian kernel function , prior probability Dynamically adjust based on historical data (e.g. historical execution times of "brighten the lights" , smoothing parameter ,but ).

[0213] In multiple matching process, the current environment (Historical range 100-1000 lux), after normalization .

[0214] Then, we get the environmental adaptability score, such as the optimal environment for the command "turn up the lights" , calculate the Mahalanobis distance , environmental adaptability score .

[0215] The final score is obtained by combining Bayesian probability (0.6), historical preference (0.7), environmental adaptability (0.85), and weight ,have to , which is determined to be a valid instruction.

[0216] After confirming the "brighten the lights" command, the mirror backlight LED (such as Cree XQ-E series) is driven to increase the brightness from 200lux to 300lux, and the indicator light flashes to feedback the operation result. Infrared ranging confirms that the operating distance is 35cm (in range), meeting the instruction execution conditions.

[0217] Then the user's voice is collected through the microphone, and the fundamental frequency change rate (normal range <0.5), change in Mel frequency cepstral coefficients , tone intensity index (Neutral), judged as satisfactory.

[0218] At the same time, the infrared sensor detects the change rate of joint movement (<threshold 0.5 m / s), it is judged to be stable.

[0219] At this point, if the user is not obviously dissatisfied (e.g., continues to put on makeup), the current lighting parameters (brightness 300 lux, color temperature 5500K) are saved to the user preference library.

[0220] If a user complains ( ) and violent movements ( ), then the step size The brightness is returned to 250 lux, and voice reassurance is activated: "The light brightness has been fine-tuned for you. Do you need further adjustment?"

[0221] In other embodiments, the cabin environment parameters are collected in real time by light sensors (photoresistors / photodiodes) and temperature sensors (digital / thermocouples) to construct an environment feature vector. ,in is the light intensity, The infrared array sensor (such as Sharp GP2Y0A21YK0F) is used to transmit infrared signals and receive reflected light intensity, and the hand position coordinates are calculated by the triangulation principle. , forming a three-dimensional motion trajectory sequence .

[0222] The data from each sensor is synchronized to a microprocessor (such as the STM32F4 series) via the SPI / I2C bus with a sampling frequency of 30 Hz to ensure temporal consistency between the action and environmental data.

[0223] Collect 1000+ hours of typical makeup action data (such as smearing, pressing, patting) in advance, extract time domain features (amplitude change rate ), frequency domain features (power spectral density PSD) and time-frequency features (wavelet transform coefficients), and construct a Gaussian mixture model ,in are model parameters.

[0224] Real-time action sequences and template sequence Calculating DTW distance ,when And feature similarity When (such as , ), which is determined to be a makeup interference signal.

[0225] The action feature vector (Intensity, frequency, duration, distance) input into the Bayesian classifier and calculate the posterior probability , where the class conditional probability By kernel density estimation (KDE) modeling, the prior probability Dynamically updated based on historical instruction frequency.

[0226] When multiple matches occur, build the environment-instruction association matrix , calculate the environmental adaptability score through the hierarchical decision tree , combined with action feature scores and historical preference scores , finally select the comprehensive score The highest instruction.

[0227] The microprocessor controls the LED driver circuit (such as TI TPS61165) through PWM signals to achieve linear brightness adjustment from 0 to 1000 lux (accuracy ±5%) and color temperature adjustment from 2700K to 6500K (step size 100K).

[0228] The infrared sensor detects the movement stability (standard deviation of joint displacement) ) and microphone array to collect speech emotions (base frequency change rate ),when or When the adaptive adjustment is triggered.

[0229] This embodiment uses the GMM-DTW hybrid model to reduce the makeup action misrecognition rate from 32% in traditional methods to below 5% (test data: only 52 false triggers in 1,000 makeup operations), significantly reducing interference from unintentional commands.

[0230] Environmental context fusion decision-making improves command accuracy by 28% and 41% in strong light (>5000lux) and low light (<200lux) environments respectively, and reduces the impact of temperature fluctuations (-10℃-40℃) to less than 3%.

[0231] Under bright midday sunlight, the accuracy of the "Dim Lights" command is improved; in low winter temperatures, the response speed of the "Switch to Warm Color Temperature" command is improved. The multi-matching processing mechanism reduces the average number of command confirmations and shortens the time it takes for a single operation. This shortens the time it takes for new users to achieve proficiency. The dynamic threshold adaptive mechanism improves stability across different user scenarios (hand motion amplitude varies by up to ±40%) and vehicle models (operation distance varies by ±15cm). It effectively filters out interference from vehicle vibration (frequency 1-10Hz) and slight passenger movement (amplitude <5cm).

[0232] This invention provides a high-precision, low-latency human-computer interaction solution for smart cockpits, particularly suitable for high-frequency operation scenarios such as makeup mirrors and central control screens. Personalized lighting adjustment enhances makeup satisfaction. Through multi-sensor fusion, intelligent interference filtering, and environmental context awareness, this embodiment significantly improves the interactive accuracy, robustness, and user experience of in-cabin makeup functions, promising broad market application prospects.

[0233] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field are aware of all common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for interacting with a makeup function in a cabin, comprising: S1 Environmental and User Information Collection Step: Ambient light intensity information and cabin temperature information are collected in real time through light sensors and temperature sensors. At the same time, user motion information is acquired through infrared array sensors. S2 makeup action interference filtering step: Combined with the preset makeup action signal model, the pattern recognition algorithm is used to filter out the interference signals generated by the makeup action in the action information; only the action signals with large differences in makeup action signal characteristics are retained and enter S3; S3 Command Judgment and Confirmation: The processed motion signal is analyzed by the microprocessor. Based on the motion signal's intensity, frequency of change, and duration characteristics, and in conjunction with a preset operation command rule library, if the motion signal characteristics fully match the preset operation command and are within a reasonable operating distance range (the operating distance is determined by the infrared ranging algorithm), the signal is confirmed to be a valid operation command and the process proceeds to S4. If the motion signal matches the preset operation command multiple times, the process returns to S2 and counts the returned motions. After the count exceeds two times, the multiple matches are combined with ambient light intensity information and cabin temperature information, and a preset rationality judgment model is used to select the one with a reasonable match as the valid operation command. If there is no reasonable match, return to S1 and remind the user; S4 function execution and feedback: According to the confirmed operation instructions, perform the corresponding makeup function operation and adjust the brightness and color temperature of the makeup mirror.

2. The interactive method for in-cabin makeup function according to claim 1, characterized in that: In said S2, when constructing the makeup action signal model, infrared sensor signal data generated by hand movements during normal makeup is collected in advance; the action information collected in real time is compared with the makeup action signal model; if the amplitude change of the signal is within the common amplitude range of makeup actions, and the frequency distribution is similar to the frequency characteristics of the makeup action signal, and the duration is consistent with the general duration of the makeup action, then the signal is determined to be an interference signal generated by the makeup action and is filtered.

3. The interactive method for in-cabin makeup function according to claim 2, characterized in that: When the pattern recognition algorithm is used for comparison in S2, the dynamic time warping algorithm is used to calculate the real-time action signal sequence. Model template sequence with preset makeup action signal The similarity is calculated as follows: ； in, for and The optimal alignment path between ,and express No. Points and No. points are aligned; when and and When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the dynamic time warping threshold, 、 is the amplitude change ratio threshold, 、 is the frequency peak ratio threshold.

4. The interactive method for in-cabin makeup function according to claim 3, characterized in that: The makeup action signal model constructed in S2 uses a Gaussian mixture model for probability density estimation, and the model parameters are obtained by iterative optimization of the expectation maximization algorithm; for the action signal feature vector collected in real time , the probability that it belongs to the makeup action interference signal is: ； in, is the number of mixed ingredients, For the The weight of the components, For the Gaussian distribution with mean , the covariance matrix is ; when When , the signal is determined to be an interference signal generated by makeup action and is filtered out. is the preset probability threshold; the parameter update formula of the Gaussian mixture model is: ； ； ； in, is a latent variable, representing the data point Belong to The posterior probability of each component is calculated as follows: 。 5. The interactive method for in-cabin makeup function according to claim 4, characterized in that: When the microprocessor in S3 analyzes the processed action signal, it uses a Bayesian decision model with multi-feature fusion to make instruction judgments. The calculation formula is as follows: ； in, is a set of preset operation instructions. is the fused feature vector, which includes the intensity, change frequency, duration of the action signal and the operation distance determined by the infrared ranging algorithm; conditional probability Estimation is performed using the Gaussian kernel function: ； in, For instructions The number of training samples, For instructions No. training samples, For instructions The covariance matrix of Dynamic adjustment based on the historical frequency of operation instructions: ； in, is the smoothing parameter, For instructions The number of historical executions of and When the signal is confirmed as a command Valid operation instructions, where: is the Bayesian decision threshold; For instructions The optimal operating distance; For instructions Standard deviation of operating distance; is the distance tolerance coefficient, and its value range is .

6. The interactive method for in-cabin makeup function according to claim 5, characterized in that: The rationality judgment model for the multiple matching items adopts a hierarchical decision tree structure and combines environmental context and user historical behavior to perform multi-dimensional scoring. The specific steps are as follows: Step 1: Build a scene containing ambient light intensity and cabin temperature The environmental feature vector , and normalize it: ； in, and are the historical observation ranges of light intensity and temperature, respectively; Step 2: Build instructions based on user historical operation data In the environment Preference probability distribution under , estimated by Gaussian process regression: ； in, is the mean function, is the covariance function, through historical data Trained, Indicates instructions In the environment The frequency of occurrence of the following; Step 3: Calculate each candidate instruction Environmental adaptability score: ； in: For instructions The optimal environment feature vector is the environmental sensitivity diagonal matrix, which is dynamically updated by the following formula: ； is the learning rate, the value range is , Indicates that the diagonal elements of the matrix are taken to form a diagonal matrix, ensuring is always a positive definite diagonal matrix; Step 4: Comprehensive Bayesian Probability , historical preferences and environmental adaptability , calculate the final rationality score: ； in, is the weight coefficient, which is dynamically adjusted by the following formula: ； is the learning rate, is the loss function based on historical decision accuracy; Step 5: Select the instruction with the highest final score as the valid operation instruction: ； when When , return to S1 and prompt the user, where is the rationality threshold.

7. The interactive method for in-cabin makeup function according to claim 6, characterized in that: The process also includes S5, collecting the user's performance after adjusting the brightness and color temperature of the makeup mirror. If the user's performance does not meet expectations, the process of adjusting the brightness and color temperature of the makeup mirror is returned to the original state; if the user's performance meets expectations, the process ends.

8. The interactive method for in-cabin makeup function according to claim 7, characterized in that: The user performance is collected through a cockpit microphone array and an infrared array sensor.

9. The interactive method for in-cabin makeup function according to claim 8, characterized in that: The evaluation of user performance in S5 uses a multimodal emotion recognition model, which specifically includes: Step 1: Synchronously collect user voice signals and action posture sequences : The sound signal is collected by the cockpit microphone array with a sampling rate of ; The action posture is acquired by the infrared array sensor in S1, with a frame rate of ; Step 2: Construct a sound vector containing prosodic, acoustic, and semantic features: ； in: Prosodic feature: fundamental frequency change rate , standard deviation of tone length : ； Acoustic features: Dynamic changes in Mel-frequency cepstral coefficients: ； Semantic features: Tone intensity index obtained by matching sentiment dictionaries : ； Step 3: Calculate the motion change rate of key joints using infrared array sensors: ； in: For the The joints are The three-dimensional coordinates of the moment; is the time window, value ; represents the Euclidean norm; Step 4: Build a Bayesian fusion model to calculate the probability of user dissatisfaction: ； Among them, the likelihood function is expressed by the Gaussian mixture model: ； is the fusion feature vector; Step 5: Adaptive adjustment of decision threshold Dynamically adjust the unsatisfactory threshold based on historical interaction data : ； in: For the The actual dissatisfaction label of the historical interaction; is the historical data window size, taking the most recent 50 interactions; is the learning rate, the value is 0.05; Step 6: When the following conditions are met at the same time, the user is deemed dissatisfied and a return is performed: The dissatisfaction probability output by the multimodal sentiment fusion model exceeds the threshold: ； If the motion change rate exceeds the preset threshold: ； Then return to the previous step with the first adjustment step: ； If the motion change rate does not exceed the threshold: ； Then return to the previous step with the second adjustment step and start the voice soothing module: ； in, , , , is the action change rate threshold, the value range is .

10. An interactive system for in-cabin makeup function, characterized in that: The method according to any one of claims 1 to 9 is adopted.

Images