Cosmetic function interaction method and system in cabin

Abstract

The invention relates to the field of light control, and particularly discloses an interaction method and system for a make-up function in a cabin, and the method comprises the steps: S1, environment and user information collection: collecting environment light intensity information and temperature information in the cabin in real time through a light sensor and a temperature sensor; meanwhile, action information of the user is obtained through an infrared array sensor; s2, a makeup action interference filtering step: in combination with a preset makeup action signal model, filtering interference signals generated by makeup actions in the action information by utilizing a mode recognition algorithm; s3, only keeping the action signal which is greatly different from the makeup action signal, and entering S3; and S3, instruction judgment and confirmation: analyzing the processed action signal through the microprocessor. According to the technical scheme, the problem of misrecognition can be effectively solved, the contradiction between the interaction distance and the user experience is balanced, and stable and efficient in-vehicle make-up interaction experience is provided for the user.

Description

Technical Field

[0001] The present invention relates to the field of lighting control, and particularly to an interaction method for the makeup function inside a cockpit. Background Art

[0002] In the wave of the intelligent development of modern vehicles, the makeup function inside the cockpit has gradually become the focus of attention of many users. To improve the convenience of users' makeup in the vehicle, non-contact interaction technology has been introduced into the cockpit makeup mirror system, hoping to bring a more hygienic and intelligent experience to users. However, in actual application scenarios, this technology exposes many problems that cannot be ignored.

[0003] When users apply makeup in the vehicle, their hands need to move frequently to complete makeup actions such as smearing and grooming. During this process, existing non-contact command recognition systems often misjudge these normal makeup actions as operation commands. For example, when a user holds a cosmetic near the makeup mirror to smear, the system may wrongly think that the user has issued an instruction to adjust the light brightness or switch the makeup mode, resulting in abnormal changes in the makeup mirror function and seriously interfering with the user's makeup process.

[0004] The non-contact command input method itself has a distance contradiction problem. If the interaction distance between the user and the makeup mirror is too close, although the misrecognition probability can be reduced to a certain extent, it will greatly limit the user's operation space, resulting in the user being unable to comfortably perform makeup operations and the user experience dropping sharply. On the contrary, if the interaction distance is too far, although the user's operation space is guaranteed, due to the complexity of the cockpit environment, infrared signals are easily reflected, blocked by in-vehicle objects such as seats and instrument panels, and interfered by electromagnetic interference from other electronic devices, making the accuracy of the signals greatly reduced, and thus misrecognition phenomena frequently occur.

[0005] It should be particularly noted that when users use the makeup function inside the cockpit, in order to observe facial details more clearly, they often get very close to the makeup mirror, and this behavior further exacerbates the misrecognition situation. At close range, the signals generated by the user's makeup actions are more easily captured and misinterpreted by the system, and at this time, even a small environmental interference may have a greater impact on the signal accuracy, making the stability and reliability of the non-contact interaction technology face severe challenges.

[0006] The current non-contact interaction technology for the makeup function inside the cockpit has obvious defects in distinguishing between users' makeup actions and operation commands, and controlling the interaction distance, etc. There is an urgent need for an innovative interaction method to effectively solve the misrecognition problem, balance the contradiction between the interaction distance and the user experience, and provide a stable and efficient in-vehicle makeup interaction experience for users. Summary of the Invention

[0007] The present invention provides an interaction method for the in-cockpit makeup function, which can effectively solve the problem of misrecognition, balance the contradiction between the interaction distance and the user experience, and provide users with a stable and efficient in-vehicle makeup interaction experience.

[0008] To solve the above technical problems, the present application provides the following technical solutions: An interaction method for the in-cockpit makeup function, comprising: S1 Environment and user information collection step: Real-time collection of ambient light intensity information and in-cockpit temperature information through a light sensor and a temperature sensor; meanwhile, obtaining the user's motion information through an infrared array sensor; S2 Makeup motion interference filtering step: Combining a preset makeup motion signal model, using a pattern recognition algorithm to filter out the interference signals generated by the makeup motion in the motion information; only retaining the motion signals with significant differences in characteristics from the makeup motion signal, and entering S3; S3 Instruction judgment and confirmation: Analyzing the processed motion signal through a microprocessor, based on the intensity, change frequency, and duration characteristics of the motion signal, combined with a preset operation instruction rule library, if the motion signal characteristics exactly match the preset operation instructions and are within a reasonable operation distance range, which is determined by an infrared ranging algorithm for example, then confirm that the signal is a valid operation instruction and transfer to S4; if there are multiple matching items between the motion signal and the preset operation instructions, return to S2, and count the returned motions. After the count exceeds 2 times, combined with the ambient light intensity information and the in-cockpit temperature information, select the reasonable matching item as the valid operation instruction according to a preset rationality judgment model among the multiple matching items; if there is no reasonable matching item, return to S1 and remind the user; S4 Function execution and feedback: According to the confirmed operation instruction, execute the corresponding makeup function operation to adjust the light brightness and color temperature of the makeup mirror.

[0009] Further, in S2, when constructing the makeup motion signal model, first collect the infrared sensor signal data generated by the hand motions during the normal makeup process; compare the real-time collected motion information with the makeup motion signal model; if the amplitude change of the signal is within the common amplitude range of the makeup motion, and the frequency distribution is similar to the frequency characteristics of the makeup motion signal, and the duration conforms to the general duration of the makeup motion at the same time, then determine that the signal is an interference signal generated by the makeup motion and filter it out.

[0010] Further, when using the pattern recognition algorithm for comparison in S2, the dynamic time warping algorithm is used to calculate the real-time motion signal sequence and the template sequence of the preset makeup motion signal model similarity, and the calculation formula is as follows: Among them, is and the optimal alignment path between, satisfying , and represents the th point of aligned with the th point of and and When, determine that the signal is an interference signal generated by a makeup action and filter it, where is the dynamic time warping threshold, , are the amplitude change ratio thresholds, , are the frequency peak ratio thresholds.

[0011] Furthermore, the makeup action signal model constructed in S2 uses a Gaussian mixture model for probability density estimation, and the model parameters are obtained through iterative optimization by the expectation maximization algorithm; for the action signal feature vector collected in real time, the probability that it belongs to a makeup action interference signal is: Among them, is the number of mixture components, is the weight of the th component, is the th Gaussian distribution, with a mean of and a covariance matrix of ; when , determine that the signal is an interference signal generated by a makeup action and filter it, where is the preset probability threshold; the parameter update formula of the Gaussian mixture model is: Among them, is the latent variable, representing the posterior probability that the data point belongs to the th component, and the calculation formula is:

[0012] Furthermore, when the microprocessor analyzes the processed action signal in S3, a Bayesian decision-making model with multi-feature fusion is used for instruction judgment, and the calculation formula is as follows: Among them, is a set of preset operation instruction categories, is the fused feature vector, including the intensity, change frequency, duration of the action signal, and the operation distance determined by the infrared ranging algorithm; the conditional probability is estimated by a Gaussian kernel function: Among them, is the number of training samples of category , is the th training sample of category , is the covariance matrix of category ; the prior probability is dynamically adjusted according to the historical usage frequency of the operation instruction: Among them, is the smoothing parameter, is the instruction 's historical execution times; when and , it is confirmed that the signal is a valid operation instruction of category , where: is the Bayesian decision threshold is the optimal operation distance of the instruction , is the standard deviation of the operation distance of the instruction , is the distance tolerance coefficient, and its value range is .

[0013] Furthermore, the rationality judgment model of the multiple matching items adopts a hierarchical decision tree structure, and combines the environmental context and the user's historical behavior for multi-dimensional scoring. The specific steps are as follows: Step 1: Construct an environmental feature vector including the environmental light intensity and the cockpit temperature , and perform normalization processing: Among them, and are the historical observation ranges of the light intensity and the temperature respectively.

[0014] Step 2: Based on the user's historical operation data, construct an instruction Preference probability distribution in the environment , estimated by Gaussian process regression: where is the mean function is the covariance function, obtained by training with historical data . represents the frequency of occurrence of instruction in the environment .

[0015] Step 3: Calculate the environmental adaptability score for each candidate instruction : where: is the optimal environmental feature vector of instruction . is the environmental sensitivity diagonal matrix, dynamically updated by the following formula: is the learning rate, with a value range . denotes taking the diagonal elements of the matrix to form a diagonal matrix, ensuring that is always a positive definite diagonal matrix.

[0016] Step 4: Combine Bayesian probability , historical preference and environmental adaptability to calculate the final rationality score: where is the weight coefficient, dynamically adjusted by the following formula: is the learning rate is the loss function based on historical decision accuracy.

[0017] Step 5: Select the instruction with the highest final score as the valid operation instruction: When , return to S1 and prompt the user, where is the rationality threshold.

[0018] Further, it also includes S5, which collects the user's performance after adjusting the light brightness and color temperature of the makeup mirror. If the user's performance does not meet the expectation, the operation of adjusting the light brightness and color temperature of the makeup mirror is rolled back; if the user meets the expectation, it ends.

[0019] Further, the user's performance is collected by the cockpit microphone array and the infrared array sensor.

[0020] Further, in S5, the evaluation of the user's performance adopts a multi-modal emotion recognition model, specifically including: Step 1: Synchronously collect the user's voice signal and the action posture sequence : Voice signal: collected by the cockpit microphone array, with a sampling rate of Action posture: obtained by the infrared array sensor in S1, with a frame rate of Step 2: Construct a voice vector containing prosodic, acoustic, and semantic features: Among them: Prosodic feature: fundamental frequency change rate and standard deviation of pitch length : Acoustic feature: dynamic change amount of Mel frequency cepstral coefficients: Semantic feature: tone intensity index obtained by emotion dictionary matching : Step 3: Calculate the motion change rate of key joint points through the infrared array sensor: Among them: is the three-dimensional coordinate of the th joint point at the moment is the time window, with a value of represents the Euclidean norm Step 4: Construct a Bayesian fusion model to calculate the probability of user dissatisfaction: Among them, the likelihood function is represented by a Gaussian mixture model: is the fused feature vector.

[0021] Step 5: Adaptive adjustment of the decision threshold Dynamically adjust the dissatisfaction threshold according to historical interaction data : where: is the actual dissatisfaction label of the th historical interaction is the size of the historical data window, taking the last 50 interactions is the learning rate, with a value of 0.05 Step 6: When the following conditions are simultaneously met, it is determined that the user is dissatisfied and a rollback operation is performed: The dissatisfaction probability output by the multi-modal sentiment fusion model exceeds the threshold: If the action change rate exceeds the preset threshold: Then roll back to the previous operation with the first adjustment step: If the action change rate does not exceed the threshold: Then roll back to the previous operation with the second adjustment step and activate the voice soothing module: where, , , , is the action change rate threshold, and the value range is .

[0022] The principle and beneficial effects of the solution are as follows: Through the relevant data of the infrared array sensor, it is converted into feature parameters such as the coordinate trajectory of the hand movement and the signal strength change rate. Based on the pre-collected normal makeup action data (such as brush application, powder puff pressing, etc.), a interference signal feature space containing features such as action amplitude, frequency, and duration is constructed. The real-time action signal needs to be compared with this feature space in multiple dimensions. If the signal feature falls within the preset makeup action feature range, it is determined as an interference signal and filtered. Compared with first identifying the instruction and then making interference judgments, the present invention first identifies interference, which can effectively solve the prior interference problem, reduce the computational amount of the judgment logic, and reduce the response time.

[0023] The microprocessor extracts eigenvalue from the filtered action signal in three dimensions: intensity, frequency, and duration, and matches them with a preset operation instruction rule base (for example, "brighten the light" corresponds to intensity > 1.5V, frequency < 3Hz, and duration 1 - 2 seconds). Calculate the operating distance between the user and the dressing mirror through an infrared ranging algorithm (such as triangulation). Only when the signal features match and the distance is within a reasonable range (such as 30 - 50cm), confirm the validity of the instruction to avoid false triggering of close - range makeup actions.

[0024] When there are multiple possible instructions, through a counting mechanism (such as triggered after 2 consecutive matching failures) combined with environmental information (light, temperature), based on a preset rationality judgment model (such as a rule engine or heuristic algorithm), screen the most likely instruction to reduce misjudgment in ambiguous scenarios (or rather, non - recognition, improving the user experience).

[0025] After confirming the valid instruction, the microprocessor adjusts the LED light brightness (0 - 1000lux) and color temperature (2700K - 6500K) of the dressing mirror through a hardware interface (such as a PWM signal) to achieve real - time feedback. If there is no reasonable match, return to the initial state and remind the user to re - enter through methods such as light flashing and voice prompts to ensure the robustness of the interaction process.

[0026] The present invention effectively filters out interference signals generated by normal makeup actions of the hand (such as holding a mascara brush) through preset makeup action feature comparison, reducing the false triggering rate of non - contact instructions. Combining three - dimensional feature matching of action intensity, frequency, and duration, the accuracy rate is improved compared to traditional single - threshold judgment (such as only relying on signal intensity). It performs better especially in complex lighting (such as backlighting, tunnel scenarios) and multi - person cockpit environments. The distance constraint mechanism further excludes long - distance interference (such as accidental touch by passengers).

[0027] The present invention only relies on low - cost hardware such as infrared arrays, light / temperature sensors, etc., without the need to deploy additional cameras or depth sensors, reducing the hardware cost. In summary, the present invention effectively solves the problem of mis - recognition, balances the contradiction between the interaction distance and the user experience, and provides a stable and efficient in - vehicle makeup interaction experience for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of an embodiment of an interaction method for the makeup function in a cockpit. DETAILED DESCRIPTION OF THE INVENTION

[0029] The following is a further detailed description through specific embodiments: An interaction method for the makeup function in a cockpit (as Figure 1 shown) includes: S1 Environment and User Information Collection Step: Real-time collect ambient light intensity information and cabin temperature information through a light sensor and a temperature sensor; at the same time, obtain the user's motion information through an infrared array sensor; S2 Makeup Action Interference Filtering Step: Combine a preset makeup action signal model and use a pattern recognition algorithm to filter out the interference signals generated by makeup actions in the motion information; only retain the motion signals with significant differences in characteristics from the makeup action signals and enter S3; In S2, when constructing the makeup action signal model, pre-collect the infrared sensor signal data generated by the hand movements during normal makeup; Compare the real-time collected motion information with the makeup action signal model; If the amplitude change of the signal is within the common amplitude range of makeup actions, and the frequency distribution is similar to the frequency characteristics of the makeup action signal, and the duration meets the general duration of makeup actions, then determine that the signal is an interference signal generated by makeup actions and filter it out.

[0030] When using the pattern recognition algorithm for comparison in S2, use the dynamic time warping algorithm to calculate the real-time motion signal sequence and the preset makeup action signal model template sequence for similarity, and the calculation formula is as follows: where, is and the optimal alignment path between, satisfying , and represents the th point of aligned with the th point of ; when and and , ,

[0031] where, is the number of mixture components, is the The weight of the component is the th Gaussian distribution with a mean of and a covariance matrix of ; when , it is determined that the signal is an interference signal generated by a makeup action and is filtered, where is a preset probability threshold; the parameter update formula of the Gaussian mixture model is: where is a latent variable representing the posterior probability that the data point belongs to the th component, and the calculation formula is:

[0032] S3 instruction judgment and confirmation: The microprocessor analyzes the processed action signal. According to the intensity, change frequency, and duration characteristics of the action signal, combined with the preset operation instruction rule library, if the action signal characteristics exactly match the preset operation instruction and are within a reasonable operation distance range (the operation distance is determined by the infrared ranging algorithm), it is confirmed that the signal is a valid operation instruction and proceeds to S4; if there are multiple matching items between the action signal and the preset operation instruction, return to S2, and count the returned actions. After the count exceeds 2 times, combined with the ambient light intensity information and the cabin temperature information, select the reasonable matching item as the valid operation instruction according to the preset rationality judgment model among the multiple matching items; if there is no reasonable matching item, return to S1 and remind the user; When the microprocessor analyzes the processed action signal in S3, a Bayesian decision model with multi-feature fusion is used for instruction judgment, and the calculation formula is as follows: where is the set of preset operation instruction categories, is the fused feature vector, including the intensity, change frequency, duration of the action signal, and the operation distance determined by the infrared ranging algorithm; the conditional probability is estimated through a Gaussian kernel function: where is the number of training samples of category , is the th training sample of category ​ For the covariance matrix of the category ; The prior probability is dynamically adjusted according to the historical usage frequency of the operation instruction: Wherein, is the smoothing parameter, is the instruction The historical execution times; When and At this time, confirm that the signal is a valid operation instruction of the category , where: is the Bayesian decision threshold is the instruction The optimal operation distance is the instruction The standard deviation of the operation distance is the distance tolerance coefficient, and the value range is .

[0033] The rationality judgment model of the multiple matching items adopts a hierarchical decision tree structure, and performs multi-dimensional scoring by combining the environmental context and the user's historical behavior. The specific steps are as follows: Step 1: Construct an environmental feature vector containing the environmental light intensity and the cockpit temperature , and perform normalization processing: Wherein, and are the historical observation ranges of the light intensity and the temperature respectively.

[0034] Step 2: Based on the user's historical operation data, construct the preference probability distribution of the instruction in the environment , estimated by Gaussian process regression: Wherein, is the mean function, is the covariance function, obtained by training with the historical data , represents the occurrence frequency of the instruction in the environment .

[0035] Step 3: Calculate the environmental adaptability score of each candidate instruction :​ Wherein: is the instruction optimal environmental feature vector is the environmental sensitivity diagonal matrix, which is dynamically updated by the following formula: is the learning rate, and the value range is , denotes taking the diagonal elements of the matrix to form a diagonal matrix, ensuring that is always a positive definite diagonal matrix.

[0036] Step 4: Synthesize Bayesian probability , historical preference and environmental adaptability , and calculate the final rationality score: Wherein, is the weight coefficient, which is dynamically adjusted by the following formula: is the learning rate, is the loss function based on the historical decision accuracy.

[0037] Step 5: Select the instruction with the highest final score as the effective operation instruction: When , return to S1 and prompt the user, where is the rationality threshold.

[0038] S4 Function Execution and Feedback: According to the confirmed operation instruction, execute the corresponding makeup function operation to adjust the light brightness and color temperature of the makeup mirror; S5, collect the user's performance after adjusting the light brightness and color temperature of the makeup mirror. If the user's performance does not meet the expectation, return to the operation of adjusting the light brightness and color temperature of the makeup mirror; if the user meets the expectation, end.

[0039] The evaluation of the user's performance in S5 adopts a multi-modal emotion recognition model, specifically including: Step 1: Synchronously collect the user's voice signal and the action posture sequence : Voice signal: Collected through the cockpit microphone array, and the sampling rate is Action gesture: Obtained by the infrared array sensor in S1, with a frame rate of Step 2: Construct a voice vector containing prosodic, acoustic, and semantic features: Among them: Prosodic feature: Fundamental frequency change rate , Standard deviation of pitch length : Acoustic feature: Dynamic change amount of Mel frequency cepstral coefficients: Semantic feature: Tone intensity index obtained by emotion dictionary matching : Step 3: Calculate the motion change rate of key joint points through the infrared array sensor: Among them: is the th joint point at moment's three-dimensional coordinates is the time window, with a value of represents the Euclidean norm Step 4: Construct a Bayesian fusion model to calculate the probability of user dissatisfaction: Among them, the likelihood function is represented by a Gaussian mixture model: is the fusion feature vector.

[0040] Step 5: Adaptive adjustment of the decision threshold Dynamically adjust the dissatisfaction threshold according to historical interaction data : Among them: is the th actual dissatisfaction label of historical interaction is the historical data window size, taking the last 50 interactions is the learning rate, with a value of 0.05 Step 6: When the following conditions are met simultaneously, it is determined that the user is dissatisfied and a return operation is performed: The dissatisfaction probability output by the multi-modal emotion fusion model exceeds the threshold: If the action change rate exceeds the preset threshold: Then return to the previous operation with the first adjustment step size: If the action change rate does not exceed the threshold: Then return to the previous operation with the second adjustment step size and activate the voice soothing module: Among them, , , , is the action change rate threshold, and the value range is .

[0041] Specifically in use: The light sensor uses a photoresistor (such as GL5528) to collect the ambient light intensity in real time, with a detection range of 100 - 10000 lux and an accuracy of ±5%. The temperature sensor uses a digital temperature sensor (such as DS18B20) to collect the cockpit temperature, with a range of -20°C - 85°C and an accuracy of ±0.5°C. The infrared array sensor deploys an 8×8 infrared emission and reception pair of tubes (such as Sharp GP2Y0A21YK0F), covering a 50cm×50cm area in front of the vanity mirror, and detecting the two-dimensional coordinates and signal intensity of hand movements.

[0042] The light and temperature data are updated 10 times per second, and the infrared action data are collected 30 frames per second to form an action signal sequence , where each sample contains features such as infrared signal intensity and coordinate offset. When the user enters the vehicle and turns on the vanity mirror, the sensor is automatically activated to collect environmental data (such as ambient light intensity , temperature °C ), and at the same time, hand movements (such as holding a mascara and moving it horizontally) are captured in real time.

[0043] Pre-collect 1000 groups of typical makeup action data, including: Brushing and applying: The amplitude voltage fluctuation is 0.5 - 1.2V, the frequency is 8 - 15Hz, and the duration is 1 - 3 seconds; Holding a powder puff and pressing: The amplitude voltage fluctuation is 0.7 - 1.8V, the frequency is 4 - 8Hz, and the duration is 1 - 4 seconds.

[0044] Preset template sequence Set for the brush-holding action feature , amplitude ratio threshold 、 , frequency threshold 、 。

[0045] Set a number of mixed components, initial mean (corresponding to amplitude, frequency, duration), iteratively optimized by the EM algorithm, preset probability threshold 。Real-time action signal The DTW distance is 0.6 (<0.8), the amplitude change ratio is 1.05 (between 0.9 - 1.1), and the frequency peak ratio is 1.1 (between 0.8 - 1.2), then it is determined as makeup action interference and filtered; if the signal feature exceeds the threshold range (such as frequency 25Hz), it is retained as a potential operation instruction.

[0046] Preset instructions include 。

[0047] The feature vector is , for example, the typical features of the "brighten the light" instruction are: intensity 1.5V, frequency 3Hz, duration 1.2 seconds, distance 35cm (optimal distance , standard deviation , distance tolerance coefficient ).

[0048] Probability calculation is to estimate the conditional probability through the Gaussian kernel function , prior probability Dynamically adjusted according to historical data (such as the historical execution times of "brighten the light" , smoothing parameter , then ).

[0049] In the processing of multiple matches, for the current environment (historical range 100 - 1000 lux), after normalization 。

[0050] Then obtain the environmental adaptability score, for example, the optimal environment for the instruction "brighten the light" , calculate the Mahalanobis distance , environmental adaptability score 。

[0051] Obtain the final score, combining the Bayesian probability (0.6), historical preference (0.7), environmental adaptability (0.85), weights , get , it is determined as a valid instruction.

[0052] After confirming the "brighten the light" instruction, drive the backlight LED of the vanity mirror (such as Cree XQ-E series) to increase the brightness from 200 lux to 300 lux, and feedback the operation result through the indicator light flashing. Infrared ranging confirms that the operation distance is 35 cm (within the range), meeting the instruction execution conditions.

[0053] Then collect the user's voice through the microphone, and the fundamental frequency change rate (normal range <0.5), the change amount of Mel-frequency cepstral coefficients , and the tone intensity index (neutral), it is determined as satisfactory.

[0054] At the same time, detect the joint point movement change rate through the infrared sensor (< threshold 0.5 m / s), it is determined as stable.

[0055] At this time, the user has no obvious dissatisfaction (such as continuing the makeup action), and save the current light parameters (brightness 300 lux, color temperature 5500K) to the user preference library.

[0056] If the user makes a complaining sound ( ) and the action is violent ( ), then step back the brightness to 250 lux with a step size of and start voice soothing: "The light brightness has been finely adjusted for you. Do you need further adjustment?"

[0057] In other embodiments, collect the cockpit environment parameters in real time through the light sensor (photoresistor / photodiode) and the temperature sensor (digital / thermocouple), and construct the environmental feature vector , where is the light intensity, is the temperature. Use an infrared array sensor (such as Sharp GP2Y0A21YK0F) to emit infrared signals and receive the reflected light intensity, and calculate the hand position coordinates through the triangulation ranging principle , forming a three-dimensional action trajectory sequence .

[0058] The data of each sensor is synchronized to the microprocessor (such as STM32F4 series) through the SPI / I2C bus, and the sampling frequency is 30 Hz to ensure the time consistency of the action and the environmental data.

[0059] Pre-collect more than 1000 hours of typical makeup action data (such as smearing, pressing, tapping), and extract the time-domain features (amplitude change rate ), frequency domain features (Power Spectral Density PSD) and time-frequency features (wavelet transform coefficients), to construct a Gaussian mixture model , where are model parameters.

[0060] For the real-time action sequence and the template sequence calculate the DTW distance , when and the feature similarity (e.g., , ), it is determined as a makeup interference signal.

[0061] Input the action feature vector (intensity, frequency, duration, distance) into the Bayesian classifier to calculate the posterior probability , where the class conditional probability is modeled by kernel density estimation (KDE), and the prior probability is dynamically updated according to the historical instruction frequency.

[0062] When there are multiple matching items, construct an environment-instruction association matrix , calculate the environmental adaptability score through a hierarchical decision tree , combine the action feature score and the historical preference score , and finally select the instruction with the highest comprehensive score .

[0063] The microprocessor controls the LED driver circuit (such as TI TPS61165) through a PWM signal to achieve linear adjustment of the brightness from 0 to 1000 lux (accuracy ±5%) and adjustment of the color temperature from 2700K to 6500K (step size 100K).

[0064] Detect the action stability through an infrared sensor (standard deviation of joint displacement ) and collect the speech emotion through a microphone array (fundamental frequency change rate ), and trigger adaptive adjustment when or .

[0065] In this embodiment, through the GMM-DTW hybrid model, the misrecognition rate of makeup actions is reduced from 32% of the traditional method to less than 5% (test data: only 52 false triggers in 1000 makeup operations), significantly reducing non-intentional instruction interference.

[0066] Environmental context fusion decision-making improves the instruction accuracy rate by 28% and 41% respectively in strong light (>5000 lux) and weak light (<200 lux) environments, and reduces the impact of temperature fluctuations (-10°C - 40°C) to within 3%.

[0067] Under strong noon light, the accuracy rate of the "dim the lights" instruction is improved; in a low-temperature environment in winter, the response speed of the "switch to warm color temperature" instruction is improved. The multi-matching item processing mechanism reduces the average number of instruction confirmations and shortens the single operation time. The time for new users to reach the proficient use state is shortened. The stability of the dynamic threshold adaptive mechanism is improved in scenarios of different users (with a ±40% difference in hand movement amplitude) and different vehicle models (with a ±15 cm difference in operating distance). It can effectively filter out the interference of vehicle vibration (frequency 1 - 10 Hz) and slight passenger shaking (amplitude <5 cm).

[0068] The present invention provides a high-precision and low-latency human-computer interaction solution for intelligent cockpits, which is particularly suitable for high-frequency operation scenarios such as makeup mirrors and central control screens. Personalized lighting adjustment improves the satisfaction with makeup. Through multi-sensor fusion, intelligent interference filtering, and environmental context perception, this embodiment significantly improves the interaction accuracy, robustness, and user experience of the cockpit makeup function, and has broad market application prospects.

[0069] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this embodiment. Common knowledge such as specific structures and characteristics known in the art is not described in detail here. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can know all the existing technologies in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can still be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be based on the content of its claims, and the specific implementation manners and the like recorded in the specification can be used to interpret the content of the claims.

Claims

1. An interaction method for the in-cockpit makeup function, comprising, characterized in that: S1 Environment and user information collection step: Real-time collect the ambient light intensity information and the in-cockpit temperature information through a light sensor and a temperature sensor; meanwhile, obtain the user's motion information through an infrared array sensor; S2 Makeup action interference filtering step: Combine a preset makeup action signal model and use a pattern recognition algorithm to filter out the interference signals generated by makeup actions in the motion information; only retain the motion signals with significant differences in characteristics from the makeup action signals, and enter S3; S3 Instruction judgment and confirmation: Analyze the processed motion signals through a microprocessor. According to the intensity, change frequency, and duration characteristics of the motion signals, combined with a preset operation instruction rule library, if the characteristics of the motion signals exactly match the preset operation instructions and are within a reasonable operation distance range, which is determined by an infrared ranging algorithm for example, then confirm that this signal is a valid operation instruction and transfer to S4; if there are multiple matching items between the motion signals and the preset operation instructions, then return to S2, and count the returned actions. After the count exceeds 2 times, combined with the ambient light intensity information and the in-cockpit temperature information, according to a preset rationality judgment model, select the reasonable matching item as the valid operation instruction; If there is no reasonable matching item, then return to S1 and remind the user; S4 Function execution and feedback: Execute the corresponding makeup function operation according to the confirmed operation instruction, and adjust the light brightness and color temperature of the makeup mirror.

2. The interactive method for the in-cockpit makeup function according to claim 1, wherein In S2, when constructing the makeup action signal model, first collect the infrared sensor signal data generated by the hand actions during the normal makeup process; compare the real-time collected motion information with the makeup action signal model; if the amplitude change of the signal is within the common amplitude range of makeup actions, and the frequency distribution is similar to the frequency characteristics of the makeup action signals, and the duration conforms to the general duration of makeup actions, then determine that this signal is an interference signal generated by makeup actions and filter it out.

3. The interactive method for the in-cockpit makeup function according to claim 2, characterized in that, When using the pattern recognition algorithm for comparison in S2, the dynamic time warping algorithm is used to calculate the real-time action signal sequence and the preset makeup action signal model template sequence of the similarity, and the calculation formula is as follows: ； Among them, is and the optimal alignment path between, satisfying , and represents the th point of aligned with the th point of ; when and and is the dynamic time warping threshold, , is the amplitude change ratio threshold, , is the frequency peak ratio threshold.

4. An interaction method for the in-cockpit makeup function according to claim 3, characterized in that The makeup action signal model constructed in S2 uses a Gaussian mixture model for probability density estimation, and the model parameters are obtained through iterative optimization by the expectation maximization algorithm; for the action signal feature vector collected in real time , the probability that it belongs to the makeup action interference signal is as follows: ； Among them, is the number of mixed components, is the weight of the -th component, is the -th Gaussian distribution with a mean of and a covariance matrix of ; when , it is determined that the signal is an interference signal generated by a makeup action and is filtered, where is a preset probability threshold; the parameter update formula of the Gaussian mixture model is: ； ； ； Among them, is a latent variable, representing the posterior probability that the data point belongs to the th component, and the calculation formula is: 。 5. The interactive method for the in-cockpit makeup function according to claim 4, wherein When the microprocessor in S3 analyzes the processed motion signals, a Bayesian decision model with multi-feature fusion is used for instruction judgment, and the calculation formula is as follows: ； Among them, is a set of preset operation instruction categories, is the fused feature vector, including the intensity, change frequency, duration of the action signal, and the operation distance determined by the infrared ranging algorithm; the conditional probability is estimated by the Gaussian kernel function: ； Among them, is the number of training samples of class , is the -th training sample of class , is the covariance matrix of class ; The prior probability is dynamically adjusted according to the historical usage frequency of the operation instruction: ； Among them, is the smoothing parameter, is the historical execution times of the instruction ; when and , confirm that the signal is a valid operation instruction of category , where: is the Bayesian decision threshold; For the instruction Optimal operating distance; is the standard deviation of the operating distance for the instruction is the distance tolerance coefficient, and its value range is .

6. The interactive method for the in-cockpit makeup function according to claim 5, wherein The rationality judgment model for the multiple matching items adopts a hierarchical decision tree structure and conducts multi-dimensional scoring by combining the environmental context and the user's historical behavior. The specific steps are as follows: Step 1: Construct an environmental feature vector including the ambient light intensity and the cockpit temperature , and perform normalization: ； Among them, and are the historical observation ranges of light intensity and temperature respectively; Step 2: Construct an instruction based on the user's historical operation data in the environment preference probability distribution , estimated by Gaussian process regression: ； Among them, is the mean function, is the covariance function, which is obtained by training with historical data and represents the instruction in the environment with the occurrence frequency; Step 3: Calculate the environmental adaptability score of each candidate instruction : ； Wherein: For the instruction Optimal environmental feature vector is the environmental sensitivity diagonal matrix, which is dynamically updated by the following formula: ； is the learning rate, and the value range is , denotes taking the diagonal elements of the matrix to form a diagonal matrix, ensuring that is always a positive definite diagonal matrix; Step 4: Comprehensive Bayesian Probability , historical preferences and environmental adaptability , and calculate the final rationality score: ； Among them, is a weight coefficient, which is dynamically adjusted by the following formula: ； is the learning rate, is the loss function based on the historical decision accuracy; Step 5: Select the instruction with the highest final score as the valid operation instruction: ； When returns S1 and prompts the user, where is the rationality threshold value.

7. An interactive method for the in-cockpit makeup function according to claim 6, characterized in that, It also includes S5, collect the user's performance after adjusting the light brightness and color temperature of the makeup mirror. If the user's performance does not meet the expectations, then roll back the operation of adjusting the light brightness and color temperature of the makeup mirror; if the user meets the expectations, then end.

8. The interactive method for the in-cockpit makeup function according to claim 7, characterized in that, The user's performance is collected through a cockpit microphone array and an infrared array sensor.

9. The interactive method for the in-cockpit makeup function according to claim 8, wherein In S5, the evaluation of the user's performance adopts a multi-modal emotion recognition model, specifically including: Step 1: Synchronously collect the user's voice signal and the action posture sequence : The sound signal is collected by the cockpit microphone array with a sampling rate of ; The motion posture is obtained by the infrared array sensor in S1, and the frame rate is ; Step 2: Construct a voice vector containing prosodic, acoustic, and semantic features: ； Wherein: Prosodic features: fundamental frequency change rate , standard deviation of pitch length : ； Acoustic feature: The dynamic change amount of the Mel frequency cepstral coefficients: ； Semantic feature: Tone intensity index obtained by matching with an emotion dictionary : ； Step 3: Calculate the motion change rate of key joint points through an infrared array sensor: ； Wherein: is the three-dimensional coordinate of the th joint point at moment; is a time window, with a value of ; represents the Euclidean norm; Step 4: Construct a Bayesian fusion model to calculate the probability of user dissatisfaction: ； Among them, the likelihood function is represented by a Gaussian mixture model: ； is the fused feature vector; Step 5: Adaptive adjustment of decision threshold Dynamically adjust the dissatisfaction threshold according to historical interaction data : ； Where: is the actual dissatisfaction label for the th historical interaction; is the historical data window size, taking the most recent 50 interactions; is the learning rate, with a value of 0.05; Step 6: When the following conditions are simultaneously satisfied, determine that the user is dissatisfied and perform a rollback operation: The dissatisfaction probability output by the multi-modal sentiment fusion model exceeds the threshold: ； If the action change rate exceeds the preset threshold: ； Then roll back to the previous operation with the first adjustment step size: ； If the action change rate does not exceed the threshold: ； Then roll back to the previous operation with the second adjustment step size and start the voice soothing module: ； Among them, , , , is the action change rate threshold, and its value range is .

10. An interactive system for the makeup function in the cockpit, characterized in that, The method described in any one of claims 1-9 is adopted.

Images