Embedded man-machine interaction optimization method, system and device

Abstract

The invention discloses an embedded man-machine interaction optimization method, system and device, and relates to the technical field of man-machine interaction, the system comprises an information acquisition unit, a central processing unit, an edge calculation unit and a feedback optimization unit, the information acquisition module is used for acquiring a multi-modal interaction instruction to obtain an abnormal score of man-machine interaction data, and the abnormal score is used for calculating the abnormal score of the man-machine interaction data; the system improves the integrity and accuracy of data monitoring, evaluates the comprehensive driving state through a central processing module, and generates an interaction optimization strategy through an edge calculation unit by using deep reinforcement learning, thereby dynamically adjusting the interaction logic in real time according to the driver state and road conditions, providing safety reminding and auxiliary functions, and improving the driving safety. The driving safety and reliability are remarkably improved, personalized interaction experience is achieved through the feedback optimization module, meanwhile, quick response and global optimization capacity of the system are ensured through application of edge calculation, and efficient, intelligent and personalized embedded man-machine interaction is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of human - machine interaction, and in particular, to an optimized method, system and device for embedded human - machine interaction. Background Art

[0002] With the rapid development of fields such as the Internet of Things, smart home, and industrial automation, embedded systems are widely used in various devices. The enhancement of hardware performance supports complex human - machine interaction, and the application of multiple sensors makes the interaction methods more diverse.

[0003] However, existing human - machine interaction - related systems, such as in - vehicle interaction systems, have problems of single interaction method, slow response speed, insufficient personalization, and insufficient security during the human - machine interaction process. Due to the lack of integrity and accuracy in monitoring human - machine interaction data, it will lead to incomplete and insufficiently personalized overall interaction logic analysis of data fusion, thus affecting the response ability of the interaction system, and further causing the defects of decreased safety and reliability in the in - vehicle interaction process.

[0004] In view of the above - mentioned technical defects, a solution is proposed. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of single interaction method, slow response speed, insufficient personalization, and insufficient security existing in the traditional system during the human - machine interaction process, and to achieve efficient, intelligent and personalized embedded human - machine interaction, which not only improves the interaction experience of the driver, but also enhances the safety and comfort of driving.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solutions: An optimized method for embedded human - machine interaction includes the following steps: Step 1, collecting human - machine interaction data through an embedded in - vehicle interaction system: setting multi - source sensors to collect human - machine interaction data during driving in the embedded in - vehicle interaction system, where the human - machine interaction data includes image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters; Step 2, pre - processing the human - machine interaction data to obtain the anomaly score of the human - machine interaction data: respectively obtaining the image instruction anomaly score, voice instruction anomaly score, action instruction anomaly score, and position status anomaly score through the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters; Step 3, performing data fusion on the anomaly scores of the human - machine interaction data to generate a comprehensive driving state parameter and obtaining a comprehensive driving state score, so as to evaluate the comprehensive driving state of the human - machine interaction process; Step 4: Dynamically adjust the human-machine interaction logic based on the comprehensive driving state: Train the state space in which the comprehensive driving state is located, obtain the state vector and perform edge computing, so as to conduct driver fatigue monitoring and complex road condition detection; Step 5: Dynamically adjust the interaction logic in real time based on the driver's state and road conditions, so as to provide safety reminders and auxiliary functions.

[0007] Furthermore, the data processing process of human-machine interaction data is as follows: Preprocess the image instruction parameters to obtain the camera data acquisition function I(t); Preprocess the voice instruction parameters to obtain the microphone data acquisition function A(t); Preprocess the action instruction parameters to obtain the action data acquisition function D(t); Preprocess the position status parameters to obtain the GPS module acquisition function G(t); Then perform data fusion on the anomaly scores of human-machine interaction data. The specific process is as follows: Take the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters as the index items of human-machine interaction data respectively; Set up an LSTM prediction model to perform anomaly analysis on the index items, sequentially input the historical data of the index items of human-machine interaction data, and perform time series analysis on the input data to obtain the prediction output : ; where, is the input data, is the prediction output, t is the time node, and T is the historical data period; Then, compare the prediction output with the actual output to obtain the anomaly scoring function Sx(t) of the index items of human-machine interaction data: ; where x refers to any one of the index items of human-machine interaction data.

[0008] Furthermore, the preprocessing process of the image instruction parameters is as follows: Mark the image instruction captured by the camera as I(t), where t represents the timestamp; Mark the camera data acquisition function as I(t)=CameraCapture(Pw, Ph, Pf, Pt), where Pw is the width pixel of the image, Ph is the height pixel of the image, Pf is the image frame rate, and Pt is the total duration of image acquisition; The function output is a four-dimensional array A, which is marked as the image instruction parameters.

[0009] Furthermore, the preprocessing process of the voice command parameters is as follows: Mark the voice command collected by the microphone as A(t); Mark the microphone data acquisition function as A(t) = MicrophoneCapture(Nr, Np, Nd, Nb, Nt), where Nr is the background noise, Np is the sound frequency, Nd is the sound amplitude, Nb is the sound decibel, and Nt is the total duration of sound acquisition; Standardize the microphone data acquisition function, output the voice command parameters, and mark them as (ψr, ψp, ψd, ψb, ψt). The standardized values of the parameters are: ; where Xi is the original value of the i-th parameter, μi is the mean of the i-th parameter, σi is the standard deviation of the i-th parameter, and ψi is the standardized parameter value.

[0010] Furthermore, the preprocessing process of the action command parameters is as follows: Mark the action command collected by the steering wheel sensor as D(t), and mark the action data acquisition function as: D(t) = Streering(Hsr, Hθ, Hv, Hf), where Hsr is the sampling rate, Hθ is the steering wheel rotation angle, Hv is the steering wheel rotation speed, and Hf is the force applied to the steering wheel; Standardize the action data acquisition function, output the action command parameters, and mark them as (ηsr, ηθ, ηv, ηf). The standardized values of the parameters are: ; where is the change rate of the j-th parameter, μj is the mean of the change rate of the j-th parameter, σj is the standard deviation of the change rate of the i-th parameter, and ηj is the standardized parameter value.

[0011] Furthermore, the preprocessing process of the position status parameters is as follows: Mark the data collected by the GPS module as G(t), and the GPS module acquisition function is: G(t) = GPSCapture(Pm, Mv, Ma, Mr); Standardize the GPS module acquisition function, output the position status parameters, and mark them as (Pm, Mv, Ma, Mr). The standardized values of the parameters are: ; where Mk is the original value of the k-th parameter, Mmin is the minimum value of the parameter, and Mmax is the maximum value of the parameter.

[0012] Furthermore, the specific process of evaluating the comprehensive driving state is as follows: Substitute the image instruction parameters, sound instruction parameters, action instruction parameters, and position status parameters into the LSTM prediction model in sequence to obtain the anomaly scoring function Sx(t) of the metric items of the human-computer interaction data; Mark the anomaly scoring functions Sx(t) of the image instruction parameters, sound instruction parameters, action instruction parameters, and position status parameters as the image instruction anomaly score S1, the sound instruction anomaly score S2, the action instruction anomaly score S3, and the position status anomaly score S4 respectively; Furthermore, integrate and generate the comprehensive driving state parameter DRIVE<S1, S2, S3, S4>, and obtain the comprehensive driving state score Scom.

[0013] Further, the specific process of the central processing module generating the interaction-optimized feature vector and then dynamically adjusting the interaction logic is as follows: The state space includes the driver's image instruction feature vector I(t), voice instruction feature vector A(t), action instruction feature vector D(t), and vehicle position status feature vector G(t). The state vector is expressed as: ; Obtain the fatigue monitoring status parameter Fatigue(t) through real-time monitoring, and set the fatigue monitoring status parameter threshold Threshold1. When it is detected that the driver is fatigued, automatically adjust the interface brightness and provide a voice reminder; when Fatigue(t) < Threshold1, it is regarded as the normal state; when Fatigue(t) > Threshold1, when the system determines that the driver is in a fatigued state, an alarm mechanism will be automatically triggered; Obtain the complex road condition status parameter ComplexRoad(t) through real-time monitoring, and set the complex road condition status parameter threshold Threshold2. When the GPS detects a complex road condition, simplify the interface information and provide navigation tips; when ComplexRoad(t) < Threshold2, it is regarded as the normal state; when ComplexRoad(t) > Threshold2, it is determined that the road condition exceeds the preset threshold, and the system will automatically trigger a multimodal alarm mechanism.

[0014] An embedded human-computer interaction optimization system includes an information collection unit, a central processor, an edge computing unit, and a feedback optimization unit. Among them, the information collection unit, the central processor, the edge computing unit, and the feedback optimization unit are communicatively connected, and this system executes the above-mentioned embedded human-computer interaction optimization method; Among them, the information collection unit is used to collect human-computer interaction data, and collect relevant data for human-computer interaction during driving through multi-source sensors. Among them, the human-computer interaction data includes image instruction parameters, sound instruction parameters, action instruction parameters, and position status parameters; The central processing unit includes a preprocessing module and a data fusion module. The central processing unit is used to preprocess and fuse human-computer interaction data. Among them, parameter preprocessing is performed through the preprocessing module to obtain the anomaly score of the parameters of the human-computer interaction data; the data fusion module combines the anomaly scores of the parameters to generate comprehensive driving state parameters and obtains the comprehensive driving state score. The edge computing unit is used to train the state space in which the comprehensive driving state dynamically locates, obtain the state vector and perform edge computing, so as to perform driver fatigue monitoring and complex road condition detection. The feedback optimization unit is used to receive the feedback data of the driver's state and road conditions, and dynamically adjust the interaction logic in real time, so as to provide safety reminders and auxiliary functions.

[0015] An embedded human-computer interaction optimization device, which includes the above-mentioned embedded human-computer interaction optimization system and applies the above-mentioned embedded human-computer interaction optimization method.

[0016] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows: The present invention uses the information collection module to collect multimodal interaction instructions. By integrating the image instructions, voice instructions, action instructions and position status data of the camera, microphone, steering wheel sensor and GPS module, the anomaly score of the human-computer interaction data is obtained, which improves the integrity and accuracy of data monitoring. The present invention preprocesses and fuses the human-computer interaction data through the central processing module, thereby evaluating the comprehensive driving state, and uses deep reinforcement learning through the edge computing unit to generate an interaction optimization strategy, so as to dynamically adjust the interaction logic in real time according to the driver's state and road conditions, and provide safety reminders and auxiliary functions, significantly improving driving safety and reliability. The present invention collects feedback data through the feedback optimization module and optimizes the human-computer interaction strategy, realizing a personalized interaction experience. At the same time, the application of edge computing ensures the fast response and global optimization ability of the system, realizing efficient, intelligent and personalized embedded human-computer interaction, which not only improves the driver's interaction experience, but also improves driving safety and comfort. Description of the Drawings

[0017] Figure 1 It shows a schematic diagram of the steps of the method flow of the present invention. Figure 2 It shows a schematic diagram of the connection of the system modules of the present invention. Detailed Embodiments

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0019] Embodiment 1: As Figure 1 - Figure 2 shown, an embedded human-computer interaction optimization method includes the following steps: S1, collect human-computer interaction data through an embedded vehicle-mounted interaction system: Set up multi-source sensors to collect human-computer interaction data during driving in the embedded vehicle-mounted interaction system. Among them, the human-computer interaction data includes image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters.

[0020] S2, preprocess the human-computer interaction data to obtain the anomaly score of the human-computer interaction data: Obtain the image instruction anomaly score, voice instruction anomaly score, action instruction anomaly score, and position status anomaly score through the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters respectively.

[0021] The data processing process of the human-computer interaction data is as follows: S2-1, preprocess the image instruction parameters to obtain the camera data acquisition function I(t); The preprocessing process of the image instruction parameters is as follows: Collect through a high-resolution camera installed in the cockpit to monitor and capture the facial expressions and posture states of the driver in real time. Mark the image instruction captured by the camera as I(t), where t represents the timestamp; Mark the camera data acquisition function as I(t)=CameraCapture(Pw, Ph, Pf, Pt), where Pw is the width pixel of the image, Ph is the height pixel of the image, Pf is the image frame rate, and Pt is the total duration of image acquisition; The function output is a four-dimensional array A, which is marked as the image instruction parameter, and its shape is (N, h, w, 3), where N is the total number of frames: N = Pf×Pt; Each element <Ai, a, b, c> of the array A represents the pixel value of the a-th row, b-th column, and c-th channel of the i-th frame image, where the range of i is from 1 to N, the range of a is from 1 to h, the range of b is from 1 to w, and the range of c is from 1 to 3, corresponding to the RGB three channels; Among them, each pixel value <Ai, a, b, c> is an integer between 0 and 255, that is, <Ai, a, b, c> ∈ {0, 1, 2,..., 255}; S2-2, preprocess the voice command parameters to obtain the microphone data acquisition function A(t); The preprocessing process of the voice command parameters is as follows: Collect the voice through the microphone installed in the cockpit, and perform noise reduction on the collected voice, so as to filter out the background noise in the vehicle, and mark the voice command collected by the microphone as A(t); Mark the microphone data acquisition function as A(t)=MicrophoneCapture(Nr,Np,Nd,Nb,Nt), where Nr is the background noise, Np is the sound frequency, Nd is the sound amplitude, Nb is the sound decibel, and Nt is the total duration of sound collection; Standardize the microphone data acquisition function, output the voice command parameters, and mark them as (ψr, ψp, ψd, ψb, ψt). The standardized values of the parameters are: ; where Xi is the original value of the i-th parameter, μi is the mean value of the i-th parameter, σi is the standard deviation of the i-th parameter, and ψi is the standardized parameter value.

[0022] S2-3, preprocess the action command parameters to obtain the action data acquisition function D(t); The preprocessing process of the action command parameters is as follows: Detect and collect through the steering wheel sensor at a sampling rate of not less than 100Hz, including the steering angle, speed and applied force of the steering wheel, so as to analyze the driver's operation habits and driving state, mark the action command collected by the steering wheel sensor as D(t), and mark the action data acquisition function as: D(t)=Streering(Hsr,Hθ,Hv,Hf), where Hsr is the sampling rate, Hθ is the steering angle of the steering wheel, Hv is the steering speed of the steering wheel, and Hf is the force applied to the steering wheel; Standardize the action data acquisition function, output the action command parameters, and mark them as (ηsr, ηθ, ηv, ηf). The standardized values of the parameters are: ; where, is the change rate of the j-th parameter, μj is the mean value of the change rate of the j-th parameter, σj is the standard deviation of the change rate of the i-th parameter, and ηj is the standardized parameter value.

[0023] S2-4, preprocess the position status parameters to obtain the GPS module acquisition function G(t); The preprocessing process of the position status parameters is as follows: Obtain the real-time position Pm and driving route of the vehicle through GPS module data acquisition, including speed Mv, acceleration Ma, and driving direction Mr. Mark the data collected by the GPS module as G(t). The GPS module acquisition function is: G(t) = GPSCapture(Pm, Mv, Ma, Mr); Standardize the GPS module acquisition function, output the position status parameters, and mark them as (Pm, Mv, Ma, Mr). The standardized values of the parameters are: ; Among them, Mk is the original value of the k-th parameter, Mmin is the minimum value of the parameter, and Mmax is the maximum value of the parameter.

[0024] S3. Perform data fusion on the abnormal scores of the human-machine interaction data, generate comprehensive driving state parameters, and obtain the comprehensive driving state score, so as to evaluate the comprehensive driving state of the human-machine interaction process; S3-1. The specific process of performing data fusion on the abnormal scores of the human-machine interaction data is as follows: Take the image instruction parameter, sound instruction parameter, action instruction parameter, and position status parameter as the index items of the human-machine interaction data respectively; Set up an LSTM prediction model to perform abnormal analysis on the index items, sequentially input the historical data of the index items of the human-machine interaction data, and perform time series analysis on the input data to obtain the prediction output : ; Among them, is the input data, is the prediction output, t is the time node, and T is the historical data period; Then, through the prediction output and the actual output are compared to obtain the abnormal score function Sx(t) of the index items of the human-machine interaction data: ; Among them, x refers to any one of the index items of the human-machine interaction data.

[0025] S3-2. The specific process of evaluating the comprehensive driving state is as follows: Substitute the image instruction parameter, sound instruction parameter, action instruction parameter, and position status parameter into the LSTM prediction model in sequence to obtain the abnormal score function Sx(t) of the index items of the human-machine interaction data; Mark the abnormal score functions Sx(t) of the image instruction parameter, sound instruction parameter, action instruction parameter, and position status parameter as the image instruction abnormal score S1, sound instruction abnormal score S2, action instruction abnormal score S3, and position status abnormal score S4 respectively; Integrate the abnormal score S1 of the image instruction, the abnormal score S2 of the voice instruction, the abnormal score S3 of the action instruction, and the abnormal score S4 of the position status to generate a comprehensive driving state parameter DRIVE<S1, S2, S3, S4>, and obtain a comprehensive driving state score Scom: ; Among them, is the weight coefficient of the abnormal score function Sx of the index item x, and The preset value of is greater than 0; when the comprehensive driving state parameter DRIVE<S1, S2, S3, S4> is lower, the comprehensive driving state score Scom is higher, indicating that the comprehensive driving state is better.

[0026] S4, dynamically adjust the human-computer interaction logic through the comprehensive driving state: through training the state space where the comprehensive driving state is located in real time, obtain the state vector and perform edge computing, so as to perform driver fatigue monitoring and complex road condition detection; The specific process of the central processing module generating an interaction-optimized feature vector and then dynamically adjusting the interaction logic is as follows: Among them, the state space includes the image instruction feature vector I(t), the voice instruction feature vector A(t), the action instruction feature vector D(t), and the vehicle position status feature vector G(t) of the driver, and the state vector is expressed as: .

[0027] S4-1, obtain the fatigue monitoring state parameter Fatigue(t) through real-time monitoring, and set the fatigue monitoring state parameter threshold Threshold1. When it is detected that the driver is fatigued, automatically adjust the interface brightness and provide a voice reminder. The fatigue detection formula is: ; Among them, β1 is the eye opening and closing frequency coefficient, β2 is the eyelid closing time coefficient, and β3 is the mouth opening time coefficient. The coefficients are obtained and optimized through training data; for example, when the detected eyelid closing time is Tyj, a corresponding conversion constant is assigned to the eyelid closing time, so as to obtain the eyelid closing time coefficient β2, and through time series analysis training of the eyelid closing time of this user, the eyelid closing time coefficient corresponding to the fatigue state of this user is obtained; When Fatigue(t) < Threshold1, it is regarded as the normal state; when Fatigue(t) > Threshold1, when the system determines that the driver is in a fatigued state, the alarm mechanism will be automatically triggered; Display eye-catching fatigue reminder information through the in-vehicle display screen, and synchronously start the voice broadcast function to send an alarm to the driver in the form of a voice prompt, so as to effectively improve driving safety.

[0028] S4-2, obtain the complex road condition state parameter ComplexRoad(t) through real-time monitoring, and set the threshold Threshold2 for the complex road condition state parameter. When the GPS detects a complex road condition, simplify the interface information and provide navigation prompts; The complex road condition detection formula is: ; Among them, is the real-time position deviation coefficient, is the driving speed standard coefficient, is the driving acceleration standard coefficient, is the driving direction deviation coefficient; the real-time position deviation coefficient is calculated through the distance deviation between the real-time position Pm of the vehicle and the preset path. When the distance deviation is higher, the real-time position deviation coefficient is higher. Similarly, the driving direction deviation coefficient is calculated through the direction deviation between the driving direction Mr of the vehicle and the preset path; the driving speed standard coefficient is trained through the difference in the excess amplitude between the driving speed Mv and the road speed limit. When the amplitude difference is higher, the driving speed standard coefficient is lower. Similarly, the driving acceleration standard coefficient is trained through the difference in the excess amplitude between the driving acceleration Ma and the vehicle standard acceleration range; When ComplexRoad(t) < Threshold2, it is regarded as the normal state; when ComplexRoad(t) > Threshold2, it is determined that the road condition exceeds the preset threshold, and the system will automatically trigger the multimodal alarm mechanism.

[0029] On the one hand, the road condition reminder information is broadcast in real time through the high-quality speech synthesis module, and the driver is clearly and accurately reminded of the current complex road environment by voice; on the other hand, the system will synchronously activate the navigation service function on the in-vehicle display screen, provide real-time road condition information, the optimal driving path planning and necessary avoidance suggestions, so as to assist the driver in safe driving in all directions and effectively reduce the driving risks brought by complex road conditions.

[0030] S5, dynamically adjust the interaction logic in real time according to the driver's state and road conditions, so as to provide safety reminders and auxiliary functions, significantly improving driving safety and reliability; Through the policy generated by deep reinforcement learning (DRL), for the detected fatigue monitoring state parameter Fatigue(t) and complex road condition state parameter ComplexRoad(t), dynamically adjust and optimize the interaction logic of the in-vehicle system to achieve the dynamic optimization of the embedded in-vehicle interaction system; By collecting feedback data and optimizing the human-machine interaction strategy, a personalized interaction experience is achieved. Meanwhile, the application of edge computing ensures the fast response and global optimization capabilities of the system.

[0031] An embedded human-machine interaction optimization system includes an information collection unit, a central processor, an edge computing unit, and a feedback optimization unit. Among them, the information collection unit, the central processor, the edge computing unit, and the feedback optimization unit are communicatively connected, and this system executes the above-mentioned embedded human-machine interaction optimization method. Among them, the information collection unit is used to collect human-machine interaction data, and collect relevant data for human-machine interaction during driving through multi-source sensors. Among them, the human-machine interaction data includes image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters. The central processor includes a preprocessing module and a data fusion module. The central processor is used to preprocess and fuse the human-machine interaction data. Among them, parameter preprocessing is performed through the preprocessing module to obtain the anomaly score of the parameters of the human-machine interaction data; the anomaly scores of the parameters are combined through the data fusion module to generate comprehensive driving state parameters and obtain the comprehensive driving state score. The edge computing unit is used to train the state space where the comprehensive driving state dynamically locates, obtain the state vector and perform edge computing, so as to perform driver fatigue monitoring and complex road condition detection. The feedback optimization unit is used to receive the feedback data of the driver state and road conditions, and dynamically adjust the interaction logic in real time, so as to provide safety reminders and auxiliary functions.

[0032] An embedded human-machine interaction optimization device includes the above-mentioned embedded human-machine interaction optimization system and applies the above-mentioned embedded human-machine interaction optimization method.

[0033] In summary, the present invention uses the information collection module to collect multi-modal interaction instructions. By integrating the image instructions, voice instructions, action instructions, and position status data of the camera, microphone, steering wheel sensor, and GPS module, the data integrity and accuracy are improved; the central processing module preprocesses and fuses the human-machine interaction data, thereby evaluating the comprehensive driving state, and the edge computing unit uses deep reinforcement learning to generate an interaction optimization strategy, so as to dynamically adjust the interaction logic in real time through the driver state and road conditions, provide safety reminders and auxiliary functions, and significantly improve driving safety and reliability; the feedback optimization module realizes a personalized interaction experience by collecting feedback data and optimizing the human-machine interaction strategy. Meanwhile, the application of edge computing ensures the fast response and global optimization capabilities of the system, realizing efficient, intelligent, and personalized embedded human-machine interaction optimization, not only improving the driver's interaction experience, but also enhancing driving safety and comfort.

[0034] The setting of the size of the interval and the threshold is for the convenience of comparison. Regarding the size of the threshold, it depends on the amount of sample data and the base quantity set by those skilled in the art for each group of sample data; as long as the proportional relationship between the parameter and the quantized value is not affected.

[0035] The above formulas are all dimensionless and take their numerical calculations. The formula is a formula obtained by collecting a large amount of data for software simulation to get the closest to the actual situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation. The above is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, through equivalent replacement or change of the technical solution and inventive concept of the present invention, should be covered within the protection scope of the present invention.

Claims

1. An embedded human-computer interaction optimization method, characterized in that: It includes the following steps: Step 1, collect human-machine interaction data through multi-source sensors: Set up multi-source sensors in the embedded vehicle interaction system to collect human-machine interaction data during driving. Among them, the human-machine interaction data includes image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters; Step 2, preprocess the human-machine interaction data to obtain the anomaly score of the human-machine interaction data: Obtain the image instruction anomaly score, voice instruction anomaly score, action instruction anomaly score, and position status anomaly score through the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters respectively; Step 3, perform data fusion on the anomaly scores of the human-machine interaction data to generate comprehensive driving state parameters and obtain the comprehensive driving state score, so as to evaluate the comprehensive driving state of the human-machine interaction process; Step 4, dynamically adjust the human-machine interaction logic through the comprehensive driving state: Train the state space where the comprehensive driving state is located, obtain the state vector and perform edge computing, so as to perform driver fatigue monitoring and complex road condition detection; Step 5, dynamically adjust the interaction logic in real time through the driver state and road conditions, so as to provide safety reminders and auxiliary functions.

2. An embedded human-computer interaction optimization method according to claim 1, characterized in that: The data processing process of the human-machine interaction data is as follows: Preprocess the image instruction parameters to obtain the camera data acquisition function I(t); Preprocess the voice instruction parameters to obtain the microphone data acquisition function A(t); Preprocess the action instruction parameters to obtain the action data acquisition function D(t); Preprocess the position status parameters to obtain the GPS module acquisition function G(t); Then perform data fusion on the anomaly scores of the human-machine interaction data. The specific process is as follows: Take the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters as the index items of the human-machine interaction data respectively; Set up an LSTM prediction model to perform anomaly analysis on the metric items, sequentially input the historical data of the metric items of the human-computer interaction data, and perform time series analysis on the input data to obtain the predicted output : ； Among them, is the input data, is the predicted output, t is the time node, and T is the historical data period; Then, through the predicted output and the actual output are compared to obtain the anomaly scoring function Sx(t) of the metric items of the human-computer interaction data: ; Among them, x refers to any index item of the human-machine interaction data.

3. An embedded human-computer interaction optimization method according to claim 2, characterized in that: The preprocessing process of the image instruction parameters is as follows: Mark the image instruction captured by the camera as I(t), where t represents the timestamp; Mark the camera data acquisition function as I(t)=CameraCapture(Pw, Ph, Pf, Pt), where Pw is the width pixel of the image, Ph is the height pixel of the image, Pf is the image frame rate, and Pt is the total duration of image acquisition; The function output is a four-dimensional array A, which is marked as the image instruction parameter.

4. A method for optimizing embedded human-computer interaction according to claim 3, characterized in that: The preprocessing process of the voice instruction parameters is as follows: Mark the voice instruction captured by the microphone as A(t); Mark the microphone data acquisition function as A(t)=MicrophoneCapture(Nr, Np, Nd, Nb, Nt), where Nr is the background noise, Np is the sound frequency, Nd is the sound amplitude, Nb is the sound decibel, and Nt is the total duration of sound acquisition; Standardize the microphone data acquisition function, output the voice command parameters, and mark them as (ψr, ψp, ψd, ψb, ψt). The standardized values of the parameters are as follows: ; Among them, Xi is the original value of the i-th parameter, μi is the mean value of the i-th parameter, σi is the standard deviation of the i-th parameter, and ψi is the parameter value after standardization.

5. A method for optimizing embedded human-computer interaction according to claim 4, characterized in that: The preprocessing process of the action instruction parameters is as follows: Mark the action instruction captured by the steering wheel sensor as D(t), and mark the action data acquisition function as: D(t) = Steering(Hsr, Hθ, Hv, Hf), where Hsr is the sampling rate, Hθ is the steering wheel rotation angle, Hv is the steering wheel rotation speed, and Hf is the force applied to the steering wheel; Standardize the action data acquisition function, output the action instruction parameters, and mark them as (ηsr, ηθ, ηv, ηf). The standardized values of the parameters are: ; wherein, is the change rate of the j-th parameter, μj is the mean of the change rate of the j-th parameter, σj is the standard deviation of the change rate of the i-th parameter, and ηj is the parameter value after standardization.

6. An embedded human-computer interaction optimization method according to claim 5, characterized in that: The preprocessing process of the position status parameters is as follows: Mark the data collected by the GPS module as G(t), and the GPS module collection function is: G(t) = GPSCapture(Pm, Mv, Ma, Mr); Standardize the GPS module acquisition function, output the position status parameters, and mark them as (Pm, Mv, Ma, Mr). The standardized values of the parameters are: ; Where Mk is the original value of the k-th parameter, Mmin is the minimum value of the parameter, and Mmax is the maximum value of the parameter.

7. An embedded human-computer interaction optimization method according to claim 6, characterized in that: The specific process of evaluating the comprehensive driving state is as follows: Substitute the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters into the LSTM prediction model in sequence to obtain the anomaly scoring function Sx(t) of the index items of the human-computer interaction data; Mark the anomaly scoring functions Sx(t) of the image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters as the image instruction anomaly score S1, the voice instruction anomaly score S2, the action instruction anomaly score S3, and the position status anomaly score S4 respectively; Furthermore, integrate to generate the comprehensive driving state parameter DRIVE<S1, S2, S3, S4>, and obtain the comprehensive driving state score Scom.

8. An embedded human-computer interaction optimization method according to claim 7, characterized in that: The specific process of the central processing module generating the feature vector for interaction optimization and then dynamically adjusting the interaction logic is as follows: The state space includes the image command feature vector I(t), the voice command feature vector A(t), the action command feature vector D(t), and the vehicle position state feature vector G(t) of the driver, and the state vector is expressed as: ; Obtain the fatigue monitoring status parameter Fatigue(t) through real-time monitoring, and set the fatigue monitoring status parameter threshold Threshold1. When it is detected that the driver is fatigued, automatically adjust the interface brightness and provide a voice reminder; When Fatigue(t) < Threshold1, it is regarded as the normal state; when Fatigue(t) > Threshold1, when the system determines that the driver is in a fatigued state, the alarm mechanism will be automatically triggered; Obtain the complex road condition status parameter ComplexRoad(t) through real-time monitoring, and set the complex road condition status parameter threshold Threshold2. When the GPS detects a complex road condition, simplify the interface information and provide navigation tips; when ComplexRoad(t) < Threshold2, it is regarded as the normal state; When ComplexRoad(t) > Threshold2, it is determined that the road condition exceeds the preset threshold, and the system will automatically trigger the multimodal alarm mechanism.

9. An embedded human-computer interaction optimization system, characterized in that: It includes an information collection unit, a central processor, an edge computing unit, and a feedback optimization unit. Among them, the information collection unit, the central processor, the edge computing unit, and the feedback optimization unit are communicatively connected, and this system executes an embedded human-computer interaction optimization method described in any one of the above claims 1-8; Among them, the information collection unit is used to collect human-computer interaction data, and collect relevant data for human-computer interaction during driving through multi-source sensors. Among them, the human-computer interaction data includes image instruction parameters, voice instruction parameters, action instruction parameters, and position status parameters; The central processing unit includes a preprocessing module and a data fusion module. The central processing unit is used to preprocess and fuse human-computer interaction data. Among them, parameter preprocessing is performed through the preprocessing module to obtain the anomaly score of the parameters of the human-computer interaction data; the anomaly scores of the parameters are combined through the data fusion module to generate comprehensive driving state parameters and obtain the comprehensive driving state score; The edge computing unit is used to train the state space in which the comprehensive driving state dynamically locates, obtain the state vector and perform edge computing, so as to perform driver fatigue monitoring and complex road condition detection; The feedback optimization unit is used to receive the feedback data of the driver's state and road conditions, and dynamically adjust the interaction logic in real time, so as to provide safety reminders and auxiliary functions.

10. An embedded human-computer interaction optimization device, characterized in that: The device includes an embedded human-computer interaction optimization system described in claim 9 above, and applies an embedded human-computer interaction optimization method described in any one of claims 1-8 above.

Images