An artificial intelligence-based backlight module operation method and system

Abstract

The application provides a backlight module operation method and system based on artificial intelligence, the method comprises the following steps: S1: acquiring the motion speed and angular speed data of the user's head, and predicting the head motion trend; S2: establishing a backlight brightness calculation model, taking the head motion trend as the input, and acquiring the theoretical value of the backlight module brightness; S3: based on the theoretical value of the backlight brightness, establishing a curve of the backlight brightness changing with time, and performing smoothing processing on the curve of the backlight brightness changing with time through smoothing filtering; S4: converting the curve of the backlight brightness changing with time after the smoothing processing into the control signal of the backlight module, and sending the control signal to the backlight driving circuit in real time, so as to realize the control of the backlight module. Through the fusion of head motion data prediction and ambient light perception, the dynamic backlight brightness calculation model is established, the 'brightness lag' problem caused by the fact that the traditional backlight control does not consider the head motion state of the user is solved, and the risk of motion sickness caused by the visual discomfort of the user is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a backlight module operation method and system based on artificial intelligence. BACKGROUND

[0002] When the head moves, the vestibular system sends a motion signal to the brain, and if the brightness of the backlight of the display device is suddenly flickered due to picture switching at this time, the brightness change perceived by the visual system does not match the motion signal of the vestibular system, forming a "conflict".

[0003] With the rise of AR technology, the traditional backlight control has limitations, and the backlight is adjusted based on the average brightness of the picture content (such as HDR technology), without considering the head movement state of the user. When the user quickly turns his head, the imaging area of the picture on the retina changes rapidly, and if the brightness of the backlight is not adjusted synchronously, "brightness lag" will occur.

[0004] When the brain cannot coordinate this lag, it will trigger motion sickness reactions such as nausea and dizziness (similar to the principle of car sickness). SUMMARY

[0005] In view of the above technical problems, the present application provides a backlight module operation method and system based on artificial intelligence, and the technical solutions adopted are as follows:

[0006] A backlight module operation method based on artificial intelligence, the method comprising:

[0007] S1: acquiring data of the motion speed and angular velocity of the user's head, and predicting the head movement trend;

[0008] S2: establishing a backlight brightness calculation model, and taking the head movement trend as input to obtain the theoretical value of the brightness of the backlight module;

[0009] S3: based on the theoretical value of the brightness of the backlight, establishing a curve of the change of the brightness of the backlight with time, and performing smoothing processing on the curve of the change of the brightness of the backlight with time through smoothing filtering;

[0010] S4: converting the curve of the change of the brightness of the backlight with time after smoothing processing into a control signal of the backlight module, and sending it to the backlight driving circuit in real time to realize the control of the backlight module.

[0011] Preferably, the S1 specifically comprises:

[0012] The motion speed and angular velocity of the user's head are acquired through the inertial measurement unit deployed in the AR head device;

[0013] According to the historical motion speed and angular velocity, time series data of the motion trajectory is formed, and a motion trend of the head is predicted; the motion trend of the head includes acceleration of the head motion at a future time, displacement of the head motion at the future time, angular velocity of the head motion at the future time, and rotation angle of the head motion at the future time.

[0014] Preferably, the inertial measurement unit includes an accelerometer and a gyroscope, the accelerometer acquires motion acceleration of the head of the user, the gyroscope acquires motion angular velocity of the head of the user, and the acceleration is integrated by time to obtain motion speed of the head of the user.

[0015] Preferably, the S1, in addition to predicting the motion trend of the head, also acquires environmental light intensity through the sensor.

[0016] Preferably, the process of establishing the S2 backlight brightness calculation model specifically includes:

[0017] The motion trend of the head and the environmental light intensity are normalized, so that each parameter in the motion trend of the head and the environmental light intensity are mapped into the interval [0, 1];

[0018] The basic brightness of the backlight module is dynamically coupled according to the environmental light intensity, to obtain the coupled basic brightness of the backlight module;

[0019] According to the predicted value of the angular velocity of the head motion at the future time, each parameter in the motion trend of the head is dynamically weighted, each parameter in the weighted motion trend of the head is fitted with the coupled basic brightness of the backlight module, and a theoretical value of the backlight module brightness is obtained.

[0020] Preferably, the S3 specifically includes:

[0021] A time window of future prediction is set with the current time as a starting point, the time window is divided into a plurality of equally spaced time points, and each time point corresponds to a theoretical value of the backlight module brightness;

[0022] The discrete backlight module brightness values are converted into a brightness curve by a linear interpolation method, and the interpolation slope is dynamically adjusted according to the acceleration of the head motion at the future time;

[0023] The brightness curve is smoothed.

[0024] Preferably, the smoothing method specifically includes exponential smoothing, and the smoothing strength is dynamically adjusted according to the angular velocity of the head motion at the future time.

[0025] Preferably, the S4 includes:

[0026] The curve of the smoothed backlight brightness changing over time is converted into a PWM dimming signal recognizable by the backlight module, and the PWM dimming signal is gamma corrected, the corrected PWM dimming signal is transmitted to the backlight driving circuit, and the timing synchronization mechanism is used to ensure that the PWM dimming signal is synchronized with the display content.

[0027] Preferably, the timing synchronization mechanism comprises:

[0028] The transmission of the PWM dimming signal is completed within a frame refresh interval.

[0029] An artificial intelligence-based backlight module operating system, the system comprising:

[0030] A head motion sensing and prediction system: obtaining data of the motion speed and angular speed of the user's head, and predicting the head motion trend;

[0031] A motion-brightness mapping calculation system: establishing a backlight brightness calculation model, and taking the head motion trend as input to obtain the theoretical value of the backlight module brightness;

[0032] A brightness curve smoothing optimization system: based on the theoretical value of the backlight brightness, a curve of the backlight brightness changing over time is established, and the curve of the backlight brightness changing over time is smoothed by smoothing filtering;

[0033] A real-time backlight driving control system: the smoothed curve of the backlight brightness changing over time is converted into a control signal of the backlight module, which is sent to the backlight driving circuit in real time to realize the control of the backlight module.

[0034] The present application has the following advantages: by fusing head motion data prediction and ambient light perception, a dynamic backlight brightness calculation model is established, the "brightness lag" problem caused by the traditional backlight control without considering the user's head motion state is solved, the matching degree of the backlight brightness and the head motion is improved, the signal conflict between the visual system and the vestibular system is reduced, the risk of motion sickness caused by visual discomfort is reduced, and the comfort and stability of the AR device are improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The present application relates to a kind of artificial intelligence-based backlight module operating method. DETAILED DESCRIPTION

[0036] The preferred embodiments of the present application will be described below with reference to the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and not to limit the present application.

[0037] One embodiment of the present application is a kind of artificial intelligence-based backlight module operating method, the method comprises:

[0038] S1: Obtain data of motion speed and angular speed of the user's head, and predict head motion trend;

[0039] S2: Establish a backlight brightness calculation model, and take the head motion trend as input to obtain a theoretical value of backlight module brightness;

[0040] S3: Based on the theoretical value of backlight brightness, a curve of backlight brightness changing with time is established, and the curve of backlight brightness changing with time is smoothed by smoothing filtering;

[0041] S4: The smoothed curve of backlight brightness changing with time is converted into a control signal of the backlight module, which is sent to the backlight driving circuit in real time to realize the control of the backlight module.

[0042] The working principle and effect of the above technical solution are as follows: the inertial measurement unit deployed on the device collects the acceleration and angular speed of the head motion in real time, and after time integration and Kalman filtering preprocessing, a motion trajectory sequence is formed, and the future motion trend parameters (including acceleration, displacement, angular speed and rotation angle) are predicted by combining the LSTM model. Then, within a 100ms prediction window, a brightness curve is constructed, the interpolation slope is dynamically adjusted according to the motion acceleration, and sudden changes are eliminated by an exponential smoothing algorithm. Finally, the curve is converted into a PWM dimming signal of 200Hz-500Hz, which is transmitted to the driving circuit through the SPI interface after gamma correction, and the synchronization mechanism ensures that the signal is aligned with the picture refresh period, realizing real-time dynamic matching of the backlight brightness and the head motion.

[0043] The method avoids the vestibular-visual conflict and reduces the probability of motion sickness by obtaining and predicting the head motion trend, establishing a backlight brightness calculation model combined with the ambient light intensity, and converting the smoothed signal into a control signal.

[0044] In an embodiment of the present application, the S1 specifically comprises:

[0045] The motion speed and angular speed of the user's head are obtained by the inertial measurement unit deployed on the AR head device.

[0046] According to the historical motion speed and angular speed, time sequence data of the motion trajectory are formed, and the motion trend of the head is predicted; the head motion trend includes the acceleration of the head motion at the future time, the displacement of the head motion at the future time, the angular speed of the head motion at the future time, and the rotation angle of the head motion at the future time.

[0047] The working principle and effect of the technical solution are as follows: head motion data is synchronously collected by an inertial measurement unit (IMU) disposed on an AR device at a frequency of 100 Hz, after noise elimination through Kalman filter fusion processing, the acceleration data is converted into linear velocity through trapezoidal integration, and the angular velocity data is converted into the world coordinate system through coordinate transformation. The motion parameters in the last 1 second are extracted to construct a time sequence, a sliding window mechanism with a window size of 50 ms is used to update the data in real time, and the motion state in the future 200 ms is predicted through an LSTM neural network. The accuracy of the motion trend is improved, and the backlight adjustment lag problem is avoided.

[0048] In an embodiment of the present application, the inertial measurement unit comprises an accelerometer and a gyroscope, the accelerometer acquires the motion acceleration of the user's head, the gyroscope acquires the motion angular velocity of the user's head, and the acceleration is integrated by time to obtain the motion velocity of the user's head.

[0049] The working principle and effect of the technical solution are as follows: the inertial measurement unit collects acceleration signals generated by head motion along the X, Y and Z axes in real time through the accelerometer, removes high-frequency noise through low-pass filtering, and then integrates the acceleration by time through the trapezoidal integration method, i.e. converts the acceleration signal into linear velocity by adding the product of the average value of the acceleration at adjacent time points and the time interval; at the same time, the gyroscope acquires the angular velocity signal of head rotation along the X, Y and Z axes, and directly outputs after eliminating the zero drift error through offset calibration. The data of the two are fused through Kalman filtering to correct the drift error generated in the integration process, and finally the accurate values of the three-axis motion velocity and angular velocity are output;

[0050] Through the cooperative work of the accelerometer and the gyroscope, the acceleration of head motion is converted into velocity through trapezoidal integration, low-pass filtering is used to reduce the noise of the acceleration signal, and Kalman filter fusion processing is combined to control the motion velocity measurement error within ±0.1 m / s, which improves the accuracy compared with the traditional integration method; the real-time data fusion mechanism of the inertial measurement unit shortens the delay of motion parameter output, and reduces the risk of motion sickness caused by data error.

[0051] In an embodiment of the present application, in addition to predicting the head motion trend, the S1 also acquires the ambient light intensity through a sensor.

[0052] The working principle and effect of the technical solution are as follows: while acquiring the head motion parameters through the inertial measurement unit, the ambient light intensity data is collected by using the ambient light sensor, because the mismatch between the backlight brightness and the ambient light in the AR / VR device can exacerbate the visual conflict, especially when the head moves quickly:

[0053] When the ambient light intensity suddenly changes, if the backlight is not adjusted in time, the brightness contrast between the screen and the real environment is unbalanced, which easily causes the focusing adjustment disorder of the human eye;

[0054] The backlight parameters are calculated in combination with the ambient light intensity, so that the screen brightness and the surrounding ambient light can be dynamically balanced, and the motion sickness caused by visual contradiction can be reduced.

[0055] In an embodiment of the present application, the establishment process of the S2 backlight brightness calculation model specifically comprises:

[0056] The head movement trend and the ambient light intensity are normalized, so that each parameter in the head movement trend and the ambient light intensity are mapped to the interval [0, 1], and the angular velocity of the head movement at the future moment is normalized according to the following formula:

[0057]

[0058] wherein, the normalized value of the angular velocity of the head movement at the future moment, the preset maximum angular velocity (corresponding to fast head turning), and the normalized values of the acceleration of the head movement at the future moment, the displacement of the head movement at the future moment and the rotation angle of the head movement at the future moment are obtained in the same way;

[0059] The basic brightness of the backlight module is dynamically coupled according to the ambient light intensity, to obtain the coupled backlight module basic brightness, and the coupled backlight module basic brightness is obtained through the following formula

[0060]

[0061] wherein, the basic brightness of the backlight module before coupling, the ambient light gain coefficient (, =0.6), the normalized ambient light intensity;

[0062] The parameters in the head movement trend are dynamically weighted according to the predicted value of the angular velocity of the head movement at the future moment, each parameter in the weighted head movement trend is fitted with the coupled backlight module basic brightness, to obtain the theoretical value of the backlight module brightness, and the theoretical value of the backlight module brightness is obtained through the following formula:

[0063]

[0064] wherein, , , , respectively represent the weight corresponding to the acceleration of head movement at the future time, the displacement of head movement at the future time, the angular velocity of head movement at the future time, and the rotation angle of head movement at the future time, and , =0.2, , wherein, and respectively represent the normalized value of the angular velocity of head movement at the future time and the rotation angle of head movement at the future time, represents a motion attenuation coefficient, and 0.3≤ ≤0.8.

[0065] The working principle and effects of the above technical solution are as follows: head movement will cause rapid changes in the retinal image, and if the backlight brightness change and the motion inertia do not match, the vestibular-visual conflict will be intensified, and the motion attenuation coefficient is introduced in the formula, so that the brightness change is synchronized with the residual image of the human eye. The change in ambient light intensity and the human eye perception show a logarithmic relationship, the logarithmic compression is realized through the normalization parameter , and the overexposure / underexposure problem of linear mapping is avoided.

[0066] The two parameters are interrelated, control the brightness baseline to ensure the visibility and comfort in the static scene, and control the brightness attenuation to optimize the dizziness suppression and energy efficiency in the dynamic scene. The traditional method does not couple the ambient light, and the backlight is easy to be over-attenuated under strong light, resulting in a dark picture. The formula avoids the content blur caused by insufficient brightness by increasing L and reducing under strong light.

[0067] One embodiment of the present application, the S3 specifically comprises:

[0068] Taking the current time as the starting point, a time window for future prediction is set, and the time window is divided into a plurality of equally spaced time points; and each time point corresponds to a theoretical value of the backlight module brightness;

[0069] The discrete backlight module brightness value is converted into a brightness curve by a linear interpolation method, and the interpolation slope is dynamically adjusted according to the acceleration of head movement at the future time;

[0070] The brightness curve is smoothed.

[0071] The working principle and effect of the above technical solution are as follows: a future prediction time window (200 ms) is set with the current moment as a starting point, is divided into 20 time points at intervals of 10 ms, each point corresponds to a backlight brightness theoretical value output by S2, and discrete values are fitted into a continuous curve through linear interpolation. In the interpolation process, the slope is dynamically adjusted according to the future head movement acceleration: the greater the acceleration (the more intense the movement), the greater the slope adjustment coefficient, so that the brightness change is more gentle to reduce flicker; the smaller the acceleration, the slope tends to the original value to ensure the response speed.

[0072] Through the dynamic time window and linear interpolation mechanism, the discrete error of the backlight brightness theoretical value is reduced, the response delay of the generated continuous curve is shortened, and the brightness jump problem caused by traditional discrete adjustment is effectively avoided; in a fast motion scene, the brightness change rate is reduced, and the visual impact caused by brightness mutation is reduced.

[0073] In one embodiment of the present application, the smoothing method specifically includes exponential smoothing, and the smoothing strength is dynamically adjusted according to the angular velocity of the head movement at the future moment, and the smoothing strength is adjusted by the following formula:

[0074]

[0075] wherein, represents a reference smoothing strength, and represents a default smoothing coefficient when there is no head rotation movement (i.e., angular velocity ω=0).

[0076] In the above technical traditional smoothing method, the fixed smoothing coefficient cannot distinguish between static and high-speed motion scenes, resulting in: at low speed: excessive smoothing introduces delay, and the brightness response lags. At high speed: insufficient smoothing, brightness jitter causes dizziness. However, in the present application, the angular velocity is used as a control variable of the smoothing strength, so that the more intense the movement, the more aggressive the smoothing. In intense motion, the smoothing strength is enhanced, the brightness curve is more gentle, and the conflict between the retinal image mutation and the vestibular signal is reduced.

[0077] In one embodiment of the present application, the S4 comprises:

[0078] The curve of the smoothed backlight brightness changing over time is converted into a PWM dimming signal recognizable by the backlight module, the PWM dimming signal is gamma corrected, the corrected PWM dimming signal is transmitted to the backlight driving circuit, and the PWM dimming signal is synchronized with the display content through a timing synchronization mechanism.

[0079] The working principle and effect of the above technical solution are: the smoothed backlight brightness curve is converted into a PWM dimming signal one by one in time sequence, and the mapping of brightness to electrical signal is realized by calculating the duty cycle corresponding to each time point. The PWM signal is gamma corrected, the signal intensity is nonlinearly transformed according to the gamma characteristic curve of the display, the nonlinear distortion of the photoelectric conversion of the display device is compensated, and the final display brightness conforms to the linear change of human eye perception; a timing synchronization mechanism is adopted to complete the transmission of the PWM dimming signal during the display refresh interval, the signal transmission is triggered by the vertical synchronization signal, the transmission timing is accurately controlled by the timer, the PWM signal update is strictly aligned with the picture refresh, and problems such as flicker and ghosting caused by the synchronization of brightness and picture content are avoided.

[0080] One embodiment of the present application is a backlight module operating system based on artificial intelligence, which comprises:

[0081] Head motion sensing and prediction system: acquire data of motion speed and angular velocity of user's head, and predict head motion trend;

[0082] Motion-brightness mapping calculation system: establish a backlight brightness calculation model, and take the head motion trend as input to obtain the theoretical value of the backlight module brightness;

[0083] Brightness curve smoothing optimization system: based on the theoretical value of the backlight brightness, a curve of the backlight brightness changing with time is established, and the curve of the backlight brightness changing with time is smoothed by smoothing filtering;

[0084] Real-time backlight driving control system: the smoothed curve of the backlight brightness changing with time is converted into a control signal of the backlight module, which is sent to the backlight driving circuit in real time to realize the control of the backlight module.

[0085] The working principle and effect of the above technology are: the acceleration and angular velocity of head motion are collected in real time by the inertial measurement unit deployed on the device, the motion trajectory sequence is formed after time integration and Kalman filtering preprocessing, and the future motion trend parameters (including acceleration, displacement, angular velocity and rotation angle) are predicted by combining the LSTM model. Then a brightness curve is constructed within a 100ms prediction window, the interpolation slope is dynamically adjusted according to the motion acceleration, and the sudden change points are eliminated by exponential smoothing algorithm. Finally, the curve is converted into a PWM dimming signal of 200Hz-500Hz, which is transmitted to the driving circuit through the SPI interface after gamma correction, and the synchronization mechanism ensures that the signal is aligned with the picture refresh cycle, realizing the real-time dynamic matching of the backlight brightness and the head motion.

[0086] The method acquires and predicts head movement trend, establishes backlight brightness calculation model in combination with ambient light intensity, converts control signal after smoothing processing, avoids vestibular-vision conflict, and reduces probability of motion sickness.

[0087] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that all such modifications and changes be included within the scope of the application as defined by the following claims and their equivalents.

Claims

1. A backlight module operation method based on artificial intelligence, characterized in that, The method comprises: S1: acquiring data of the motion speed and angular velocity of the user's head and predicting the head motion trend; S2: establishing a backlight brightness calculation model and taking the head motion trend as input to obtain the theoretical value of the backlight module brightness; S3: based on the theoretical value of the backlight brightness, a curve of the backlight brightness changing with time is established, and the curve of the backlight brightness changing with time is smoothed by smoothing filtering; S4: the smoothed backlight brightness curve changing with time is converted into a control signal of the backlight module, which is sent to the backlight driving circuit in real time to realize the control of the backlight module; In addition to predicting the head motion trend, S1 also acquires the ambient light intensity through the sensor; The establishment process of the backlight brightness calculation model in S2 specifically comprises: The head motion trend and the ambient light intensity are normalized, so that each parameter in the head motion trend and the ambient light intensity are mapped to the interval [0, 1]; According to the ambient light intensity, the basic brightness of the backlight module is dynamically coupled to obtain the coupled basic brightness of the backlight module; According to the predicted value of the angular velocity of the head motion at the future time, each parameter in the weighted head motion trend is fitted with the coupled basic brightness of the backlight module to obtain the theoretical value of the backlight module brightness. 2.The AI-based backlight module operation method of claim 1, wherein, S1 specifically comprises: The motion speed and angular velocity of the user's head are acquired through the inertial measurement unit deployed on the AR head device; According to the historical motion speed and angular velocity, time series data of the motion trajectory are formed, and the head motion trend is predicted; the head motion trend includes the acceleration of the head motion at the future time, the displacement of the head motion at the future time, the angular velocity of the head motion at the future time, and the rotation angle of the head motion at the future time. 3.The AI-based backlight module operation method of claim 2, wherein, The inertial measurement unit comprises an accelerometer and a gyroscope, the accelerometer acquires the motion acceleration of the user's head, and the gyroscope acquires the motion angular velocity of the user's head, and the acceleration is integrated by time to obtain the motion speed of the user's head. 4.The AI-based backlight module operation method of claim 1, wherein, S3 specifically comprises: Taking the current time as the starting point, a future prediction time window is set, and the time window is divided into several equally spaced time points; and each time point corresponds to a theoretical value of the backlight module brightness; The discrete backlight module brightness values are converted into a brightness curve by linear interpolation, and the interpolation slope is dynamically adjusted according to the acceleration of the head motion at the future time; The brightness curve is smoothed.

5. The method of claim 4, wherein the method further comprises: The smoothing method specifically comprises exponential smoothing, and the smoothing strength is dynamically adjusted according to the angular velocity of the head motion at the future time.

6. The method of claim 1, wherein the method further comprises: S4 comprises: The smoothed backlight brightness curve changing with time is converted into a PWM dimming signal recognizable by the backlight module, the PWM dimming signal is gamma corrected, the corrected PWM dimming signal is transmitted to the backlight driving circuit, and the timing synchronization mechanism is used to ensure that the PWM dimming signal is synchronized with the display content.

7. The method of claim 6, wherein the method further comprises: The timing synchronization mechanism comprises: The refresh frequency of the picture is acquired, and the signal transmission timing is triggered by the vertical synchronization signal, and the data generation period is controlled by the timer.

8. An artificial intelligence-based backlight module operating system, characterized in that, The system comprises: a head motion sensing and prediction system: acquiring data of the motion speed and angular velocity of the user's head, and predicting the head motion trend; a motion-brightness mapping calculation system: establishing a backlight brightness calculation model, and taking the head motion trend as input to obtain the theoretical value of the backlight module brightness; a brightness curve smoothing optimization system: based on the theoretical value of the backlight brightness, establishing a curve of the backlight brightness changing over time, and smoothing the curve of the backlight brightness changing over time through smoothing filtering; a backlight driving real-time control system: converting the smoothed curve of the backlight brightness changing over time into a control signal of the backlight module, and sending it to the backlight driving circuit in real time to realize the control of the backlight module; In addition to predicting the head motion trend, the head motion sensing and prediction system also acquires the ambient light intensity through the sensor; The establishment process of the backlight brightness calculation model of the motion-brightness mapping calculation system specifically comprises: normalizing the head motion trend and the ambient light intensity, so that each parameter in the head motion trend and the ambient light intensity are mapped to the interval [0, 1]; According to the ambient light intensity, the basic brightness of the backlight module is dynamically coupled to obtain the coupled basic brightness of the backlight module; According to the predicted value of the angular velocity of the head motion at the future time, each parameter in the weighted head motion trend is fitted with the coupled basic brightness of the backlight module to obtain the theoretical value of the backlight module brightness.

Images