Mobile phone screen display image enhancement method and system

Abstract

The application discloses a mobile phone screen display image enhancement method and system, and relates to the technical field of mobile phone screen control.The method comprises the following steps: acquiring a user face video stream and mobile phone space motion data; identifying head posture information according to the user face video stream; identifying mobile phone posture information by using an extended Kalman filter according to the mobile phone space motion data; calculating a polarization axis included angle between a polarization axis of polarized sunglasses and a polarization axis of a mobile phone screen according to the head posture information and the mobile phone posture information; determining mobile phone screen adjustment parameters according to the polarization axis included angle; and performing compensation processing on the mobile phone screen according to the mobile phone screen adjustment parameters, wherein the compensation processing comprises brightness compensation and color compensation.The application can determine mobile phone screen adjustment parameters in combination with head posture information and mobile phone posture information, and perform brightness compensation and color compensation, so as to realize display image enhancement, and improve accuracy and user experience.

Description

Technical Field

[0001] This invention relates to the field of mobile phone screen control technology, and in particular to a method and system for enhancing images displayed on a mobile phone screen. Background Technology

[0002] In daily life, the display effect of mobile phone screens is often affected by the ambient lighting. To ensure users can see the screen content clearly under different lighting conditions, current mobile phones are usually equipped with a light sensor that can detect the intensity and color of ambient light and automatically adjust the screen's brightness and color accordingly. This automatic adjustment mechanism works normally in most cases. However, the situation becomes complicated when users wear special glasses such as polarized sunglasses. These glasses change the way light enters the user's eyes, causing the adjustments made by the phone system based on the light information sensed by its own sensors to deviate from the actual screen effect seen by the user through the glasses. This deviation is more pronounced, especially when outdoor lighting changes rapidly or when the user's body and the phone's position are constantly changing. It can even lead to problems such as inaccurate colors, unstable brightness, or loss of image detail, seriously affecting the user experience, especially in professional scenarios where high display accuracy is required.

[0003] Existing display adjustment systems typically rely on ambient light information collected by the phone's built-in ambient light sensor. This method has inherent limitations when dealing with scenarios where the user is wearing optical accessories (such as polarized sunglasses). The system cannot obtain detailed information about the additional optical medium between the user and the screen (i.e., polarized sunglasses), nor can it obtain the dynamic angle information between this optical medium and the phone screen caused by changes in the user's posture. This lack of information leads to a significant disconnect between the system's compensation adjustments and the user's actual visual perception. Especially under the combined conditions of rapidly changing ambient light and an unpredictable user posture, the display parameter adjustments based on incomplete information not only fail to optimize the viewing experience but can also cause severe color distortion, abnormal brightness fluctuations, and even loss of image information due to physical phenomena such as cross-polarization effects. This fails to meet the accuracy requirements of specific professional fields (such as outdoor photography), resulting in low image display accuracy and a poor user experience.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this invention is to propose a method and system for enhancing mobile phone screen display images. This method can combine head posture information and mobile phone posture information to determine mobile phone screen adjustment parameters and perform brightness and color compensation to enhance the display image, thereby improving accuracy and user experience.

[0006] On one hand, embodiments of the present invention provide a method for enhancing images displayed on a mobile phone screen, comprising the following steps: Acquire user facial video stream and mobile phone spatial motion data, wherein the mobile phone spatial motion data includes linear acceleration data, angular velocity data and motion speed data; Based on the user's facial video stream, head posture information is identified, which reflects the relative posture of the user's head with respect to the mobile phone. Based on the mobile phone spatial motion data, an extended Kalman filter is used to identify the mobile phone's attitude information. Based on the head posture information and the mobile phone posture information, the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen is calculated, and the polarized sunglasses are worn on the user's head. Based on the polarization axis angle, the mobile phone screen adjustment parameters are determined. The mobile phone screen adjustment parameters include a brightness attenuation factor and a color offset vector of the RGB three primary colors, which include red, blue, and green. The mobile phone screen is compensated according to the adjustment parameters of the mobile phone screen. The compensation process includes brightness compensation and color compensation.

[0007] In some embodiments, identifying head pose information based on the user's facial video stream includes: Facial feature point detection is performed on the user's facial video stream to identify facial feature points, including the tip of the nose and the corners of the mouth. Based on the facial feature points, facial position change analysis is performed to obtain two-dimensional displacement data; Based on the two-dimensional displacement data, the three-dimensional rotation angle and the three-dimensional translation vector are calculated using an attitude estimation algorithm; The head posture information is identified based on the three-dimensional rotation angle and the three-dimensional translation vector.

[0008] In some embodiments, the step of performing facial position change analysis based on the facial feature points to obtain two-dimensional displacement data includes: Sunglasses feature point detection is performed on the user's facial video stream to identify sunglasses feature points, including the center point of the frame and the connection point between the frame and the temple. The target feature points are obtained by combining the sunglasses feature points and the facial feature points; The two-dimensional displacement data is obtained by analyzing the two-dimensional positional changes of the target feature points between consecutive frames of the user's facial video stream.

[0009] In some embodiments, calculating the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information includes: The relative orientation of the first polarization axis of polarized sunglasses and the relative orientation of the second polarization axis of a mobile phone screen are obtained. The relative orientation of the first polarization axis is used to represent the directional relationship between the polarization axis of the polarized sunglasses and the direction of the user's eyes, and the relative orientation of the second polarization axis is used to represent the directional relationship between the polarization axis of the mobile phone screen and the direction of the mobile phone screen. Based on the mobile phone attitude information and the relative direction relationship of the second polarization axis, the polarization direction vector of the mobile phone screen in the global spatial coordinate system is determined; Based on the head posture information and the relative direction relationship of the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined. The polarization axis angle is calculated based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen.

[0010] In some embodiments, determining the polarization direction vector of the mobile phone screen in the global spatial coordinate system based on the mobile phone attitude information and the relative direction relationship of the second polarization axis includes: Based on the phone's attitude information, determine the phone's absolute attitude in the global spatial coordinate system; Determine the orientation of the phone screen based on the phone's absolute orientation; Based on the relative orientation of the mobile phone screen and the second polarization axis, the polarization direction vector of the mobile phone screen in the global spatial coordinate system is determined.

[0011] In some embodiments, determining the polarization direction vector of the polarized sunglasses in the global spatial coordinate system based on the head posture information and the relative direction relationship of the first polarization axis includes: Based on the head pose information and the absolute pose of the mobile phone, determine the absolute pose of the user's head in the global spatial coordinate system. Determine the user's eye orientation based on the absolute head posture; Based on the relationship between the user's eye orientation and the relative direction of the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined.

[0012] In some embodiments, calculating the polarization axis angle based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen includes: Perform a vector dot product operation on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen to obtain the direction vector dot product value; Perform a modulus operation on the polarization direction vector of the sunglasses to obtain the modulus of the polarization direction vector of the sunglasses; Perform a modulus operation on the polarization direction vector of the mobile phone screen to obtain the modulus of the polarization direction vector of the mobile phone screen; The polarization axis angle is calculated based on the dot product of the direction vectors, the magnitude of the polarization direction vector of the sunglasses, and the magnitude of the polarization direction vector of the mobile phone screen.

[0013] In some embodiments, determining the mobile phone screen adjustment parameters based on the polarization axis angle includes: Based on the polarization axis angle, the channel intensity change is determined by consulting the mapping table between the polarization axis angle and the channel intensity changes of the RGB three primary colors; Based on the polarization axis angle, the chromaticity coordinate offset is determined by consulting the mapping table between the polarization axis angle and the chromaticity coordinate offset of the RGB three primary colors; The color offset vector is generated based on the channel intensity change and the chromaticity coordinate offset; The brightness attenuation factor is determined by consulting a mapping table between the polarization axis angle and the brightness attenuation factor.

[0014] In some embodiments, when the compensation process is brightness compensation, the step of performing compensation processing on the mobile phone screen according to the mobile phone screen adjustment parameters includes: Get the current brightness value of the phone screen; Calculate the target brightness gain based on the brightness attenuation factor; Calculate the target brightness value based on the current brightness value and the target brightness gain; Based on the target brightness value, the driving current of the screen backlight module is adjusted to compensate for the brightness.

[0015] On the other hand, embodiments of the present invention provide a mobile phone screen display image enhancement system, including: The data acquisition module is used to acquire user facial video stream and mobile phone spatial motion data, wherein the mobile phone spatial motion data includes linear acceleration data, angular velocity data and motion speed data; The head posture information recognition module is used to recognize head posture information based on the user's facial video stream. The head posture information is used to reflect the relative posture of the user's head with respect to the mobile phone. The mobile phone posture information recognition module is used to identify mobile phone posture information using an extended Kalman filter based on the mobile phone spatial motion data. The polarization axis angle calculation module is used to calculate the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information. The polarized sunglasses are worn on the user's head. The mobile phone screen adjustment parameter determination module is used to determine the mobile phone screen adjustment parameters based on the polarization axis angle. The mobile phone screen adjustment parameters include a brightness attenuation factor and a color offset vector of the RGB three primary colors, which include red, blue and green. The compensation processing module is used to perform compensation processing on the mobile phone screen according to the adjustment parameters of the mobile phone screen. The compensation processing includes brightness compensation and color compensation.

[0016] The embodiments of this application include at least the following beneficial effects: First, the embodiments of this application acquire user facial video stream and mobile phone spatial motion data. Then, based on the user facial video stream, head posture information is identified, and based on the mobile phone spatial motion data, an extended Kalman filter is used to identify mobile phone posture information. Next, based on the head posture information and mobile phone posture information, the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen is calculated. Based on the polarization axis angle, the mobile phone screen adjustment parameters are determined. Finally, based on the mobile phone screen adjustment parameters, brightness compensation and color compensation are performed on the mobile phone screen. Thus, the mobile phone screen adjustment parameters can be determined by combining head posture information and mobile phone posture information, and brightness compensation and color compensation can be performed to enhance the displayed image, improve accuracy and user experience.

[0017] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0019] Figure 1 This is a flowchart of a mobile phone screen image enhancement method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a mobile phone screen display image enhancement system according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0021] In related technologies, the display effect of mobile phone screens is often affected by the ambient light environment in daily life. To ensure that users can see the screen content clearly under different lighting conditions, modern mobile phones are usually equipped with a light sensor that can sense the intensity and color of ambient light and automatically adjust the screen's brightness and color accordingly. This automatic adjustment mechanism works well in most cases. However, the situation becomes complicated when users wear special glasses such as polarized sunglasses. These glasses change the way light enters the user's eyes, causing the adjustments made by the mobile phone system based on the light information sensed by its own sensors to deviate from the actual screen effect seen by the user through the glasses. This deviation is more pronounced, especially when outdoor light changes rapidly or when the user's body and the phone's position are constantly changing. It can even lead to problems such as inaccurate colors, unstable brightness, or loss of image details, seriously affecting the user experience, especially in professional scenarios where high display accuracy is required.

[0022] For example, a photographer shooting outdoors needs to use their phone to view the footage in real time to precisely control composition and exposure. These professional applications place extremely high demands on the screen's color reproduction and detail. To protect their eyes in strong sunlight and better observe their environment, this photographer typically wears polarized sunglasses. Polarized sunglasses effectively filter horizontal glare, which is very useful outdoors. In this case, the phone's ambient light sensor, located on the front of the device, can directly sense the external light conditions, but it cannot detect that the user is wearing sunglasses. Therefore, the phone system still adjusts the screen display based on the raw light information unfiltered by the sunglasses. However, for the photographer wearing sunglasses, the light intensity and color spectrum of the external world and the phone screen seen through the lenses have changed. This leads to the first mismatch between the phone's automatic adjustment and the user's actual visual perception. The phone system perceives the ambient light as very strong and implements a high-brightness, high-contrast display strategy, but these adjustments may appear too bright or distorted to the user wearing sunglasses.

[0023] A deeper problem lies in the interference between polarized sunglasses and the physical characteristics of the LCD screen itself. The LCD screen of a mobile phone contains a polarizer, and the light it emits is polarized in a specific direction. When the polarization direction of the user's polarized sunglasses is at an angle to the polarization direction of the light emitted from the screen, brightness attenuation and color shift occur; this phenomenon is called the "cross-polarization effect." At certain angles, such as when the two polarization directions are perpendicular, the screen may even appear completely black to the user. In actual use, photographers cannot maintain a fixed relative posture between their head and the phone. During movement, composition, and operation, they will unconsciously turn their heads or tilt the phone. Every tiny change in angle alters the relative angle between the sunglasses' polarizer and the screen's polarizer. This causes unpredictable changes in the screen display effect in the user's field of vision due to changes in posture: a scene that was clearly visible one second might suddenly show rainbow stripes or a sharp drop in brightness in a certain area with the next slight head turn. The phone's display enhancement system is unaware of this, unable to detect what optical accessories the user is wearing, and even less aware of the real-time relative posture between the user's head and the phone. The system is still making what it considers to be correct display adjustments based on the collected ambient light information, but these adjustments may appear completely wrong to the user, or even exacerbate visual confusion.

[0024] Existing mobile phone screen enhancement technologies rely solely on ambient light information collected by the phone's built-in ambient light sensor. This approach has inherent limitations when dealing with scenarios where users wear optical accessories (such as polarized sunglasses). The system cannot obtain detailed information about the additional optical medium (i.e., sunglasses) between the user and the screen, nor can it obtain dynamic angle information between this optical medium and the phone screen caused by changes in the user's posture. This lack of information leads to a significant disconnect between the compensation adjustments made by the display system and the user's actual visual perception. Especially under the combined conditions of rapidly changing ambient light and fluctuating user posture, the display parameter adjustments based on incomplete information not only fail to optimize the viewing experience but can also cause severe color distortion, abnormal brightness fluctuations, and even loss of image information due to physical phenomena such as cross-polarization effects. This makes it impossible to meet the accuracy requirements of display content in specific professional fields (such as outdoor photography monitoring).

[0025] The embodiments of this application will be explained in detail below with reference to the accompanying drawings: Figure 1 This is an optional flowchart of a mobile phone screen display image enhancement method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S106.

[0026] Step S101: Obtain the user's facial video stream and mobile phone spatial motion data, including linear acceleration data, angular velocity data and motion speed data; Step S102: Based on the user's facial video stream, identify head posture information, which reflects the relative posture of the user's head with respect to the mobile phone. Step S103: Based on the mobile phone's spatial motion data, use an extended Kalman filter to identify the mobile phone's attitude information; Step S104: Based on the head posture information and the mobile phone posture information, calculate the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen, and then wear the polarized sunglasses on the user's head. Step S105: Determine the mobile phone screen adjustment parameters based on the polarization axis angle. The mobile phone screen adjustment parameters include the brightness attenuation factor and the color offset vector of the RGB three primary colors. The RGB three primary colors include red, blue and green. Step S106: Adjust the parameters according to the mobile phone screen and perform compensation processing on the mobile phone screen. The compensation processing includes brightness compensation and color compensation.

[0027] Steps S101 to S106 as shown in the embodiments of this application can combine head posture information and mobile phone posture information to determine mobile phone screen adjustment parameters, and perform brightness compensation and color compensation to enhance the displayed image, thereby improving accuracy and user experience.

[0028] In some embodiments, steps S101-S106 may first acquire user facial video stream and mobile phone spatial motion data. The mobile phone spatial motion data includes linear acceleration data, angular velocity data, and motion speed data. The user facial video stream can be continuously acquired through the mobile phone's front-facing camera, for example, by capturing video data at a rate of 30 frames per second and transmitting it to the image processing unit for analysis. The mobile phone spatial motion data can be acquired in real time by sensors such as accelerometers, gyroscopes, and magnetometers built into the mobile phone. These sensors provide linear acceleration, angular velocity, and motion speed data at a high sampling rate (e.g., 200Hz). This raw data forms the basis for subsequent posture recognition. It is understood that the user facial video stream refers to the dynamic image data of the user's face captured in real time through the mobile phone's front-facing camera or other visual sensors, primarily used to analyze the user's head movements and posture. The mobile phone spatial motion data refers to the data collected by sensors such as the mobile phone's built-in inertial measurement unit (IMU), including linear acceleration data, angular velocity data, and motion speed data. This data accurately reflects the mobile phone's motion state and posture changes in three-dimensional space.

[0029] Then, based on the user's facial video stream, head pose information is identified. This head pose information reflects the user's head's relative posture to the phone. For example, computer vision technology can be used to estimate the user's three-dimensional head pose by analyzing the motion trajectories and relative position changes of key facial feature points (such as eyes, nose, and mouth) in the video stream. This can be achieved using a trained deep learning model (such as existing convolutional neural network-based facial pose estimation models), which can directly output the rotation and translation parameters of the head in three-dimensional space from the image, thus reflecting the user's head's relative posture to the phone. It can be understood that head pose information refers to the relative position and orientation of the user's head relative to the phone, such as the pitch, yaw, and roll angles of the head relative to the phone, as well as the displacement of the head's center point relative to the phone.

[0030] Then, based on the phone's spatial motion data, an extended Kalman filter (EPF) is used to identify the phone's attitude information. The EPF is a widely used algorithm for attitude estimation that can fuse noisy data from different sensors to provide an accurate estimate of the phone's attitude. For example, accelerometer data can be used to measure the direction of gravity to determine pitch and roll angles, gyroscope data can be used to measure angular velocity to track attitude changes, and magnetometer data can be used to measure the direction of the Earth's magnetic field to determine the yaw angle. Through prediction and update steps, the EPF continuously optimizes the estimate of the phone's attitude in the global spatial coordinate system, thus obtaining accurate phone attitude information. It can be understood that phone attitude information refers to the phone's absolute position and orientation in the global spatial coordinate system, such as the phone's attitude angles (e.g., Euler angles or quaternions) and spatial coordinates.

[0031] Based on head and phone posture information, the polarization angle between the polarization axis of the polarized sunglasses and the polarization axis of the phone screen is calculated. The polarized sunglasses are worn on the user's head. For example, the direction relationship between the polarization axis of the polarized sunglasses and the user's eye orientation, and the direction relationship between the polarization axis of the phone screen and the phone screen orientation, can be pre-defined. Combining the identified head and phone posture information, these relative directional relationships are transformed into a unified global coordinate system, resulting in the polarization axis direction vectors of the polarized sunglasses and the phone screen. By calculating the angle between these two vectors, the polarization axis angle is obtained. This angle is a key parameter determining the strength of the polarization effect. It can be understood that the polarization axis of polarized sunglasses refers to the physical axis on the lens that allows light polarized in a specific direction to pass through. The polarization axis of the phone screen refers to the polarization direction of the polarization layer inside the phone screen display panel.

[0032] Based on the polarization axis angle, the adjustment parameters for the mobile phone screen are determined. These parameters include a brightness attenuation factor and color shift vectors for the RGB primary colors (red, blue, and green). For example, experiments can be conducted beforehand to measure the degree of brightness attenuation and the color shift of the RGB primary colors under different polarization axis angles, and a mapping table between the polarization axis angle and these parameters can be established. This mapping table can be set according to actual needs and is not specifically limited. Once the current polarization axis angle is calculated, the corresponding brightness attenuation factor and color shift vector can be obtained by querying these mapping tables. The brightness attenuation factor quantifies the brightness loss caused by the polarization effect, and the color shift vector describes the color deviation caused by the polarization effect, where the RGB primary colors include red, blue, and green.

[0033] Finally, the phone screen is compensated based on the adjusted parameters. This compensation includes brightness and color compensation. For example, for brightness compensation, the driving current of the phone screen's backlight module can be adjusted according to the brightness attenuation factor to increase screen brightness and offset the brightness loss caused by polarization. For color compensation, the RGB channel intensity and chromaticity coordinates of each pixel in the displayed image can be adjusted according to the color offset vectors of the RGB primary colors to correct color distortion caused by polarization. In this way, even when the user is wearing polarized sunglasses, the phone screen can still present a near-realistic and color-accurate display effect.

[0034] This embodiment first uses multimodal data fusion to accurately acquire the user's head posture and the phone's posture. Specifically, the user's facial video stream provides the relative posture information of the user's head relative to the phone, while the phone's spatial motion data provides the absolute posture information of the phone in global space through an extended Kalman filter. The system can then calculate the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the phone screen. This angle is a core parameter that determines the intensity and direction of the polarization effect. Through a pre-established mapping relationship, the system can dynamically determine the phone screen adjustment parameters based on this polarization axis angle. The brightness attenuation factor quantifies the brightness loss caused by the polarization effect, while the color shift vector describes the specific direction and degree of color distortion. Finally, the system performs compensation processing on the phone screen based on these adjustment parameters. Brightness compensation offsets brightness attenuation by adjusting the screen backlight brightness, ensuring that the screen still maintains sufficient visibility when wearing polarized sunglasses. Color compensation corrects color deviation by adjusting the RGB channel intensity and chromaticity coordinates of the displayed image, ensuring the accuracy of image colors. The entire process is real-time and dynamic, continuously adjusting to changes in the user's head and phone posture to ensure optimal phone screen viewing in any scenario where polarized sunglasses are worn. This method considers the user, phone, and polarized sunglasses as a whole, achieving an intelligent closed loop from perception to compensation, significantly improving display accuracy and user satisfaction.

[0035] Through the above technical solution, this embodiment achieves precise optimization of the mobile phone screen display effect in scenarios where polarized sunglasses are worn. It not only solves the problems of insufficient or excessive compensation caused by missing information in existing technologies, but also provides a dynamic and intelligent solution that can adapt to real-time changes in the user's head and phone posture, ensuring a clear, accurate, and comfortable viewing experience under any circumstances. This method technically breaks through the limitations of traditional ambient light perception, opening up new paths for the application of mobile phone display technology in special usage scenarios, and has significant practical value and market prospects.

[0036] In some embodiments, step S102, identifying head pose information based on the user's facial video stream, may include, but is not limited to, the following steps: Step S201: Perform facial feature point detection on the user's facial video stream and identify facial feature points, including the tip of the nose and the corners of the mouth. Step S202: Based on facial feature points, perform facial position change analysis to obtain two-dimensional displacement data; Step S203: Based on the two-dimensional displacement data, calculate the three-dimensional rotation angle and the three-dimensional translation vector using the attitude estimation algorithm; Step S204: Identify head posture information based on the three-dimensional rotation angle and the three-dimensional translation vector.

[0037] In some embodiments, facial feature point detection can be performed on the user's facial video stream to identify facial feature points, including the tip of the nose and the corners of the mouth. Key facial feature points, such as the tip of the nose and the corners of the mouth, can be automatically identified and located from the user's facial video stream using image processing and computer vision techniques. These facial feature points, such as the tip of the nose and the corners of the mouth, are salient and stable points in the facial structure, effectively representing the geometry and pose of the face. The purpose is to provide accurate input data for subsequent facial position change analysis.

[0038] Then, based on facial feature points, facial position change analysis is performed to obtain two-dimensional displacement data. The movement of the user's face on the two-dimensional image plane can be quantified by tracking the positional changes of these identified facial feature points in consecutive video frames. The two-dimensional displacement data reflects the pixel-level displacement of facial feature points in the horizontal and vertical axes, aiming to capture the dynamic changes of the user's head in the video stream.

[0039] Then, based on the two-dimensional displacement data, a pose estimation algorithm is used to calculate the three-dimensional rotation angle and the three-dimensional translation vector. The displacement information of feature points on the two-dimensional image plane can be mapped to three-dimensional space through specific mathematical models and algorithms (such as perspective-n-point (PnP) algorithms or deep learning pose estimation models). The three-dimensional rotation angle represents the rotational attitude (such as pitch) of the user's head relative to the phone camera, and the three-dimensional translation vector represents the change in the position of the user's head in three-dimensional space. The goal is to recover the complete three-dimensional pose information of the user's head from the two-dimensional image information.

[0040] Finally, head pose information is identified based on the three-dimensional rotation angle and the three-dimensional translation vector. The calculated three-dimensional rotation angle and the three-dimensional translation vector can be used as the final representation of the user's head pose. These three-dimensional parameters can comprehensively and accurately describe the orientation and position of the user's head in space, thus reflecting the relative pose of the user's head with respect to the mobile phone.

[0041] This embodiment obtains accurate two-dimensional displacement data by performing refined facial feature point detection on the user's facial video stream and analyzing continuous facial position changes based on these feature points. Furthermore, using advanced pose estimation algorithms, this two-dimensional displacement data is converted into three-dimensional rotation angles and three-dimensional translation vectors, thereby accurately identifying the user's head pose information in three-dimensional space. This step-by-step and refined processing flow ensures the accuracy and robustness of head pose information recognition, providing reliable input for subsequent polarization axis angle calculations.

[0042] Through the above technical solution, this embodiment can achieve high-precision recognition of user head posture information. By capturing subtle changes in facial feature points and combining them with posture estimation algorithms, potential recognition errors and environmental interference problems are effectively overcome. This not only improves the accuracy of head posture information but also lays a solid foundation for subsequent calculation of the polarization axis angle between the polarization axis of polarized sunglasses and the polarization axis of the mobile phone screen, thereby enhancing the overall effect of image enhancement on the mobile phone screen and the user experience.

[0043] In some embodiments, in step S202, facial position change analysis is performed based on facial feature points to obtain two-dimensional displacement data, which may include, but is not limited to, the following steps: Sunglasses feature point detection is performed on the user's facial video stream to identify sunglasses feature points, including the center point of the frame and the connection point between the frame and the temple. The target feature points are obtained by combining the feature points of sunglasses and facial features. Two-dimensional displacement data is obtained by analyzing the two-dimensional positional changes of target feature points between consecutive frames of a user's facial video stream.

[0044] In some embodiments, when a user wears polarized sunglasses, facial feature points may be obscured or their recognition accuracy may decrease, affecting the accuracy of two-dimensional displacement data and consequently the accuracy of head posture information recognition. To address this, sunglasses feature point detection can be performed on the user's facial video stream to identify these feature points, including the center point of the frame and the connection point between the frame and temples. Image processing and computer vision techniques can be used to identify and locate key points on the polarized sunglasses within the user's facial video stream. These key points, i.e., sunglasses feature points, can include the center point of the frame and the connection point between the frame and temples, and can stably reflect the position and posture of the sunglasses on the face. The center point of the frame refers to the geometric center of the sunglasses lens area, and the connection point between the frame and temples refers to the key location at the junction of the frame and temples.

[0045] The sunglasses feature points and facial feature points are then combined to obtain the target feature points. Target feature points can be formed by combining the identified sunglasses feature points with facial feature points (such as the tip of the nose and the corners of the mouth). This combination aims to provide a more comprehensive and robust set of facial and object-related features.

[0046] Further analysis of the two-dimensional positional changes of target feature points across consecutive frames of the user's facial video stream yields two-dimensional displacement data. By tracking the pixel coordinate changes of these combined feature points over time, the displacement of the user's face in the two-dimensional plane can be precisely quantified. For example, optical flow or feature matching algorithms can be used to track these feature points and calculate their two-dimensional displacement vectors between adjacent frames.

[0047] This embodiment introduces sunglasses feature point detection and combines it with facial feature points to form a more comprehensive set of target feature points. When a user wears polarized sunglasses, although some facial feature points may be obscured, the sunglasses themselves have a stable geometric structure and identifiable feature points. By simultaneously identifying and utilizing these sunglasses feature points, even if the accuracy of facial feature point recognition decreases, the system can still rely on sunglasses feature points to assist in facial position change analysis. Therefore, the robustness of the target feature point set is significantly improved, making the acquisition of two-dimensional displacement data more accurate and stable under various wearing conditions. This combined feature point analysis method effectively compensates for the limitations of a single facial feature point in specific scenarios, ensuring the accuracy of subsequent head posture information recognition.

[0048] To illustrate this technical solution more clearly, a specific example is used below. When a user wears polarized sunglasses to view their phone, the system first acquires a video stream of the user's face through the phone's camera. Next, this video stream is processed, identifying facial feature points such as the tip of the nose and the corners of the mouth. Simultaneously, feature point detection is specifically performed on the sunglasses, identifying feature points such as the center point of the frame and the connection point between the frame and the temples. Subsequently, these facial and sunglasses feature points are combined into a unified set of target feature points. For example, an existing deep learning-based facial feature point detection model can be used, which, after training, can simultaneously identify key points on both the face and the sunglasses. In consecutive video frames, by tracking the two-dimensional positional changes of these target feature points on the image plane—for example, calculating the pixel displacement of each feature point between adjacent frames—accurate two-dimensional displacement data is obtained. Even if the user's facial expression changes or part of the face is obscured by the sunglasses, the calculation of the two-dimensional displacement data maintains high accuracy due to the introduction of sunglasses feature points, avoiding pose estimation bias caused by feature point loss or misidentification.

[0049] Through the above technical solution, this embodiment effectively addresses the issues of occlusion and decreased accuracy in facial feature point recognition when users wear polarized sunglasses. By fusing sunglasses feature points with facial feature points, the resulting target feature point set exhibits higher robustness and stability, thereby making the two-dimensional displacement data obtained from facial position change analysis more accurate. This not only improves the recognition accuracy of head posture information but also provides more reliable basic data for subsequent calculation of polarization axis angles and determination of mobile phone screen adjustment parameters, ultimately enhancing the overall performance and user experience of the mobile phone screen display image enhancement method.

[0050] In some embodiments, in step S104, calculating the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information may include, but is not limited to, the following steps: Step S301: Obtain the relative orientation relationship of the first polarization axis of the polarized sunglasses and the relative orientation relationship of the second polarization axis of the mobile phone screen. The relative orientation relationship of the first polarization axis is used to represent the orientation relationship between the polarization axis of the polarized sunglasses and the direction of the user's eyes, and the relative orientation relationship of the second polarization axis is used to represent the orientation relationship between the polarization axis of the mobile phone screen and the direction of the mobile phone screen. Step S302: Based on the phone's attitude information and the relative direction relationship between the second polarization axis, determine the polarization direction vector of the phone screen in the global spatial coordinate system. Step S303: Based on the head posture information and the relative direction relationship of the first polarization axis, determine the polarization direction vector of the polarization axis of the polarized sunglasses in the global spatial coordinate system. Step S304: Calculate the polarization axis angle based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen.

[0051] In some embodiments, the relative directional relationship of the first polarization axis of the polarized sunglasses and the relative directional relationship of the second polarization axis of the mobile phone screen can be obtained first. The first polarization axis relative directional relationship represents the directional relationship between the polarization axis of the polarized sunglasses and the direction of the user's eyes, and the second polarization axis relative directional relationship represents the directional relationship between the polarization axis of the mobile phone screen and the direction of the mobile phone screen. Specifically, the first polarization axis relative directional relationship refers to a fixed or preset directional relationship between the polarization axis of the polarized sunglasses and the direction of the user's eyes. For example, when wearing polarized sunglasses, their polarization axis usually has a fixed angular relationship with the horizontal or vertical direction of the user's eyes. The second polarization axis relative directional relationship refers to a fixed or preset directional relationship between the polarization axis of the mobile phone screen and the direction of the mobile phone screen. For example, the polarization axis of the mobile phone screen usually has a fixed angular relationship with the direction of the physical long or short side of the mobile phone screen. These relative directional relationships can be preset and obtained during system initialization.

[0052] Then, based on the phone's attitude information and the relative direction of the second polarization axis, the polarization direction vector of the phone screen in the global spatial coordinate system is determined. Specifically, the phone's attitude information and the relative direction of the second polarization axis are used to determine the polarization direction vector of the phone screen in the global spatial coordinate system. The phone's attitude information provides the phone's position and orientation in three-dimensional space; combined with the relative direction of the phone screen's polarization axis, the polarization axis of the phone screen can be accurately transformed from the phone's own coordinate system to a globally unified coordinate system.

[0053] Then, based on the head posture information and the relative direction relationship of the first polarization axis, the polarization direction vector of the polarization axis of the polarized sunglasses in the global spatial coordinate system is determined. The head posture information reflects the relative posture of the user's head with respect to the phone. By further combining this with the phone's posture information, the user's head posture in the global spatial coordinate system can be derived. Based on this, using the relative direction relationship between the polarization axis of the polarized sunglasses and the user's eye orientation, the direction vector of the polarization axis of the polarized sunglasses in the global spatial coordinate system can be determined.

[0054] Finally, the polarization axis angle is calculated based on the polarization direction vectors of the sunglasses and the phone screen. Ultimately, the polarization axis angle between the two can be obtained by calculating the polarization direction vectors of the sunglasses and the phone screen, which are determined in the global coordinate system. This calculation typically involves vector operations, such as dot product, to obtain the cosine of the angle between the two vectors, and thus the angle itself.

[0055] This embodiment first obtains the relative orientation of the polarization axes of the polarized sunglasses and the mobile phone screen, and then combines this with the identified head posture information and mobile phone posture information to unify the orientation of these two polarization axes into a global spatial coordinate system. Specifically, the mobile phone posture information provides the absolute orientation of the mobile phone in space. Combined with the relative orientation of the second polarization axis of the mobile phone screen, the polarization direction vector of the mobile phone screen can be accurately constructed. Simultaneously, the head posture information reflects the relative orientation of the user's head. By combining this with the mobile phone posture information, the absolute orientation of the user's head in global space can be derived. Then, combined with the relative orientation of the first polarization axis of the polarized sunglasses, the polarization direction vector of the sunglasses can be constructed. Therefore, under a unified global spatial coordinate system, by calculating these two polarization direction vectors, the included angle of the polarization axes can be accurately determined, providing accurate input parameters for subsequent mobile phone screen compensation processing.

[0056] Through the above technical solution, this embodiment can systematically and accurately unify the polarization axis directions of polarized sunglasses and mobile phone screens into a global spatial coordinate system for consideration, thereby overcoming the problem of inaccurate calculation of the polarization axis angle caused by changes in posture. This embodiment ensures that the polarization axis angle can still be accurately calculated even when the user's head and the mobile phone's posture are constantly changing, providing reliable basic data for subsequent brightness compensation and color compensation, and significantly improving the display effect and user experience of the mobile phone screen under polarized sunglasses.

[0057] In some embodiments, in step S302, determining the polarization direction vector of the mobile phone screen in the global spatial coordinate system based on the mobile phone posture information and the relative direction relationship of the second polarization axis may include, but is not limited to, the following steps: Based on the phone's attitude information, determine the phone's absolute attitude in the global coordinate system; Determine the orientation of the phone screen based on the phone's absolute orientation; Based on the orientation of the phone screen and the relative direction of the second polarization axis, determine the polarization direction vector of the phone screen in the global spatial coordinate system.

[0058] In some embodiments, since the phone's attitude information usually reflects the overall motion state of the phone, the precise determination of the polarization axis direction of the phone screen requires more specific knowledge of the phone's absolute attitude in global space and the orientation of the phone screen itself.

[0059] To this end, we can first determine the phone's absolute attitude in the global coordinate system based on the phone's attitude information. The phone's attitude information, identified using an extended Kalman filter, can be converted into the phone's precise three-dimensional orientation and position in the global coordinate system. This attitude information is typically represented as quaternions, Euler angles, or rotation matrices relative to a reference coordinate system. By transforming this attitude information with the global coordinate system, we can obtain the phone's absolute attitude in global space, providing a stable global reference for subsequent screen orientation determination.

[0060] Then, based on the phone's absolute orientation, the phone screen's orientation is determined. After obtaining the phone's absolute orientation in the global coordinate system, the screen's normal direction in the global coordinate system can be calculated using the fixed geometric relationship between the screen and the phone itself. For example, if the phone's absolute orientation is represented by a rotation matrix, and the screen's normal direction is known in the phone's body coordinate system, then by applying this rotation matrix to the screen normal vector in the phone's body coordinate system, the screen's orientation vector in the global coordinate system can be obtained. The purpose is to accurately represent the actual orientation of the phone screen in three-dimensional space.

[0061] Then, based on the phone screen orientation and the relative direction of the second polarization axis, the polarization direction vector of the phone screen in the global spatial coordinate system is determined. The direction vector of the phone screen's polarization axis in the global spatial coordinate system can be calculated by combining the determined phone screen orientation and the preset relative direction of the second polarization axis. The relative direction of the second polarization axis is an inherent geometric relationship between the phone screen's polarization axis and its orientation; for example, the polarization axis may be parallel to, perpendicular to, or at a fixed angle to a specific edge of the screen. By mapping this relative relationship to the phone screen orientation in the global coordinate system, the absolute direction of the phone screen's polarization axis can be accurately determined.

[0062] This embodiment first explicitly derives the absolute attitude of the phone in the global spatial coordinate system from the phone's attitude information. Then, based on this absolute attitude, it further determines the precise orientation of the phone screen in global space. Finally, it calculates the direction vector of the phone screen's polarization axis in the global spatial coordinate system by combining a preset second polarization axis relative direction relationship. This step-by-step processing method establishes a clear and logically rigorous derivation chain, ensuring higher accuracy and traceability in the conversion process from the original phone motion data to the final phone screen polarization direction vector. Specifically, the phone's attitude information (e.g., output by an extended Kalman filter) is usually the attitude of the phone itself in a certain reference frame, while determining the polarization axis direction of the phone screen requires the screen's precise spatial orientation. By introducing two intermediate quantities—the phone's absolute attitude and the phone screen's orientation—the attitude information of the phone itself can be accurately mapped to the geometric orientation of the screen. Then, combined with the preset second polarization axis relative direction relationship, the direction vector of the phone screen's polarization axis in the global spatial coordinate system can be accurately calculated.

[0063] To illustrate this technical solution more clearly, a specific example is used below. Assume the phone's attitude information is output as a set of quaternions from the extended Kalman filter, representing the rotation of the phone's body coordinate system relative to the global coordinate system. First, based on this quaternion, it can be converted into a 3x3 rotation matrix R, which represents the phone's absolute attitude in the global coordinate system. Second, assuming the phone screen's normal direction is (0, 0, 1) (i.e., the Z-axis direction) in the phone's body coordinate system, then the phone screen's orientation vector in the global coordinate system... It can be done The value is calculated using R * [0; 0; 1]. Finally, if the relative direction relationship of the second polarization axis specifies that the polarization axis of the phone screen is parallel to the X-axis direction of the phone screen, then in the phone's body coordinate system, the polarization axis direction vector is (1, 0, 0). At this time, the polarization direction vector of the phone screen in the global spatial coordinate system is... It can be done The polarization direction vector of the mobile phone screen is calculated using the formula = R * [1;0; 0]. This step-by-step calculation ensures the accuracy of the polarization direction vector.

[0064] Through the above technical solution, this embodiment can significantly improve the accuracy and reliability of determining the direction vector of the polarization axis of the mobile phone screen in the global spatial coordinate system. Specifically, by refining the mobile phone attitude information into the absolute attitude of the mobile phone and the orientation of the mobile phone screen, the ambiguity and error accumulation that may be caused by direct derivation are avoided, making the subsequent calculation of the polarization axis angle more accurate. This layered and progressive determination method not only enhances the robustness of the entire image enhancement method, but also provides a clearer and more operable path for the actual system implementation, thereby ensuring that when the user is wearing polarized sunglasses, the brightness compensation and color compensation of the mobile phone screen can be performed more accurately and effectively, significantly improving the user's viewing experience.

[0065] In some embodiments, determining the polarization direction vector of the polarization axis of the polarized sunglasses in the global spatial coordinate system based on head posture information and the relative direction relationship of the first polarization axis may include, but is not limited to, the following steps: Based on the head pose information and the absolute pose of the mobile phone, determine the absolute pose of the user's head in the global spatial coordinate system. Determine the user's eye orientation based on the absolute head posture; Based on the relationship between the user's eye orientation and the relative direction of the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined.

[0066] In some embodiments, the absolute head pose in the global coordinate system can be determined first based on head pose information and the absolute pose of the phone. This can be achieved by combining the relative pose of the user's head with respect to the phone (i.e., head pose information) and the absolute pose of the phone in the global coordinate system (i.e., phone absolute pose). Head pose information reflects the rotation and translation of the user's head relative to the phone, while the phone absolute pose provides the phone's position and orientation in three-dimensional space. By transforming and fusing these two coordinates, the precise pose of the user's head in the global coordinate system can be obtained.

[0067] Then, based on the absolute head pose, the user's eye orientation is determined. Once the absolute head pose is identified in the global coordinate system, the specific orientation of the user's eyes in the global coordinate system can be calculated based on a pre-defined user head model or the relative position of the eyes within the head. This typically involves extracting the gaze direction or line-of-sight vector from the head pose.

[0068] Then, based on the relative direction between the user's eye orientation and the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined. This relative direction between the user's eye orientation and the first polarization axis predefines a fixed or variable directional relationship between the polarization axis of the polarized sunglasses and the user's eye orientation. By combining the user's eye orientation with this relative directional relationship, the direction vector of the polarization axis of the polarized sunglasses can be accurately calculated in the global spatial coordinate system.

[0069] This embodiment first transforms the user's head posture from the phone's relative coordinate system to the global spatial coordinate system, thereby obtaining the absolute posture of the user's head in the real world. Based on this, the actual orientation of the user's eyes is further derived. Finally, combining the inherent polarization axis of polarized sunglasses with the relative orientation of the eyes, the polarization axis direction of the polarized sunglasses is accurately located in the global spatial coordinate system. This series of detailed steps ensures the accuracy and reliability of the polarization direction vector calculation for polarized sunglasses, laying the foundation for the subsequent accurate calculation of the polarization axis angle.

[0070] Through the above technical solution, this embodiment can more precisely determine the direction vector of the polarization axis of polarized sunglasses in the global spatial coordinate system. This step-by-step refinement process avoids the potential accumulation of errors that may exist when directly deriving the global polarization direction from the relative attitude, significantly improving the calculation accuracy of the sunglasses' polarization direction vector. This provides a more accurate input for subsequent calculations of the polarization axis angle, enabling the compensation processing of the mobile phone screen to more accurately match the user's actual visual needs, thus improving the effectiveness of image enhancement and the user experience.

[0071] In some embodiments, step S304, calculating the polarization axis angle based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen, may include, but is not limited to, the following steps: Perform a vector dot product operation on the polarization direction vectors of the sunglasses and the mobile phone screen to obtain the dot product value of the direction vectors. Perform a modulus operation on the polarization direction vector of the sunglasses to obtain the modulus of the polarization direction vector of the sunglasses; Perform a modulus operation on the polarization direction vector of the mobile phone screen to obtain the modulus of the polarization direction vector of the mobile phone screen; The polarization axis angle is calculated based on the dot product of the direction vectors, the magnitude of the polarization direction vector of the sunglasses, and the magnitude of the polarization direction vector of the mobile phone screen.

[0072] In some embodiments, a vector dot product operation can be performed on the polarization direction vectors of the sunglasses and the mobile phone screen to obtain the direction vector dot product value. Specifically, both the polarization direction vectors of the sunglasses and the mobile phone screen are three-dimensional vectors determined in the global spatial coordinate system, representing the directions of the polarization axes of the polarized sunglasses and the mobile phone screen in three-dimensional space, respectively. The vector dot product operation, also known as the inner product, results in a scalar, which is the sum of the products of the corresponding components of the two vectors, reflecting the degree of similarity between the two vectors in direction.

[0073] Then, a modulo operation is performed on the polarization direction vector of the sunglasses to obtain the modulus of the sunglasses' polarization direction vector. Similarly, a modulo operation is performed on the polarization direction vector of the mobile phone screen to obtain the modulus of the mobile phone screen's polarization direction vector. Modulo operation refers to calculating the length or magnitude of a vector, that is, the square root of the sum of the squares of its components.

[0074] Then, based on the dot product of the direction vectors, the magnitude of the polarization direction vector of the sunglasses, and the magnitude of the polarization direction vector of the mobile phone screen, the polarization axis angle is calculated. The polarization axis angle is the angle between two polarization axes, and this angle is a key parameter affecting brightness attenuation and color shift when viewing a mobile phone screen with polarized sunglasses. The formula for calculating the polarization axis angle is: In the formula, The angle between the polarization axes. The dot product of the direction vectors. Let be the magnitude of the polarization direction vector of the mobile phone screen. Let be the magnitude of the polarization direction vector of the sunglasses.

[0075] This embodiment obtains the dot product of the polarization direction vectors of the sunglasses and the mobile phone screen, which is proportional to the cosine of the angle between the two vectors. Simultaneously, the magnitudes of the two direction vectors are obtained by performing a modulo operation. Dividing the dot product by the product of the magnitudes of the two direction vectors yields the cosine of the polarization axis angle, which is then calculated using the inverse cosine function to determine the precise polarization axis angle. This calculation method is based on rigorous vector algebra principles, ensuring the accuracy and reliability of the results.

[0076] Through the above technical solution, this embodiment can calculate the angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen using a precise and standardized mathematical method. This precise calculation method provides reliable basic data for subsequently determining the adjustment parameters of the mobile phone screen, thereby enabling more accurate brightness and color compensation. This effectively solves the problem of poor display effect when users wear polarized sunglasses, improving the user experience.

[0077] In some embodiments, in step S105, determining the mobile phone screen adjustment parameters based on the polarization axis angle may include, but is not limited to, the following steps: Based on the polarization axis angle, the channel intensity change is determined by consulting the mapping table between the polarization axis angle and the channel intensity changes of the RGB three primary colors; Based on the polarization axis angle, the chromaticity coordinate offset is determined by consulting the mapping table between the polarization axis angle and the chromaticity coordinate offset of the RGB three primary colors; Generate a color offset vector based on channel intensity changes and chromaticity coordinate shifts; Based on the polarization axis angle, the brightness attenuation factor is determined by consulting the mapping table between the polarization axis angle and the brightness attenuation factor.

[0078] In some embodiments, the channel intensity change can be determined first by consulting a mapping table between the polarization axis angle and the channel intensity changes of the RGB primary colors. Specifically, the mapping table pre-stores the expected changes in the channel intensity of each of the RGB primary colors (red, blue, and green) under different polarization axis angles. Channel intensity change refers to the relative attenuation or enhancement of the light intensity of each color channel due to the polarization effect at a specific polarization axis angle. By consulting this mapping table, the corresponding intensity change values ​​of the red, green, and blue channels can be directly obtained based on the currently calculated polarization axis angle. This mapping table can be set according to actual needs and is not specifically limited.

[0079] Then, based on the polarization axis angle, the chromaticity coordinate offset is determined by consulting a mapping table between the polarization axis angle and the chromaticity coordinate offsets of the RGB primary colors. This mapping table pre-stores the required offsets of the RGB primary colors in the chromaticity space (e.g., the CIE xy chromaticity diagram) for different polarization axis angles. Chromaticity coordinate offset refers to the change in hue and saturation of a color due to polarization effects at a specific polarization axis angle. By consulting this mapping table, the corresponding chromaticity coordinate offset value can be obtained based on the currently calculated polarization axis angle for subsequent color adjustments. This mapping table can be set according to actual needs and is not specifically limited.

[0080] Then, based on the channel intensity changes and chromaticity coordinate shifts, a color shift vector is generated. In practical applications, the color shift vector is a comprehensive parameter used to describe the overall adjustment direction and magnitude of the hue, saturation, and brightness of the colors displayed on the mobile phone screen. This vector is generated based on the determined channel intensity changes and chromaticity coordinate shifts. Specifically, channel intensity changes mainly affect the brightness component of the color, while chromaticity coordinate shifts directly affect the hue and saturation components. By integrating and transforming these two pieces of information, a unified color shift vector can be obtained to guide subsequent color compensation processing.

[0081] Finally, based on the polarization axis angle, the brightness attenuation factor is determined by consulting a mapping table between the polarization axis angle and the brightness attenuation factor. This mapping table pre-stores the expected attenuation factor for the overall brightness of the phone screen under different polarization axis angles. The brightness attenuation factor is a value between 0 and 1, representing the proportion of brightness reduction relative to the maximum brightness of the phone screen at a specific polarization axis angle. By consulting this mapping table, the corresponding brightness attenuation factor can be directly obtained based on the currently calculated polarization axis angle for subsequent brightness compensation processing. This mapping table can be set according to actual needs and is not specifically limited.

[0082] This embodiment achieves precise and efficient determination of mobile phone screen adjustment parameters by pre-establishing a mapping table between the polarization axis angle and various adjustment parameters (including channel intensity changes of the RGB three primary colors, chromaticity coordinate offset, and luminance attenuation factor). Once the polarization axis angle is calculated, the system can directly query these mapping tables to quickly obtain the corresponding channel intensity changes, chromaticity coordinate offsets, and luminance attenuation factors. The channel intensity changes and chromaticity coordinate offsets are further integrated to generate a comprehensive color offset vector to guide color compensation. The luminance attenuation factor is directly used for luminance compensation. This mapping table-based method avoids complex real-time physical model calculations, thereby improving the efficiency and accuracy of parameter determination and ensuring optimized mobile phone screen display effects under different polarization axis angles.

[0083] Through the above technical solution, this embodiment can achieve rapid and accurate determination of mobile phone screen adjustment parameters. By using a preset mapping table, complex real-time calculations during runtime can be avoided, significantly improving the efficiency of parameter determination. Furthermore, by determining the brightness attenuation factor and the color offset vectors of the RGB primary colors separately, and further refining the color offset vector generation process, the compensation processing can more precisely adjust for the different effects of polarized sunglasses on brightness and color. This effectively improves the visibility and color reproduction of the image displayed on the mobile phone screen when the user is wearing polarized sunglasses, thus enhancing the user's viewing experience.

[0084] In some embodiments, in step S106, when the compensation process is brightness compensation, the compensation process for the mobile phone screen is performed according to the adjustment parameters of the mobile phone screen, which may include, but is not limited to, the following steps: Get the current brightness value of the phone screen; Calculate the target brightness gain based on the brightness attenuation factor; Calculate the target brightness value based on the current brightness value and the target brightness gain; The driving current of the screen backlight module is adjusted according to the target brightness value to compensate for the brightness.

[0085] In some embodiments, the current brightness value of the mobile phone screen can be obtained first. This can be achieved by reading register information from the mobile phone display driver chip or through API interfaces provided by the operating system.

[0086] Then, the target brightness gain is calculated based on the brightness attenuation factor. The brightness attenuation factor is a parameter determined by the polarization axis angle and is used to quantify the degree to which polarized sunglasses attenuate the brightness of the phone screen. Based on the brightness attenuation factor, the required factor or proportion of brightness enhancement to offset the brightness loss caused by the polarized sunglasses can be determined. For example, if the brightness attenuation factor is 0.5, the target brightness gain might be set to 2 to restore the final brightness to its original level. The formula for calculating the target brightness gain is: In the formula, For target brightness gain, This is the brightness attenuation factor.

[0087] Then, based on the current brightness value and the target brightness gain, the target brightness value is calculated. The current brightness value can be multiplied by the calculated target brightness gain to obtain the ideal brightness level that the screen should achieve after compensation.

[0088] Finally, based on the target brightness value, the driving current of the screen backlight module is adjusted to compensate for the brightness. By controlling the display driver chip or power management unit according to the calculated target brightness value, the current supplied to the screen backlight module can be precisely adjusted, thereby changing the backlight brightness and achieving brightness compensation.

[0089] This embodiment obtains the current brightness value of the mobile phone screen and calculates the precise target brightness gain by combining it with a brightness attenuation factor determined by the polarization axis angle, thereby obtaining the target brightness value. This allows for direct and precise adjustment of the drive current of the screen backlight module. This mechanism enables brightness compensation to dynamically adapt to changes in the polarization axis angle, ensuring that the brightness of the mobile phone screen is effectively enhanced when the user wears polarized sunglasses, thus offsetting the brightness loss caused by polarization and avoiding problems of insufficient or excessive brightness compensation.

[0090] To illustrate this technical solution more clearly, a specific example is used below. Assume a user is wearing polarized sunglasses while using their mobile phone outdoors. The system first obtains the current brightness value of the phone screen as 500 nits. Simultaneously, based on the user's head posture and the phone's posture information, it calculates that the angle between the polarization axis of the polarized sunglasses and the polarization axis of the phone screen is 45 degrees. According to a preset mapping table, this 45-degree angle corresponds to a brightness attenuation factor of 0.5. Based on this, the system calculates a target brightness gain of 2 (i.e., 1 / 0.5). Subsequently, the system multiplies the current brightness value of 500 nits by the target brightness gain of 2, obtaining a target brightness value of 1000 nits. Finally, the system sends a command to the screen backlight module to adjust its drive current, so that the actual brightness output of the phone screen reaches 1000 nits. In this way, when the user wears polarized sunglasses, the perceived screen brightness will be restored to approximately 500 nits, close to the level when not wearing sunglasses, thus providing a clear and comfortable viewing experience.

[0091] Through the above technical solution, this embodiment can achieve precise and dynamic compensation for mobile phone screen brightness. This not only significantly improves the visual comfort of users wearing polarized sunglasses and effectively solves the problem of screen dimming caused by polarization effect, but also avoids unnecessary power consumption waste by precisely adjusting the backlight drive current, thus ensuring both display effect and device energy efficiency.

[0092] The beneficial effects of implementing the embodiments of the present invention include: First, the embodiments of this application acquire user facial video stream and mobile phone spatial motion data. Then, based on the user facial video stream, head posture information is identified, and based on the mobile phone spatial motion data, an extended Kalman filter is used to identify mobile phone posture information. Then, based on the head posture information and mobile phone posture information, the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen is calculated. Based on the polarization axis angle, the mobile phone screen adjustment parameters are determined. Finally, based on the mobile phone screen adjustment parameters, brightness compensation and color compensation are performed on the mobile phone screen. Thus, the mobile phone screen adjustment parameters can be determined by combining head posture information and mobile phone posture information, and brightness compensation and color compensation can be performed to achieve image enhancement, improve accuracy and user experience.

[0093] like Figure 2 As shown, this embodiment of the invention also provides a mobile phone screen display image enhancement system, including: Data acquisition module 401 is used to acquire user facial video stream and mobile phone spatial motion data, including linear acceleration data, angular velocity data and motion speed data; The head posture information recognition module 402 is used to recognize head posture information based on the user's facial video stream. The head posture information is used to reflect the relative posture of the user's head relative to the mobile phone. The mobile phone posture information recognition module 403 is used to recognize the mobile phone posture information based on the mobile phone's spatial motion data using an extended Kalman filter. The polarization axis angle calculation module 404 is used to calculate the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information. The polarized sunglasses are worn on the user's head. The mobile phone screen adjustment parameter determination module 405 is used to determine the mobile phone screen adjustment parameters based on the polarization axis angle. The mobile phone screen adjustment parameters include the brightness attenuation factor and the color offset vector of the RGB three primary colors, which include red, blue and green. The compensation processing module 406 is used to perform compensation processing on the mobile phone screen according to the adjustment parameters of the mobile phone screen. The compensation processing includes brightness compensation and color compensation.

[0094] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0095] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

Claims

1. A method for enhancing images displayed on a mobile phone screen, characterized in that, Includes the following steps: Acquire user facial video stream and mobile phone spatial motion data, wherein the mobile phone spatial motion data includes linear acceleration data, angular velocity data and motion speed data; Based on the user's facial video stream, head posture information is identified, which reflects the relative posture of the user's head with respect to the mobile phone. Based on the mobile phone spatial motion data, an extended Kalman filter is used to identify the mobile phone's attitude information. Based on the head posture information and the mobile phone posture information, the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen is calculated, and the polarized sunglasses are worn on the user's head. Based on the polarization axis angle, the mobile phone screen adjustment parameters are determined. The mobile phone screen adjustment parameters include a brightness attenuation factor and a color offset vector of the RGB three primary colors, which include red, blue, and green. Based on the mobile phone screen adjustment parameters, compensation processing is performed on the mobile phone screen, including brightness compensation and color compensation.

2. The method according to claim 1, characterized in that, The step of identifying head pose information based on the user's facial video stream includes: Facial feature point detection is performed on the user's facial video stream to identify facial feature points, including the tip of the nose and the corners of the mouth. Based on the facial feature points, facial position change analysis is performed to obtain two-dimensional displacement data; Based on the two-dimensional displacement data, the three-dimensional rotation angle and the three-dimensional translation vector are calculated using an attitude estimation algorithm; The head posture information is identified based on the three-dimensional rotation angle and the three-dimensional translation vector.

3. The method according to claim 2, characterized in that, The step of analyzing facial position changes based on the facial feature points to obtain two-dimensional displacement data includes: Sunglasses feature point detection is performed on the user's facial video stream to identify sunglasses feature points, including the center point of the frame and the connection point between the frame and the temple. The target feature points are obtained by combining the sunglasses feature points and the facial feature points; The two-dimensional displacement data is obtained by analyzing the two-dimensional positional changes of the target feature points between consecutive frames of the user's facial video stream.

4. The method according to claim 1, characterized in that, The step of calculating the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information includes: The relative orientation of the first polarization axis of polarized sunglasses and the relative orientation of the second polarization axis of a mobile phone screen are obtained. The relative orientation of the first polarization axis is used to represent the directional relationship between the polarization axis of the polarized sunglasses and the direction of the user's eyes, and the relative orientation of the second polarization axis is used to represent the directional relationship between the polarization axis of the mobile phone screen and the direction of the mobile phone screen. Based on the mobile phone attitude information and the relative direction relationship of the second polarization axis, the polarization direction vector of the mobile phone screen in the global spatial coordinate system is determined; Based on the head posture information and the relative direction relationship of the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined. The polarization axis angle is calculated based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen.

5. The method according to claim 4, characterized in that, The step of determining the polarization direction vector of the mobile phone screen in the global spatial coordinate system based on the mobile phone attitude information and the relative direction relationship of the second polarization axis includes: Based on the phone's attitude information, determine the phone's absolute attitude in the global spatial coordinate system; Determine the orientation of the phone screen based on the phone's absolute orientation; Based on the relative orientation of the mobile phone screen and the second polarization axis, the polarization direction vector of the mobile phone screen in the global spatial coordinate system is determined.

6. The method according to claim 5, characterized in that, The step of determining the polarization direction vector of the polarized sunglasses in the global spatial coordinate system based on the head posture information and the relative direction relationship of the first polarization axis includes: Based on the head pose information and the absolute pose of the mobile phone, determine the absolute pose of the user's head in the global spatial coordinate system. Determine the user's eye orientation based on the absolute head posture; Based on the relationship between the user's eye orientation and the relative direction of the first polarization axis, the polarization direction vector of the polarized sunglasses in the global spatial coordinate system is determined.

7. The method according to claim 4, characterized in that, The step of calculating the polarization axis angle based on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen includes: Perform a vector dot product operation on the polarization direction vector of the sunglasses and the polarization direction vector of the mobile phone screen to obtain the direction vector dot product value; Perform a modulus operation on the polarization direction vector of the sunglasses to obtain the modulus of the polarization direction vector of the sunglasses; Perform a modulus operation on the polarization direction vector of the mobile phone screen to obtain the modulus of the polarization direction vector of the mobile phone screen; The polarization axis angle is calculated based on the dot product of the direction vectors, the magnitude of the polarization direction vector of the sunglasses, and the magnitude of the polarization direction vector of the mobile phone screen.

8. The method according to claim 1, characterized in that, The step of determining the mobile phone screen adjustment parameters based on the polarization axis angle includes: Based on the polarization axis angle, the channel intensity change is determined by consulting the mapping table between the polarization axis angle and the channel intensity changes of the RGB three primary colors; Based on the polarization axis angle, the chromaticity coordinate offset is determined by consulting the mapping table between the polarization axis angle and the chromaticity coordinate offset of the RGB three primary colors; The color offset vector is generated based on the channel intensity change and the chromaticity coordinate offset; The brightness attenuation factor is determined by consulting a mapping table between the polarization axis angle and the brightness attenuation factor.

9. The method according to claim 1, characterized in that, When the compensation process is brightness compensation, the step of performing compensation processing on the mobile phone screen according to the adjustment parameters of the mobile phone screen includes: Get the current brightness value of the phone screen; Calculate the target brightness gain based on the brightness attenuation factor; Calculate the target brightness value based on the current brightness value and the target brightness gain; Based on the target brightness value, the driving current of the screen backlight module is adjusted to compensate for the brightness.

10. A mobile phone screen display image enhancement system, characterized in that, include: The data acquisition module is used to acquire user facial video stream and mobile phone spatial motion data, wherein the mobile phone spatial motion data includes linear acceleration data, angular velocity data and motion speed data; The head posture information recognition module is used to recognize head posture information based on the user's facial video stream. The head posture information is used to reflect the relative posture of the user's head with respect to the mobile phone. The mobile phone posture information recognition module is used to identify mobile phone posture information using an extended Kalman filter based on the mobile phone spatial motion data. The polarization axis angle calculation module is used to calculate the polarization axis angle between the polarization axis of the polarized sunglasses and the polarization axis of the mobile phone screen based on the head posture information and the mobile phone posture information. The polarized sunglasses are worn on the user's head. The mobile phone screen adjustment parameter determination module is used to determine the mobile phone screen adjustment parameters based on the polarization axis angle. The mobile phone screen adjustment parameters include a brightness attenuation factor and a color offset vector of the RGB three primary colors, which include red, blue and green. The compensation processing module is used to perform compensation processing on the mobile phone screen according to the adjustment parameters of the mobile phone screen. The compensation processing includes brightness compensation and color compensation.

Images