Display method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a display method and device, computer equipment, a computer readable storage medium and a computer program product, and is applied to the technical field of artificial intelligence. The method comprises the following steps: monitoring mobile data of a user face relative to terminal equipment in real time; identifying the facial expression of the user when the mobile data meets an adjustment condition; under the condition that the facial expression of the user is a target facial expression, determining a target font adjustment strategy based on the mobile data; and performing font adjustment on display content in a display interface according to the target font adjustment strategy. By adopting the method, the font adjustment efficiency and precision can be effectively improved, and the user information acquisition efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a display method, device, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the progress of technology, terminal devices penetrate all aspects of people's lives. People can use the client installed on the terminal device to watch videos, read, send messages, make calls, etc.

[0003] During the use of the client, a large amount of text information is provided to the customer through the display interface. Due to the limitation of the size of the display interface by the terminal device, it is easy for users to experience visual fatigue during the reading process. Moreover, for users with presbyopia, myopia, visual impairment, etc., currently, the font size can only be set by the user's own adjustment method. The font size adjustment method is not only cumbersome and single, but also the adjustment is not timely enough, resulting in a significant reduction in the efficiency of users' information acquisition. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a display method, device, computer device, computer-readable storage medium, and computer program product that can simplify the font size adjustment operation and improve the timeliness of font size adjustment.

[0005] In a first aspect, this application provides a display method, and the method includes:

[0006] Real-time monitor the movement data of the user's face relative to the terminal device;

[0007] When the movement data meets the adjustment condition, identify the facial expression of the user;

[0008] When the facial expression of the user is the target facial expression, determine the target font adjustment strategy based on the movement data;

[0009] Adjust the font of the display content in the display interface according to the target font adjustment strategy.

[0010] In one embodiment, the method further includes:

[0011] When the movement data meets the movement condition, obtain the sensor data of the terminal device;

[0012] When the sensor data indicates that the movement behavior of the terminal device is true, determine that the movement data meets the adjustment condition;

[0013] When the sensor data indicates that the movement behavior of the terminal device is false, determine that the movement data does not meet the adjustment condition.

[0014] In one embodiment, determining the target font adjustment strategy based on the mobile data includes:

[0015] Determining a target movement behavior based on the mobile data, where the target movement behavior includes a pulling - closer behavior or a pulling - away behavior;

[0016] Determining a target font adjustment strategy according to the target movement behavior.

[0017] In one embodiment, when the target movement behavior is a pulling - closer behavior, the target font adjustment strategy is a first adjustment strategy. Adjusting the font of the display content in the display interface according to the target font adjustment strategy includes:

[0018] In response to the first adjustment strategy, enlarging the font size of the display content in the display interface to a target font size.

[0019] In one embodiment, when the target movement behavior is a pulling - away behavior, the target font adjustment strategy is a second adjustment strategy. Adjusting the font of the display content in the display interface according to the target font adjustment strategy includes:

[0020] In response to the second adjustment strategy, enlarging the font resolution of the display content in the display interface to a target font resolution.

[0021] In one embodiment, after adjusting the font of the display content in the display interface according to the target font adjustment strategy, the method further includes:

[0022] Determining the change in the user's facial expression;

[0023] When the change in the user's facial expression does not meet the stop condition, jump to the step of adjusting the font of the display content in the display interface using the target font adjustment strategy until the change in the user's facial expression meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of display anomalies of the display content in the display interface reaches the anomaly - count threshold.

[0024] In one embodiment, the method further includes:

[0025] After enlarging the font size of the display content in the display interface, if there is content overlap of text elements or a scroll bar appears, or non - text content is cropped or deformed, it is determined that the display content has a display anomaly;

[0026] Starting from the first detection of display anomalies, within a preset time period, the number of times of the display anomalies is accumulated in real time to obtain the number of display anomalies of the display content.

[0027] In one embodiment, after the font size of the display content in the display interface is enlarged to the target font size, the method further includes:

[0028] According to the target font size, synchronously perform layout adaptability adjustment on the display content.

[0029] In one embodiment, the method further includes:

[0030] Determine a target keyword from the display content in the display interface;

[0031] Circle the target display content from the display content according to the target keyword;

[0032] Display the target display content according to the display mode corresponding to the target keyword.

[0033] In a second aspect, the present application further provides a display device, and the device includes:

[0034] A movement monitoring unit, configured to monitor in real time the movement data of the user's face relative to the terminal device;

[0035] An identification unit, configured to identify the facial expression of the user when the movement data meets the adjustment condition;

[0036] A first determination unit, configured to determine a target font adjustment strategy based on the movement data when the facial expression of the user is the target facial expression;

[0037] A first font adjustment unit, configured to perform font adjustment on the display content in the display interface according to the target font adjustment strategy.

[0038] In one embodiment, the device further includes:

[0039] An acquisition unit, configured to acquire the sensor data of the terminal device when the movement data meets the movement condition;

[0040] A second determination unit, configured to determine that the movement data meets the adjustment condition when the sensor data indicates that the movement behavior of the terminal device is true;

[0041] A third determination unit, configured to determine that the movement data does not meet the adjustment condition when the sensor data indicates that the movement behavior of the terminal device is false.

[0042] In one embodiment, the first determination unit is further configured to:

[0043] Determine a target movement behavior based on the mobile data, where the target movement behavior includes a pulling - closer behavior or a pulling - farther behavior;

[0044] Determine a target font adjustment strategy according to the target movement behavior.

[0045] In one embodiment, when the target movement behavior is a pulling - closer behavior, the target font adjustment strategy is a first adjustment strategy, and the first font adjustment unit is further configured to:

[0046] In response to the first adjustment strategy, enlarge the font size of the content displayed in the display interface to a target font size.

[0047] In one embodiment, when the target movement behavior is a pulling - farther behavior, the target font adjustment strategy is a second adjustment strategy, and the first font adjustment unit is further configured to:

[0048] In response to the second adjustment strategy, enlarge the font resolution of the content displayed in the display interface to a target font resolution.

[0049] In one embodiment, the device further includes:

[0050] A fourth determination unit, configured to determine the change in the user's facial expression;

[0051] A second font adjustment unit, configured to, when the change in the user's facial expression does not meet the stop condition, jump to the step of adjusting the font of the content displayed in the display interface by using the target font adjustment strategy until the change in the user's facial expression meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of display anomalies of the content displayed in the display interface reaches the anomaly - count threshold.

[0052] In one embodiment, the device further includes:

[0053] A fifth determination unit, configured to determine that a display anomaly occurs if, after enlarging the font size of the content displayed in the display interface, there is content overlap or a scroll bar appears for text elements, or there is clipping or distortion for non - text content;

[0054] An accumulation unit, configured to, starting from the first detection of a display anomaly, accumulate the number of display anomalies in real time within a preset duration to obtain the number of display anomalies of the content displayed.

[0055] In one embodiment, the device further includes:

[0056] A layout adjustment unit for synchronously performing layout adaptation adjustment on the displayed content according to the target font size.

[0057] In one embodiment, the device further includes:

[0058] A sixth determination unit for determining a target keyword from the displayed content in the display interface;

[0059] A circumscribing unit for circumscribing the target displayed content from the displayed content according to the target keyword;

[0060] A display unit for displaying the target displayed content according to the display manner corresponding to the target keyword.

[0061] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, any of the above methods is implemented.

[0062] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any of the above methods is implemented.

[0063] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0064] The above display method, device, computer device, computer-readable storage medium, and computer program product can real-time monitor the movement data of the user's face relative to the terminal device, and identify the user's facial expression when the movement data meets the adjustment condition. In the case where the user's facial expression is the target facial expression, a target font adjustment strategy is determined based on the movement data, and the font of the displayed content in the display interface is adjusted according to the target font adjustment strategy. By using the display method, device, computer device, computer-readable storage medium, and computer program product provided by the embodiments of the present disclosure, the font of the displayed content in the display interface can be automatically adjusted based on the movement data of the user's face relative to the terminal device and the user's facial expression, without the need for the user to manually adjust the font, enriching the font adjustment method, simplifying the font adjustment operation, and being able to timely adjust the font to a font size suitable for the user to read, which can greatly improve the efficiency of font adjustment, and further improve the efficiency of user information acquisition. Description of the Drawings

[0065] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present application or related technologies. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related accompanying drawings can also be obtained based on these drawings.

[0066] Figure 1 It is a schematic flowchart showing the method in an embodiment;

[0067] Figure 2 It is a schematic flowchart showing the method in an embodiment;

[0068] Figure 3 It is a schematic flowchart of step 106 in an embodiment;

[0069] Figure 4 It is a schematic flowchart showing the method in an embodiment;

[0070] Figure 5 It is a schematic flowchart showing the method in an embodiment;

[0071] Figure 6 It is a schematic flowchart showing the method in an embodiment;

[0072] Figure 7 It is a schematic diagram showing the method in an embodiment;

[0073] Figure 8 It is a structural block diagram showing the device in an embodiment;

[0074] Figure 9 It is the internal structure diagram of a computer device in an embodiment. Specific Embodiments

[0075] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following further details the present application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0076] In an embodiment, as Figure 1 shown, a display method is provided. In this embodiment, it is exemplified that the method is applied to a terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0077] Step 102, monitor the movement data of the user's face relative to the terminal device in real time.

[0078] In the embodiments of the present disclosure, the mobile data may include the distance by which the user's face moves relative to the terminal device. The distance between the user's face and the terminal device can be continuously calculated, and the difference between the distance between the user's face and the terminal device calculated at the current moment and the previous moment is used as the distance by which the user's face moves relative to the terminal device. Here, both the current moment and the previous moment are time points for calculating the distance, and the time interval between them is a threshold duration, for example: an interval of 1 second, an interval of 2 seconds, etc.

[0079] Exemplarily, the built-in depth sensor and front camera of the terminal can be used to continuously calculate and record the distance between the user's face and the terminal screen, including: accurately calculating the distance between the user's face and the terminal screen by calculating the time difference of the infrared light from emission to reflection back to the receiver, or measuring the depth by projecting a specific pattern and analyzing its deformation.

[0080] In one example, the straight-line distance between the center point of the face (or key feature points) and the terminal screen can be calculated based on the depth sensor data combined with the coordinate information of the face in the image. The three-dimensional coordinates (Xf, Yf, Zf) of the face on the depth image are calculated, where Zf represents the distance from the face to the sensor. Combining the terminal camera parameters (focal length, principal point coordinates, etc.), the three-dimensional coordinates are converted into two-dimensional coordinates (Xs, Ys) of the terminal screen. Considering the physical size and screen size of the terminal, the distance Ds from the face to the terminal screen can be calculated as Ds = Zf × (screen diagonal length / diagonal length corresponding to the field of view angle).

[0081] To improve the measurement accuracy, a multi-point measurement and average value strategy can be adopted, while considering the influence of factors such as ambient light and temperature, and the distance is corrected for errors through a machine learning model, including:

[0082] First, a large number of actual measurement sample data are collected, including but not limited to: raw readings of the depth sensor, position information of the face in the image, environmental factors (such as light intensity, temperature, etc.), and true distance values measured by known standard methods or devices. The collected data is preprocessed and feature extracted. Possible features include but are not limited to: depth value Zf, two-dimensional coordinates (Xs, Ys), mobile phone hardware status parameters, environmental parameters, etc. In addition, additional geometric features or statistical characteristics can be calculated to assist in modeling.

[0083] A regression model (such as a neural network regression model) is trained using a supervised learning method. The goal of the model is to predict the true distance between the face and the screen based on the existing feature inputs. The data set is divided into a training set and a validation set, and the training set is used to train the model to minimize the difference between the prediction result and the true distance. The performance of the model is evaluated through cross-validation or an independent validation set, and the model hyperparameters are tuned.

[0084] The trained model can output a corrected distance estimate based on the currently collected sensor data and other relevant features. When there is a deviation in the distance measured by the original sensor, the model will correct this value based on the previously learned rules. During actual application, the model can continuously learn from newly collected data and perform real-time updates to adapt to system changes caused by different customer habits, mobile phone usage scenarios, and hardware aging, etc.

[0085] In the embodiments of the present disclosure, the measurement result can be updated and displayed in real time. If the distance is too far or too close to meet the security authentication requirements, the detection will be stopped and a marker point will be generated and transmitted to the background for use when there are special requirements. In one example, face recognition technology can also be used to detect and track the customer's face in real time, and obtain information such as the position and size of the face in the camera's field of view to ensure that the customer is always the same person during continuous operation.

[0086] Step 104, when the mobile data meets the adjustment condition, identify the user's facial expression.

[0087] In the embodiments of the present disclosure, the adjustment condition can be a condition for determining that the user actively adjusts the distance between the face and the terminal screen. In the embodiments of the present disclosure, when the mobile data meets the adjustment condition, it can be determined that the user actively adjusts the distance between the face and the terminal screen. This operation may be because the user cannot clearly see the information content displayed on the screen, or it may be due to other user behavior habits. Therefore, at this time, the user's facial expression can be identified, and the motivation of the user's operation behavior can be further judged in combination with the user's facial expression.

[0088] In the embodiments of the present disclosure, a deep learning-based facial expression recognition model can be used to analyze the user's facial expression in real time. Exemplarily, if the user shows expressions such as squinting or frowning, indicating blurred vision or efforts to focus, it may indicate that their operation behavior is for adjusting the viewing distance. Further, by setting a threshold or establishing a probability model, when the probability values of expressions such as squinting or frowning, indicating blurred vision or efforts to focus, are detected, the probability values can be compared with the set probability threshold. When the detected probability exceeds the set probability threshold (the probability threshold can be flexibly configured according to actual needs), the system can further confirm that the user's operation behavior is for adjusting the viewing distance.

[0089] It should be noted that in the embodiments of the present disclosure, no specific limitation is imposed on the method for recognizing the user's facial expression, and any method that can implement facial expression recognition is applicable to the embodiments of the present disclosure.

[0090] Step 106, when the user's facial expression is the target facial expression, determine the target font adjustment strategy based on the mobile data.

[0091] In the embodiments of the present disclosure, when it is recognized and determined that the facial expression of the current user is the target facial expression, it can be determined that the user is indeed performing reading distance adjustment currently, that is, it can be determined that the current font is not suitable for the user, so font adjustment is required. The mobile behavior of the user can be determined based on the mobile data, and the corresponding target font adjustment strategy can be matched according to the mobile behavior. The target font adjustment strategy is a strategy for adjusting the font size and / or the font resolution of the text. Among them, the target facial expression is any expression in the preset expression set, for example, expressions such as squinting and frowning that indicate that the user is currently focusing.

[0092] Step 108, adjust the font of the display content in the display interface according to the target font adjustment strategy.

[0093] In the embodiments of the present disclosure, after the target font adjustment strategy is determined, the font of the current display content in the display interface can be adjusted based on the target font adjustment strategy, including: enlarging the font size and / or enlarging the font resolution, etc.

[0094] The above display method can monitor the mobile data of the user's face relative to the terminal device in real time, and recognize the user's facial expression when the mobile data meets the adjustment conditions. When the user's facial expression is the target facial expression, the target font adjustment strategy is determined based on the mobile data, and the font of the display content in the display interface is adjusted according to the target font adjustment strategy. By using the display method provided in the embodiments of the present disclosure, the font of the display content in the display interface can be automatically adjusted based on the mobile data of the user's face relative to the terminal device and the user's facial expression, without the user manually adjusting the font, enriching the font adjustment method, simplifying the font adjustment operation, and being able to timely adjust the font to a font size suitable for the user to read, which can greatly improve the efficiency of font adjustment and thus improve the efficiency of user information acquisition.

[0095] In an exemplary embodiment, referring to Figure 2 as shown, the above method may further include:

[0096] Step 202, when the mobile data meets the movement conditions, obtain the sensor data of the terminal device;

[0097] Step 204, when the sensor data indicates that the movement behavior of the terminal device is true, determine that the mobile data meets the adjustment conditions;

[0098] Step 206, when the sensor data indicates that the movement behavior of the terminal device is false, determine that the mobile data does not meet the adjustment conditions.

[0099] In the embodiments of the present disclosure, the movement condition may be a condition for determining whether the user is adjusting the distance between the user's face and the terminal device. Exemplarily, the movement condition may include that the amplitude of distance change is greater than the amplitude threshold. After the distance between the user's face and the terminal screen detected in real time, it can be determined whether the user is adjusting the distance between the face and the terminal screen at this time by the amplitude of real-time distance change (the difference between the distance between the user's face and the terminal device calculated at the current moment and the previous moment). For example, when the amplitude of change exceeds the amplitude threshold, it can be determined that the user is adjusting the distance between the face and the terminal screen at this time, and the amplitude threshold can be flexibly configured according to actual needs.

[0100] In the embodiments of the present disclosure, on the basis of obtaining the distance between the user's face and the terminal screen in real time, a more in-depth dynamic analysis of the distance can be further performed to identify whether the customer is actively moving the screen closer or farther away, including: in order to eliminate the distance data fluctuations caused by factors such as slight shaking of the device and measurement errors, a moving average filter can be used to smooth the calculated distance between the user's face and the terminal screen to obtain distance data. First-order difference (differentiation) is performed on the continuous time-series distance data to obtain the speed change, and second-order difference (second-order differentiation) can obtain the acceleration change. The change in speed reflects the trend of distance change, and the change in acceleration usually corresponds to the intensity and direction of the customer's actions.

[0101] Set the speed change threshold and the slope change threshold. When the slope (i.e., the rate of change) of the speed change continuously exceeds the speed change threshold for a period of time, it can be determined that the user is making a large-amplitude and purposeful action. At the same time, if the continuously monitored speed values show an obvious positive or negative growth trend (for example, increasing or decreasing a certain distance unit per second), and this trend has been maintained for a period of time, then it can be preliminarily judged that the customer is actively moving the screen closer or farther away. Exemplarily, the daily operation habits of the customer can also be combined to learn and establish a distance change model for normal use. For example, the distance change during typing usually has specific amplitude and frequency characteristics, while the behavior of actively adjusting the distance often has a larger amplitude and a relatively stable rate. In the embodiments of the present disclosure, the above analysis can be performed within a short time window (such as 0.5 seconds to 2 seconds), because the distance change caused by normal operation swings often quickly returns to its original state within a short time, while the behavior of actively changing the distance will continue to be reflected within the window period.

[0102] Meanwhile, to more accurately determine whether the customer is actively adjusting the distance between the face and the screen, sensors such as the gyroscope and accelerometer of the terminal device can be further combined to make a further judgment. When it is determined that the mobile data currently meets the movement conditions, it can be determined that there is indeed an adjustment in the distance between the face and the screen. At this time, the sensor data of the terminal device can be further obtained, and the sensor data can be used to determine whether the movement behavior of the terminal device is a behavior for a reading event.

[0103] Exemplarily, the built-in gyroscope of the mobile phone can be used to detect the rotation angle and dynamic changes of the device, and the movement trend and speed of the terminal device can be evaluated by calculating the change in angular velocity between consecutive time points. The accelerometer can sense the acceleration changes of the mobile phone in three axes (X, Y, Z). When the user brings the mobile phone closer to or farther from the face, specific patterns of acceleration changes usually occur. When it is determined that the movement trend and speed of the terminal device meet the movement requirements of the reading event (for example: the movement trend and speed match the preset trend and speed), and / or the acceleration change meets the movement requirements of the reading event (for example: the acceleration change is greater than or equal to the change threshold), it can be determined that the movement behavior of the terminal device is a behavior for the reading event, that is, the current sensor data indicates that the movement behavior of the terminal device is true; otherwise, it can be determined that the movement behavior of the terminal device is not a behavior for the reading event, that is, the current sensor data indicates that the movement behavior of the terminal device is false.

[0104] When the sensor data indicates that the movement behavior of the terminal device is true, it can be determined that the mobile data meets the adjustment conditions, and the user's facial expression can be further recognized to further confirm whether font adjustment is required based on the user's facial expression. Or, when the sensor data indicates that the movement behavior of the terminal device is false, it can be determined that the mobile data does not meet the adjustment conditions, that is, the current movement behavior is not a behavior for the reading event, so there is no need to perform font adjustment, that is, there is no need to perform user facial expression recognition.

[0105] The display method provided by the embodiments of the present disclosure can further confirm the user's movement behavior through sensor data, and then realize the recognition of the user's facial expression only when it is determined that the user's movement behavior is a behavior for the reading event, which can greatly reduce the calculation amount and save the resources of the terminal device. Moreover, for the display method provided by the embodiments of the present disclosure, font adjustment can be performed only when both the distance change amplitude, the gyroscope acceleration, and the user's facial expression reflect that the user makes a distance adjustment for the reading event, which can greatly improve the accuracy and efficiency of font adjustment.

[0106] In the embodiments of the present disclosure, with reference to Figure 3 As shown, in step 106, determining the target font adjustment strategy based on the mobile data may include:

[0107] Step 302: Determine a target movement behavior based on mobile data, where the target movement behavior includes a pulling - closer behavior or a pulling - away behavior.

[0108] Step 304: Determine a target font adjustment strategy according to the target movement behavior.

[0109] In the embodiments of the present disclosure, a target movement behavior can be determined based on mobile data, and the target movement behavior includes a pulling - closer behavior or a pulling - away behavior. Exemplarily, the current target movement behavior being a pulling - closer behavior or a pulling - away behavior can be determined by the direction corresponding to the acceleration (a pulling - closer behavior is towards the user, and a pulling - away behavior is towards the terminal device), or it can also be determined based on the change in the distance between the user's face and the terminal device (a pulling - closer behavior is when it gets closer and closer, and a pulling - away behavior is when it gets farther and farther).

[0110] After determining the target movement behavior, a corresponding target adjustment strategy can be determined based on the target movement behavior. Exemplarily, when the target movement behavior is a pulling - closer behavior, the font size can be enlarged, and when the target movement behavior is a pulling - away behavior, the font resolution can be enlarged.

[0111] The display method provided by the embodiments of the present disclosure can match a corresponding adjustment strategy based on the user's movement behavior for font adjustment, which can improve the accuracy and efficiency of font adjustment.

[0112] In an exemplary embodiment, when the target movement behavior is a pulling - closer behavior, the target font adjustment strategy is the first adjustment strategy. Adjusting the font of the display content in the display interface according to the target font adjustment strategy includes:

[0113] In response to the first adjustment strategy, enlarge the font size of the display content in the display interface to the target font size.

[0114] In an embodiment of the present disclosure, when the target adjustment strategy is the first adjustment strategy, the font size of the displayed content can be enlarged to the target font size in response to the first adjustment strategy. In one example, the font size can be directly enlarged by one level. For example, if the current font size is "small four", the font size can be adjusted to "size four". In another example, the correspondence between the font size and the distance between the user's face and the terminal device can be pre-set. According to the distance between the user's face and the terminal device, the corresponding font size is matched as the target font size, and the font size of the displayed content in the terminal device display interface is adjusted to the target font size. In another example, the magnification ratio can be determined, and the font size can be enlarged based on the magnification ratio, for example, the font size can be enlarged at a magnification ratio of 1.2 times, wherein the magnification ratio can be a pre-set ratio, or a ratio determined based on the movement speed and / or acceleration corresponding to the target movement behavior, wherein the magnification ratio can be positively correlated with the movement speed and / or acceleration, that is, the greater the movement speed and / or acceleration, the greater the magnification ratio.

[0115] In an exemplary embodiment, after the font size of the displayed content in the display interface is enlarged to the target font size, the layout of the displayed content can be adaptively adjusted synchronously according to the target font size. Exemplarily, in the embodiment of the present disclosure, relative units such as em, rem or percentage can be used to define the size and spacing of elements instead of fixed pixel values. In this way, when the font size changes, the size and spacing of the elements will also be adjusted accordingly. Alternatively, different layout rules and font size ranges can be pre-configured, and media queries in CSS (Cascading Style Sheets) can be used to apply different layout styles according to font size or screen size. Alternatively, JavaScript can be used to monitor changes in font size and adjust the page layout accordingly. The embodiment of the present disclosure does not specifically limit the method of layout adaptive adjustment.

[0116] In an exemplary embodiment, when the target movement behavior is a zooming behavior, the target font adjustment strategy is a second adjustment strategy, and the font of the displayed content in the display interface is adjusted according to the target font adjustment strategy, including: in response to the second adjustment strategy, the font resolution of the displayed content in the display interface is enlarged to the target font resolution.

[0117] In an embodiment of the present disclosure, when the target adjustment strategy is the second adjustment strategy, the font resolution of the displayed content can be enlarged in response to the second adjustment strategy. In one example, the font resolution can be directly enlarged by one level; alternatively, the corresponding relationship between the font resolution and the distance between the user's face and the terminal device can be preset in advance, and according to the distance between the user's face and the terminal device, the corresponding font resolution can be matched as the target font resolution, and the font resolution of the displayed content in the display interface of the terminal device can be adjusted to the target font resolution. In another example, the resolution magnification factor can be determined, and the font resolution can be enlarged based on the resolution magnification factor, where the resolution magnification factor can be a preset magnification factor or a magnification factor determined based on the moving speed and / or acceleration corresponding to the target moving behavior, and the resolution magnification factor can be positively correlated with the moving speed and / or acceleration, that is, the greater the moving speed and / or acceleration, the greater the resolution magnification factor.

[0118] By using the display method provided by the embodiments of the present disclosure, the font size can be adaptively enlarged when the user performs a zoom-in operation, and the font resolution can be adaptively enlarged when the user performs a zoom-out operation, which can greatly improve the font adjustment accuracy and efficiency.

[0119] In an exemplary embodiment, referring to Figure 4 as shown, after the font of the displayed content in the display interface is adjusted according to the target font adjustment strategy, the method further includes:

[0120] Step 402, determining the change in the user's facial expression;

[0121] Step 404, when the change in the user's facial expression does not meet the stop condition, jump to the step of adjusting the font of the displayed content in the display interface by using the target font adjustment strategy until the change in the user's facial expression meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of abnormal display times of the displayed content in the display interface reaches the abnormal number threshold.

[0122] In an embodiment of the present disclosure, after the font is adjusted, the user's facial expression can be continuously recognized, and the change in the user's facial expression can be analyzed. When it is recognized that the user's facial expression relaxes, it can be determined that the change in the user's facial expression meets the stop condition, and the font enlargement or resolution increase can be stopped.

[0123] Exemplarily, a front camera can be used to capture continuous user facial images, and an expression recognition algorithm can be applied to analyze the user's micro-expression features, such as the degree of eye closure (used to determine whether the eyes are squinted), the curvature of the eyebrows (used to determine whether frowning), etc. By calculating the comparison between the above features and a preset threshold, if it is found that the degree of squinting and / or frowning of the customer is reduced to the normal level, it is determined that the user's facial expression is in a relaxed state. Alternatively, the facial expressions before and after font adjustment can be compared in real time, such as comparing the degree of eye closure and / or the curvature of the eyebrows. In the case where the difference in the degree of eye closure and / or the curvature of the eyebrows is greater than the preset threshold, it can be determined that the user's facial expression tends to be relaxed.

[0124] In the case where the change in the user's facial expression does not meet the stop condition, it can be determined that the adjusted font is still not suitable for the user. Therefore, the font of the display content in the display interface can continue to be adjusted based on the target font adjustment strategy until the change in the user's facial expression meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of display anomalies of the display content in the display interface reaches the anomaly count threshold, at which point the font adjustment is stopped. At the same time, the user's font habit preference at this time is automatically recorded and saved for automatic application when accessing next time.

[0125] If the user's relaxed expression is never recognized, the following method is used to stop font enlargement and resolution increase at an appropriate time. Exemplarily, the font size scaling or resolution increase coefficient can be initialized to 1 (representing the original font size or original resolution), and the maximum magnification limit is set. The current screen resolution, window size, and page layout information are obtained. The distance data between the human face and the screen is obtained in real time. When the distance decreases, the font automatic enlargement mechanism is triggered, and when the distance increases, the font resolution increase mechanism is triggered, and the font scaling or resolution increase coefficient is adjusted according to the distance change. After each font enlargement, the text and non-text content on the interface are re-layouted and evaluated.

[0126] In one exemplary embodiment, referring to Figure 5 as shown, the above method further includes:

[0127] Step 502, after enlarging the font size of the display content in the display interface, if the text elements have content overlap or a scroll bar appears, or the non-text content is cropped or deformed, it is determined that the display content has a display anomaly;

[0128] Step 504, starting from the first detection of the display anomaly, the number of display anomalies of the display content is accumulated in real time within a preset duration to obtain the number of display anomalies of the display content.

[0129] In the embodiments of the present disclosure, after the font size of the displayed content in the display interface is enlarged, if content overlap occurs in the text elements or a scroll bar appears, it can be determined that there is an abnormal display of the displayed content. Exemplarily, the lengths, number of lines, and display integrity of all text elements in the interface can be calculated and compared. When a certain text element has content overlap due to font enlargement, or the number of lines is too many to exceed the screen area and a scroll bar appears, or the sentence is truncated and cannot form a complete semantic meaning, it is recorded as an abnormal display.

[0130] For non-text content, such as pictures, charts, etc., when non-text content is cropped or deformed due to font enlargement, affecting its normal display and understanding, it is recorded as an abnormal display. The occupied space of non-text content at different font sizes can be calculated, and it is recorded as an abnormal display when it exceeds its container area.

[0131] Starting from the occurrence of the first abnormal display, within a preset duration, an accumulator is enabled to accumulate the number of times of abnormal display that occurs, and the number of abnormal displays of the displayed content is obtained. When the number of abnormal displays reaches the abnormal number threshold, the font enlargement operation can be stopped. If the number of abnormal displays does not reach the abnormal number threshold within the preset duration, the accumulator can be cleared.

[0132] If the distance between the subsequent user's face and the terminal device continues to decrease, but the font size has reached the upper limit size, other methods can be tried to optimize the user experience. For example, a prompt message is displayed, and through this prompt message, it is prompted to switch the reading mode, or the displayed content of the display interface is read out by voice, etc.

[0133] By using the display method provided in the embodiments of the present disclosure, during the font adjustment process, the clarity and integrity of the displayed content can be fully ensured, and the user's reading experience can be greatly improved.

[0134] In an exemplary embodiment, with reference to Figure 6 as shown, the method may further include:

[0135] Step 602, determine a target keyword from the displayed content in the display interface;

[0136] Step 604, circle the target displayed content from the displayed content according to the target keyword;

[0137] Step 606, display the target displayed content according to the display method corresponding to the target keyword.

[0138] In the embodiments of the present disclosure, a keyword set can be preset in advance to identify whether a keyword in the keyword set is hit in the currently displayed interface, and the hit keyword is used as the target keyword. Exemplarily, based on the target keyword, the target display content can be circled from the display content of the display interface. For example, the preset number of texts adjacent to the target keyword before and after and the target keyword are used as the target display content.

[0139] A corresponding display method is set for each target keyword. The display method can include ways such as using boldface, a specific font, a specific font color, setting an underline, and highlighting. In the embodiments of the present disclosure, the display method is not specifically limited. After determining the target display content, the target display content can be displayed in the display method corresponding to the target keyword. For example, the target display content is highlighted to ensure that the delivery of key information is not affected by font adjustment.

[0140] To enable those skilled in the art to better understand the embodiments of the present disclosure, the embodiments of the present disclosure are illustrated below through specific examples.

[0141] Refer to Figure 7 As shown, the distance between the user's face and the terminal device can be continuously detected. Based on the distance between the user's face and the terminal device, sensor data, and the user's facial expression, it can be determined whether the user makes a reading distance adjustment. After determining that the user makes a reading distance adjustment, the user's movement behavior can be determined. The movement behavior can include a pulling-close behavior and a pulling-away behavior, and the font adjustment strategy corresponding to the movement behavior is executed for font adjustment. After adjusting the font, the user's facial expression can be continuously monitored, and it is determined whether to continue font adjustment based on the user's facial expression, including: before the user's facial expression relaxes, continue to adjust the font based on the font adjustment strategy until the user's facial expression relaxes, or the font size reaches the upper limit, or the number of times of display abnormality reaches the abnormality times threshold, and then stop font adjustment.

[0142] By using the display method provided in the embodiments of the present disclosure, when the user uses the client, the font adjustment mechanism is started, the distance between the user's face and the mobile phone screen is continuously detected, and it is determined whether the user has an action of pulling the mobile phone closer or farther. If so, the corresponding screen display is adjusted and the adjustment is stopped in a timely manner, so as to improve the presentation effect for the user, automatically adjust the presentation methods such as font size for users with presbyopia, myopia, visual impairment, etc., improve the efficiency of the user's information acquisition, and enhance the user experience.

[0143] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0144] Based on the same inventive concept, an embodiment of the present application further provides a display device for implementing the above-mentioned display method. The solution for solving the problem provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following display devices can refer to the limitations on the display method in the above text and will not be repeated here.

[0145] In an exemplary embodiment, as Figure 8 shown, a display device is provided, including: a movement monitoring unit 802, an identification unit 804, a first determination unit 806, and a first font adjustment unit 808, where:

[0146] The movement monitoring unit 802 is configured to monitor in real time the movement data of the user's face relative to the terminal device;

[0147] The identification unit 804 is configured to identify the user's facial expression when the movement data meets the adjustment condition;

[0148] The first determination unit 806 is configured to determine a target font adjustment strategy based on the movement data when the user's facial expression is a target facial expression;

[0149] The first font adjustment unit 808 is configured to adjust the font of the display content in the display interface according to the target font adjustment strategy.

[0150] The above display device can monitor in real time the movement data of the user's face relative to the terminal device, and when the movement data meets the adjustment condition, identify the user's facial expression. When the user's facial expression is the target facial expression, based on the movement data, determine the target font adjustment strategy, and adjust the font of the display content in the display interface according to the target font adjustment strategy. By using the display device provided in the embodiments of the present disclosure, the font of the display content in the display interface can be automatically adjusted based on the movement data of the user's face relative to the terminal device and the user's facial expression, without the need for the user to manually adjust the font, which enriches the font adjustment method, simplifies the font adjustment operation, and can timely adjust the font to a font size suitable for the user to read, which can greatly improve the efficiency of font adjustment, and further improve the user's information acquisition efficiency.

[0151] In one embodiment, the device further includes:

[0152] An acquisition unit, configured to acquire the sensor data of the terminal device when the movement data meets the movement condition;

[0153] A second determination unit, configured to determine that the movement data meets the adjustment condition when the sensor data indicates that the movement behavior of the terminal device is true;

[0154] A third determination unit, configured to determine that the movement data does not meet the adjustment condition when the sensor data indicates that the movement behavior of the terminal device is false.

[0155] In one embodiment, the first determination unit 806 is further configured to:

[0156] Determine a target movement behavior based on the movement data, where the target movement behavior includes a closer behavior or a farther behavior;

[0157] Determine a target font adjustment strategy according to the target movement behavior.

[0158] In one embodiment, when the target movement behavior is a closer behavior, the target font adjustment strategy is a first adjustment strategy, and the first font adjustment unit 808 is further configured to:

[0159] In response to the first adjustment strategy, enlarge the font size of the display content in the display interface to the target font size.

[0160] In one embodiment, when the target movement behavior is a farther behavior, the target font adjustment strategy is a second adjustment strategy, and the first font adjustment unit is further configured to:

[0161] In response to the second adjustment strategy, enlarge the font resolution of the display content in the display interface to the target font resolution.

[0162] In one embodiment, the device further includes:

[0163] A fourth determination unit, configured to determine a change in the user's facial expression;

[0164] A second font adjustment unit, configured to, when the change in the user's facial expression does not meet the stop condition, jump to the step of adjusting the font of the display content in the display interface by using the target font adjustment strategy until the change in the user's facial expression meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of display anomalies of the display content in the display interface reaches the anomaly number threshold.

[0165] In one embodiment, the device further includes:

[0166] A fifth determination unit, configured to, after enlarging the font size of the display content in the display interface, if content overlap occurs in the text elements or a scroll bar appears, or clipping or deformation occurs in the non-text content, determine that a display anomaly occurs in the display content;

[0167] An accumulation unit, configured to start from the first detection of a display anomaly and accumulate the number of display anomalies in real time within a preset time period to obtain the number of display anomalies of the display content.

[0168] In one embodiment, the device further includes:

[0169] A layout adjustment unit, configured to synchronously perform layout adaptability adjustment on the display content according to the target font size.

[0170] In one embodiment, the device further includes:

[0171] A sixth determination unit, configured to determine a target keyword from the display content in the display interface;

[0172] A circling unit, configured to circle the target display content from the display content according to the target keyword;

[0173] A display unit, configured to display the target display content according to the display mode corresponding to the target keyword.

[0174] Each module in the above display device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form so that the processor can call and execute the operations corresponding to the above respective modules.

[0175] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in Figure 9 . The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. The computer program, when executed by the processor, implements a display method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0176] Those skilled in the art can understand that Figure 9 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0177] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0179] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0181] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0182] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0183] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A display method, characterized in that, The method includes: Real-time monitoring of the movement data of the user's face relative to the terminal device; When the movement data meets the adjustment condition, identifying the facial expression of the user; When the facial expression of the user is the target facial expression, determining a target font adjustment strategy based on the movement data; Adjusting the font of the display content in the display interface according to the target font adjustment strategy.

2. The method according to claim 1, wherein The method further includes: When the movement data meets the movement condition, obtaining the sensor data of the terminal device; When the sensor data indicates that the movement behavior of the terminal device is true, determining that the movement data meets the adjustment condition; When the sensor data indicates that the movement behavior of the terminal device is false, determining that the movement data does not meet the adjustment condition.

3. The method according to claim 1 or 2, characterized in that, The determining of the target font adjustment strategy based on the movement data includes: Determining a target movement behavior based on the movement data, where the target movement behavior includes a pulling-near behavior or a pulling-away behavior; Determining a target font adjustment strategy according to the target movement behavior.

4. The method according to claim 3, characterized in that When the target movement behavior is the pulling-near behavior, the target font adjustment strategy is the first adjustment strategy, and the adjusting of the font of the display content in the display interface according to the target font adjustment strategy includes: In response to the first adjustment strategy, enlarging the font size of the display content in the display interface to the target font size.

5. The method according to claim 3, wherein When the target movement behavior is the pulling-away behavior, the target font adjustment strategy is the second adjustment strategy, and the adjusting of the font of the display content in the display interface according to the target font adjustment strategy includes: In response to the second adjustment strategy, enlarging the font resolution of the display content in the display interface to the target font resolution.

6. The method according to claim 4, wherein After the adjusting of the font of the display content in the display interface according to the target font adjustment strategy, the method further includes: Determining the change in the facial expression of the user; When the change in the facial expression of the user does not meet the stop condition, jumping to the step of adjusting the font of the display content in the display interface using the target font adjustment strategy until the change in the facial expression of the user meets the stop condition, or the current font size is enlarged to the upper limit size, or the number of display anomalies of the display content in the display interface reaches the anomaly number threshold.

7. The method according to claim 6, wherein The method further includes: After enlarging the font size of the display content in the display interface, if there is content overlap or a scroll bar appears for text elements, or there is cropping or deformation for non-text content, then determining that the display content has a display anomaly; Starting from the first detection of a display anomaly, in a preset time period, accumulating the number of display anomalies in real time to obtain the number of display anomalies of the display content.

8. The method according to claim 4, characterized in that After enlarging the font size of the display content in the display interface to the target font size, the method further includes: According to the target font size, synchronously performing a layout adaptability adjustment on the display content.

9. The method according to claim 1, wherein The method further includes: Determining a target keyword from the display content in the display interface; Circling the target display content from the display content according to the target keyword; The target display content is displayed according to the display mode corresponding to the target keyword.

10. A display device, characterized in that, The device includes: A movement monitoring unit, configured to monitor in real time movement data of a user's face relative to a terminal device; An identification unit, configured to identify a facial expression of the user when the movement data meets an adjustment condition; A first determination unit, configured to determine a target font adjustment strategy based on the movement data when the facial expression of the user is a target facial expression; A first font adjustment unit, configured to adjust the font of the display content in a display interface according to the target font adjustment strategy.

11. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images