Display method and device and electronic equipment

Abstract

The invention provides a display method and device and electronic equipment. The method comprises the steps of obtaining feature data; inputting the image data into an image feature extraction layer of the initial model, and outputting image features; inputting the environment data into an environment feature extraction layer of the initial model, and outputting environment features; inputting the static data into a static feature extraction layer of the initial model, and outputting static features; inputting the image features, the environment features and the static features into a feature fusion layer of the initial model to obtain target fusion features, thereby adjusting model parameters of the initial model to obtain a trained display model; the display model represents the visual fatigue degree of the user and a screen display parameter adjustment strategy. According to the method, the related data of the user can be collected in real time, the model parameters are continuously updated and adjusted according to the related data of the user to obtain the display model, and the display model can identify visual fatigue symptoms possibly occurring to the user, adjust the parameters of the display equipment in time and relieve eye burdens of the user.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a display, device and electronic equipment. Background Art

[0002] The widespread use of display technology in education, healthcare and entertainment, as well as people's dependence on electronic devices, which can cause visual fatigue to the eyes, has made visual health issues a key concern for researchers around the world. Although eye adaptation mechanisms allow the human visual system to work within the recommended range of ambient light levels, long-term use of electronic devices has led to an increasing number of vision health problems, with symptoms such as sore eyes, pain, tearing, and other more serious symptoms including severe vision loss.

[0003] Existing eye protection methods applied to electronic devices are usually based on fixed parameter settings and cannot dynamically adapt to the user's actual usage. Summary of the invention

[0004] The present disclosure provides a display, a device and an electronic device to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, a display method is provided, the method comprising:

[0006] Acquire feature data; wherein the feature data includes image data, environmental data and static data;

[0007] Input the image data into the image feature extraction layer of the initial model and output the image features; input the environment data into the environment feature extraction layer of the initial model and output the environment features; input the static data into the static feature extraction layer of the initial model and output the static features;

[0008] Inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features;

[0009] The model parameters of the initial model are adjusted based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

[0010] In one possible implementation, the image data includes user image data acquired by an image acquisition device; the user image data includes user facial data and user eye data;

[0011] The environmental data includes the ambient brightness, ambient humidity, the distance between the user and the device, and the time the user uses the device;

[0012] The static data includes user age and gender, device type, and screen content.

[0013] In one possible implementation manner, inputting the image features, environmental features, and static features into the feature fusion layer of the initial model to obtain target fusion features includes:

[0014] Fusing the image feature and the environment feature to obtain a first fused feature;

[0015] The first fusion feature is fused with the static feature to obtain a target fusion feature.

[0016] In one possible implementation manner, the model parameters of the initial model include: a first model parameter of an image feature extraction layer of the initial model, a second model parameter of an environment feature extraction layer of the initial model, and a third model parameter of a static feature extraction layer of the initial model; and adjusting the model parameters of the initial model based on the target fusion feature includes:

[0017] According to the loss function between the target fusion feature and the target task, the first model parameter, the second model parameter and the third model parameter of the initial model are adjusted.

[0018] In one possible implementation, the image feature extraction layer uses a convolutional neural network;

[0019] The environmental feature extraction layer adopts a long short-term memory network or an attention mechanism module;

[0020] The static feature extraction layer adopts a deep learning neural network.

[0021] In one possible implementation, the screen display parameter adjustment strategy includes:

[0022] Suggestions for adjusting screen display parameters and reminders for users to move away from the screen;

[0023] The screen display parameters include contrast, display brightness and color temperature.

[0024] In one possible implementation manner, after obtaining the trained display model, the method further includes:

[0025] New feature data and user feedback data are collected regularly to retrain the display model.

[0026] In one possible implementation manner, after acquiring the characteristic data, the method further includes: preprocessing the characteristic data, specifically including:

[0027] Normalizing, cropping and gray-scaling the image data;

[0028] Handling missing values ​​and outliers in the environmental data;

[0029] The categorical data in the static data is one-hot encoded, and the numerical data in the static data is normalized.

[0030] According to a second aspect of the present disclosure, a display device is provided, the device comprising:

[0031] An acquisition module, used to acquire feature data; wherein the feature data includes image data, environmental data and static data;

[0032] An extraction module, used for inputting the image data into the image feature extraction layer of the initial model and outputting the image features; inputting the environment data into the environment feature extraction layer of the initial model and outputting the environment features; inputting the static data into the static feature extraction layer of the initial model and outputting the static features;

[0033] A fusion module, used for inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features;

[0034] The output module is used to adjust the model parameters of the initial model based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

[0035] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0036] at least one processor; and

[0037] a memory communicatively connected to the at least one processor; wherein,

[0038] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0039] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0040] The display, device and electronic device disclosed in the present invention utilize multimodal data acquired in real time to train a model and improve the model parameters of the model. The obtained display model can identify the visual fatigue symptoms that may occur in the user, adjust the parameters of the display device in time, and reduce the burden on the user's eyes.

[0041] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0043] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0044] Figure 1 The schematic diagram of the implementation process of the display method of the embodiment of the present disclosure is shown Figure 1 ;

[0045] Figure 2 The schematic diagram of the implementation process of the display method of the embodiment of the present disclosure is shown Figure 2 ;

[0046] Figure 3 A schematic diagram showing the structure of a display device according to an embodiment of the present disclosure is shown;

[0047] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0049] Existing electronic devices usually have low blue light modes on their displays, which need to be adjusted manually. Software solutions have night modes, which need to be turned on manually, but cannot be adaptive. The main disadvantages are as follows:

[0050] Subjectivity: Users may not be able to perceive the effectiveness of this mode or its impact on visual comfort.

[0051] High cost: The price of eye protection screens on the market is relatively high and requires special material support.

[0052] No personalized customization: Traditional eye protection screens can only be used according to the parameters set by the manufacturer and cannot be customized to suit your own needs.

[0053] Environmental impact: Traditional eye protection screens have different effects in different environments, which greatly reduces the eye protection effect.

[0054] Low display quality: After turning on the eye protection mode, the display quality of all content may decline. Therefore, there is an urgent need for a solution that can promptly identify user eye fatigue and provide adjustment strategies. The technical solution provided in this application first performs multi-dimensional data collection, extracts data features of the collected multi-dimensional data, and optimizes the model parameters of the initial model after fusing the data features, thereby obtaining a method that can identify possible visual fatigue symptoms of users and promptly adjust the parameters of the display device.

[0055] A display method, device and electronic device provided by the present application are described below in conjunction with the accompanying drawings.

[0056] like Figure 1 As shown, the present application provides a display method, the method comprising:

[0057] S101, acquiring feature data; wherein the feature data includes image data, environment data and static data;

[0058] Specifically, Figure 2 As shown, feature data is data used to extract features from the input initial model, such as image data, environmental data, and static data.

[0059] In some embodiments, the image data includes user image data acquired by an image acquisition device; the user image data includes user facial data and user eye data;

[0060] The environmental data includes the ambient brightness, ambient humidity, the distance between the user and the device, and the time the user uses the device;

[0061] The static data includes user age and gender, device type, and screen content.

[0062] The image data may be a user's facial image and a user's eye image collected by a camera, and the user's blink rate and the user's pupil adjustment speed may be calculated through the user's facial image data and the user's eye image data. A reduced user blink rate will lead to poor tear film quality and increase corneal exposure time, leading to dry eyes. A slowdown in pupil adjustment speed is a sign of increased visual fatigue, and monitoring this parameter can help prevent it in advance.

[0063] Environmental data can be ambient brightness, humidity, etc., and static data can be user age and gender, display device status, etc. Low ambient humidity can easily cause dry eyes, and high ambient brightness can easily cause eye fatigue.

[0064] Static data can be the user's age and gender. Considering the user's age and gender, as the age increases, the transmittance of the lens decreases, and women are more likely to show visual fatigue than men. The display state includes the type of display device; for example, the difference between organic light-emitting diode (OLED) and light-emitting diode (LED) displays, the display content, and the specific parameters of the display device. The content and device type directly affect the user's eye fatigue level.

[0065] The collected feature data may also include user feedback data, specifically using a visual fatigue scale, which is widely used in subjective visual fatigue testing and provides powerful user subjective feedback data.

[0066] Among them, image data, environmental data and static data are multimodal perception data, through which the user's eye condition can be dynamically tracked, that is, the user's visual fatigue state can be comprehensively evaluated from multiple dimensions using rich perception data. For example, the adopted data is shown in Table 1.

[0067] Table 1 Characteristic data

[0068]

[0069]

[0070] S102, inputting the image data into the image feature extraction layer of the initial model, and outputting the image features; inputting the environment data into the environment feature extraction layer of the initial model, and outputting the environment features; inputting the static data into the static feature extraction layer of the initial model, and outputting the static features;

[0071] The initial model provided in this application includes an image feature extraction layer, an environmental feature extraction layer, and a static feature extraction layer, and the image features of the input image data are extracted through the image feature extraction layer. The image feature extraction layer uses a convolutional neural network (CNN), which processes image input, extracts features such as blink rate and pupil distance, and generates feature vectors. The convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer. Image features are extracted through a convolutional layer (Conv2D) and a pooling layer (MaxPooling2D), and a feature vector is generated using a fully connected layer (Dense).

[0072] Specifically, the blink rate is calculated by capturing eye images in consecutive frames, extracting eye features using CNN, and detecting the opening and closing status of the eyes, and calculating the number of blinks per unit time. A low blink rate may indicate visual fatigue of the user. The blink rate is calculated in the following way:

[0073]

[0074] Wherein, T is the number of image frames captured per minute, and the value of T is 1400; t is the time, and the unit is minute; for example, the calculated number of blinks can be 15 times / minute.

[0075] The pupil distance can be used to calculate the frequency of the user's visual focus shift by detecting the changes in the user's pupil position. Frequent or slow pupil adjustments may indicate the user's visual fatigue. The pupil distance is calculated in the following way:

[0076]

[0077] Among them, (x 1 ,y 1 ), (x 2 ,y 2 ) are the pupil position coordinates of the current frame and the previous frame, for example, the two coordinates are (32, 24) and (30, 20) pixels. The calculated pupil adjustment distance is (PA=4.47) pixels, indicating the change of pupil adjustment speed.

[0078] The environmental features of the input environmental data are extracted through the environmental feature extraction layer, including: environmental brightness and humidity, the distance between the user and the device, and the time the user uses the device. The environmental feature extraction layer can use a long short-term memory network (Long Short-Term Memory, LSTM) or an attention mechanism module Transformer. LSTM or Transformer processes time series data and analyzes the changing trend of user status.

[0079] Among them, the ambient brightness and humidity are used to analyze the impact of changes in ambient brightness and humidity on the user's eye condition. Low humidity can easily lead to dry eyes, and high brightness may increase visual fatigue. The following method is used to calculate the impact coefficient of ambient brightness and humidity on the eyes:

[0080] Impact=α×Brightness+β×Humidity

[0081] Among them, a is the empirical coefficient of ambient brightness, which is 0.5; β is the empirical coefficient of ambient humidity, which is 0.3; Brigheness is ambient brightness; Humidity is ambient humidity. For example, if the ambient brightness is 700 lux and the humidity is 40%, the calculated impact coefficient is Impact = 410.

[0082] The distance between the user and the device can be obtained by monitoring the distance between the user and the screen in real time through sensors and input into the initial model. A closer distance may cause pressure on the user's eyesight. The distance between the user and the device is calculated in the following way:

[0083]

[0084] Wherein, Distance Impact is the distance between the user and the device, Distance is the distance between the user and the screen, and γ is the distance coefficient, which can be 1.2. For example, when the distance between the user and the screen is 35 cm, the calculated distance impact value is Distance Impact = 0.034.

[0085] The time a user uses the device is to track the time the user continuously uses the device. Long-term use may cause eye fatigue. The following method is used to calculate the time the user uses the device:

[0086] Fatigue Accumulation=λ×Usage Time

[0087] Wherein, λ is the contribution coefficient, which is 0.8, and Usage Time is the continuous usage time. For example, when the continuous usage time is 120 minutes, the calculated fatigue accumulation is Fatigue Accumulation=96.

[0088] The static features of the input static data are extracted through the static feature extraction layer: visual fatigue, fatigue score. Among them, the static feature extraction layer in the present application can adopt a deep learning neural network (Deep Neural Networks, DNN).

[0089] The DNN model is used to process the user's age and gender. Assume that there are two hidden layers in the DNN, with 64 and 32 neurons respectively. The predicted value of visual fatigue is calculated in the following way:

[0090] Fatigue Score=DNN Output=f(Age,Gender,Other Feqtures)

[0091] Among them, (f) is the nonlinear activation function of the DNN model.

[0092] For example, if the user is 30 years old and female, the fatigue score output by DNN is (Fatigue Score = 0.75), indicating that the user is in a moderate fatigue state.

[0093] The fatigue score can be analyzed based on the type of screen content (such as text, images, videos) to determine its effect on the eyes. For example, fast-changing video content may increase the fatigue score. The fatigue score is calculated in the following way:

[0094] Content Impact=δ×Content Type

[0095] Among them, δ is the influence coefficient of content type.

[0096] S103, inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features;

[0097] In some embodiments, the step of inputting the image features, environmental features, and static features into the feature fusion layer of the initial model to obtain target fusion features includes:

[0098] Fusing the image feature and the environment feature to obtain a first fused feature;

[0099] The first fusion feature is fused with the static feature to obtain a target fusion feature.

[0100] The initial model provided in the present application also includes a feature fusion layer. After the image features and the environmental features are fused, the fused first fusion features are fused with the feature fusion layer to obtain the target fusion features.

[0101] Specifically, the output features of the CNN and LSTM / Transformer modules are fused and combined with the static features, and the final target fusion features are output through the DNN layer. For example, the weights of the entire network can be fine-tuned through a back-propagation algorithm, such as the Adam optimizer, to optimize the overall performance of the model.

[0102] S104, adjusting the model parameters of the initial model based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

[0103] In some embodiments, the model parameters of the initial model include: a first model parameter of an image feature extraction layer of the initial model, a second model parameter of an environment feature extraction layer of the initial model, and a third model parameter of a static feature extraction layer of the initial model; and adjusting the model parameters of the initial model based on the target fusion feature includes:

[0104] According to the loss function between the target fusion feature and the target task, the first model parameter, the second model parameter and the third model parameter of the initial model are adjusted.

[0105] It is understandable that the model parameters of a single feature extraction layer (such as CNN, LSTM / Transformer module) are first fine-tuned, and then joint training is performed to ensure that the outputs of each feature extraction layer can be effectively fused. The initial model provided in this application uses an appropriate loss function (such as mean square error MSE, cross entropy loss, etc.) and is adjusted according to task requirements. Then, using learning rate scheduling, a lower initial learning rate is used for fine-tuning, and the learning rate is gradually increased to prevent the initial model from falling into a local optimum during training. When the initial model is trained, after obtaining the fusion feature, the fusion feature can also be expanded, and the expanded feature is used as a training set to determine that the trained model maintains good generalization ability on the migrated data. Alternatively, data enhancement can also be performed on the fusion feature, such as image flipping, scaling, brightness adjustment, etc. to improve the robustness of the model.

[0106] In some embodiments, the screen display parameter adjustment strategy includes:

[0107] Suggestions for adjusting screen display parameters and reminders for users to move away from the screen;

[0108] The screen display parameters include contrast, display brightness and color temperature.

[0109] The display model outputs two settings in this application, namely, suggestions for adjusting screen display parameters and reminder information for prompting users to move away from the screen; the reminder information for users to move away from the screen is based on the user's eye status and device usage, and the model will provide a recommended distance range between the user and the screen to ensure the best viewing experience.

[0110] The display model in this application comprehensively judges the user's current visual fatigue state by fusing images, time series and static features. According to the user's visual fatigue level, the model dynamically adjusts the screen display parameters and provides distance recommendations. The screen display parameter adjustment strategy outputs a mapping to specific operation suggestions suitable for the user, such as lowering the brightness, increasing the contrast or reminding the user to move away from the screen.

[0111] In some embodiments, after obtaining the trained display model, the method further includes:

[0112] New feature data and user feedback data are collected regularly to retrain the display model.

[0113] In this application, user feedback can also be collected through questionnaires, user ratings, behavioral analysis, etc., and user feedback can be analyzed to identify deficiencies in model predictions, adjust model parameters, or improve data processing methods. Adjust model structure, parameters, or training strategies based on user feedback. For example, if feedback indicates that a certain feature has a greater impact on predictions, the modeling of that feature can be strengthened. Regularly retrain the model using user feedback and new data.

[0114] In some embodiments, after acquiring the characteristic data, the method further includes: preprocessing the characteristic data, specifically including:

[0115] Normalizing, cropping and gray-scaling the image data;

[0116] Handling missing values ​​and outliers in the environmental data;

[0117] The categorical data in the static data is one-hot encoded, and the numerical data in the static data is normalized.

[0118] It is understandable that in order to ensure the accuracy of feature data for model training, the feature data can be preprocessed after obtaining the feature data. Among them, the pixel values ​​of the image data are normalized to the range of 0-1, and then the eye area is cropped from the image to reduce the interference of irrelevant information on the model. Assume that the size of the cropped image is 64×64 pixels. Convert the color image to a grayscale image and then perform feature extraction. Process missing values ​​and outliers for environmental data to ensure data integrity and consistency. The preprocessing of static data is to uniquely encode categorical features (such as gender, device type) and normalize numerical features (such as age).

[0119] like Figure 3 As shown, an embodiment of the present application provides a display device, the device comprising:

[0120] The acquisition module 301 is used to acquire feature data; wherein the feature data includes image data, environment data and static data;

[0121] Extraction module 302, used for inputting the image data into the image feature extraction layer of the initial model and outputting the image features; inputting the environment data into the environment feature extraction layer of the initial model and outputting the environment features; inputting the static data into the static feature extraction layer of the initial model and outputting the static features;

[0122] A fusion module 303 is used to input the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features;

[0123] The output module 304 is used to adjust the model parameters of the initial model based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

[0124] The display device provided by the present application acquires image data, environmental data and static data through the acquisition module 301, and the extraction module 302 inputs the image data into the image feature extraction layer of the initial model to output image features; inputs the environmental data into the environmental feature extraction layer of the initial model to output environmental features; inputs the static data into the static feature extraction layer of the initial model to output static features; the fusion module inputs the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features; finally, the output module 304 adjusts the model parameters of the initial model based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

[0125] It should be noted that the display device of the embodiment of the present application solves the problem based on a principle similar to that of the aforementioned display method. Therefore, the implementation process, implementation principle, and beneficial effects of the display device can all be referred to the description of the implementation process, implementation principle, and beneficial effects of the aforementioned method, and the repeated parts will not be repeated.

[0126] An embodiment of the present application provides an electronic device, including:

[0127] at least one processor; and

[0128] a memory communicatively connected to the at least one processor; wherein,

[0129] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of the above embodiments.

[0130] An embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0131] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0132] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0133] like Figure 4 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0134] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as display methods. For example, in some embodiments, the display method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the display method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the display method in any other appropriate manner (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0141] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0143] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0144] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A display method, characterized in that: The method comprises: Acquire feature data; wherein the feature data includes image data, environmental data and static data; Input the image data into the image feature extraction layer of the initial model and output the image features; input the environment data into the environment feature extraction layer of the initial model and output the environment features; input the static data into the static feature extraction layer of the initial model and output the static features; Inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features; The model parameters of the initial model are adjusted based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

2. The method according to claim 1, characterized in that The image data includes user image data collected by an image collection device; the user image data includes user facial data and user eye data; The environmental data includes the ambient brightness, ambient humidity, the distance between the user and the device, and the time the user uses the device; The static data includes user age and gender, device type, and screen content.

3. The method according to claim 1, characterized in that The inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain the target fusion features includes: fusing the image feature and the environment feature to obtain a first fused feature; The first fusion feature is fused with the static feature to obtain a target fusion feature.

4. The method according to claim 1, characterized in that: The model parameters of the initial model include: a first model parameter of the image feature extraction layer of the initial model, a second model parameter of the environment feature extraction layer of the initial model, and a third model parameter of the static feature extraction layer of the initial model; and adjusting the model parameters of the initial model based on the target fusion feature includes: According to the loss function between the target fusion feature and the target task, the first model parameter, the second model parameter and the third model parameter of the initial model are adjusted.

5. The method according to claim 4, characterized in that The image feature extraction layer adopts a convolutional neural network; The environmental feature extraction layer adopts a long short-term memory network or an attention mechanism module; The static feature extraction layer adopts a deep learning neural network.

6. The method according to claim 4, characterized in that The screen display parameter adjustment strategy includes: Suggestions for adjusting screen display parameters and reminders for users to move away from the screen; The screen display parameters include contrast, display brightness and color temperature.

7. The method according to claim 1, characterized in that After obtaining the trained display model, it also includes: New feature data and user feedback data are collected regularly to retrain the display model.

8. The method according to claim 1, characterized in that: After acquiring the characteristic data, the method further includes: preprocessing the characteristic data, specifically including: Normalizing, cropping and gray-scaling the image data; Handling missing values ​​and outliers in the environmental data; The categorical data in the static data is one-hot encoded, and the numerical data in the static data is normalized.

9. A display device, characterized in that: The device comprises: An acquisition module, used to acquire feature data; wherein the feature data includes image data, environmental data and static data; An extraction module, used for inputting the image data into the image feature extraction layer of the initial model and outputting the image features; inputting the environment data into the environment feature extraction layer of the initial model and outputting the environment features; inputting the static data into the static feature extraction layer of the initial model and outputting the static features; A fusion module, used for inputting the image features, environmental features and static features into the feature fusion layer of the initial model to obtain target fusion features; The output module is used to adjust the model parameters of the initial model based on the target fusion features to obtain a trained display model; the display model is used to characterize the user's visual fatigue level and the screen display parameter adjustment strategy.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images