Screen display adjustment method, device, electronic device and storage medium

Abstract

The present application discloses a method, device, electronic device and computer-readable storage medium for adjusting screen display. The method for adjusting screen display of electronic device comprises: obtaining external image data collected by an image acquisition device; obtaining screen display content of the current screen; determining backlight adjustment parameters and screen display adjustment parameters based on a preset neural network model according to the external image data and the screen display content; adjusting the screen display of the electronic device according to the backlight adjustment parameters and the screen display adjustment parameters. The method for adjusting screen display proposed in the present application performs intelligent collaborative calculation of multiple adjustment parameters through a neural network model, thereby realizing synchronous adjustment of screen display content while adjusting the screen backlight, ensuring the coordination between screen display content and screen backlight during dynamic adjustment of screen display, and improving screen display effect.

Description

Technical Field

[0001] The present invention relates to the field of terminal display technology, and in particular to a method, device, electronic equipment and storage medium for adjusting screen display. Background Art

[0002] With the rapid development of hardware and software, mobile terminals with touch screens, such as smart phones and tablet computers, have become indispensable tools in people's lives. Users can use terminal devices to chat, watch videos, read, play games, browse the web, etc. in different occasions and at different times.

[0003] Users have different display requirements for mobile terminal screens in different situations. For example, when the user is in a brightly lit environment, the screen brightness needs to be increased to avoid poor readability of the screen due to the screen brightness being too low. Conversely, when the user is in a dimly lit environment, the screen brightness needs to be lowered to avoid eye irritation due to the screen brightness being too high. Currently, some mobile terminals have an automatic screen brightness adjustment function, but after adjusting the terminal screen using this general automatic screen brightness adjustment method, the screen display effect is not ideal, and the user needs to readjust the screen display parameters. Summary of the invention

[0004] Embodiments of the present application provide a method, device, electronic device, and non-volatile computer-readable storage medium for adjusting screen display.

[0005] In a first aspect, an embodiment of the present application provides a method for adjusting screen display, which is applied to an electronic device, comprising:

[0006] Acquiring external image data acquired by an image acquisition device;

[0007] Get the screen display content of the current screen;

[0008] Determine backlight adjustment parameters and screen display adjustment parameters based on a preset neural network model according to external image data and screen display content;

[0009] The screen display of the electronic device is adjusted according to the backlight adjustment parameters and the screen display adjustment parameters.

[0010] In a second aspect, an embodiment of the present application provides a device for adjusting screen display, comprising:

[0011] A first acquisition module, used to acquire external image data acquired by an image acquisition device;

[0012] A second acquisition module is used to acquire the screen display content of the current screen;

[0013] A processing module, used to determine backlight adjustment parameters and screen display adjustment parameters based on a preset neural network model according to external image data and screen display content;

[0014] The adjustment module is used to adjust the screen display of the electronic device according to the backlight adjustment parameters and the screen display adjustment parameters.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including an image acquisition device, a memory, and a processor;

[0016] The image acquisition device is used to acquire external image data;

[0017] The memory stores a computer program;

[0018] The processor is used to execute the screen display adjustment method provided in any embodiment of the present application by calling a computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, a method for adjusting the screen display provided in any embodiment of the present application is implemented.

[0020] The screen display adjustment method, device, electronic device and non-volatile computer-readable storage medium provided in the embodiments of the present application determine backlight adjustment parameters and screen display adjustment parameters based on neural network model calculation according to external image data and screen display content, thereby realizing synchronous adjustment of screen backlight parameters and screen display parameters, ensuring the coordination between screen display content and screen backlight during dynamic adjustment of screen display, and improving screen display effect.

[0021] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0024] Figure 2 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 3 It is a flowchart of a method for adjusting screen display provided in an embodiment of the present application;

[0026] Figure 4 It is a flowchart of a method for adjusting screen display provided in an embodiment of the present application;

[0027] Figure 5 It is a flowchart of a method for adjusting screen display provided in an embodiment of the present application;

[0028] Figure 6 It is a flowchart of a method for identifying scene features of screen display content provided by an embodiment of the present application;

[0029] Figure 7 It is a flowchart of a method for adjusting screen display provided in an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of a screen display adjustment system provided in an embodiment of the present application;

[0031] Fig. 9 It is a structural schematic diagram of a screen display adjustment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to facilitate understanding of the embodiments of the present application, the elements that may be introduced in the embodiments of the present application are first introduced here. It should be noted that the following elements and their introduction do not constitute a limitation on the technical solution of the present application.

[0033] Brightness: Indicates the brightness of the image. For light source color, the brightness value is related to the brightness of the light source. For object color, this value is related to the transmittance or reflectance of the object. The value range is usually 0% (black) to 100% (white). An image gives people an intuitive feeling. If it is a grayscale image, the higher the grayscale value, the brighter the image. Brightness is an important indicator to measure the luminous intensity of the display screen. For the display panel, high brightness also means higher anti-interference ability to its working environment.

[0034] Color temperature: A unit of measurement that indicates the color components contained in light. In blackbody radiation, the color of light varies with temperature, and the blackbody presents a gradual process of red-orange-red-yellow-yellow-white-white-blue-white. When the color of light emitted by a certain light source looks the same as the color of light emitted by a blackbody at a certain temperature, the temperature of the blackbody is called the color temperature of the light source. The higher the temperature of the "blackbody", the more blue components in the spectrum and the less red components. When shooting with natural light, the color of the photos taken is different because the color temperature of light at different time periods is different. For example, when shooting under a clear blue sky, the photos are cold because the color temperature of the light is high; if shooting at dusk, the photos are warm because the color temperature of the light is low. When shooting with artificial light, the color of the photos taken will also be different depending on the type of light source. Color temperature is a performance indicator of a display. The color temperature can be adjusted by changing the proportion of different color components in the display screen.

[0035] Contrast: It refers to the measurement of the different brightness levels between the brightest white and the darkest black in the light and dark areas of an image, that is, the size of the grayscale contrast of an image. Generally, the higher the contrast of an image, the brighter and more prominent it feels; the lower the contrast, the less obvious the contrast.

[0036] Saturation: Indicates the degree to which a color is close to a spectral color. A color can be seen as the result of a mixture of a certain spectral color and white. The greater the proportion of the spectral color, the closer the color is to the spectral color, and the higher the saturation of the color. A high saturation gives people the feeling that the color is deep and bright.

[0037] Grayscale: It means dividing the brightness change between the brightest and darkest into several parts, so as to facilitate the screen brightness control corresponding to the signal input. Generally, the more intermediate levels there are, the more delicate the picture effect can be presented. Taking an 8-bit display screen as an example, the display screen is composed of many pixels, and usually each pixel is composed of three sub-pixels of red (R), green (G), and blue (B). For a single sub-pixel, there can be 2 8 The brightness levels are called 256 grayscales. The color change of each pixel on the screen is actually caused by the grayscale change of the three RGB sub-pixels that make up this pixel.

[0038] Grayscale histogram: It indicates the number of pixels with a certain grayscale level in the image, reflecting the frequency of a certain grayscale in the image. In this histogram, the left side of the horizontal axis is the pure black, darker area, and the right side is the brighter, pure white area. Therefore, the data in the grayscale histogram of a darker image is mostly concentrated on the left and middle parts, and the data in the grayscale histogram of an overall bright image is mostly concentrated on the right and middle parts. For the grayscale histogram of a picture with a large contrast, the data is distributed on the left and right sides.

[0039] Color gamut: refers to the range of colors that a certain color representation mode can express, and also refers to the range of colors that a specific device, such as a display, can express. For example, for a gray-mode image, each pixel is represented by one byte, so a gray-mode image can have at most 2 8 = 256 colors, and its color gamut is (0, 255). For an RGB image, if one base color is represented by one byte, then the RGB image can have at most 2 24 colors, its color gamut is (0, 2 24 -1).

[0040] Color calibration is the process of correcting the color cast of an image, so that the image display is more consistent with what the human eye sees on the spot, and follows the laws of photographic imaging and color formation. For example, the color cast of a standard color temperature (daylight) photo is based on gray balance, which can be dark tones, mid-tones, and highlights. Because the human eye reacts more slowly to dark and bright parts, the gray of the mid-tone is considered to be calibrated in place when it is close to 1:1:1 in color calibration. For example, the sampling parameter color scale is 128:124:130, and generally, the color calibration of images with low requirements is allowed to have a color calibration deviation within 10%.

[0041] Color management: Calibrate all input / output devices in each link of the image processing chain so that the expected color can always be obtained on different devices.

[0042] Liquid crystal display (LCD): It is a flat and thin display device, including a display screen and a light source. The display screen is placed in front of the light source and consists of multiple pixels, each of which consists of three sub-pixels. The pixel value of each sub-pixel is the corresponding grayscale brightness. Each sub-pixel corresponds to a pixel channel. For example, the R sub-pixel corresponds to the R pixel channel. A color filter that only allows light in the red band to pass through is added between the R sub-pixel and the backlight. The grayscale brightness of the R sub-pixel is determined by the transmittance of the R sub-pixel. The transmittance of the R sub-pixel is related to the deflection angle of the corresponding liquid crystal molecule of the R sub-pixel. The deflection angle of the liquid crystal molecule is related to the pixel voltage loaded on the brightness of the liquid crystal molecule. Generally speaking, the higher the pixel voltage, the larger the deflection angle of the corresponding liquid crystal molecule, the higher the transmittance, and the higher the corresponding grayscale brightness.

[0043] OLED (organic light-emitting diode) display: also known as organic electric laser display, organic light-emitting semiconductor, has many advantages such as high color, wide viewing angle, high contrast, high response speed, light weight, low power consumption, etc., and has been widely used in terminal display screens such as mobile phones and tablets. OLED can be divided into AMOLED (Active-matrix OLED) and PMOLED (Passive-matrix OLED) according to the driving method.

[0044] AMOLED display is a display screen that uses a self-luminous diode matrix arrangement. It uses a very thin organic material coating and a glass substrate. When current passes through, these organic materials will emit light. Thousands of light-emitting diodes that can only emit one color of red, green or blue are placed on the screen substrate in a specific arrangement. These light emitters will emit red, green or blue light when voltage is applied. Each pixel is composed of three diodes representing the three primary colors of RGB, which can be switched on and off individually. For example, RGB (0, 0, 0) represents a black pixel, and RGB (255, 255, 255) represents a white pixel. The integrated circuit IC provides 256 levels of adjustable voltage to limit the upper limit voltage. Each level of voltage is linearly related to the brightness of the screen. Each level corresponds to a brightness. The higher the level, the brighter the white light seen by the human eye.

[0045] LUT: Short for Look Up Table. In Wikipedia, LUT is considered as a function. After the color information of each pixel is repositioned by LUT, a new color value can be obtained. LUT is usually used in three aspects: for color calibration of display devices; for conversion of image color space; for design and simulation of special color effects.

[0046] Subpixel Rendering (SPR): represents sub-pixel rendering operation.

[0047] Demura: refers to the operation of compensating and eliminating mura. Mura specifically refers to the problem of uneven brightness.

[0048] Deburn in: Indicates reducing the possibility of screen burn-in by processing display data.

[0049] IR drop: IR refers to the problem of voltage drop or increase on the power and ground networks in integrated circuits. IRdrop refers to the problem of reducing IR in integrated circuits.

[0050] SOC (System on Chip): usually referred to as "system on chip". SoC has two significant characteristics: one is that the hardware is large in scale, usually based on IP design mode; the other is that the software accounts for a large proportion, which requires hardware and software co-design. The composition of the system-level chip can be a system-level chip control logic module, a microprocessor / microcontroller CPU core module, a digital signal processor DSP module, an embedded memory module, an interface module for communicating with the outside, an analog front-end module containing ADC / DAC, a power supply and power consumption management module. For a wireless SoC, there is also a radio frequency front-end module, user-defined logic (which can be implemented by FPGA or ASIC) and a micro-electromechanical module. More importantly, a SoC chip is embedded with basic software (RDOS or COS and other application software) modules or loadable user software.

[0051] Hardware acceleration: refers to the technology of assigning very computationally intensive tasks to special hardware components (chips or processors) to reduce the workload of the central processing unit in the computer.

[0052] In order to make the explanation of the technical problems, technical solutions and technical effects to be solved by the present method clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the post-positioned terms "unit" and "module" are only for the convenience of description, and these post-positioned terms do not have mutually distinguishing meanings or functions. The method provided in the present application can be implemented by hardware or software.

[0053] See also Figure 1 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 100 includes an image acquisition device 110, an analysis unit 120, an interface unit 130, a processor 140, a memory 150, an adjustment unit 160 and a screen 170. Each part will be described in detail below.

[0054] Image acquisition device 110: In one embodiment of the present application, the image acquisition device 110 may be a front-mounted image acquisition device in an electronic device, having an always-on (AON) function, and is mainly used to monitor the user and environmental status in real time. The image acquisition device 110 includes but is not limited to an image sensor and an optical sensor. The image sensor outputs image data for identification and detection of user feature information, including but not limited to information such as user age, eye position, pupil size, eye blinking frequency, and the scene in which the user is located. The optical sensor is mainly used to sense ambient light information around the device, such as ambient light brightness, color temperature, etc.

[0055] Analysis unit 120: Mainly used for analyzing and processing the screen display content of the current screen, such as performing grayscale statistics on the screen display content, and determining the screen display parameters of the screen display content based on the obtained grayscale statistics. In one embodiment of the present application, the analysis unit 120 may be a frame-level analysis unit, which is used to perform frame-level analysis and processing on the screen display content of the current screen. Since the single-frame processing method has high requirements on computing power, the frame-level analysis unit 120 may be implemented in the form of external hardware, and of course, it may also be set in a built-in processor, and this application does not impose too many restrictions.

[0056] Interface unit 130: used to receive key information, touch information or user feature information (such as fingerprint) input by the user, and generate interface data corresponding to the user input. For example, the user can manually adjust the screen backlight parameters through the interface unit 130, or manually adjust the screen display parameters through the interface unit 130.

[0057] Processor 140: In the processor 140, the backlight adjustment parameters and the screen display adjustment parameters adapted to the backlight adjustment parameters can be calculated based on the neural network model in the processor 140 according to the external image data collected by the image acquisition device 110 and the screen display content of the current screen analyzed and acquired by the analysis unit 120. In the processor, the local training library of the neural network model can also be continuously trained and updated through the interface data collected by the interface unit 130, such as the screen backlight parameters and / or screen display parameters manually adjusted by the user.

[0058] In an embodiment of the present application, the analysis and processing of the screen display content and the method for adjusting the screen display can be at the frame level. Therefore, in order to achieve real-time and frame-level processing, the demand for computing power is relatively large, so a hardware accelerator based on an AI engine can be used as the processor 140. Currently, the processor 140 can also use a processor built into the electronic device 100, and the present application does not make too many restrictions here.

[0059] Compared with traditional AI hardware, for example, Google's TPU has a computing power utilization rate of only 28%, and the other 70% is waiting for memory access. If a dedicated display hardware accelerator is used, targeted matching can be made on the memory size and channel, improving the channel reuse and access mechanism, shortening the memory access waiting time, and thus greatly improving the algorithm processing time. The use of a dedicated display hardware accelerator can increase the utilization rate to more than 90%. Based on the current traditional TPU's computing power of 5 trillion times per second (5Tops), a dedicated display hardware accelerator can increase the computing power to about 15Tops to achieve the purpose of real-time computing, thereby greatly improving the processing speed of display effects.

[0060] Memory 150: can be used to store computer programs and data. The computer programs stored in the memory 150 contain instructions that can be executed in the processor 140. The computer programs can form various functional modules. The processor 140 executes various functional applications and data processing by calling the computer programs stored in the memory 150. The memory 150 can be one or more of the following types: flash memory, hard disk type memory, micro multimedia card type memory, card memory (such as SD or XD memory), random access memory (random access memory, RAM), static random access memory (static RAM, SRAM), read only memory (read only memory, ROM), electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), programmable read-only memory (programmable ROM, PROM) or magnetic memory.

[0061] The adjustment unit 160 is used to adjust the display of the screen 170 of the electronic device 100 according to the backlight adjustment parameters and the screen display adjustment parameters output by the processor 140 .

[0062] In an embodiment of the present application, the adjustment unit 160 may further include a display driver chip and a system-level chip.

[0063] Display driver IC (DDIC): used to adjust the backlight of the display screen according to the backlight adjustment parameters output by the processor 140. In LCD displays, the display driver chip can adjust the backlight source voltage by pulse width modulation (PWM) dimming or linear dimming, thereby adjusting the backlight brightness. In AMOLED displays, the display driver chip adjusts the display screen voltage according to the voltage-brightness conversion model by adjusting the luminous time, PWM ratio, gamma mapping, etc., thereby changing the backlight brightness or backlight color temperature.

[0064] System on Chip (SOC): used to adjust the screen display parameters according to the screen display adjustment parameters output by the processor 140. The screen display adjustment parameters indicate how much offset adjustment is made based on the screen display parameters of the current screen. Of course, the screen display adjustment parameters can also directly give new values ​​for the screen display parameters. If the screen display parameters are adjusted at the frame level, the system on chip can be implemented using an external SOC combined with a system hardware platform. Of course, the adjustment of the screen display parameters can also be implemented using a built-in system hardware platform, and there are no excessive restrictions in this application.

[0065] Screen 170: used to display screen content, which may be implemented as an LCD display or an OLED display.

[0066] like Figure 2 As shown, the electronic device 100 may further include a color calibration unit and / or a color management unit 180, which pre-processes the screen display content of the current screen and then inputs the processed screen display content into the analysis unit 120 for analysis and processing.

[0067] For example, when the user is in different scene modes, such as sunlight screen, dark mode, eye protection mode, and color temperature adaptive mode, the user has different requirements for color temperature corresponding to different scene modes. And the standard image data may be different for different color temperatures. Therefore, the current screen display content and the standard image data corresponding to the color temperature are analyzed to obtain color calibration parameters such as color deviation, and the screen display content is preprocessed according to different color calibration parameters to obtain the processed screen display content, and then the processed screen display content is input to the analysis unit 120 for analysis and processing. At the same time, for different display screens, due to different color rendering capabilities, that is, the color management parameters such as display color gamut will be different, so for the same image, the color effects presented on different display screens may be different. Therefore, the screen display content can be preprocessed according to the color management parameters of the display screen. The method for adjusting the screen display content may include one or more of SPR, demura, Deburn in and IR drop.

[0068] The embodiment of the present application provides a method for adjusting screen display, and the execution subject of the method for adjusting screen display may be the screen display adjustment device 300 provided in the embodiment of the present application, or the electronic device 100 integrated with the screen display adjustment device 300. The electronic device 100 may be a device with display function such as a smart phone, a tablet computer, a PDA, a laptop computer, or a desktop computer.

[0069] See also Figure 3 , is a flow chart of a method for adjusting screen display provided in an embodiment of the present application. The method for adjusting screen display includes the following process:

[0070] 210: Obtaining external image data and the screen display content of the current screen, including:

[0071] 211: The image acquisition device 110 acquires external image data, and the acquired external image data is analyzed to obtain user characteristic information and / or ambient light information. The user characteristic information includes, but is not limited to, one or more characteristic information such as user age, eye position, pupil size, eye blinking frequency, and the environment scene where the user is located. The ambient light information may include ambient light brightness and / or ambient light color temperature.

[0072] 212: Obtain the screen display content of the current screen through the analysis unit 120. The screen display content may indicate the RGB value of the pixel array of the current screen display image.

[0073] It is understandable that there is no specific order for obtaining the above data.

[0074] 230: Determine backlight adjustment parameters and screen display adjustment parameters according to external image data and screen display content based on a preset neural network model.

[0075] See also Figure 4 The method of step 230 may specifically include the following steps:

[0076] 232: Determine screen display parameters of the screen display content according to the screen display content;

[0077] 233: Determine backlight adjustment parameters and screen display adjustment parameters according to external image data and screen display parameters based on a preset neural network model.

[0078] 240: Adjust the screen display of the electronic device according to the backlight adjustment parameter and the screen display adjustment parameter.

[0079] The screen 170 is actually composed of a plurality of pixels, each of which is composed of sub-pixels such as R, G, and B. By setting different grayscales for the R, G, and B sub-pixels, the entire screen can present different color information such as brightness, contrast, and saturation. For example, the screen display parameters of the screen display content can be determined by performing grayscale statistics on the screen display content, including but not limited to contrast, saturation, user preferred color, and other parameters according to the grayscale statistics results. The backlight adjustment parameters are used to adjust the screen backlight parameters. The backlight adjustment parameters can represent the offset based on the initial screen backlight parameters, and of course, they can also directly represent the new value of the screen backlight parameters. The screen backlight parameters can include one or both of the screen backlight brightness and the screen backlight color temperature. That is, for the adjustment of the screen backlight parameters, it can be either to adjust only the screen backlight brightness, or to adjust only the screen backlight color temperature, or to adjust both the screen backlight brightness and the screen backlight color temperature. For different application scenarios, the emphasis on brightness and color temperature adjustment is different. For example, to achieve eye comfort and improve readability, the brightness needs to be adjusted. Depending on the user's environment or state, the color temperature needs to be adjusted.

[0080] If the screen 170 is simply adjusted for screen backlight brightness and / or screen backlight color temperature, the adjusted backlight brightness and / or backlight color temperature may not match the screen display content of the current screen, resulting in an unsatisfactory display effect of screen 170. The technical solution proposed in the embodiment of the present application can determine the backlight adjustment parameters and the screen display adjustment parameters based on the external image data and the screen display content, and can be calculated based on a preset neural network model. While adjusting the screen backlight parameters based on the backlight adjustment parameters, the screen display parameters can be synchronously adjusted according to the screen display adjustment parameters, thereby realizing collaborative analysis and collaborative setting of multiple adjustment parameters, avoiding algorithm conflicts that exist when multiple parameters are adjusted separately, and reducing the trouble of users repeating adjustment operations. At the same time, the synchronous adjustment of the screen backlight parameters and the screen display parameters can ensure that the screen backlight is more compatible with the screen display content after adjustment, improve the comfort of the human eye and the effect of the screen display, and enhance the user experience.

[0081] See also Figure 5 , which is a flow chart of a method for adjusting screen display provided in one embodiment of the present application.

[0082] and Figure 4 Compared with the embodiment shown, step 230 may further include:

[0083] 234: identifying scene feature information of the screen display content based on a scene classifier according to the screen display content;

[0084] 235: Determine backlight adjustment parameters and screen display adjustment parameters based on a preset neural network model according to external image data, screen display parameters and scene feature information.

[0085] For the entire image, different scene details may require different display effects after the brightness or color temperature changes. Therefore, based on the global analysis of the screen display content, the recognition of scene feature information is added, and then based on different scene feature information, different adjustments are made to different details of the entire image, thereby refining the adjustment effect and improving the display effect.

[0086] For the recognition process of scene feature information, please refer to Figure 6 , Figure 6A flow chart of a scene feature information identification method provided by an embodiment of the present application is shown. For the input screen display content, preprocessing is first performed, including performing partition grayscale statistics on the screen display content according to the initial screen partitioning rule, calculating the grayscale statistical global information of the entire screen display content, and the grayscale statistical local information of the display content in each screen partition. According to the comparative analysis of the local information and the global information, the edge and grayscale information of the area of ​​interest in the screen display content can be identified. The initial partition boundary is adjusted so that the screen display content is re-divided according to the area of ​​interest, and the scene feature information in the area of ​​interest is extracted respectively. The extracted scene feature information is input into the scene classifier, and the scene classifier performs feature matching analysis on the input scene feature information and the second AI training library. If the scene feature information is matched, the scene feature information is output. If the scene feature information is not matched, the feature is stored in the second AI training library, and the second AI training library data is further optimized and trained. By continuously updating and training the second AI training library, the neural network model proposed in the embodiment of the present application can process more scene features, and give corresponding screen adjustment strategies according to more scene features to meet more personalized needs of users.

[0087] To make more detailed adjustments to the screen display, please refer to Figure 7 , is a flow chart of a method for adjusting screen display provided in an embodiment of the present application. Compared with the above-mentioned embodiment, a block division process for screen display content is added, specifically including:

[0088] 231: Divide the screen display content into blocks according to a preset rule;

[0089] 237: for the single screen block obtained after the block division, determine the backlight adjustment parameter and the screen display adjustment parameter based on the preset neural network model according to the external image data and the screen display content in the single screen block;

[0090] 242: Adjust the screen display of the electronic device according to the backlight adjustment parameters and the screen display adjustment parameters of all screen blocks.

[0091] The preset rules may include manually set division methods, such as dividing the screen display content in the manner of M*N (M, N are integers greater than or equal to 1), or training the optimal division parameters through neural network model training based on manually set historical parameters.

[0092] By first dividing the screen display content into blocks and then determining targeted display adjustment strategies for each block, the display adjustment is made more refined and the display effect is further improved.

[0093] An embodiment of the present application further provides a method for adjusting screen display, the technical solution comprising:

[0094] Determine scene feature information based on screen display content;

[0095] The scene feature information includes first scene feature information and second scene feature information, and the first scene feature information corresponds to the first user interface, and the second scene feature information corresponds to the second user interface. There is a time difference between the adjustment of the screen display corresponding to the first scene feature information and the adjustment of the screen display corresponding to the second scene feature information.

[0096] Taking the split-screen mode as an example, if the first scene feature information of the first screen display area is determined to correspond to the reading mode according to the current screen display content, the reading user interface is displayed accordingly, and at the same time, the second scene feature information of the second screen display area is determined to correspond to the game mode, and the game user interface is displayed accordingly. It can be understood that the user's requirements for the screen display effect of the game user interface are different from those for the screen display effect of the reading user interface. Therefore, different user display interfaces can be processed with time-division differences, and interfaces with high screen display effect requirements can be processed first, thereby reducing the requirements for processor computing power and processing rate.

[0097] Of course, it is understandable that the method for adjusting the time-division difference of different display areas of the screen is not limited to the split-screen mode. For the entire display interface, the time-division difference adjustment method can be used for the display areas corresponding to different scene features to reduce the pressure on the processor computing power and processing rate.

[0098] The display content of the screen 170 is refreshed and displayed at a preset refresh frequency. For example, the preset refresh frequency can be set to 60 Hz, that is, 60 frames of images are refreshed and displayed in 1 second, and one frame of image is refreshed on the display screen every 16.67 ms. In an embodiment of the present application, the display adjustment frequency of the screen 170 of the electronic device 100 is the same as the display refresh frequency of the screen 170.

[0099] By analyzing the single-frame screen display content of screen 170 and dynamically adjusting the screen backlight parameters and screen display parameters based on a single-frame image, missed analysis of differences between multiple-frame image data can be avoided, thereby increasing the accuracy of screen display adjustment.

[0100] In some embodiments, external image data and screen display content can be collected and analyzed at regular intervals or at preset time intervals, and multi-frame image processing can be used to adjust screen backlight parameters and screen display parameters based on the results of multi-frame image processing, thereby reducing the requirements for hardware computing power.

[0101] In some embodiments, a certain threshold can also be set for the screen backlight parameters or the screen display parameters. Only when the backlight parameter difference or the display parameter difference between the two frames of displayed images is greater than the threshold, the screen backlight parameters and the screen display parameters are adjusted to reduce the requirements for hardware computing power.

[0102] See also Figure 8 , is a schematic diagram of a screen display adjustment system provided in an embodiment of the present application. The adjustment system may include: raw data, input data, a neural network model, output data, interface data, and a first AI training library.

[0103] The original data includes external image data acquired by the image acquisition device 110 and screen display content acquired by the analysis unit 120 . The screen display content may be acquired directly from the screen 170 or may be obtained after preprocessing by the color calibration unit and / or the color management unit 180 .

[0104] The input data is obtained by further processing the original data and is used as the input data of the neural network model. Specifically, the external image data can be analyzed to obtain user feature information and ambient light information. The user feature information can be information such as the user's age, eye position, pupil size, eye blinking frequency, and the scene the user is in. The ambient light information can be ambient light brightness, ambient light chromaticity, etc. The screen display content is analyzed and processed by grayscale statistics, scene recognition, etc., to further obtain the screen display parameters, scene feature information, etc. of the screen display content.

[0105] The neural network model can perform calculations based on multiple input data such as user feature information, ambient light information, screen display parameters, scene feature information, etc., and output corresponding backlight adjustment parameters and screen display adjustment parameters. In addition, the neural network model can perform model calculations based on the first AI training library, and can also be continuously trained and optimized by the first AI training library based on parameters manually adjusted by the user.

[0106] The first AI training library includes a local AI training library and a cloud AI training library. The local AI training library can update the training library data according to the parameters manually adjusted by the user, specifically including but not limited to:

[0107] Obtain the screen backlight parameters manually adjusted by the user, and determine that the adjusted screen backlight parameters are the reference screen backlight parameters; obtain the external image data and screen display content corresponding to the user manually adjusting the screen backlight parameters; update the data in the local AI training library according to the reference screen backlight parameters and the corresponding external image data and screen display content, and train the neural network model according to the reference screen backlight parameters, the external image data and the screen display content.

[0108] In addition to manually adjusting the screen backlight parameters, users can also manually set the screen display parameters such as user preferred color, contrast, saturation, etc. Therefore, the data of the local AI training library can be continuously updated based on the user's manual adjustment data of the screen display parameters. Specifically:

[0109] Obtain screen display parameters manually adjusted by the user, and determine that the adjusted screen display parameters are reference screen display parameters; obtain external image data and screen display content corresponding to when the user manually adjusts the screen display parameters; update data in the local AI training library according to the reference screen display parameters and the corresponding external image data and screen display content, and train the neural network model according to the reference screen display parameters, the external image data and the screen display content.

[0110] The neural network model proposed in the embodiment of the present application can learn and train based on the user's previous manual adjustment behavior in the automatic adjustment mode of the screen display, so as to output a screen display effect that is closer to the user's usage needs, reduce the user's actions of autonomously adjusting the screen display in the automatic adjustment mode, improve the screen display performance, and enhance the user experience.

[0111] Furthermore, in order to prevent some extreme display adjustment behaviors of the user from affecting the learning of the neural network model, it is possible to pre-judge whether the user's display adjustment behavior is a valid adjustment behavior. If it is a valid adjustment behavior, the adjustment behavior is learned and trained; if it is an invalid adjustment behavior (such as behavior that obviously deviates from normal display adjustment), the adjustment behavior is not learned.

[0112] See also Fig. 9 , a screen display adjustment device 300 provided in an embodiment of the present application, comprising:

[0113] A first acquisition module 310 is used to acquire external image data acquired by the image acquisition device 110;

[0114] The second acquisition module 320 is used to acquire the screen display content of the current screen 170;

[0115] The processing module 340 is used to determine the backlight adjustment parameters and the screen display adjustment parameters based on the preset neural network model according to the external image data and the screen display content, wherein the neural network model is trained by the first AI training library according to the parameters manually adjusted by the user;

[0116] The adjustment module 350 is used to adjust the screen display of the electronic device according to the backlight adjustment parameters and the screen display adjustment parameters.

[0117] Furthermore, if Fig. 9 As shown, the screen display adjustment device 300 may also include:

[0118] The third acquisition module 330 is used to acquire the screen backlight parameters and / or screen display parameters manually adjusted by the user, and the processing module 340 determines the adjusted screen backlight parameters and / or screen display parameters as reference screen backlight parameters and / or reference screen display parameters;

[0119] The first acquisition module 310 acquires external image data when the user manually adjusts the screen backlight parameters and / or the screen display parameters;

[0120] The second acquisition module 320 acquires the screen display content when the user manually adjusts the screen backlight parameters and / or the screen display parameters;

[0121] The processing module 340 is also used to train a neural network model based on reference screen backlight parameters and / or reference screen display parameters, external image data and screen display content.

[0122] An embodiment of the present application further provides a non-volatile computer-readable storage medium storing a computer program. When the computer program runs on a computer, the computer executes the screen display adjustment method in any of the above embodiments.

[0123] It should be noted that, a person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the storage medium can include but is not limited to: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0124] In addition, the terms "first", "second", and "third" in this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but some embodiments also include steps or modules that are not listed, or some embodiments also include other steps or modules inherent to these processes, methods, products, or devices.

[0125] The above is a detailed introduction to the screen display adjustment method, device, electronic device and non-volatile computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for adjusting screen display, applied to electronic equipment, characterized in that: include: Acquiring external image data acquired by an image acquisition device; Get the screen display content of the current screen; Determine a backlight adjustment parameter and a screen display adjustment parameter based on a preset neural network model according to the external image data and the screen display content, wherein the backlight adjustment parameter includes one or both of screen backlight brightness and screen backlight color temperature; The screen display of the electronic device is adjusted according to the backlight adjustment parameters and the screen display adjustment parameters; wherein the screen backlight parameters and the screen display parameters are dynamically adjusted in units of single-frame images; or, the screen backlight parameters and the screen display parameters are adjusted according to the results of multi-frame image processing; or, when the backlight parameter difference or display parameter difference between two frames of display images is greater than a threshold, the screen backlight parameters and the screen display parameters are adjusted.

2. The method for adjusting screen display according to claim 1, characterized in that: The determining of the backlight adjustment parameters and the screen display adjustment parameters according to the external image data and the screen display content based on a preset neural network model comprises: Determining user feature information and / or ambient light information according to the external image data; According to the user characteristic information and / or ambient light information, the screen display content, and based on the preset neural network model, the backlight adjustment parameters and the screen display adjustment parameters are determined.

3. The method for adjusting screen display according to claim 1, characterized in that: The determining of the backlight adjustment parameters and the screen display adjustment parameters according to the external image data and the screen display content based on a preset neural network model comprises: Determining screen display parameters of the screen display content according to the screen display content; Determining the backlight adjustment parameter and the screen display adjustment parameter based on the preset neural network model according to the external image data and the screen display parameter; The screen display parameters include one or more of user preferred color, contrast, and saturation.

4. The method for adjusting screen display according to claim 3, characterized in that: The method further comprises: According to the screen display content, identifying scene feature information of the screen display content based on a scene classifier; According to the external image data, the screen display parameters and the scene feature information, based on the preset neural network model, the backlight adjustment parameters and the screen display adjustment parameters are determined.

5. The method for adjusting screen display according to claim 1, characterized in that: The method further comprises: Dividing the screen display content into blocks according to preset rules; For a single screen block obtained after the block division, determining the backlight adjustment parameter and the screen display adjustment parameter based on the preset neural network model according to the external image data and the screen display content in the single screen block; The screen display of the electronic device is adjusted according to the backlight adjustment parameters and the screen display adjustment parameters of all screen blocks.

6. The method for adjusting screen display according to claim 3, characterized in that: Acquiring screen backlight parameters and / or screen display parameters manually adjusted by a user, and determining the adjusted screen backlight parameters and / or screen display parameters as reference screen backlight parameters and / or reference screen display parameters; Acquiring the external image data and the screen display content corresponding to when the user manually adjusts the screen backlight parameters and / or the screen display parameters; The neural network model is trained according to the reference screen backlight parameters and / or reference screen display parameters, the external image data and the screen display content.

7. The method for adjusting screen display according to claim 1, characterized in that: The adjustment frequency of the screen display of the electronic device is the same as the display refresh frequency of the screen.

8. A device for adjusting screen display, characterized in that: include: A first acquisition module, the first acquisition module is used to acquire external image data acquired by the image acquisition device; A second acquisition module, the second acquisition module is used to acquire the screen display content of the current screen; A processing module, the processing module is used to determine a backlight adjustment parameter and a screen display adjustment parameter based on a preset neural network model according to the external image data and the screen display content, wherein the backlight adjustment parameter includes one or both of screen backlight brightness and screen backlight color temperature; An adjustment module, wherein the adjustment module is used to adjust the screen display of the electronic device according to the backlight adjustment parameters and the screen display adjustment parameters; wherein the screen backlight parameters and the screen display parameters are dynamically adjusted in units of single-frame images; or, the screen backlight parameters and the screen display parameters are adjusted according to the results of multi-frame image processing; or, when the backlight parameter difference or display parameter difference between two frames of display images is greater than a threshold, the screen backlight parameters and the screen display parameters are adjusted.

9. An electronic device, comprising an image acquisition device, a memory and a processor, characterized in that: The image acquisition device is used to acquire external image data; The memory stores a computer program; The processor is used to execute the screen display adjustment method as described in any one of claims 1 to 7 by calling the computer program.

10. A non-volatile computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the screen display adjustment method as described in any one of claims 1 to 7 is implemented.

Images