Image processing method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method, electronic equipment and a storage medium. The method comprises the steps that a to-be-displayed first image and a currently-displayed second image are acquired; the first image is the next frame of image of the second image; according to the first image, the second image and historical display color gradation information of the ink screen, a first display result corresponding to the first image is determined, and the historical display color gradation information is used for representing a migration rule of historical display color gradation of pixel points of the ink screen; determining a first area with color gradation deviation based on the first display result; determining a target color gradation of each pixel point corresponding to the first area in the first image according to historical display color gradation information of the ink screen; adjusting the current color gradation of each pixel point corresponding to the first area in the first image to the target color gradation of the corresponding pixel point; and the third image is displayed, and the third image is a corresponding display result after the first image subjected to color gradation adjustment is sent and displayed, so that the effect of reducing visual ghost shadows is achieved.

Description

Technical Field

[0001] This application relates to the field of electronic technologies, and in particular, to an image processing method, an electronic device, and a storage medium. Background Art

[0002] Electronic paper displays (EPDs), also known as ink screens, are being adopted by more and more mobile devices because they are friendly to human eye health, have low power consumption, and are more energy-efficient. Although EPDs have many advantages, there are also significant limitations that hinder the widespread application of EPDs in devices other than e-readers. EPDs have two typical display modes. One is the slow refresh mode. In the slow refresh mode, EPDs can display static content with good quality. However, due to problems such as long refresh time and screen flicker, when displaying dynamic content, the user experience is poor, especially in scenarios where EPDs need to have a higher update frequency, such as web browsing and video playback scenarios. Users cannot accept problems such as long refresh time and screen flicker existing in EPDs. For this reason, another refresh mode, that is, the fast refresh mode, has been introduced in the prior art. The fast refresh mode uses short waveforms to drive the display, skips the activation step in the waveform, has no flicker problem, and the refresh time usually only requires one-fifth of the time required for a complete waveform. Compared with the slow refresh mode, the refresh time can be greatly shortened. Currently, fast refresh has become a standard option for EPD devices. Although the fast refresh mode can solve problems such as long refresh time and screen flicker of ink screens, because the activation step is missing in the short waveform, it becomes more difficult to control the particles in the EPD, which may lead to inaccurate positioning of the particles, causing the image color of the previous frame to remain on the image of the current frame, resulting in a ghost image on the screen, that is, the "ghosting effect", seriously affecting the screen readability and the user experience is poor. Summary of the Invention

[0003] This application provides an image processing method, an electronic device, and a storage medium to solve the problem of ghosting in ink screen displays in the fast refresh mode.

[0004] In a first aspect, the present application provides an image processing method, which can be executed by an electronic device with an e-ink screen (hereinafter referred to as an e-ink screen device) or a component in the e-ink screen device. Taking the case where the method can be executed by the e-ink screen device as an example, in this method, the e-ink screen device acquires a first image to be displayed and a second image currently being displayed; the first image is the next frame image of the second image; according to the first image, the second image, and the historical display gradation information of the e-ink screen, a first display result corresponding to the first image is determined, and the historical display gradation information is used to characterize the migration information of the historical display gradations of the pixel points of the e-ink screen. The e-ink screen device determines a first area with a gradation deviation based on the first display result, and determines the target gradation of each pixel point in the first image corresponding to the first area according to the historical display gradation information of the e-ink screen; then, adjusts the current gradation of each pixel point in the first image corresponding to the first area to the target gradation of the corresponding pixel point; and displays a third image, where the third image is the display result corresponding to the first image after gradation adjustment is sent for display.

[0005] Based on this method, by determining the first display result corresponding to the first image to be displayed, the second image currently being displayed, and the historical display gradation information of the e-ink screen, and after determining the first area with a gradation deviation in the first display result, performing gradation adjustment on the first area in the first image, so as to eliminate the afterimage visual effect of the display result corresponding to the first image after gradation adjustment, thereby solving the problem of e-ink screen display afterimage in the brush mode.

[0006] In a possible design, the e-ink screen device determines the first area with a gradation deviation based on the first display result, including: if the gradation of each pixel point in the first area in the first display result is different from the gradation of each pixel point in the first image corresponding to the first area, the e-ink screen device determines the first area with a gradation deviation. Through this design, a method for easily determining the first area with a gradation deviation is provided.

[0007] In a possible design, the first display result may further include a second area, the second area and the first area belong to the same semantic area, and no gradation deviation occurs in the second area. The above method may further include: the e-ink screen device adjusts the gradation of each pixel point in the first image corresponding to the second area to the target gradation of the corresponding pixel point according to the historical display gradation information corresponding to each pixel point in the second area. In this way, the e-ink screen device can perform gradation adjustment on the second area in the first image where no gradation deviation occurs, so that the gradations corresponding to the first area and the second area in the display result corresponding to the first image after gradation adjustment are the same or similar, thereby solving the problem of e-ink screen display afterimage.

[0008] In a possible design, the method further includes: the e-ink screen device sending and displaying multiple preset images to the e-ink screen; for each pixel point in the e-ink screen, performing: the e-ink screen device obtaining multiple color levels corresponding to the display of each pixel point when the e-ink screen displays multiple preset images; clustering the multiple color levels to obtain at least one key color class; and constructing a migration rule between at least one key color class based on the multiple color levels displayed by each pixel point in the e-ink screen to obtain the historical display color level information of the e-ink screen. Through this design, the color levels that the pixel points in the e-ink screen may display can be represented by a small number of key color classes, thereby avoiding misinterpretation of color level deviation caused by minor color level deviations during the image processing process.

[0009] In a possible design, the e-ink screen device constructs a migration rule between at least one key color class based on the multiple color levels displayed by each pixel point in the e-ink screen to obtain the historical display color level information corresponding to each pixel point, including: the e-ink screen device automatically constructs a migration rule between at least one key color class based on the multiple color levels displayed by each pixel point in the e-ink screen to obtain a state transition automaton corresponding to each pixel point, and the state transition automaton corresponding to each pixel point is used to represent the migration rule between at least one key color class historically displayed by the corresponding pixel point. Through this design, using an automaton to model the rule of color level deviation can accurately and efficiently describe the color level deviation phenomenon, and can improve the speed and efficiency of afterimage analysis.

[0010] In a possible design, the e-ink screen device determines a first display result corresponding to the first image according to the first image, the second image, and the historical display color level information of the e-ink screen, including: for the first pixel point in the first image, where the first pixel point is any pixel point in the e-ink screen, performing: determining the color level of the first pixel point on the e-ink screen when displaying the first image according to the color level of the first pixel point in the first image, the color level of the corresponding first pixel point in the second image, and the state transition automaton corresponding to the first pixel point to obtain the first display result. Through this design, according to the historical display color level information of the e-ink screen and the color levels displayed by each pixel point in the currently displayed second image, the color levels of the corresponding pixel points in the first image to be displayed subsequently can be determined, thereby obtaining the first display result corresponding to the first image.

[0011] In a second aspect, the present application further provides a device, and the device includes modules / units for executing the method according to any one of the possible designs in any of the above aspects. These modules / units can be implemented by hardware or by hardware executing corresponding software.

[0012] In a third aspect, the present application provides an electronic device, including an e-ink screen, a processor, and a memory. The e-ink screen is used to display a user interface, and the memory is used to store one or more programs; when the one or more programs stored in the memory are executed by the processor, the electronic device can implement the methods in the above-mentioned various aspects and any possible design involved in each aspect.

[0013] In a fourth aspect, the present application further provides a readable storage medium, where the readable storage medium includes a program, and when the program runs on an electronic device, the electronic device is enabled to execute the methods in the above-mentioned various aspects and any possible design involved in each aspect.

[0014] In a fifth aspect, the present application further provides a computer program product, and when the computer program product runs on an electronic device, the electronic device is enabled to execute the methods in the above-mentioned various aspects and any possible design involved in each aspect. Description of the Drawings

[0015] Figure 1 It is a schematic diagram of an e-ink screen structure provided by an embodiment of the present application;

[0016] Figure 2 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0017] Figure 3 It is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0018] Figure 4 It is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0019] Figure 5 It is a schematic diagram of a state transition automaton provided by an embodiment of the present application;

[0020] Figure 6 It is a schematic diagram of the flowchart of an image processing method provided by an embodiment of the present application;

[0021] Figure 7 It is a schematic diagram of image sending and displaying provided by an embodiment of the present application;

[0022] Figure 8 It is a schematic diagram of image sending and displaying provided by an embodiment of the present application;

[0023] Figure 9 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0024] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above", "said", "this" are also intended to include the forms such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the embodiments of the present application, "one or more" means one, two or more than two; "and / or" describes the association relationship of associated objects and indicates that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship.

[0025] References to "some embodiments" and the like described in this specification mean that specific features, structures, or characteristics described in connection with that embodiment are included in one or more embodiments of the present application. Thus, the phrases "in some embodiments", "in other some embodiments", "in still other some embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0026] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0027] For ease of understanding, illustrative descriptions of concepts related to the present application are given for reference.

[0028] In some embodiments of the present application, the electronic device may be a portable device with an e-ink screen, which may be simply referred to as an e-ink screen device, such as a mobile phone with an e-ink screen, a tablet computer, a wearable device with wireless communication functions (e.g., a watch, a bracelet, a helmet, headphones, etc.), a vehicle-mounted terminal device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart home device (e.g., a smart TV, a smart speaker, etc.), a smart robot, a workshop device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, or a wireless terminal in a smart home, a flying device (e.g., a smart robot, a hot air balloon, a drone, an airplane), etc.

[0029] In some embodiments of the present application, the electronic device may also be a portable terminal device that further includes other functions such as a personal digital assistant and / or a music player function. Exemplary embodiments of the portable terminal device include, but are not limited to, a portable terminal device equipped with or other operating systems. The above portable terminal device may also be other portable terminal devices, such as a laptop with a touch-sensitive surface (e.g., a touch panel), etc. It should also be understood that in some other embodiments of the present application, the above electronic device may not be a portable terminal device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0030] An e-ink screen is a reflective display screen. Different from the traditional LCD display screen, the e-ink screen realizes the display ability by reflecting the light incident on the screen panel. Each pixel of the e-ink screen consists of several tiny colored particle capsules. By controlling the voltage waveform applied to the e-ink screen, the particles can be suspended at different positions in the capsules, thus presenting different color gradations.

[0031] Figure 1 This is a schematic structural diagram of the e-ink screen provided by the embodiments of the present application. As Figure 1As shown, the ink screen includes a large number of ink droplets 13. The ink droplets 13 are also called microcapsules, and the size of these microcapsules is about the diameter of a human hair. The ink droplets 13 contain a clear liquid and white particles 15 and black particles 16 suspended in the clear liquid. The white particles 15 carry a positive charge, and the black particles 16 carry a negative charge. The ink screen also includes a top transparent electrode 11 and a bottom electrode 12 for applying an electric field to control the movement of the white particles 15 and black particles 16 in the ink droplets 13. For example, as Figure 1 shown by the ink droplet 13 on the left in Figure 1 , when the electric field is set to positive and the bottom electrode 12 applies a positive voltage, the white particles 15 in the ink droplet 13 move to the top of the ink droplet, that is, in the upward direction of the screen, so that the ink screen presents white to the user. At the same time, the black particles 16 in the ink droplet 13 are pulled to the bottom of the ink droplet and thus hidden. For example, as

[0032] shown by the ink droplet 13 on the right in

[0033] , when the electric field is set to negative and the bottom electrode 12 applies a negative voltage, the black particles 16 in the ink droplet 13 move to the top of the ink droplet, that is, in the upward direction of the screen, so that the ink screen presents black to the user. At the same time, the white particles 15 in the ink droplet 13 are pulled to the bottom of the ink droplet and thus hidden. It can be seen that by applying an electric field through the top transparent electrode 11 and the bottom electrode 12, the movement of the white particles and black particles in the ink droplets can be controlled, so that the ink screen can present colors and display the picture content. Among them, the colors include white, black, and grayscale levels (also called color levels) between white and black. The number of color levels is not limited in the embodiments of the present application. For example, 16 color levels, 32 color levels, etc.

[0032] It should be understood that in the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally means that the associated objects before and after are an "or" relationship. "At least one (item) of the following" or its similar expression refers to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, c can be single or multiple.

[0033] Currently, although the fast brush mode adopted by the ink screen can solve problems such as the long refresh time and screen flicker of the ink screen, it also brings the problem of display afterimage of the ink screen, affecting the user experience. To solve the problem of display afterimage of the ink screen, several possible methods are provided below.

[0034] In one solution, a brand-new driving waveform can be designed to reduce the phenomenon or degree of color level deviation, thereby reducing the afterimage effect. However, since there is no particle oscillation process in the fast refresh mode, regardless of the waveform, the particles will not be driven in place in the fast refresh mode, resulting in color level deviation. Therefore, this method has extremely limited corrective effect on color deviation and poor improvement effect on display afterimages.

[0035] In another solution, noise is incorporated into the image to be displayed, attempting to use the noise to confuse the afterimage. This method itself cannot prevent the occurrence of afterimages, and a lot of display content unrelated to the image to be displayed is doped into the image to be displayed in order to confuse the afterimage, which will cause the display content of the entire screen to be more chaotic, the image quality of the screen display to be damaged, and it is difficult to read.

[0036] In yet another solution, all possible color level deviations of each pixel point during the display process of the screen are recorded, and then, based on the recorded information, color level correction is performed on the pixel points with color level deviations, so that the color displayed by the pixel points with color level deviations is close to the color that the pixel points should actually display. Since the e-ink screen is a display device different from the LCD and realizes color level display by driving the particles in the screen hardware, a considerable proportion of the particles cannot be accurately driven. Therefore, for a pixel point, although the device only supports a limited number of color levels (such as 16), there are many possibilities for the actual gray scale results displayed by the pixel point. Therefore, it is not practical to record all possible color levels in a real system. Moreover, some color level deviations cannot be corrected. For example, in some specific historical display situations of the e-ink screen, white will be displayed as light gray, and then, even if pure white is continuously sent to such pixel points, it is still impossible to correct the light gray to white. In this case, the appearance of afterimages is almost inevitable.

[0037] In view of the above problems, the present application provides an image processing method, an electronic device, and a storage medium to solve the problem of afterimage display on the e-ink screen in the fast refresh mode. Among them, the method and the electronic device are based on the same technical concept. Since the principles of the method and the electronic device for solving the problem are similar, the implementation of the electronic device and the method can be referred to each other, and the repeated parts will not be elaborated.

[0038] Figure 2 Only the schematic hardware structure diagram of an electronic device 100 provided by the application embodiment is shown. On the basis Figure 2 shown, there may also be other variant structural forms. Such as Figure 2As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0039] The sensor module 180 may include one or more of the following: a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0040] Next, a detailed introduction will be given to Figure 2 the components of the illustrated electronic device 100.

[0041] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0042] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory, thereby avoiding repeated access, reducing the waiting time of the processor 110, and thus improving the efficiency of the system.

[0043] The execution of the image processing method provided by the embodiments of the present application can be controlled by the processor 110 or other components can be called to complete it. For example, the processing program of the embodiments of the present application stored in the internal memory 121 can be called, or the processing program of the embodiments of the present application stored in a third-party device can be called through the external memory interface 120 to control the wireless communication module 160 to perform data communication with other devices, improving the intelligence and convenience of the electronic device 100 and enhancing the user experience. The processor 110 may include different devices. For example, when the CPU and GPU are integrated, the CPU and GPU can cooperate to execute the image processing method provided by the embodiments of the present application. For example, some algorithms in the image processing method are executed by the CPU, and another part of the algorithms are executed by the GPU to obtain a faster processing efficiency.

[0044] In some embodiments, the processor 110 may include one or more interfaces. For example, the interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. It can be understood that the interface connection relationship between the modules schematically shown in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100.

[0045] The charging management module 140 is used to receive a charging input from a charger. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0046] Antenna 1 and Antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, Antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0047] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through Antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through Antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be provided in the same device.

[0048] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor can be an independent device. In some other embodiments, the modulation and demodulation processor can be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0049] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be sent from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic waves via the antenna 2 for radiation.

[0050] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technologies.

[0051] The electronic device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The display screen 194 is used to display images, videos, etc. In the embodiments of the present application, the display screen 194 is an e-ink screen. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. The display screen 194 may be used to display the information input by the user or the information provided to the user, as well as various graphical user interfaces (GUIs). For example, the display screen 194 may display photos, videos, web pages, or files, etc.

[0052] In the embodiments of the present application, the display screen 194 may be an integrated flexible display screen, or may be a spliced display screen composed of two rigid screens and a flexible screen located between the two rigid screens.

[0053] In some other embodiments, the electronic device 100 may implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0054] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0055] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system and application programs required for at least one function. In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor.

[0056] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0057] The electronic device 100 can receive the input of the key 190 and generate a key signal input related to the user settings and function control of the electronic device 100. For example, the key 190 can include a power key, a volume up key, a volume down key, etc. provided on the right side of the electronic device.

[0058] Although Figure 2 not shown in the figure, the electronic device 100 may further include a Bluetooth device, a positioning device, a flashlight, a micro projection device, a near field communication (NFC) device, etc., which will not be elaborated here.

[0059] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than Figure 2 shown in the figure, or combine certain components, or split certain components, or have different component arrangements. Figure 2 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0060] The following embodiments can all be implemented in the electronic device 100 having the above hardware structure.

[0061] The software system of an electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of this application, taking a software system with a layered architecture (such as the Android system) as an example, the software structure of the electronic device is exemplarily described.

[0062] Refer to Figure 3 , which is a schematic diagram of the layered software structure of the electronic device provided in the embodiments of this application. The layered architecture divides the software system of the electronic device into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces.

[0063] In some embodiments, the Android system can be divided into four layers, namely the application layer (applications), the application framework layer (application framework), the Android runtime, and the system libraries and the kernel layer (kernel). The embodiments of this application do not limit the layering of the software structure of the electronic device.

[0064] It should be understood that Figure 3 the modules included in each layer shown are the modules involved in the embodiments of this application. The modules included in the following layers do not constitute a limitation on the structure and module deployment hierarchy (exemplary description) of the electronic device. In one embodiment, Figure 3 the modules shown in can be deployed separately, or several modules can be deployed together. Figure 3 The division of modules in is an example. In one embodiment, Figure 3 the names of the modules shown in are for exemplary description.

[0065] The application layer can include a series of application packages. For example Figure 3 the application 1, application 2, and application 3 shown. These applications can include, but are not limited to: camera, gallery, calendar, Bluetooth, music, video, information, etc. The applications involved in the application layer can include system applications and third-party applications. Any application can generate data to be displayed, and after the electronic device processes the data to be displayed, it is displayed on the display screen. In the embodiments of this application, the display screen is an e-ink screen.

[0066] The application framework layer can provide application programming interfaces (application programming interfaces, APIs) and programming frameworks for the applications in the application layer. The application framework layer can include some predefined functions. Refer to Figure 3 , the application framework layer can include window management services, graphics rendering systems, etc.

[0067] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0068] The graphics drawing system can be used as an interface for providing system drawing services in an electronic device. An application can draw graphics, images and other drawing contents required for application services by calling the graphics drawing system. In the embodiments of the present application, the application can send the drawing contents to the display system for content display. Exemplarily, the graphics drawing system can store each generated frame of image into a buffer queue, and the display system sequentially obtains each frame of image from the buffer queue and then sends it to the e-ink screen for display. Optionally, the application can be a system application or a third-party application.

[0069] Since the Android runtime, system libraries, and kernel layer are regarded as one layer in the embodiments of the present application, the system layer may include the Android runtime, system libraries, and functional modules in the kernel layer. The system layer may include multiple functional modules, such as: a display system, media libraries, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc. Refer to Figure 3 The system layer may include multiple functional modules, such as: a surface manager, media libraries, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc.

[0070] The display system is used to synthesize and output the drawing contents drawn by the application by calling the graphics drawing system to the e-ink screen for display. Specifically, the display system may include a display control module, a ghosting analysis module, and a ghosting reduction module. Among them, the display control module is used to control the content display related to the content synthesis method, display path, etc. For example, the display control module can obtain a frame of image to be displayed from the buffer queue and the image of the previous frame of the image to be displayed on the e-ink screen, and send the image to be displayed to the ghosting analysis module. The ghosting analysis module can also obtain the image of the previous frame displayed on the e-ink screen, and obtain the possible ghosting information of the image to be displayed by performing ghosting analysis on the image to be displayed and the image of the previous frame displayed on the e-ink screen. The ghosting reduction module is used to perform ghosting reduction processing on the image to be displayed to obtain a processed image. The ghosting reduction module can send the processed image after the ghosting reduction processing to the display control module, and the display control module updates the image to be displayed in the buffer queue with the processed image after the ghosting reduction processing. The display control module can also send the processed image after the ghosting reduction processing to the e-ink screen.

[0071] Optionally, the display system may further include a dithering algorithm module for performing dithering processing on the image to be displayed, generating a dithered image, and the display color of each pixel point in the dithered image is pure black or pure white. For example, the dithering algorithm module may perform dithering processing on the processed image after ghosting reduction processing, and the display control module may further control the dithering algorithm module to update the image to be displayed in the buffer queue to the processed image after ghosting reduction processing and dithering processing, and send the processed image to the ink screen for display.

[0072] Taking the image B to be displayed as an example below, combined with Figure 4 the image processing process will be described.

[0073] As Figure 4 shown, the display control module obtains the image B to be displayed from the buffer queue, and performs dithering on the image B to be displayed through the dithering algorithm module to obtain the dithered image B to be displayed D , and then sends the dithered image B to be displayed D to the ghosting analysis module. The ghosting analysis module can perform ghosting analysis based on the dithered image B to be displayed D and the actually displayed image A of the previous frame D to obtain the analyzed ghosting information, and the ghosting information is used to characterize the relevant information of the possible ghosting of the normal image B to be displayed, such as the color level deviation value of each pixel point in the area where the ghosting appears. Then, the ghosting analysis module can send the analyzed ghosting information to the ghosting reduction module, and the ghosting reduction module performs ghosting reduction processing on the image B to be displayed to generate the modified image B' to be displayed. This ghosting reduction processing can achieve ghosting reduction when the image B to be displayed is displayed on the ink screen. The ghosting reduction module sends the modified image B' to be displayed to the dithering algorithm module, and performs dithering on the modified image B' to be displayed through the dithering algorithm module to obtain the dithered image B to be displayed D ', and the dithered image B to be displayed D ' is the final image to be displayed. After that, the display control module can also update the image B to be displayed in the buffer queue to the dithered image B to be displayed D '. The display result of the dithered image B to be displayed in the buffer queue D ' sent to the ink screen is actually the result of superimposing the dithered image B to be displayed D ' and the actually displayed image A of the previous frame D , that is, as Figure 4 the actually displayed image B in S ', and the ghosting phenomenon is reduced.

[0074] In addition, the system layer further includes hardware drivers for driving the hardware to work. For example, the driver layer includes: sensor drivers, display drivers (display driver), graphics processing unit drivers (graphics processing unit driver, GPU driver), etc., which are not limited in the embodiments of the present application. It can be understood that Figure 3 The display driver involved in the embodiments of the present application is shown in Figure 3 . The display driver is used to drive the ink screen to display the user interface.

[0075] In one example, when the electronic device adopts the Android system, the graphics drawing system can be implemented as a render thread in the Android system, and the display system can be implemented as a surface flinger service in the Android system.

[0076] It should be noted that Figure 3 The system architecture shown is only an exemplary illustration of the system architecture applicable to the solution of the present application, Figure 3 and the shown system architecture does not limit the system architecture applicable to the solution of the present application. The system architecture applicable to the solution of the present application may include fewer or more modules than Figure 3 those shown, and no specific limitation is made in the embodiments of the present application.

[0077] Next, specific embodiments will be combined to detail the solution provided by the present application. The image processing method provided by the embodiments of the present application can be applied to an ink screen device.

[0078] In the embodiments of the present application, the historical display gradation information corresponding to each pixel point in the ink screen can be obtained. The historical display gradation information is used to characterize the migration law of the historical display gradation of each pixel point. Then, the display prediction result corresponding to the image to be displayed can be predicted according to the historical display gradation information, so as to determine whether the display prediction result corresponding to the image to be displayed has a gradation deviation according to the display prediction result corresponding to the image to be displayed and the sending display gradation corresponding to the image to be displayed. Then, the gradation of the image to be displayed with a gradation deviation is adjusted to eliminate the ghosting phenomenon of the image to be displayed.

[0079] In a possible implementation manner, the ghosting analysis module in the ink screen device can perform ghosting analysis on the display results of the ink screen for sending and displaying multiple preset images to obtain the historical display gradation information corresponding to each pixel point in the ink screen. It should be understood that ghosting analysis can be performed on each ink screen. For multiple ink screens of the same batch, ghosting analysis can be performed on one or more ink screens in the same batch to obtain the historical display gradation information corresponding to each pixel point in this batch of ink screens.

[0080] In a possible implementation, the image processing method can also be used to analyze the ghosting of different batches of e-ink screens.

[0081] The following takes the analysis of the ghosting of the target e-ink screen as an example for illustration.

[0082] First, a large number of color switching experiments are carried out on the target e-ink screen hardware. For example, a plurality of different preset images are sequentially sent to the target display screen, so that each pixel on the target e-ink screen experiences various display histories, and the color levels actually displayed by each pixel are recorded. For each pixel, during the color switching experiment, there may be a deviation between the color level actually displayed by this pixel (which can be called the actual display color level) and the expected display color level (which can be called the sent display color level).

[0083] Exemplarily, taking the example of sequentially sending 1000 preset images to the target e-ink screen, for a pixel A on the target e-ink screen, record the color levels actually displayed by pixel A when the e-ink screen displays these 1000 preset images respectively. For example, when sending picture B to the target display screen, the sent display color level corresponding to pixel A is 13, but the actual display color level corresponding to pixel A when the target e-ink screen displays picture B is 11. It can be seen that there is a color level deviation when the target e-ink screen displays picture B, and the color level deviation value is 2.

[0084] Secondly, after the e-ink screen device obtains the multiple color levels corresponding to each pixel when the display screen displays each preset image, it can cluster the multiple color levels displayed by each pixel obtained, obtain at least one key color class, and construct the migration rules between at least one key color class to obtain the historical display color level information corresponding to each pixel.

[0085] Exemplarily, taking four key color classes as an example, for example, cluster the color levels that pixel A may display to obtain four different key color classes (or called color level classes). Among them, the color level value from 0 to 2 is pure white, represented by the color level value from 3 to 7 is light gray, represented by W-, the color level value from 8 to 12 is dark gray, represented by B + the color level value from 13 to 15 is pure black, represented by It should be understood that the gradation values from 0 to 15 can be classified into four gradation classes, or can be classified into a greater or smaller number of gradation classes. This application does not limit this. Through clustering, it is possible to obtain the main information of color deviation during the image processing process, and exclude the misinterpretation of the gradation deviation behavior caused by minor gradation deviations. Then, the e-ink screen device can construct the migration rules between the four key color classes based on the key color classes to which the multiple gradations historically displayed by each pixel belong, so as to obtain the historical display gradation information of the e-ink screen. Among them, the migration rule is essentially a formal mathematical expression, equivalent to a mathematical function. That is, when the display history of pixel point A is given, according to the migration rule, it can be determined what kind of gradation deviation pixel point A will have in the display result corresponding to the subsequent image to be sent for display.

[0086] In a possible implementation manner, the e-ink screen device can automatically construct the migration rules between at least one key color class to obtain a state transition automaton corresponding to each pixel point. The state transition automaton is used to represent the migration rules between at least one key color class historically displayed by the corresponding pixel point.

[0087] Exemplarily, the migration rule corresponding to pixel point A is, for example: Pixel point A displayed pure black in the previous frame, and the current image to be sent for display is white, and the current frame will be displayed as light gray instead of white. After displaying light gray, when sending black for display again, this pixel point A will be displayed as dark gray in the next frame. It should be understood that due to the physical characteristics of the e-ink screen device, pure black is generally not displayed.

[0088] For migration rules like this, a state transition automaton as shown in Figure 5 can be used to formally express the extracted gradation deviation rules. As shown in Figure 5 , the circles represent states, and the specific physical meaning is the actual display result of a pixel point. The letters in the circles represent a certain gradation. For example, represents pure white, represents pure black, W - represents light gray, B + represents dark gray, and the arrows represent state transitions. The letters on the arrows represent the gradations to be sent for display. For example, Figure 5Among them, B represents the transmitted display black, and W represents the transmitted display white. This state transition automaton can express what color level the ink screen will finally present starting from a certain color of the transmitted display, such as transmitted display black or white. In this way, all possible color levels that may appear on the ink screen are expressed with a small and finite number of states, and the transition rules between color levels are clearly expressed. In an actual system, the ink screen device may transmit and display pictures at a very high frequency, for example, once every 100 ms. Then, all analyses of afterimages and processing of the original image must be completed within these 100 ms. Using an automaton to model the law of color level deviation can accurately and efficiently describe the phenomenon of color level deviation, and can also improve the speed and efficiency of afterimage analysis. In addition, using an automaton to store color level deviation information requires less storage space.

[0089] See Figure 6 , which is a schematic flowchart of an image processing method provided by an embodiment of this application. This method can be executed by an ink screen device or a chip or component inside the ink screen device. The following embodiments will be described by taking the ink screen device as an example.

[0090] Step 601, the ink screen device obtains a first image to be displayed and a second image currently being displayed, where the first image is the next frame image of the second image.

[0091] Exemplarily, the buffer queue in the ink screen device may include multiple frames of images. The display system can sequentially obtain each frame of image from the buffer queue for transmission and display. For example, taking the second image as the first frame image in the buffer queue and the display result after transmitting it to the ink screen, and the first image as the second frame image in the buffer queue as an example, the display system can obtain the second frame image to be displayed from the buffer queue, and then execute the following step 602, which can be implemented by the afterimage analysis module.

[0092] Step 602, the ink screen device determines a first display result corresponding to the first image according to the first image, the second image, and the historical display color level information of the ink screen, where the historical display color level information is used to characterize the migration information of the historical display color levels of the pixel points of the ink screen.

[0093] In a way that can implement step 602, the ink screen device can determine the color level of each pixel point when the first image is displayed on the ink screen according to the color level of each pixel point in the first image, the color level of the corresponding pixel point in the second image, and the state transition automaton corresponding to each pixel point, so as to obtain the first display result corresponding to the first image.

[0094] Taking the second image as Figure 7Taking the first frame display image C shown as an example and the second frame display image D as the first image, after the first frame display image C is displayed on the e-ink screen, the afterimage analysis module obtains the second frame display image D, and based on the first frame display image C, the second frame display image D, and the state transition automaton shown as Figure 5 determines the first display result of the second frame display image D on the e-ink screen, such as the image E shown as Figure 7 .

[0095] Taking pixel point 1 and pixel point 2 in the e-ink screen as an example, when the e-ink screen displays the first frame display image C, pixel point 1 displays white and pixel point 2 displays light gray. When the second frame display image D is sent to the e-ink screen, the sending color levels corresponding to pixel point 1 and pixel point 2 are both black. For pixel point 1, the first frame displays white and the second frame is sent with black (i.e., the sending color level is black). According to the Figure 5 state transition automaton shown, it can be determined that pixel point 1 will display black when the e-ink screen displays the second frame, that is, the actual display color level is black. Therefore, the sending color level and the actual display color level corresponding to pixel point 1 are the same, indicating that there is no color level deviation for pixel point 1 when the e-ink screen displays the second frame. For pixel point 2, the first frame displays light gray, and when the second frame is sent with black (i.e., the sending color level is black), according to the Figure 5 state transition automaton shown, it can be determined that pixel point 2 will display dark gray when the e-ink screen displays the second frame, that is, the actual display color level is dark gray. Therefore, the sending color level and the actual display color level corresponding to pixel point 2 are different, indicating that there is a color level deviation for pixel point 2 when the e-ink screen displays the second frame. By analogy, the color level displayed by each pixel point when the second frame display image D is sent to the e-ink screen can be determined, so as to obtain the Figure 7 first display result: image E, in which there are areas with color level deviation in the hair area and the clothing area, as shown in Figure 7 . The hair area includes area 1 and area 2, and the clothing area includes area 3 and area 4. Among them, area 2 and area 3 have color level deviation, which will cause the user to have the visual effect of afterimage. Area 1 and area 4 do not have color level deviation and will not cause the user to have the visual effect of afterimage.

[0096] After obtaining the first display result corresponding to the first image, the e-ink screen device can determine whether there is a color level deviation in the first display result.

[0097] In a possible implementation manner, the afterimage analysis module can divide the first display result into multiple semantic regions, and the specific division method is not limited. For example Figure 7The first display result: Image E can be divided into semantic regions such as hair, face, clothes, background, etc. Then, it is determined whether the color level of each pixel in each semantic region of Image E is the same as that of the corresponding pixel in the second-frame display image D. If they are the same, it means that the color level of the pixel in the first display result has not deviated; if they are different, it means that the color level of the pixel in the first display result has deviated. Exemplarily, as Figure 7 The color level of pixel 1 in region 1 of the first display result (Image E) is the same as that of the corresponding pixel 1 in the first image, indicating that pixel 1 in region 1 of the first display result (Image E) has not deviated in color level; the color level of pixel 2 in region 2 of the first display result is different from that of the corresponding pixel 2 in the first image, indicating that pixel 2 in region 2 of the first display result has deviated in color level.

[0098] Step 603, the e-ink screen device determines the first region with a color level deviation based on the first display result.

[0099] If the color level of each pixel in the first region of the first display result is different from that of the corresponding pixel in the first image, it is determined that the first display result includes the first region with a color level deviation. Taking Figure 7 the first display result corresponding to the second-frame display image D as an example, there are two semantic regions in the first display result (Image E) that have afterimages. Among them, one semantic region is the hair region in the figure. The upper half of the hair region (i.e., Figure 7 region 1 of Image E in Figure 7 does not have a color level deviation, so it is normal black. However, the lower half of the hair region (i.e., Figure 7 region 2 of Image E in Figure 7 ) has a color level deviation (due to the reason of the display history), and the result shows dark gray. For the user, what the user expects is that the entire hair region, that is, region 1 and region 2, both show black, but the above phenomenon causes the hair region to show two colors, thus resulting in a significant afterimage phenomenon. The other region is the clothes region. When the second-frame display image D is displayed, most of the clothes region has a color level deviation. The clothes should show white, but region 3 in the first display result (Image E) shows light gray. And there is a region under the clothes, such as Figure 7 region 4 in Image E shown in Figure 7 , which does not have a color level deviation, so it still shows pure white. Therefore, for the user, the user can see that part of the clothes region that should all show white shows light gray and part shows white, which presents an afterimage visual effect.

[0100] Step 604, the e-ink screen device determines the target color level of each pixel in the first image corresponding to the first region according to the historical display color level information corresponding to each pixel in the first region.

[0101] Taking the first pixel point in the first region as an example, how to determine the target gray level of the first pixel point will be described. Exemplarily, the historical display gray level information corresponding to the first pixel point is, for example, as Figure 5 shown state transition automaton, and the ink screen device can, according to Figure 5 the shown state transition automaton, determine the target gray level to be sent for display of the first pixel point during the transition process from the gray level state of the first pixel point in the second image to the expected gray level state to be displayed corresponding to the first image. It should be understood that the first pixel point can be any pixel point in the first region.

[0102] Exemplarily, taking the first pixel point in the first region as an example for description, the sent display gray level of the first pixel point in the first region of the first image is light gray, that is, the expected display gray level of the first pixel point is light gray, while the actually displayed gray level of the first pixel point in the previous frame is black. According to Figure 5 the shown state transition state machine, it can be predicted that the actual display result of the first pixel point is dark gray, rather than the expected light gray to be displayed, indicating that there is a gray level deviation of the first pixel point in the first display result corresponding to the first image.

[0103] The ink screen device can, according to Figure 5 the shown state transition automaton, determine the target gray level to be sent for display of the first pixel point during the transition from the gray level state of the first pixel point when displaying the second image to the expected gray level state to be displayed when sending and displaying the first image. Specifically, according to Figure 5 the shown state transition automaton, it can be determined that the sent display color on the arrow from the gray level state representing black to the gray level state representing light gray (W-) is white (W). Therefore, it can be determined that the target gray level of the first pixel point is white (W).

[0104] Step 605, the ink screen device adjusts the sent display gray level of each pixel point in the first region corresponding to the first image to the target gray level of the corresponding pixel point.

[0105] In step 605, for any pixel point in the first region, taking the first pixel point as an example, the ink screen device adjusts the sending display level of the first pixel point corresponding to the first region in the first image to the target level. Continuing with the example in step 604, adjusting the sending display level of the first pixel point to white (W) can make the final display level of the first pixel point light gray, which is consistent with the color level state that the first pixel point is expected to display. In addition, for other pixel points in the first region other than the first pixel point, the target gray level corresponding to the other pixel points is determined. The specific implementation method can refer to the relevant description of determining the target gray level of the first pixel point, which will not be elaborated here. Then, the sending display levels of the other pixel points are adjusted to the target gray levels of the corresponding pixel points, which will not be elaborated here.

[0106] Step 606, the ink screen device displays the third image, and the third image is the sending display result corresponding to the first image after the color level adjustment.

[0107] In step 606, after the ink screen of the ink screen device sends and displays the first image after the color level adjustment, the third image is displayed on the ink screen, and there is no area with color level deviation in the third image.

[0108] In the above embodiment, when there is a first region with color level deviation in the first display result corresponding to the first image, the color levels of each pixel point in the first region of the first image are adjusted to achieve the visual effect of eliminating afterimages. In some other scenarios, the same semantic region in the first display result may include a part with color level deviation and a part without color level deviation. The part with color level deviation in the same semantic region can be regarded as the first region, and the part without color level deviation can be regarded as the second region. The ink screen device can adjust the color levels of each pixel point in the first region of the first image to achieve the visual effect of eliminating afterimages.

[0109] It should be understood that, for example, Figure 7 The hair, face, background, etc. in the portrait in the second-frame sending display image D shown are all different semantic regions.

[0110] In another possible implementation manner, the ink screen device can adjust the color levels of each pixel point in the second region of the first image so that the color levels of the first region and the second region in the third image are similar or equal to achieve the visual effect of eliminating afterimages.

[0111] In this embodiment, if the e-ink screen device determines that the first display result includes a first area with color level deviation and a second area without color level deviation, and the first area and the second area belong to the same semantic area, the e-ink screen device determines the target color level of each pixel corresponding to the second area in the first image according to the historical display color level information of each pixel in the second area. The specific implementation method can refer to the relevant description in step 604 above and will not be elaborated here. Then, the color level of each pixel corresponding to the second area in the first image is adjusted to the target color level of the corresponding pixel, so that the color levels of the first area and the second area in the third image are consistent, so as to achieve the visual effect of eliminating afterimages.

[0112] Exemplarily, taking Figure 7 the second-frame display image D shown as the first image as an example, the hair area in the first display result corresponding to the first image includes area 1 without color level deviation and area 2 with color level deviation. The e-ink screen device can adjust the color level of the pixels corresponding to area 1 in the first display result in the first image, so that the color levels of the pixels in the upper half area of the hair and the color levels of the pixels in the lower half area of the hair in the third image are consistent, so as to achieve the visual effect of eliminating afterimages.

[0113] In another possible implementation manner, the e-ink screen device can adjust the color level of each pixel in the first area of the first image and adjust the color level of each pixel in the second area of the first image, so that the color levels of the first area and the second area in the third image are similar or equal, so as to achieve the visual effect of eliminating afterimages.

[0114] If the e-ink screen device determines that the first display result includes a first area with color level deviation and a second area without color level deviation, and the first area and the second area belong to the same semantic area, the e-ink screen device determines the target color level of each pixel corresponding to the first area in the first image according to the historical display color level information of each pixel in the first area, and determines the target color level of each pixel corresponding to the second area in the first image according to the historical display color level information of each pixel in the second area.

[0115] Next, in combination with Figure 8 the example shown, the color level adjustment of the area without color level deviation is introduced.

[0116] Taking the second image as Figure 8 the first-frame display image C shown and the first image as the second-frame display image D as an example, after the e-ink screen displays the first-frame display image C, the afterimage analysis module obtains the second-frame display image D, and according to the first-frame display image C, the second-frame display image D and as Figure 5The state transition automaton shown in the figure determines the first display result of the second frame of image D on the ink screen. There are two semantic areas with residual images in the first display result. One area with residual images is the hair area in the figure. The upper half of the hair area has no color deviation, so it is normal black. However, the lower half of the hair area (due to display history) has a color deviation, and the result is dark gray. The normal-colored hair in the upper half and the lighter hair in the lower half show a color contrast, allowing the user to visually feel the residual image. Figure 8 In the second frame of the image D' after the adjustment process shown in FIG. 1 , since the color level of the hair is pure black, it is impossible to compensate the color level by adjusting the color of the area where the color level deviation occurs. In other words, it is impossible to eliminate the residual image visual effect. Figure 8 The afterimage of the hair area in the first display result image E shown still exists. In the embodiment of the present application, for the semantic area of ​​hair, the color scale of the upper half of the hair area (i.e., the area corresponding to the first display result where no color scale deviation occurs) is adjusted so that the displayed color of this part becomes lighter, for example Figure 8 In the second frame of the image D" after the adjustment process shown, the white clothes (area 1) without the color gradation deviation area is adjusted to dark gray, so that the upper half of the hair area and the lower half of the hair area appear to be similar colors. Visually, the user will not clearly feel the color difference between the upper and lower parts of the hair, which means that the residual image is weakened.

[0117] Another area where afterimages appear is the clothing area. When the second frame of image D is displayed, color gradation deviation appears in most areas of the clothing. The clothing should be displayed in white, but it appears in light gray in the first display result. However, there is an area below the clothing where there is no color gradation deviation, so it still appears in pure white. Therefore, for the user, the user can see that part of the clothing area that should have been displayed in white appears in light gray, and part appears in white, which presents a visual effect of afterimages. Figure 8 In the second frame of the displayed image D' after the adjustment process shown in FIG. 1 , since the displayed color level corresponding to the clothes is pure white, it is impossible to compensate the color level by adjusting the color of the area where the color level deviation occurs, that is, it is impossible to eliminate the residual image visual effect, such as Figure 8 The afterimage of the clothing area in the first display result image E shown still exists. In the embodiment of the present application, for the semantic area of ​​clothing, the color scale of the area where the color scale does not deviate is adjusted, so that the color of the area where the color scale does not deviate is darker, for example Figure 8In the second frame display image D” after the adjustment process shown, the white clothes (region 4) in the area where there is no color level deviation are adjusted to light gray, so as to eliminate the color difference between this region 4 and the surrounding regions with color level deviation in the second display result: image F. Visually, the user will not significantly feel the color difference between the upper and lower parts of the clothes, thus eliminating the visual effect of the afterimage in the clothes area.

[0118] By performing color level adjustment on the sub-regions in the hair area where there is no color level deviation, and performing color level adjustment on the sub-regions in the clothes area where there is no color level deviation, so as to obtain Figure 8 In the second frame display image D” after the adjustment process shown, and then the second display result obtained by displaying the second frame display image D”: the afterimage visually perceived in image F disappears.

[0119] Through the above example, when adjusting the areas where color level deviation has occurred, in many cases, when reaching the adjustment limit and being unable to complete the adjustment target, the areas where there is no color level deviation can be adjusted, so as to make the color levels of an image semantic area show better consistency.

[0120] In the above embodiments provided by the present application, the method provided by the embodiments of the present application is introduced from the perspective of the e-ink screen device as the execution subject. In order to implement the various functions in the method provided by the above embodiments of the present application, the e-ink screen device may include a hardware structure and / or software modules, and implement the above various functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the above various functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraint conditions of the technical solution.

[0121] Exemplarily, when implemented in hardware, the hardware implementation of the e-ink screen device can refer to Figure 2 and its related description.

[0122] As Figure 9 shown, the e-ink screen device 900 may include: an e-ink screen 901, a memory 902, one or more processors 903, and one or more computer programs (not shown in the figure). The above components can be coupled through one or more communication buses 904.

[0123] Among them, the e-ink screen 901 is used to display relevant user interfaces such as application interfaces.

[0124] The memory 902 stores one or more computer programs (codes), and one or more computer programs include computer instructions; one or more processors 903 call the computer instructions stored in the memory 902, so that the e-ink screen device 900 executes the image processing method provided by the embodiments of the present application.

[0125] In a specific implementation, the memory 902 may include a high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 902 may store an operating system (hereinafter referred to as the system), such as an embedded operating system like ANDROID, IOS, WINDOWS, or LINUX. The memory 902 may be used to store the implementation program of the embodiments of the present application. The memory 902 may also store a network communication program, which may be used to communicate with one or more additional devices, one or more user devices, and one or more network devices.

[0126] One or more processors 903 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the solution of the present application.

[0127] It should be noted that Figure 9 This is merely one implementation manner of the e-ink screen device 900 provided by the embodiments of the present application. In practical applications, the e-ink screen device 900 may further include more or fewer components, which are not limited herein.

[0128] The embodiments of the present application further provide a computer storage medium, in which computer instructions are stored. When the computer instructions run on an electronic device, the electronic device is caused to execute the above-related method steps to implement the method in the above embodiments.

[0129] The embodiments of the present application further provide a computer program product. When the computer program product runs on a computer, the computer is caused to execute the above-related steps to implement the method in the above embodiments.

[0130] In addition, the embodiments of the present application further provide a device, which may specifically be a chip, a component, or a module. The device may include a processor and a memory connected thereto; wherein, the memory is used to store computer execution instructions. When the device runs, the processor may execute the computer execution instructions stored in the memory, so that the chip executes the image processing method in the above method embodiments.

[0131] Among them, the electronic device, computer storage medium, computer program product, or chip provided by the embodiments of the present application are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved may refer to the beneficial effects in the corresponding method provided above, and will not be elaborated herein.

[0132] From the description of the above embodiments, those skilled in the art can understand that for the convenience and brevity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0133] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the device or unit can be in electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated. The components displayed as units may be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks or optical discs and other various media that can store program codes.

[0137] The above content is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device with an e-ink screen, including: Obtain a first image to be displayed and a second image currently being displayed; the first image is the next frame image of the second image; Determine a first display result corresponding to the first image according to the first image, the second image, and the historical display gradation information of the e-ink screen, where the historical display gradation information is used to characterize the migration information of the historical display gradations of the pixel points of the e-ink screen; Determine a first area where color gradation deviation occurs based on the first display result; Determine the target gradation of each pixel point in the first image corresponding to the first area according to the historical display gradation information of the e-ink screen; Adjust the current gradation of each pixel point in the first image corresponding to the first area to the target gradation of the corresponding pixel point; Display a third image, where the third image is the display result corresponding to the first image after color gradation adjustment is sent for display.

2. The method according to claim 1, characterized in that, The determining the first area where color gradation deviation occurs based on the first display result includes: If the gradation of each pixel point in the first area in the first display result is different from the gradation of each pixel point in the first image corresponding to the first area, determine the first area where color gradation deviation occurs.

3. The method according to claim 1 or 2, characterized in that, The first display result further includes a second area, the second area and the first area belong to the same semantic area, and no color gradation deviation occurs in the second area. The method further includes: Adjust the gradation of each pixel point in the first image corresponding to the second area to the target gradation of the corresponding pixel point according to the historical display gradation information corresponding to each pixel point in the second area.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Send multiple preset images to the e-ink screen; For each pixel point in the e-ink screen, execute: Obtain multiple gradations corresponding to the display of each pixel point when the e-ink screen displays the multiple preset images; Cluster the multiple gradations to obtain at least one key color class; Construct the migration rule between the at least one key color class according to the multiple gradations displayed by each pixel point in the e-ink screen to obtain the historical display gradation information of the e-ink screen.

5. The method according to claim 4, characterized in that, The constructing the migration rule between the at least one key color class according to the multiple gradations displayed by each pixel point in the e-ink screen to obtain the historical display gradation information corresponding to each pixel point includes: According to the multiple gradations displayed by each pixel point in the e-ink screen, use an automaton to construct the migration rule between the at least one key color class to obtain a state transition automaton corresponding to each pixel point. The state transition automaton corresponding to each pixel point is used to characterize the migration rule between at least one key color class historically displayed by the corresponding pixel point.

6. The method according to claim 5, characterized in that, The determining the first display result corresponding to the first image according to the first image, the second image, and the historical display gradation information of the e-ink screen includes: For a first pixel point in the first image, the first pixel point is any pixel point in the e-ink screen, execute: Determine the color level of the first pixel point on the ink screen when displaying the first image based on the color level of the first pixel point in the first image, the color level corresponding to the first pixel point in the second image, and the state transition automaton corresponding to the first pixel point, to obtain the first display result.

7. An electronic device, characterized in that, Comprising a processor, a memory, and an ink screen; The ink screen is used for displaying a user interface; The memory is used for storing one or more computer programs; when the computer programs are executed by the processor, the electronic device is caused to execute: Obtain a first image to be displayed and a second image currently being displayed; the first image is the next frame image of the second image; Determine a first display result corresponding to the first image according to the first image, the second image, and the historical display color level information of the ink screen, where the historical display color level information is used to characterize the migration information of the historical display color levels of the pixel points on the ink screen; Based on the first display result, determine a first area where color level deviation occurs; According to the historical display color level information of the ink screen, determine the target color level of each pixel point corresponding to the first area in the first image; Adjust the current color level of each pixel point corresponding to the first area in the first image to the target color level of the corresponding pixel point; Display a third image, where the third image is the display result corresponding to the first image after color level adjustment is sent for display.

8. The electronic device according to claim 7, characterized in that, When the computer programs are executed by the processor, the electronic device is specifically caused to execute: If the color level of each pixel point in the first area in the first display result is different from the color level of each pixel point corresponding to the first area in the first image, determine that the first display result includes a first area where color level deviation occurs.

9. The electronic device according to claim 7 or 8, characterized in that, The first display result further includes a second area, the second area and the first area belong to the same semantic area, and no color level deviation occurs in the second area. When the computer programs are executed by the processor, the electronic device is further caused to execute: According to the historical display color level information corresponding to each pixel point in the second area, adjust the color level of each pixel point corresponding to the second area in the first image to the target color level of the corresponding pixel point.

10. The electronic device according to any one of claims 7 to 9, characterized in that, When the computer programs are executed by the processor, the electronic device is further caused to execute: Send multiple preset images to the ink screen; For each pixel point on the ink screen, execute: Obtain multiple color levels corresponding to the display of each pixel point when the ink screen displays the multiple preset images; Cluster the multiple color levels to obtain at least one key color class; According to the multiple color levels displayed by each pixel point on the ink screen, construct the migration rule between the at least one key color class to obtain the historical display color level information of the ink screen.

11. The electronic device according to claim 10, characterized in that, The constructing the migration rule between the at least one key color class to obtain the historical display color level information corresponding to each pixel point includes: According to the multiple color levels displayed by each pixel in the ink screen, an automatic mechanism is used to construct the migration rules between the at least one key color class, and a state transition automaton corresponding to each pixel is obtained. The state transition automaton corresponding to each pixel is used to characterize the migration rules between at least one key color class displayed by the corresponding pixel in history.

12. The electronic device according to claim 11, characterized in that, When the computer program is executed by the processor, the electronic device is specifically caused to execute: For the first pixel in the first image, where the first pixel is any pixel in the ink screen, execute: determine the color level of the first pixel when the ink screen displays the first image according to the color level of the first pixel in the first image, the color level of the corresponding first pixel in the second image, and the state transition automaton corresponding to the first pixel, so as to obtain the first display result.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program runs on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 6.

Images