Image processing method, apparatus and electronic device

Abstract

The application discloses an image processing method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a content change area of a display area; in the case that the content change area is a local area, controlling a partition refresh function to be in an open state, acquiring a first local image to be refreshed and displayed in the local area, and a cached first global image, the first global image being a global image acquired in the case that the content change area is a global area last time; in the case that an image algorithm is open, generating a second global image according to the first local image and the first global image, the image algorithm being used for detecting feature information of the display area, the second global image comprising the first local image and a second local image, the second local image being all images except a third local image corresponding to the local area in the first global image; and detecting the second global image by using the image algorithm to obtain a detection result of the display area.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processing method, apparatus, and electronic device. Background Technology

[0002] In the use of electronic devices, when only a local area changes, partitioned refresh technology can be used to update the content of that local area in order to reduce the power consumption of the electronic device. Partitioned refresh technology can transmit, process, and update only the data in the changed area.

[0003] Image algorithms based on full-screen detection rely on full-screen image data for operation. However, when a partition refresh is triggered, the system only processes a portion of the image data, creating a conflict between the operation of the image algorithm and the partition refresh.

[0004] Currently, in order to ensure the normal operation of the image algorithm, the partition refresh function is disabled when the image algorithm is enabled, resulting in a limited scope of application for partition refresh. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method, apparatus, and electronic device that can solve the problem that existing methods disable the partition refresh function to ensure the operation of image algorithms, resulting in a limited scope of application for partition refresh.

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0007] Get the area where the content of the display area changes;

[0008] When the content change area is a local area, the control partition refresh function is enabled, and the first local image to be refreshed and displayed in the local area and the cached first global image are obtained, wherein the first global image is the global image obtained when the content change area was the last global area.

[0009] When the image algorithm is enabled, a second global image is generated based on the first local image and the first global image. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the images in the first global image except for the third local image corresponding to the local area.

[0010] The image algorithm is used to detect the second global image to obtain the detection result of the display area.

[0011] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0012] The acquisition module is used to acquire the content change areas of the display area;

[0013] The first operation module is used to control the partition refresh function to be enabled when the content change area is a local area, and to obtain the first local image to be refreshed and displayed in the local area, as well as the cached first global image, wherein the first global image is the global image obtained when the content change area was the last global area.

[0014] The generation module is used to generate a second global image based on the first local image and the cached first global image, wherein the image algorithm is used to detect feature information of the display area, and the second global image includes the first local image and the second local image, wherein the second local image is all the other images in the first global image except for the third local image corresponding to the local area;

[0015] The first processing module is used to detect the second global image using the image algorithm to obtain the detection result of the display area.

[0016] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0017] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0018] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0019] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0020] In this embodiment, when the image algorithm is enabled, the content change area of ​​the display area can be obtained. If the content change area is a local area, the partition refresh function can be enabled, and a first local image to be refreshed and displayed in that local area, along with a cached first global image, can be obtained. Then, a second global image can be generated using the first local image and the cached first global image. The first global image is the global image displayed when the content change area was last a global area. The second global image includes the first local image and the second local image, which are all images in the first global image except for the third local image displayed in the content change area. Then, the image algorithm is used to detect the second global image to obtain the detection result of the display area. Therefore, through this embodiment, a second global image matching the actual global image displayed in the display area can be generated using the first local image to be refreshed and the cached first global image. Thus, even if the partition refresh function is enabled simultaneously with the image algorithm, detecting the second local image using the image algorithm ensures the normal operation of the image algorithm. This achieves the coexistence of the image algorithm and the partition refresh function, expanding the applicability of partition refresh. Attached Figure Description

[0021] Figure 1 This is one of the flowcharts of the image processing method provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the content change area provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram illustrating the switching of the displayed image in the display area provided in an embodiment of this application;

[0024] Figure 4 This is a structural diagram of the AP provided in the embodiments of this application;

[0025] Figure 5 This is a schematic diagram of the display area segmentation provided in an embodiment of this application;

[0026] Figure 6 This is the second flowchart of the image processing method provided in the embodiments of this application;

[0027] Figure 7 This is a structural diagram of the image processing apparatus provided in the embodiments of this application;

[0028] Figure 8 This is one of the structural diagrams of the electronic device provided in the embodiments of this application;

[0029] Figure 9This is the second structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0031] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0032] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0033] Figure 1 This is one of the flowcharts of an image processing method provided in the embodiments of this application. The image processing method may include:

[0034] Step 101: Obtain the content change area of ​​the display area.

[0035] In practice, the area where the content of the display area changes can be determined by comparing the currently displayed image with the next frame image to be displayed. This area of ​​content change can be represented as a local region or a global region.

[0036] Specifically, when only a portion of the content in the display area changes—that is, when the next frame image to be displayed changes only partially relative to the currently displayed image—the area of ​​content change in the display area is called the local area. When all content in the display area changes—that is, when the next frame image to be displayed changes completely relative to the currently displayed image—the area of ​​content change in the display area is called the global area.

[0037] In some embodiments, the electronic device may determine whether the content change area of ​​the display area is a local area or a global area based on the display interface of the display area and / or the input received.

[0038] Example 1: Assume the display area is the stopwatch interface of an alarm clock application. In this interface, such as... Figure 2 As shown, only the content of the dashed area 21 changes. Therefore, it can be determined that the area where the content of the displayed area changes in this scenario is a local area.

[0039] Example 2: Suppose the current display area is an application interface. When the electronic device receives a touch input for the home button, it can determine that the display area will switch from the application interface to the desktop. The content displayed in the display area will change. Therefore, it can be determined that the area where the content of the display area changes in this scenario is the global area.

[0040] The partition refresh function is used to update data in a localized area. It can be activated when the content change area of ​​the display area is localized. In other words, the activation condition for the partition refresh function includes: the content change area of ​​the display area is localized. Therefore, to determine the status of the partition refresh function, the electronic device can acquire the content change area of ​​the display area at a preset frequency and determine the subsequent execution steps based on the specific manifestation of the content change area. In some embodiments, the preset frequency may be equal to the refresh frequency of the display area. It is understood that the content change areas acquired in different iterations may be the same or different.

[0041] Step 102: When the content change area is a local area, the control partition refresh function is enabled, and the first local image to be refreshed and displayed in the local area and the cached first global image are obtained, wherein the first global image is the global image obtained when the content change area was the last global area.

[0042] In step 102, the content change area of ​​the display area is a local area, which means that only a local area of ​​the display area changes. In this case, the partition refresh function can be turned on so that the display area only refreshes the content of the local area that has changed, thereby reducing the power consumption of the electronic device.

[0043] It should be noted that when the partition refresh function is enabled, controlling the partition refresh function to remain enabled will result in the partition refresh function being kept running. Conversely, when the partition refresh function is disabled, controlling the partition refresh function to remain enabled will result in the partition refresh function being enabled.

[0044] After the control partition refresh function is enabled, a partial image of the local area where the content has changed and is to be refreshed can be obtained, namely the first partial image.

[0045] In addition, in this embodiment, historically acquired global images can be further cached. These global images can be generated from local and global images when the partition refresh function is enabled, or they can be directly drawn and synthesized when the partition refresh function is disabled. The specific settings can be determined according to actual needs, and this embodiment does not limit them.

[0046] It should be noted that, considering the purpose of caching the global image is to generate a global image for image algorithm detection, the caching of the global image can occur when the image algorithm is enabled. That is, if the image algorithm is not enabled, caching the global image can be discarded, thus reducing the operational burden. Furthermore, this application embodiment does not limit the timing of enabling the image algorithm. Enabling the image algorithm can occur before, after, or during the step of determining whether to enable the partition refresh function, depending on the actual situation. This application embodiment does not impose any limitations on this.

[0047] The first global image is the global image obtained when the last time the content change area of ​​the display area was a global area, which is equivalent to the global image displayed in the display area when the last time the content change area of ​​the display area was a global area. In some embodiments, the first global image may be: the last global image obtained when the last time the content change area was a global area, which is equivalent to the last global image displayed in the display area. In other embodiments, the first global image may be: the first global image obtained when the last time the content change area was a global area, which is equivalent to the first global image displayed in the display area when the last time the content change area was a global area.

[0048] For easier understanding, please refer to Figure 3 .exist Figure 3 In this example, the images displayed in the display area are arranged in the following order: F0 → F1 → F2 → F3. Specifically, F0 undergoes a global change relative to the previous frame, with the changed content area being the global region; F1 changes only the dashed line area relative to F0, with the changed content area being the local region; F2 changes only the dashed line area relative to F1, with the changed content area being the local region; and F3 changes globally relative to F2, with the changed content area being the global region. If the electronic device obtains that the changed content area of ​​the display area is a local region when the display area displays F1, then the first local image is the local image displayed in the dashed line area of ​​F2, and the first global image can be either F0 or F1.

[0049] After obtaining the first local image and the first global image, step 103 can be executed.

[0050] Step 103: With the image algorithm enabled, a second global image is generated based on the first local image and the first global image. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the other images in the first global image except for the third local image corresponding to the local area.

[0051] The image algorithm of this application embodiment is used to detect feature information of the display area and obtain the detection result of the display area. It is worth noting that the image algorithm of this application embodiment is a full-screen detection-based image algorithm. That is to say, the image algorithm in principle needs to calculate the image data of the entire display area. Therefore, the normal operation of the image algorithm depends on the global image of the display area.

[0052] In some embodiments, the image algorithm may include, but is not limited to, at least one of the following: a grayscale detection algorithm, a blue light detection algorithm, and a burn-in detection algorithm. For the grayscale detection algorithm, its feature information can be represented as grayscale values; for the blue light detection algorithm, its feature information can be represented as blue light values; for the burn-in detection algorithm, its feature information can be represented as the number of dead pixels.

[0053] like Figure 4 As shown, the image algorithm can be located in the backend "display" stage. Figure 4 In this context, the Application Processor (AP) can include a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), Dynamic Random Access Memory (DRAM), and a Data Processing Unit (DPU). The CPU is used for image rendering, implementing the image rendering stage (also known as the image drawing stage); the GPU is used for image compositing, implementing the image compositing stage (also known as the image compositing stage); DRAM includes a framebuffer, which can be used to cache images; the DPU can integrate image algorithms and a Display Serial Interface (DSI), enabling it to detect images using image algorithms, update the detection results in the display area, and also transmit images to the display area via the DSI, implementing the image display stage (also known as the image display stage).

[0054] It should be noted that, in Figure 4In this embodiment, the image rendering stage is implemented by the CPU, the image synthesis stage is implemented by the GPU, and the image display stage is implemented by the DPU. However, it is understood that the embodiments of this application do not limit the execution devices of each image processing stage. In other embodiments, other devices can also be used to implement each image processing stage.

[0055] It is worth noting that the objects of the image rendering and image compositing stages are related to the state of the partition refresh function. Specifically, when the partition refresh function is enabled, the display area only refreshes the content of the areas where the content has changed. The image rendering and image compositing stages only render and composite the local images of the areas where the content has changed, which are to be refreshed and displayed. In other words, the objects of the image rendering and image compositing stages are the local images of the local areas of the display area where the content has changed, which are to be refreshed and displayed. When the partition refresh function is disabled, the display area refreshes and displays the global content. The image rendering and image compositing stages render and composite the global image of the display area, which is to be refreshed and displayed. In other words, the objects of the image rendering and image compositing stages are the global image of the entire display area, which is to be refreshed and displayed.

[0056] In related technologies, image algorithms detect images synthesized during the image synthesis stage. However, considering that when the partition refresh function is enabled, the image synthesized during the image synthesis stage is a local image of the display area, while the detection by the image algorithm depends on the global image of the display area, the accuracy of the detection results will be low when the partition refresh function is enabled. Therefore, this embodiment of the application can generate a second global image from a first local image and a first global image obtained when the partition refresh function is enabled. The difference between the second global image and the first global image is that the content of the local areas where the content changes changes, while other areas remain the same.

[0057] Understandably, when the partition refresh function is enabled, the AP only sends the content that has changed, and the display area only refreshes the content of the area where the content has changed, while other content remains unchanged from the previous frame image. Therefore, the second global image is the same as the image to be refreshed and displayed in the display area.

[0058] Thus, by executing step 104, the normal operation of the image algorithm can be guaranteed, thereby realizing the coexistence of the image algorithm and partition refresh and expanding the applicable scope of partition refresh.

[0059] In practice, the second global image can be generated by the CPU or GPU, then stored in DRAM, and transferred from DRAM to the DPU's image algorithm to execute step 104.

[0060] The third local image corresponding to the local region in the first global image is the local image that will be displayed in the local region when the first global image is displayed in the display area.

[0061] Step 104: Use the image algorithm to detect the second global image to obtain the detection result of the display area.

[0062] As can be seen from the foregoing, the second global image is the same as the image to be refreshed in the display area. Based on this, "the detection result of the display area obtained by using the image algorithm to detect the second global image" is the same as "the detection result of the display area obtained by using the image algorithm to detect the global image obtained after disabling the partition refresh function". This ensures the normal operation of the image algorithm, thereby realizing the coexistence of the image algorithm and partition refresh, and expanding the applicable scope of partition refresh.

[0063] For grayscale detection algorithms, the detection result of the display area can be represented as the grayscale value of the display area; for blue light detection algorithms, the detection result of the display area can be represented as the blue light value of the display area; for anti-burn-in detection algorithms, the detection result of the display area can be represented as the number of dead pixels in the display area.

[0064] The image processing method of this application embodiment, when the image algorithm is enabled, can obtain the content change area of ​​the display area, and when the content change area is a local area, can enable the partition refresh function and obtain a first local image to be refreshed and displayed in the local area; then, a second global image can be generated using the first local image and the cached first global image, wherein the first global image is the global image displayed in the display area when the content change area was last a global area; the second global image includes the first local image and the second local image, and the second local image is all the images in the first global image except for the third local image displayed in the content change area; then, the image algorithm is used to detect the second global image to obtain the detection result of the display area. It can be seen that, through this application embodiment, a second global image that matches the actual global image displayed in the display area can be generated using the first local image to be refreshed and the cached first global image of the local area of ​​the display area. Thus, even if the partition refresh function is enabled at the same time as the image algorithm, the normal operation of the image algorithm can be guaranteed by detecting the second local image through the image algorithm. In this way, the coexistence of the image algorithm and the partition refresh function is achieved, which can expand the applicability of partition refresh.

[0065] This application does not limit the method of generating the second global image. In some embodiments, generating the second global image based on the first local image and the cached first global image includes:

[0066] Delete the third local image from the first global image to obtain the intermediate image;

[0067] The intermediate image and the first local image are combined to obtain the second global image.

[0068] For easier understanding, please refer to Figure 5 .exist Figure 5 In the image, the display area is divided into regions A and B, with region A being the area where content changes. Assume the first global image is F0 and the second global image is F1.

[0069] To generate F1, the local image corresponding to region A in F0 can be deleted to obtain an intermediate image. Then, the first local image is added to region A of the intermediate image to obtain F1.

[0070] By using the above method, the generated second global image can be matched with the next frame image to be refreshed and displayed in the display area, thereby ensuring the reliability of the image algorithm and realizing the coexistence of the image algorithm and the partition refresh function.

[0071] In other embodiments, the first local region can be directly overlaid onto the content change area of ​​the first global image to obtain an intermediate image; or, all other images except the third local image can be extracted from the first global image and then stitched together with the first local image to obtain an intermediate image. The specific settings can be determined according to actual needs, and this application embodiment does not limit this.

[0072] In some embodiments, obtaining the first local image to be refreshed and displayed in the local area, and the cached first global image, includes:

[0073] In the image rendering stage and the image compositing stage, the first local image of the local area to be refreshed and displayed is rendered and composed respectively;

[0074] The first global image is cached during the image rendering phase;

[0075] The step of generating a second global image based on the first local image and the first global image includes:

[0076] After the image synthesis stage, a second global image is generated based on the first local image and the first global image.

[0077] As can be seen from the foregoing, when the partition refresh function is enabled, the image drawing stage and the image compositing stage only process the local areas of the image whose content has changed and which need to be refreshed and displayed. Specifically, the first local image is drawn in the image drawing stage and the first local image is composited in the image compositing stage.

[0078] Additionally, the first global image can be cached during the image rendering stage. After the image synthesis stage and before running the image algorithm, the second global image can be generated using the first local image and the first global image.

[0079] For easier understanding, please continue to refer to Figure 4 With the partition refresh function enabled, Figure 4 The CPU and GPU each draw and synthesize a local image of a local area to be refreshed and displayed, namely the first local image. In addition, the first local image can be cached in DRAM for use in the synthesis of the global image.

[0080] In this way, on the one hand, the image rendering stage and the image synthesis stage can process only local images, which can reduce image processing data and reduce the image processing burden; on the other hand, after the image synthesis stage, the local images and global images are used to generate a global image to be refreshed and displayed in the display area for the image algorithm to detect, thereby ensuring the detection reliability of the image algorithm.

[0081] In some embodiments, after obtaining the content change area of ​​the display area, the method may further include:

[0082] When the content change area is a global area, the partition refresh function is turned off, and the third global image to be refreshed and displayed in the display area is drawn and synthesized in the image drawing stage and the image compositing stage, respectively.

[0083] The image algorithm is used to detect the third global image to obtain the detection result of the display area.

[0084] In this embodiment, the content change area is the global area, meaning that all content in the display area will change. The partition refresh function can be turned off to refresh all content in the display area and ensure the accuracy of content display.

[0085] It is worth noting that when the partition refresh function is enabled, the control for the partition refresh function is in a disabled state, specifically: the partition refresh function is turned off. When the partition refresh function is not enabled, the control for the partition refresh function is in a disabled state, specifically: the partition refresh function remains disabled even when enabled.

[0086] With the partition refresh function disabled, the electronic device can directly acquire the global image to be refreshed in the display area, i.e., the third global image. Specifically, the third global image can be drawn and composited in the image drawing stage and the image compositing stage, respectively. Figure 4In this process, the CPU and GPU respectively draw and composite the global image to be refreshed and displayed in the display area, i.e., the third global image. Afterwards, the third global image can be transferred to the DPU.

[0087] After acquiring the third global image, the DPU can, on the one hand, transmit the third global image to the display area for display. On the other hand, it can use image algorithms to detect the third global image, obtain the detection results for the display area, and update the detection results for the display area.

[0088] By using the above method, if the content change area of ​​the display area is a local area when the image algorithm is enabled, the partition refresh function can be enabled; if the content change area of ​​the display area is a global area, the partition refresh function can be disabled. In this way, the coexistence of the image algorithm and partition refresh can be maximized while ensuring the normal operation of the image algorithm, thus improving the applicability of partition refresh.

[0089] In some embodiments, after generating the second global image based on the first local image and the cached first global image, the method may further include at least one of the following:

[0090] If the area where the content changes is still a local area, cache the second global image;

[0091] If the area where the content changes is a global area, the second global image is discarded from caching.

[0092] After step 102, the AP sends a first partial image to the display area so that the display area refreshes and displays the first partial image in the area where the content changes, while other content remains unchanged. It can be understood that the global image displayed in the display area is the same as the second global image.

[0093] After refreshing the displayed image in the display area, the electronic device can return to and re-execute step 101.

[0094] When the content change area is still a local area, a second global image can be cached to generate the same global image as the next frame image to be refreshed and displayed in the display area, thereby enabling the detection of the image algorithm. In this way, the partition refresh function can continue to be enabled, extending the coexistence time of the image algorithm and partition refresh.

[0095] When the content change area is a global area, the partition refresh function is switched to the off state. At this time, the image detected by the image algorithm and the next frame image to be refreshed and displayed in the display area are directly drawn and synthesized by the CPU and GPU. Therefore, the second global image can be abandoned to improve the storage utilization of DRAM.

[0096] In some embodiments, the second global image may be abandoned during the image rendering stage, but this does not limit the timing of abandoning the second global image.

[0097] It should be noted that the various optional embodiments described in this application can be combined with each other or implemented individually without conflict, and this application does not limit the implementation of these embodiments.

[0098] To facilitate understanding of the image processing method provided in the above embodiments, the following describes the image processing method using a specific scenario embodiment.

[0099] When a user is using their phone, taking the stopwatch interface of an alarm clock app as an example, if only part of the interface changes, such as... Figure 2 As shown in the dashed area, only the image data in this area needs to be updated.

[0100] In mobile phone systems, detection-type image algorithms (common algorithms include grayscale detection, blue light detection, and screen burn-in prevention) in principle require calculating full-screen image data. Image algorithms reside in the backend "display delivery" stage, such as... Figure 4 .

[0101] When only a portion of the content on the phone screen (i.e. the display area) changes, the partition refresh function is enabled. The AP only processes the changed parts during rendering and compositing, causing the image algorithm to only receive the image of the changed area. Since the image algorithm does not receive the full-screen data, the calculation results will be inaccurate. Therefore, the image algorithm and the partition refresh function are mutually exclusive. In order to ensure that the image algorithm works properly, the traditional method is to disable the partition refresh function.

[0102] This application enables the coexistence of the two. For example... Figure 3 As shown, during the transition from F0 to F1, only the dashed line area changes when drawing the F1 image, triggering a partial refresh function, also known as a partition refresh function. This caches the full-screen F0 image in DRAM, and only processes the dashed line area during the drawing and compositing stages, while simultaneously updating the image of the dashed line area in DRAM. The goal of calculating the full-screen F1 image can be achieved by subtracting the dashed line area of ​​F0 from the cached full-screen F0 image in DRAM and adding the dashed line area of ​​F1. Compared to directly calculating the full-screen F1 image, this method requires less data. Furthermore, the image algorithm can still function normally when the partition refresh is triggered, achieving coexistence of both methods.

[0103] The processing of F1→F2 is the same as that of F0→F1.

[0104] When the F2→F3 transition occurs and the entire screen changes, the rendering phase detects that a full-screen change has occurred. In this case, the cached F2 full-screen image can be discarded, and the normal processing flow can be restored.

[0105] For easier understanding, please refer to Figure 6 , Figure 6 This is a schematic flowchart of a scenario embodiment of the image processing method provided in this application. Figure 6 As shown, the image processing method may include the following steps:

[0106] Step 601: Image algorithm activated.

[0107] Step 602: Determine whether the partition refresh function is enabled.

[0108] If yes, proceed to step 603; otherwise, proceed to step 607.

[0109] Step 603: Cache the first global image into DRAM.

[0110] Step 604: Overlay the first local image to be refreshed and displayed in the local area onto the corresponding area of ​​the first global image to obtain the second global image.

[0111] Step 605: Detect the second global image using an image algorithm.

[0112] Step 606: Update the detection results of the display area.

[0113] Step 607: Use image algorithms to detect the third global image.

[0114] In this way, on the one hand, image algorithms and partition refresh can coexist, expanding the scope of application of partition refresh; on the other hand, when partition refresh exists, only a local image can be drawn, reducing the amount of data and power consumption.

[0115] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0116] like Figure 7 As shown, the image processing apparatus may include:

[0117] The acquisition module 701 is used to acquire the content change area of ​​the display area;

[0118] The first operation module 702 is used to control the partition refresh function to be enabled when the content change area is a local area, and to obtain the first local image to be refreshed and displayed in the local area, and the cached first global image, wherein the first global image is the global image obtained when the content change area was the last global area.

[0119] The generation module 703 is used to generate a second global image based on the first local image and the first global image when the image algorithm is enabled. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the images in the first global image except for the third local image corresponding to the local area.

[0120] The first processing module 704 is used to detect the second global image using the image algorithm to obtain the detection result of the display area.

[0121] In some embodiments, the generation module includes:

[0122] The deletion unit is used to delete the third local image in the first global image to obtain an intermediate image;

[0123] A synthesis unit is used to synthesize the intermediate image and the first local image to obtain a second global image.

[0124] In some embodiments, the first operation module includes:

[0125] The first operation unit is used to draw and synthesize the first local image to be refreshed and displayed in the local area during the image drawing stage and the image synthesis stage, respectively.

[0126] A caching unit is used to cache the first global image during the image rendering phase;

[0127] The generation module is specifically used for:

[0128] After the image synthesis stage, a second global image is generated based on the first local image and the first global image.

[0129] In some embodiments, the image processing apparatus further includes at least one of the following:

[0130] The first caching module is used to cache the second global image when the content change area is still a local area;

[0131] The second caching module is used to discard caching the second global image when the content change area is a global area.

[0132] In some embodiments, the apparatus further includes:

[0133] The second operation module is used to control the partition refresh function to be turned off when the content change area is a global area, and to draw and synthesize the third global image to be refreshed and displayed in the display area during the image drawing stage and the image synthesis stage, respectively.

[0134] The second processing module is used to detect the third global image using the image algorithm to obtain the detection result of the display area.

[0135] In some embodiments, the first global image is: the last global image obtained when the content change area was last a global area; or, the first global image obtained when the content change area was last a global area.

[0136] The image algorithm includes at least one of the following: grayscale detection algorithm, blue light detection algorithm, and anti-burn-in detection algorithm.

[0137] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0138] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0139] The image processing apparatus provided in this application embodiment can implement the various processes implemented in the method embodiment, and will not be described again here to avoid repetition.

[0140] Optionally, such as Figure 8As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0141] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0142] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0143] The electronic device 900 includes, but is not limited to, components such as: radio frequency unit 901, network module 902, audio output unit 903, input unit 904, sensor 905, display unit 906, user input unit 907, interface unit 908, memory 909, and processor 910.

[0144] Those skilled in the art will understand that the electronic device 900 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 910 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0145] The processor 910 is used for:

[0146] Get the area where the content of the display area changes;

[0147] When the content change area is a local area, the control partition refresh function is enabled, and the first local image to be refreshed and displayed in the local area and the cached first global image are obtained, wherein the first global image is the global image obtained when the content change area was the last global area.

[0148] When the image algorithm is enabled, a second global image is generated based on the first local image and the first global image. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the images in the first global image except for the third local image corresponding to the local area.

[0149] The image algorithm is used to detect the second global image to obtain the detection result of the display area.

[0150] In some embodiments, the processor 910 is configured to:

[0151] Delete the third local image from the first global image to obtain the intermediate image;

[0152] The intermediate image and the first local image are combined to obtain the second global image.

[0153] In some embodiments, the processor 910 is configured to:

[0154] In the image rendering stage and the image compositing stage, the first local image of the local area to be refreshed and displayed is rendered and composed respectively;

[0155] The first global image is cached during the image rendering phase;

[0156] After the image synthesis stage, a second global image is generated based on the first local image and the first global image.

[0157] In some embodiments, the processor 910 is configured to perform at least one of the following:

[0158] If the area where the content changes is still a local area, cache the second global image;

[0159] If the area where the content changes is a global area, the second global image is discarded from caching.

[0160] In some embodiments, the processor 910 is configured to:

[0161] When the content change area is a global area, the partition refresh function is turned off, and the third global image to be refreshed and displayed in the display area is drawn and synthesized in the image drawing stage and the image compositing stage, respectively.

[0162] The image algorithm is used to detect the third global image to obtain the detection result of the display area.

[0163] In some embodiments, the first global image is: the last global image obtained when the content change area was last a global area; or, the first global image obtained when the content change area was last a global area.

[0164] The image algorithm includes at least one of the following: grayscale detection algorithm, blue light detection algorithm, and anti-burn-in detection algorithm.

[0165] It should be understood that, in this embodiment, the input unit 904 may include a graphics processing unit (GPU) 9041 and a microphone 9042. The GPU 9041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0166] The memory 909 can be used to store software programs and various data. The memory 909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0167] Processor 910 may include one or more processing units; optionally, processor 910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 910.

[0168] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0169] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0170] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0171] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0172] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0173] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0175] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: Get the area where the content of the display area changes; When the content change area is a local area, the control partition refresh function is enabled, and the first local image to be refreshed and displayed in the local area and the cached first global image are obtained, wherein the first global image is the global image obtained when the content change area was the last global area. When the image algorithm is enabled, a second global image is generated based on the first local image and the first global image. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the images in the first global image except for the third local image corresponding to the local area. The image algorithm is used to detect the second global image to obtain the detection result of the display area.

2. The method according to claim 1, characterized in that, The step of generating a second global image based on the first local image and the cached first global image includes: Delete the third local image from the first global image to obtain the intermediate image; The intermediate image and the first local image are combined to obtain the second global image.

3. The method according to claim 1, characterized in that, The step of obtaining the first local image to be refreshed and displayed in the local area, and the cached first global image, includes: In the image rendering stage and the image compositing stage, the first local image of the local area to be refreshed and displayed is rendered and composed respectively; The first global image is cached during the image rendering phase; The step of generating a second global image based on the first local image and the first global image includes: After the image synthesis stage, a second global image is generated based on the first local image and the first global image.

4. The method according to claim 1, characterized in that, After generating the second global image based on the first local image and the cached first global image, the method further includes at least one of the following: If the area where the content changes is still a local area, cache the second global image; If the area where the content changes is a global area, the second global image is discarded from caching.

5. The method according to claim 1, characterized in that, After obtaining the content change area of ​​the display area, the method further includes: When the content change area is a global area, the partition refresh function is turned off, and the third global image to be refreshed and displayed in the display area is drawn and synthesized in the image drawing stage and the image compositing stage, respectively. When the image algorithm is enabled, the third global image is detected using the image algorithm to obtain the detection result of the display area.

6. The method according to any one of claims 1 to 5, characterized in that, The first global image is: the last global image obtained when the content change area was last a global area; or, the first global image obtained when the content change area was last a global area. The image algorithm includes at least one of the following: grayscale detection algorithm, blue light detection algorithm, and anti-burn-in detection algorithm.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the content change areas of the display area; The first operation module is used to control the partition refresh function to be enabled when the content change area is a local area, and to obtain the first local image to be refreshed and displayed in the local area, as well as the cached first global image, wherein the first global image is the global image obtained when the content change area was the last global area. The generation module is used to generate a second global image based on the first local image and the first global image when the image algorithm is enabled. The image algorithm is used to detect feature information of the display area. The second global image includes the first local image and the second local image. The second local image is all the images in the first global image except for the third local image corresponding to the local area. The first processing module is used to detect the second global image using the image algorithm to obtain the detection result of the display area.

8. The apparatus according to claim 7, characterized in that, The generation module includes: The deletion unit is used to delete the third local image in the first global image to obtain an intermediate image; A synthesis unit is used to synthesize the intermediate image and the first local image to obtain a second global image.

9. The apparatus according to claim 7, characterized in that, The first operation module includes: The first operation unit is used to draw and synthesize the first local image to be refreshed and displayed in the local area during the image drawing stage and the image synthesis stage, respectively. A caching unit is used to cache the first global image during the image rendering phase; The generation module is specifically used for: After the image synthesis stage, a second global image is generated based on the first local image and the first global image.

10. The apparatus according to claim 7, characterized in that, It also includes at least one of the following: The first caching module is used to cache the second global image when the content change area is still a local area; The second caching module is used to discard caching the second global image when the content change area is a global area.

11. The apparatus according to claim 7, characterized in that, The device further includes: The second operation module is used to control the partition refresh function to be turned off when the content change area is a global area, and to draw and synthesize the third global image to be refreshed and displayed in the display area during the image drawing stage and the image synthesis stage, respectively. The second processing module is used to detect the third global image using the image algorithm when the image algorithm is enabled, and to obtain the detection result of the display area.

12. The apparatus according to any one of claims 7 to 11, characterized in that, The first global image is: the last global image obtained when the content change area was last a global area; or, the first global image obtained when the content change area was last a global area. The image algorithm includes at least one of the following: grayscale detection algorithm, blue light detection algorithm, and anti-burn-in detection algorithm.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as described in any one of claims 1 to 6.

Images