An image processing method and apparatus

Abstract

The application discloses an image processing method and device, and relates to the technical field of image processing, in particular to an image processing method and device, which are characterized in that the method is suitable for an image processing device with an operating system, and the method comprises the following steps: an image processing module of the operating system receives an instruction in which a first application program calls the image processing module of the operating system, the instruction carries a to-be-displayed image and a resolution of the to-be-displayed image; when the resolution of the to-be-displayed image is lower than a first threshold value, the image processing module performs super-resolution processing on the to-be-displayed image, and displays the image after the super-resolution processing, so as to solve the problem of poor image definition in a traditional display method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and in particular, to an image processing method and device. BACKGROUND

[0002] In recent years, with the rapid development of electronic industry and communication technology, new services based on data, voice and video develop rapidly. The rapid development of microelectronic technology, computer software and hardware technology lays a foundation for image processing devices to handle more and more complex work, and provides a possibility for image processing device individualization, so that the terminal can have more and more powerful functions to some extent and can be free from the restriction of network. In addition, users have an urgent need for terminal, hoping that the terminal has more powerful, more flexible and more simple functions. The development of information technology makes the terminal technology intelligent, mobile and multifunctional.

[0003] With the popularity of mobile terminals, especially the rapid development of smart phones, people's life is more convenient, and people enjoy the achievements brought by high technology. Smart mobile terminals such as smart phones are more and more accepted by people because of their powerful operating systems, large storage spaces, and the convenience of installing various software and other advantages. Compared with traditional mobile terminals, smart mobile terminals can install more third-party applications. The operating system of a smart mobile terminal with an Android system usually includes, from high to low, an application layer, a framework layer, a runtime, a core library, a hardware abstraction layer and a Linux kernel layer. Generally, the core function implementation of the Android system, including the framework layer and the core library, is based on which each Android application developer develops applications. The application layer of the Android system is composed of all applications running on the Android device, which not only includes system applications (pre-installed on the smart mobile terminal with the Android system) such as calls, short messages and contacts, but also includes other third-party applications subsequently installed in the device. Third-party applications are developed based on the software development kit (SDK) provided by Android and are subject to the constraints of the SDK interface. The system applications pre-installed in the device can call the interfaces and modules of the entire framework layer. The existing smart mobile terminal operating system can directly call the control interface from the Android framework layer after the third-party application is installed and authorized by the system, for example, an application program usually calls the Image View control of the framework layer of the Android system to draw an image when displaying an image. However, because the image is displayed to save traffic, many image details are compressed based on bandwidth considerations, resulting in poor image clarity. SUMMARY

[0004] The present application provides an image processing method and device to solve the problem of poor definition of application display image information in existing terminal devices.

[0005] In a first aspect, the embodiments of the present application provide an image processing method, which is applicable to an image processing device with an operating system, and includes: an image processing module of the operating system receiving an instruction of a first application program calling the image processing module of the operating system, the instruction carrying a to-be-displayed image; and the image processing module performing image optimization processing on the to-be-displayed image and displaying the image after the image optimization processing.

[0006] The above method perfects the image processing module of the operating system of the image processing device, increases the function of image optimization, and performs image optimization processing on the to-be-displayed image in different application programs in the application layer. That is, when a multimedia file in an application program calls the interface of the image processing module of the operating system to display an image, the image will first undergo the image optimization processing process of the image processing module, and the finally displayed image is an optimized image, for example, the resolution of the optimized image is improved, and therefore the definition is better.

[0007] In a possible design, the operating system is an Android operating system, and the image processing module is an image view class in a framework layer of the Android operating system. In this way, the image view class can obtain the width and height of the to-be-displayed image from the instruction, and then determine that the width and height of the to-be-displayed image satisfy a set condition, the set condition being that a difference between the height of a display screen of the image processing device and the height of the to-be-displayed image is less than a second threshold value, and / or a difference between the width of the display screen of the image processing device and the width of the to-be-displayed image is less than a third threshold value. That is, the image view class can perform image optimization only on images of a large enough size, which helps to improve the efficiency of image optimization.

[0008] In a possible design, the operating system is an Android operating system, and the image processing module is a bitmap factory class in a framework layer of the Android operating system. In this way, the image processing module decodes the to-be-displayed image before performing image optimization processing on the to-be-displayed image. This method mainly makes up for image optimization in a scenario where an application program in the application layer itself has an image view class. When the application program in the application layer itself has an image view class, the application program no longer calls the ImageView in the framework layer, but still calls the bitmap factory class (BitmapFactory) in the framework layer to decode the to-be-displayed image and obtain a decoded image. Therefore, the image optimization function can be expanded on the BitmapFactory, and thus the image optimization is triggered once the application program in the application layer calls this interface.

[0009] In a possible design, the image processing module determines to perform super-resolution processing on the to-be-displayed image when the resolution of the to-be-displayed image is lower than a first threshold value. The super-resolution processing can adjust the resolution of the original image, and thus the image clarity of the image after image optimization is higher.

[0010] In a possible design, the image processing module can further determine, according to the identifier of the first application in the instruction, that the first application has the permission of super-resolution processing. Apparently, this can improve the efficiency of image optimization and is helpful to centralized processing of multimedia files having image elements.

[0011] Optionally, the image processing module determines whether the identifier of the first application exists in a preset white list; if the identifier exists in the white list, the image processing module determines that the first application has the permission of image optimization, otherwise, the image of the application is not subjected to image optimization, because the white list can be updated by the user at any time, and thus it is more convenient to control.

[0012] Specifically, the method for the image processing module to perform super-resolution processing on the to-be-displayed image can be: the image processing module adds the to-be-displayed image as a task object into a task queue.

[0013] The image processing module determines an image optimization algorithm corresponding to the to-be-displayed image according to the resolution of the to-be-displayed image; and the image processing module performs super-resolution processing on the task object corresponding to the to-be-displayed image in the task queue by using the corresponding image optimization algorithm.

[0014] In addition, the image processing module releases the memory space corresponding to the task object corresponding to the to-be-displayed image after the image processing module determines that the task object completes the super-resolution processing, so as to facilitate memory management.

[0015] In addition, a first task in the task queue is allocated to a first processor to perform super-resolution processing, and a second task in the task queue is allocated to a second processor to perform super-resolution processing, so that heterogeneous acceleration can be implemented.

[0016] In a second aspect, an embodiment of the present application provides an image processing apparatus having a function of implementing the image processing module behavior of the operating system in the method embodiments. The function can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0017] In a possible design, the image processing apparatus includes a receiving unit and a processing unit, where:

[0018] receive a first application program calling an instruction of an image processing module of an operating system, the instruction carrying a to-be-displayed image;

[0019] a processing unit, configured to perform image optimization processing on the to-be-displayed image, and display the image after the image optimization processing.

[0020] In a possible design, the processing unit is specifically configured to: when the resolution of the to-be-displayed image is lower than a first threshold, perform super-resolution processing on the to-be-displayed image.

[0021] In another possible design, the image processing apparatus further includes a determination unit, configured to determine, according to the identifier of the first application program, that the first application program has the permission of super-resolution processing.

[0022] Further, the determination unit is specifically configured to: determine, according to whether the identifier of the first application program exists in a preset white list, whether the first application program has the permission of image optimization.

[0023] In a possible design, the processing unit is specifically configured to:

[0024] add the to-be-displayed image as a task object into a task queue;

[0025] determine, according to the resolution of the to-be-displayed image, an image optimization algorithm corresponding to the to-be-displayed image;

[0026] perform super-resolution processing on the task object corresponding to the to-be-displayed image in the task queue by using the corresponding image optimization algorithm.

[0027] Further, the processing unit is further configured to: after the task object corresponding to the to-be-displayed image completes the super-resolution processing, release the memory space corresponding to the task object.

[0028] In addition, in a possible design, a first task in the task queue is allocated to a first processor to perform super-resolution processing, and a second task in the task queue is allocated to a second processor to perform super-resolution processing, so as to accelerate image processing.

[0029] In a possible design, the image processing apparatus is an image view class of an operating system, the instruction further includes a width and a height of the to-be-displayed image, and the processing unit is further configured to: determine, according to whether the width and the height of the to-be-displayed image satisfy a set condition, that the difference between the height of a display screen of the terminal device and the height of the to-be-displayed image is less than a second threshold, and / or the difference between the width of the display screen of the terminal device and the width of the to-be-displayed image is less than a third threshold.

[0030] In a possible design, the image processing apparatus is a bitmap factory class in a framework layer of the Android operating system, and the image processing apparatus decodes the image to be displayed before performing the image optimization processing on the image to be displayed.

[0031] In a third aspect, an embodiment of the present application provides an image processing device, which includes a memory, a display, and a processor. The processor can be a central processing unit (CPU) or a digital processing unit, etc. The processor executes an image optimization function according to an instruction of a first application program to call an image processing module of an operating system. The memory is used to store an instruction of the first application program, a program instruction of the operating system, and a program executed by the processor.

[0032] The display is used to display an image processed by the processor on a human-computer interaction interface of the first application program.

[0033] Specifically, the processor is configured to perform super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than a first threshold.

[0034] In a possible design, the instruction of the first application program further includes an identifier of the first application program, and the processor is further configured to determine that the first application program has the permission of super-resolution processing according to the identifier of the first application program.

[0035] In a possible design, the processor is specifically configured to determine whether the identifier of the first application program exists in a preset white list, and determine that the first application program has the permission of image optimization if the identifier exists in the white list.

[0036] In a possible design, the processor is specifically configured to add the image to be displayed as a task object into a task queue, and determine an image optimization algorithm corresponding to the image to be displayed according to the resolution of the image to be displayed.

[0037] The processor performs super-resolution processing on the task object corresponding to the image to be displayed in the task queue by using the corresponding image optimization algorithm.

[0038] In a possible design, the operating system is an Android operating system, the image processing module is an image view class or a bitmap factory class in a framework layer of the Android operating system, and the processor can optimize the image in the following two ways.

[0039] In a first mode, the processor determines that the width and height of the image to be displayed satisfy a set condition, and performs super-resolution processing on the image satisfying the condition, wherein the set condition is that a difference between the height of the display screen of the image processing device and the height of the image to be displayed is less than a second threshold value, and / or a difference between the width of the display screen of the image processing device and the width of the image to be displayed is less than a third threshold value.

[0040] In a second mode, the processor decodes the image to be displayed, and then performs super-resolution processing on the image satisfying the condition.

[0041] In a possible design, the processor is further configured to release the memory space corresponding to the task object after the task object corresponding to the image to be displayed completes the super-resolution processing.

[0042] In a possible design, a first task in the task queue is assigned to a first processor to perform super-resolution processing, and a second task in the task queue is assigned to a second processor to perform super-resolution processing, so that heterogeneous acceleration can be implemented.

[0043] In a fourth aspect, the embodiments of the present application further provide a computer storage medium, which stores a software program. The software program, when read and executed by one or more processors, can implement the method provided in any one of the designs of the first aspect.

[0044] In a fifth aspect, the embodiments of the present application further provide a computer program product. When the computer program product is executed by a computer, the computer performs the method provided in any one of the designs of the first aspect.

[0045] In the scheme provided in the embodiments of the present application, the improvement is made on the image processing module in the operating system, and each application program does not need to be developed separately, so the scheme has strong reusability. In addition, the image information of the multimedia file from different application programs is automatically optimized when displayed, and the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 FIG. 1 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0047] Figure 2 FIG. 2 is a structural schematic diagram of a mobile phone provided in an embodiment of the present application;

[0048] Figure 3 FIG. 3 is a method flow schematic diagram of image processing provided in an embodiment of the present application;

[0049] Figure 4 FIG. 4 is a screen judgment method schematic diagram provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of an Android operating system framework with image optimization function is provided for the embodiment of the present application.

[0051] Figure 6 A schematic diagram of an image view class integrated with image optimization function is provided for the embodiment of the present application.

[0052] Figure 7 A schematic diagram of a bitmap factory class integrated with image optimization function is provided for the embodiment of the present application.

[0053] Figure 8 A schematic diagram of an image optimization method flow of an image view class is provided for the embodiment of the present application. Figure 1

[0054] Figure 8a A schematic diagram of an image optimization method flow of an image view class is provided for the embodiment of the present application. Figure 2

[0055] Figure 9 A schematic diagram of an image optimization method flow of a bitmap factory class is provided for the embodiment of the present application.

[0056] Figure 10 A schematic diagram of a management strategy method of an image optimization queue is provided for the embodiment of the present application.

[0057] Figure 11 A schematic diagram of an image processing device structure is provided for the present application.

[0058] Figure 12 Another schematic diagram of an image processing device structure is provided for the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0060] The method for image processing provided by the embodiments of the present application can be applied to an image processing device as shown in the following figure. Figure 1 The image processing device includes an application layer 110 and a framework layer 120, wherein the application layer 110 includes a first application program 111 and other application programs, and the framework layer 120 includes an image processing module 122. The image processing module 122 receives an image transmitted by the first application program 111, and performs image optimization processing on the image. The image processed by the image processing module 122 is transmitted to the first application program 111, so that the first application program 111 displays the processed image. It can be understood that the framework layer 120 further includes a storage module for caching the image to be processed and the processed image. The following image processing process is taken as an example to illustrate the working process of the image processing module.​​

[0061] After the image processing module 122 receives the image to be displayed, the resolution of the image to be displayed is first acquired, and it is determined whether the resolution is lower than a threshold value. If yes, the image to be displayed is subjected to super-resolution processing, and the image after super-resolution processing is displayed. The super-resolution processing refers to a method of improving the resolution of an original image by hardware or software. A process of obtaining a high-resolution image from a series of low-resolution images is super-resolution reconstruction. The image processing module 122 returns the image after super-resolution processing to the first application program 111 for interface display.

[0062] The image processing method provided in the embodiments of the present application is also applicable to a mobile phone as shown in Figure 2 The specific structure of the mobile phone will be first introduced briefly below.

[0063] Referring to Figure 2 FIG. 1 shows a hardware structure diagram of a mobile phone to which the embodiments of the present application are applied. As shown in Figure 2 The mobile phone 200 includes a display device 210, a processor 220, and a memory 230. The memory 230 can be used to store software programs and data, and the processor 220 can execute various function applications and data processing of the mobile phone 200 by running the software programs and data stored in the memory 230. The memory 230 can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as an image acquisition function), and the like; and the data storage area can store data created according to the use of the mobile phone 200 (such as audio data, a phonebook, images, and the like). In addition, the memory 230 can include a high-speed random access memory, and can also include a nonvolatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. The processor 220 is the control center of the mobile phone 200, and connects all parts of the mobile phone through various interfaces and lines, and executes various functions and processes data of the mobile phone 200 by running or executing the software programs and / or data stored in the memory 230, thereby monitoring the mobile phone as a whole. The processor 220 can include one or more general-purpose processors, and can also include one or more DSPs (digital signal processors), and can also include one or more ISPs (image signal processors) for executing related operations to implement the technical solutions provided in the embodiments of the present application.

[0064] The mobile phone 200 also includes a camera 260 for shooting images or videos. The camera 260 can be a normal camera or a focus camera.

[0065] The mobile phone 200 can further include an input device 240 for receiving inputted digital information, character information or contact touch operation / non-contact gesture, and generating signal input related to user settings and function control of the mobile phone 200, etc.

[0066] The display device 210 includes a display panel 211 for displaying information inputted by a user or provided to the user and various menu interfaces of the mobile phone 200, etc., and in the embodiments of the present application, is mainly used for displaying the to-be-detected image acquired by the camera or sensor in the mobile phone 100. Optionally, the display panel 211 can be configured in the form of a liquid crystal display (LCD) or an OLED (organic light-emitting diode), etc.

[0067] In addition to the above, the mobile phone 200 can further include a power supply 250 for supplying power to other modules. The mobile phone 200 can further include one or more sensors 270, such as an image sensor, an infrared sensor, a laser sensor, etc. The mobile phone 200 can further include a radio frequency (RF) circuit 280 for network communication with a wireless network device, and can further include a WiFi module 290 for WiFi communication with other devices, for acquiring images or data transmitted by other devices, etc.

[0068] Based on the above introduction, the embodiments of the present application provide a method and an image processing device for image processing, to solve the problem of unclear image display of multimedia files in an application. The method and the image processing device are based on the same inventive concept, and the implementation of the image processing device and the method can be referred to each other, and the repeated parts will not be described here.

[0069] In the embodiments of the present application, the image processing module of the operating system of the image processing device is improved, and the function of image optimization is added to perform image optimization processing on the to-be-displayed images in different application programs in the application layer. That is, the embodiments of the present application improve the layer (for example, the framework layer) except the application layer in the operating system, add the image optimization program and call the related hardware to realize the optimization processing. When the multimedia file in the application program calls the image processing module of the operating system to display the image, the image will first pass through the image optimization processing process of the image processing module, and the finally displayed image is the optimized image. At this time, the resolution has been improved, and therefore the definition is better. Because the embodiments of the present application improve the image processing module in the operating system, the respective development of the application programs is not needed, and the user who operates the application program in the application layer is not aware of the whole image optimization process. Therefore, the method has strong reusability and is more automatic.

[0070] Hereinafter, some terms in the present application are explained and described to facilitate the understanding of the skilled in the art.

[0071] 1) The application program involved in the embodiments of the present application refers to the software with a visual user interface, which can perform human-computer interaction with the user. For example, the short message application, the multimedia message application, various mailbox applications, microblog, WeChat, Tencent chat software (QQ), Line, photo sharing (instagram), DingDing, headlines news, browser and the like. The user can share the information such as text, voice, image, video file and other various files to other contacts through the application, or obtain the above information.

[0072] 3) The image processing device involved in the embodiments of the present application can also be referred to as user equipment (User Equipment, UE), which is a device that can install various communication applications or has a communication function. For example, a smart phone, a tablet computer, various wearable devices, a vehicle-mounted device, a computer and the like.

[0073] 4) The image involved in the embodiments of the present application is derived from a multimedia file, wherein the multimedia file is an image, an image set or a video file composed of multiple images.

[0074] In combination with Figure 1 The operating system architecture of the image processing device is described in detail in the following embodiments of the present application, and the specific process of the image processing method is described in detail with reference to Figure 3 The specific flow of the method can include:

[0075] Step 201a: The first application program sends an instruction to the image processing module of the operating system to call the image processing module to perform image optimization.

[0076] Specifically, when detecting that a user views an image, the first application sends an instruction to the image processing module. For example, when a user receives an image sent by a friend using WeChat, the WeChat interface displays the received image in a thumbnail. When the user clicks the thumbnail to view the large image, WeChat calls the image processing module.

[0077] In step 202a, when the image processing module determines that the resolution of the image to be displayed is lower than the first threshold, the image processing module performs super-resolution processing on the image to be displayed, and displays the image after super-resolution processing.

[0078] In step 202a, because the instruction carries the image to be displayed and the resolution of the image to be displayed, the image processing module can first obtain the resolution of the image to be displayed, and then determine the resolution. If it is determined that the resolution is lower than the first threshold, the image processing module starts super-resolution processing. Otherwise, the image processing module does not perform super-resolution processing. Here, the image optimization processing is described by taking super-resolution processing as an example.

[0079] It should be noted that, in addition to the super-resolution processing, other image optimization methods can also be used to optimize the image, such as adjusting the brightness, saturation, color of the image, or beautifying the face, etc. These image optimization methods can also be used together, or different image optimization methods can be triggered according to different detection conditions, such as identifying that the current mode is a night reading mode, and then reducing the brightness of the image, or identifying that there is a face in the image, and then performing the image optimization method of face beautification, etc.

[0080] It should be noted that, in addition to the super-resolution processing, other image optimization methods can also be used to optimize the image, such as adjusting the brightness, saturation, color of the image, or beautifying the face, etc. These image optimization methods can also be used together, or different image optimization methods can be triggered according to different detection conditions, such as identifying that the current mode is a night reading mode, and then reducing the brightness of the image, or identifying that there is a face in the image, and then performing the image optimization method of face beautification, etc.

[0081] That is, the image processing module pre-stores a white list with image optimization permission, and the white list contains the identification of the application layer application. If the image processing module determines that the identification of the application in the instruction is in the white list, it is obvious that the image has the image optimization permission, and the super resolution processing can be further triggered, otherwise, the super resolution processing is not performed. Alternatively, the image processing module authenticates the identification of the application. If the authentication is passed, it proves that the application has the image optimization permission, and the super resolution processing can be further triggered, otherwise, the super resolution processing is not performed.

[0082] In another possible design, in order to improve the efficiency of image processing, before performing the super resolution processing, other conditions can be used to further determine whether the image to be displayed needs image optimization. For example, after the image processing module obtains the image, it determines whether the image has been processed by the super resolution processing. If it has been processed, the super resolution processing is not performed on the image.

[0083] When there are more images to be displayed in the application, the image processing module will have great pressure on performance and memory, so at this time, part of the image can be filtered, such as processing the image whose size meets the preset condition. For example, the image processing module determines whether the image size of the image to be displayed is close to the full screen. Only in the case that the image size is close to the screen size, the super resolution processing is performed. As shown in Figure 4 The width of the image to be displayed is consistent with the screen width, and this image to be displayed meets the condition of performing the super resolution processing. Because in actual operation, the image width of part of the application is slightly smaller than the screen width, when the difference between the height of the display screen and the height of the image to be displayed is less than a second threshold, or the difference between the width of the display screen of the image processing device and the width of the image to be displayed is less than a third threshold, any one of the two conditions is met, the super resolution processing is performed on the image to be displayed. The second threshold and the third threshold can be the same. For example, when the width of the image to be displayed meets [0.95, 1.0] times of the screen width, or the height of the image to be displayed meets [0.95, 1.0] times of the height, the super resolution processing can be continued.

[0084] At present, the commonly used operating system of the image processing device is the Android operating system. The embodiment of the application further combines the system architecture of the Android operating system to describe the above image processing method. The Android operating system usually includes, from high layer to low layer, application layer, framework layer, runtime, core library, hardware abstraction layer and Linux kernel layer. The image processing module of the embodiment of the application belongs to the functional module in the framework layer.

[0085] As Figure 5As shown, the Android operating system architecture with image optimization function includes an application layer 301, a framework layer 302, a hardware abstraction layer 303, and a kernel chip 304; wherein, the scene recognition module 305 in the image processing module in the framework layer is provided with various conditions for determining whether image optimization is required, such as determining whether the first application that calls the image processing module is in the whitelist for image optimization, whether the image to be displayed needs to be displayed on the screen, such as whether the width or height of the image to be displayed is close to the width or height of the screen; whether the resolution of the image to be displayed meets the resolution threshold, whether there is a face in the image to be displayed, etc. The image view class (ImageView) 306 and the bitmap factory class (BitmapFactory) 307 are used to perform image optimization on the image to be displayed, and specifically can manage the image optimization task queue and memory call, and perform optimization processing on the image by calling the image optimization algorithm in the HiAI service. The HiAI service platform 308 in the framework layer includes various image optimization algorithms, such as DNN hard algorithm and Raisr soft algorithm. The DNN hard algorithm is a super-resolution image hard algorithm that improves performance and effect by interacting with the neural network processor (IPU); the Raisr-like soft algorithm is a super-resolution image soft algorithm that is mainly aimed at low-end image processing devices without IPU hardware. In addition, the framework layer also includes a storage module 309, which acts on Figure 1 This has been explained in , so I won’t repeat it here.

[0086] Because the applications in the application layer are different, some applications will call the image view class (ImageView) 306 to draw images, but some applications will not call ImageView, but call the bitmap factory class (BitmapFactory) 307, so the image processing module mainly has the following two ways to implement image optimization.

[0087] Method 1: Since most applications in the application layer will call the image view class 306 in the framework layer to display an image, the image optimization function can be expanded on the image view class. In this way, the application in the application layer can call the image view class to trigger the above image processing method. Figure 6 As shown, different applications in the application layer 401 will create ImageView classes respectively when displaying images, such as ImageView1 of the first application, ImageView2 of the second application, and ImageView3 of the third application, which will trigger the image processing module 404 in the framework layer 402 to call the heterogeneous processor 403 to perform image optimization, and the image view class 405 in the image processing module 404 will call the image optimization algorithm to perform image optimization processing. During the image optimization processing, the image processing module will perform heterogeneous acceleration through multiple processors, thereby speeding up the image optimization process.

[0088] Method 2: When the application in the application layer 401 already has an image view class 405, it will no longer call the ImageView in the framework layer 402, but will still call the bitmap factory class (BitmapFactory) 406 in the framework layer to decode the image to be displayed and obtain the decoded image. Therefore, the image optimization function can be extended on the BitmapFactory. In this way, once the application in the application layer calls the bitmap factory class, the above image processing method will be triggered. Figure 7 As shown, in step 501, the first application in the application layer calls the BitmapFactory in the framework layer of the operating system to decode the multimedia file during the image display process; in step 502, the decoded image is judged by a series of conditions of the scene recognition module, such as resolution size recognition, and the image that meets the conditions is added to the image optimization queue; in step 503, considering that the information contained in some images, such as the first few frames or the last few frames is usually not important, the image optimization can be directly abandoned; in step 504, after filtering, the remaining images are super-resolution processed in turn.

[0089] Regarding the first method, specifically, in the Android operating system, the most commonly used image display module is ImageView (image view module). By analyzing the Android ImageView source code, no matter how the ImageView of each application is created, it will all go to initImageView() (initialize the image view interface) in the end to perform general data initialization. Therefore, the embodiment of the present application adds a conditional judgment process related to the image optimization permission in this interface to determine whether the current application has the permission for image optimization. The following is an example of super-resolution processing. Figure 8 As shown:

[0090] Step 601: The application creates an ImageView and calls the initImageView() interface to initialize data.

[0091] Step 602: In initImageView(), multiple conditions are set to determine whether the current application has permission for image optimization. The conditions are as follows:

[0092] (1) Whether the preset image optimization capability switch is turned on; (2) Whether the mobile phone model supports image optimization; (3) Whether the application package name is in the whitelist.

[0093] If the three conditions are met, it can be determined that the ImageView created by the current application has the image optimization permission, and the subsequent judgment logic can be continued, i.e., step 603 is executed; otherwise, the image optimization processing is not continued, i.e., step 605 is executed to draw and display the image.

[0094] In the image optimization processing, the image quality is improved, which has different influences and meanings on users in different applications, and needs to be distinguished according to the importance and functional value. Therefore, the current image optimization processing is only started for specific applications. In addition, in the Android application, the package name and the application are usually one-to-one corresponding, so the package name of the current application can be obtained by calling getPackageName(). If the package name of the application is in the whitelist, it is determined that the ImageView allows image optimization, and the subsequent scene recognition logic is continued; if not, the subsequent behavior of the ImageView is consistent with that without the image optimization feature.

[0095] In step 603, the image size is obtained. In order to continue to determine whether the image needs image optimization according to the image size, the getIntrinsicWidth() and getIntrinsicHeight() methods of Drawable in the Android View drawing system can obtain the "width and height" of an image, so the width and height of the image to be displayed can be measured by these methods. For example, in the case of BitmapDrawable, the width and height of the held Bitmap are actually returned.

[0096] In step 604, when the difference between the height of the display screen and the height of the image to be displayed is less than a first threshold, or the difference between the width of the display screen of the image processing device and the width of the image to be displayed is less than a second threshold, any one of the two conditions is met, the super-resolution processing is performed on the image to be displayed, i.e., step 605 is executed, otherwise, step 609 is executed. For example, when the width of the image to be displayed satisfies [0.95, 1.0] times of the screen width, or the height of the image to be displayed satisfies [0.95, 1.0] times of the screen height, it can be determined that the size of the image to be displayed is large enough, and it is necessary to optimize the image.

[0097] Step 605, the ImageView draws the set Drawable data in the standard method onDraw() of the View, and draws the content through the standard method draw() of the Drawable, for example, the BitmapDrawable draws the content of a Bitmap onto the ImageView, and the ColorDrawable draws a color to the ImageView, etc.

[0098] Step 606, it is judged whether the super-resolution optimization algorithm device state is normal, because when the image batch display, it will cause the resolution optimization algorithm device task queue congestion, so it is needed to determine the current super-resolution optimization algorithm device state is normal, if yes, then step 507 is executed, otherwise step 509 is executed.

[0099] Step 607, the image optimization processing is only performed on the image with a certain range of resolution, considering the performance, memory and effect. The resolution size setting rule is different for different application programs, for example, the image resolution requirement of the microblog is higher than that of other applications. Only when the resolution of the application program is lower than the corresponding size setting rule, the image optimization is started, so it is needed to continue to judge whether the resolution meets the set condition, if yes, then step 608 is executed, otherwise step 609 is executed.

[0100] Step 608, the super-resolution processing is performed on the image to be displayed which meets the above condition, wherein the super-resolution algorithm executed by the super-resolution corresponds to the image resolution size setting of the application program. Then it returns to execute step 605 to re-draw and display the optimized image. It is needed to be explained that the image optimization process is ended after step 605 is executed.

[0101] Step 609, the Imageview does not perform the optimization on the image, and it is also needed to draw the set Drawable data in the standard method onDraw() of the View, and draw the content through the standard method draw() of the Drawable, that is, as shown in step 505.

[0102] It is needed to be explained that the above judgment order does not exist strict sequence, generally, the embodiment of the application first judges whether the switch is opened, and then measures whether the size of the image is large enough, and further judges whether the resolution is lower than the set standard.

[0103] For example, the image size is large enough, and the resolution is lower than the set standard, then the image optimization is started. Figure 8aThe framework and process of implementing image optimization using the ImageView class are shown, including an application layer A10, a framework layer A20, and a HAL layer / hardware A30 in the operating system. The application layer A10 has an application A11 using the ImageView. The trigger image optimization module in the ImageView A21 in the framework layer A20 is used to determine the timing of image optimization. The scene recognition A211 is used to determine which images need to be processed for image optimization, such as whether the image size meets the preset condition, whether the image resolution meets the resolution threshold, whether there is a face in the image, and the like. Specifically, it can include steps 502-504, 506-507, and different conditions can be set according to the purpose of image optimization to determine which images need to be processed for image optimization. The optimization task allocation and content management module A212 is used to manage the multi-threaded tasks and content of image optimization processing. Specifically, it can be managed in the manner of Figure 10 The optimization algorithm A22 provides multiple image optimization models A221, i.e., image optimization algorithms. The heterogeneous optimization processing module A222 is used to call different processors A32 to accelerate image optimization processing. The HAL layer / hardware is used to implement display and hardware driver of the processor. The application A11 sends the image to the ImageView A21. The trigger image optimization module in the ImageView A21 confirms the timing of image optimization processing. The scene recognition module A211 determines whether the image needs to be optimized. The images that need to be optimized are put into the queue of optimization tasks, which are managed by the optimization task allocation and memory management module A212. The optimization algorithm A22 is called to optimize the images in the optimization task queue. When the optimization processing is implemented, the heterogeneous optimization processing module A222 is used to call different processors A32 to run the optimization algorithm. The updated image is used to update the screen and displayed on the display A31.

[0104] For the second mode, specifically, the image optimization processing of BitmapFactory provides a prediction model to predict that the image in the display interface needs to be processed for super-resolution, and filters out the images outside the display interface, so that the images that meet the conditions can be processed for image optimization, and the performance end-to-end requirements are met. The specific steps are shown in Figure 9

[0105] In step 701, the image of the application is decoded by calling BitmapFactory.decodeFile() of BitmapFactory to decode the image to be displayed, and the source bitmap object is output, for example, the image to be displayed in JPEG format is decoded into a bitmap.

[0106] ​Step 702, BitmapFactory calls the BitmapFactory class of the framework layer, and substitutes the source bitmap object.

[0107] Step 703, the image processing module internally creates / manages the Destination Bitmap, takes the source bitmap and the Destination Bitmap as parameters, and calls the Native API of the HisiDDK to perform data processing.

[0108] Step 704, the HisiDDK internally transmits the sourceBitmap and the Destination Bitmap to the HIAI service process through the Binder interface and inter-process communication (IPC), and calls the IPU to perform super-resolution algorithm processing.

[0109] Step 705, the super-resolution image processed by the HIAI service process is stored in the Destination Bitmap, and the Binder process is called back to the gallery process through the Binder interface asynchronously.

[0110] Step 706, after the Binder receives that the image processing module has completed processing, the Destination Bitmap is transmitted to the view display interface for redrawing.

[0111] Step 707, the image after redrawing is displayed on the display interface of the application.

[0112] Because the above-mentioned image optimization process involves the creation of a Bitmap, in most application scenarios, the Bitmap is the most likely factor to cause memory exceptions. The Bitmap follows the Java GC mechanism, and when there is no strong reference to the Bitmap object, the Bitmap object will be released. If there is an unreasonable pointer holding the Bitmap, it will cause memory occupation or leakage. Therefore, the embodiments of the present application explicitly give the life cycle of the Bitmap in the image optimization process, so as to facilitate memory management. At the same time, there are many image optimization algorithms at present, and the running calculation requirement is high. Different heterogeneous processors such as GPU and FPGA can be further managed, and task allocation can be performed to accelerate the processing performance of image optimization.

[0113] For example, assuming that the first application program is WeChat, user A sends a selfie to user B through WeChat, and a thumbnail of the selfie is displayed in the WeChat chat box of user B. When user B clicks to view the selfie, the image processing module of the operating system is triggered to perform the image display process, and the ImageView or BitmapFactory of the image processing module performs image optimization on the selfie, so that the resolution of the selfie viewed by user B is higher and the definition is better.

[0114] In summary, the above image processing method is more general for third-party applications, and each third-party application does not need to repeatedly develop image optimization functions. By directly calling the existing interface on the operating system, the super-resolution processing can be triggered.

[0115] In addition, the embodiment of the application further adds a control strategy to the image optimization queue of the image processing module, for example, as shown in the following table: Figure 10 As shown in the following table, in step 801, the control strategy is that the task queue receives a super-resolution task request initiated by the image processing module, and then adds the task to the queue. In step 802, if a cancel super-resolution task request initiated by the image processing module is received, the task is deleted from the queue. In step 803, when the super-resolution state machine is detected to be in a ready state, the task is taken out from the task queue and sent to the DDK for processing.

[0116] Meanwhile, considering that the creation of a Bitmap is involved in the complete image optimization process, and in most application scenarios, the Bitmap is the most likely factor to cause memory exceptions. The Bitmap follows the Java GC mechanism, and when there is no strong reference to the Bitmap object, the Bitmap object is released. If there is an unreasonable pointer holding the Bitmap, memory occupation / leakage will be caused. Therefore, the embodiment of the application clearly defines the life cycle of the Bitmap in the image optimization process, so as to manage the memory.

[0117] Based on the same inventive concept as the method embodiment, the embodiment of the application provides an image processing device 900, which belongs to the framework layer of the operating system of an image processing device and is specifically used for implementing the method described in the above embodiment. Figure 3 The structure of the device is as shown in the following table: Figure 11 As shown in the following table, the device includes a receiving unit 901 and a processing unit 902, wherein:

[0118] The receiving unit 901 is configured to receive an instruction in which a to-be-displayed image is carried, the instruction being used for calling an image processing module of an operating system by a first application program.

[0119] The processing unit 902 is configured to perform image optimization processing on the image to be displayed, and display the image after the image optimization processing.

[0120] In a possible design, the processing unit 902 is specifically configured to perform super-resolution processing on the image to be displayed when the resolution of the image to be displayed is lower than a first threshold.

[0121] In another possible design, the image processing apparatus further includes a determination unit 903 configured to determine, according to the identifier of the first application program, that the first application program has the permission of super-resolution processing.

[0122] Further, the determination unit 903 is specifically configured to determine, according to the identifier of the first application program, that the first application program has the permission of image optimization when the identifier of the first application program exists in a preset white list.

[0123] In a possible design, the processing unit 902 is specifically configured to:

[0124] add the image to be displayed as a task object into a task queue;

[0125] determine an image optimization algorithm corresponding to the image to be displayed according to the resolution of the image to be displayed;

[0126] perform super-resolution processing on the task object corresponding to the image to be displayed in the task queue by using the corresponding image optimization algorithm.

[0127] Further, the processing unit 902 is further configured to release the memory space corresponding to the task object after the task object corresponding to the image to be displayed completes the super-resolution processing.

[0128] In addition, in a possible design, a first task in the task queue is allocated to a first processor to perform super-resolution processing, and a second task in the task queue is allocated to a second processor to perform super-resolution processing, so as to accelerate the image processing.

[0129] In a possible design, assuming that the image processing apparatus is an image view class of an operating system, the instructions further include the width and height of the image to be displayed; and the processing unit 902 is further configured to determine that the width and height of the image to be displayed satisfy a set condition, the set condition being that a difference between the height of a display screen of the terminal device and the height of the image to be displayed is less than a second threshold, and / or a difference between the width of the display screen of the terminal device and the width of the image to be displayed is less than a third threshold.

[0130] In a possible design, the image processing apparatus is a bitmap factory class in a framework layer of the Android operating system, and the image processing apparatus decodes the image to be displayed before performing the image optimization processing on the image to be displayed.

[0131] Based on the above embodiments, the embodiments of the present application further provide an image processing device for implementing Figure 3 The method described in the embodiments is described with reference to the Figure 12 As shown in the figure, the device includes a processor 1001, a memory 1002, and a display 1003.

[0132] The processor 1001 can be a central processing unit (CPU) or a digital processing unit, etc.

[0133] The memory 1002 is configured to store instructions of the first application and program instructions of an operating system.

[0134] The display 1003 is configured to display the image processed by the processor 1001 on a human-computer interaction interface of the first application.

[0135] The embodiments of the present application do not limit the specific connection medium between the processor 1001 and the memory 1002. Figure 12 In the embodiments of the present application, the memory 1002, the processor 1001, and the display 1003 are connected through a bus 1004. Figure 12 In the embodiments of the present application, the bus is represented by a thick line, and the connection mode between other components is only schematically described and is not limited. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 12 In the embodiments of the present application, only one thick line is used, but it does not mean that there is only one bus or only one type of bus.

[0136] The memory 1002 can be a volatile memory such as a random-access memory (RAM), or a non-volatile memory such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 1002 can be any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 1003 can be a combination of the above memories.

[0137] The processor 1001 executes program instructions in the memory 1002 for implementing the image processing method as shown in the method, comprising: a first application program sends an instruction to an image processing module in an operating system to call the image processing module to execute an image optimization algorithm, and the image processing module performs image optimization processing on the to-be-displayed image and returns to the first application program for display. Figure 3 The processor 1001 adopts the image optimization processing method, which can be implemented in the following manner: when the resolution of the to-be-displayed image is lower than a first threshold, performing super-resolution processing on the to-be-displayed image.

[0138] The processor 1001 adopts the image optimization processing method, which can be implemented in the following manner: when the resolution of the to-be-displayed image is lower than a first threshold, performing super-resolution processing on the to-be-displayed image.

[0139] In a possible implementation, the instruction of the first application program further comprises an identifier of the first application program, and the processor 1001 is further configured to determine, according to the identifier of the first application program, that the first application program has the permission of super-resolution processing.

[0140] The embodiment of the present application further provides a computer readable storage medium, which stores computer software instructions required for execution of the processor, and contains programs required for execution of the processor.

[0141] The embodiment of the present application further provides a computer program product, which, when executed by a computer, causes the computer to perform the image processing method as described above.

[0142] In the embodiment of the present application, the image processing module of the operating system of the image processing device is improved, the function of image optimization is added, and the to-be-displayed image in different application programs in the application layer is subjected to image optimization processing. That is, when the multimedia file in the application program calls the interface of the image processing module of the operating system to display the image, the image will first undergo the image optimization processing process of the image processing module, and the finally displayed image is an optimized image, and the resolution has been improved, so the clarity is better. Since the image processing module in the operating system is improved, the respective application programs do not need to be developed, and the user operating the application program in the application layer does not perceive the whole image optimization process, so the method has strong reusability and is more automatic.

[0143] Those skilled in the art can clearly understand that the descriptions of the embodiments provided by the present application can be mutually referred to, for the convenience and brevity of description, the functions of the devices and the steps executed by the devices provided by the embodiments of the present application can refer to the related descriptions of the method embodiments of the present application, which will not be repeated here.

[0144] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0145] The various illustrative logic blocks, modules, and circuits described in the embodiments of the present application can be implemented or operated by a general processing unit, a digital signal processing unit, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general processing unit can be a microprocessing unit, optionally, the general processing unit can also be any traditional processing unit, controller, microcontroller, or state machine. The processing unit can also be implemented by a combination of computing devices, such as a digital signal processing unit and a microprocessing unit, a plurality of microprocessing units, one or more microprocessing units in conjunction with a digital signal processing unit core, or any other similar configuration.

[0146] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, a software module executed by a processing unit, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium in the art. Exemplarily, a storage medium can be connected to the processing unit so that the processing unit can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium can also be integrated into the processing unit. The processing unit and the storage medium can be configured in an ASIC, which can be configured in a user terminal. Alternatively, the processing unit and the storage medium can also be configured in different components in the user terminal.

[0147] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a computer. By way of example, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or other wire-based, fiber-based, or other over-the-air technologies, then the coaxial cable, fiber optic cable, twisted pair, DSL, or other wire-based, fiber-based, or other over-the-air technologies are included in the definition of computer-readable medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, DVD, floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0148] The foregoing description of the present application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of the disclosure. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto.

Claims

1. An image processing method, characterized by, The method is suitable for an image processing device with an operating system, and comprises the following steps: An image processing module of the operating system receives an instruction of a first application program calling the image processing module of the operating system, and the instruction carries a to-be-displayed image; The image processing module performs image optimization processing on the to-be-displayed image, and displays the image after the image optimization processing; The image processing module performs image optimization processing on the to-be-displayed image, comprising: when the image processing module determines that the resolution of the to-be-displayed image is lower than a first threshold, the image processing module performs super-resolution processing on the to-be-displayed image; The instruction further comprises an identification of the first application program, and before the image processing module performs super-resolution processing on the to-be-displayed image, the image processing module further comprises: the image processing module determines, according to the identification of the first application program, that the first application program has the permission of super-resolution processing; The image processing module determines, according to the identification of the first application program, that the first application program has the permission of super-resolution processing, comprising: the image processing module judges whether the identification of the first application program exists in a preset white list; if yes, the image processing module determines that the first application program has the permission of image optimization.

2. The image processing method according to claim 1, wherein: The image processing module performs super-resolution processing on the to-be-displayed image, comprising: The image processing module adds the to-be-displayed image as a task object into a task queue; The image processing module determines, according to the resolution of the to-be-displayed image, an image optimization algorithm corresponding to the to-be-displayed image; The image processing module performs super-resolution processing on the task object corresponding to the to-be-displayed image in the task queue by using the corresponding image optimization algorithm.

3. The image processing method of claim 2, wherein, Further comprising: The image processing module releases the memory space corresponding to the task object after the image processing module determines that the task object corresponding to the to-be-displayed image completes the super-resolution processing.

4. The image processing method of claim 2, wherein, A first task in the task queue is allocated to a first processor to perform super-resolution processing, and a second task in the task queue is allocated to a second processor to perform super-resolution processing.

5. The image processing method of any one of claims 1 to 4, wherein, The operating system is an Android operating system, the image processing module is an image view class in a framework layer of the Android operating system, and the instruction further comprises the width and height of the to-be-displayed image; After the image display module receives the instruction of the first application program calling the image processing module of the operating system, and before the image processing module performs image optimization processing on the to-be-displayed image, the method further comprises: The image view class determines that the width and height of the to-be-displayed image satisfy a set condition, the set condition being that the difference between the height of the display screen of the image processing device and the height of the to-be-displayed image is less than a second threshold, and / or the difference between the width of the display screen of the image processing device and the width of the to-be-displayed image is less than a third threshold.

6. The image processing method of any one of claims 1 to 4, wherein, The operating system is an Android operating system, and the image processing module is a bitmap factory class in a framework layer of the Android operating system; Before the image processing module receives the instruction of the first application program calling the image processing module of the operating system, and before the image processing module performs the image optimization processing on the to-be-displayed image, the image processing module further comprises the following steps of: The bitmap factory class decodes the to-be-displayed image.

7. An image processing apparatus characterized by comprising: The image processing device comprises a memory, a processor and a display; The memory is configured to store instructions of the first application program, program instructions of the operating system and program executed by the processor; the instructions of the first application program further comprise an identification of the first application program; The processor is configured to perform image optimization processing on the to-be-displayed image according to the instruction of the first application program calling the image processing module of the operating system; The display is configured to display the image after the image optimization processing. The processor is specifically configured to: When the resolution of the to-be-displayed image is lower than a first threshold, perform super-resolution processing on the to-be-displayed image; the processor is further configured to: According to the identification of the first application program, determine that the first application program has the permission of super-resolution processing; The processor is specifically configured to: Determine whether the identification of the first application program exists in a preset whitelist; If yes, determine that the first application program has the permission of image optimization.

8. The image processing apparatus of claim 7, wherein The processor is specifically configured to: Add the to-be-displayed image as a task object into a task queue; According to the resolution of the to-be-displayed image, determine the corresponding image optimization algorithm of the to-be-displayed image; Use the corresponding image optimization algorithm to perform super-resolution processing on the task object corresponding to the to-be-displayed image in the task queue.

9. The image processing apparatus of claim 8, wherein The processor is further configured to: After the task object corresponding to the to-be-displayed image completes the super-resolution processing, release the memory space corresponding to the task object.

10. The image processing apparatus of claim 8, wherein The first task in the task queue is allocated to a first processor to perform super-resolution processing, and the second task in the task queue is allocated to a second processor to perform super-resolution processing.

11. The image processing apparatus of any one of claims 7 to 10, wherein The operating system is an Android operating system, the image processing module is an image view class in a framework layer of the Android operating system, and the instructions of the first application program further comprise the width and height of the to-be-displayed image. The processor is further configured to: determine that the width and height of the to-be-displayed image meet a set condition, the set condition being that a difference between the height of the display screen of the image processing device and the height of the to-be-displayed image is less than a second threshold, and / or a difference between the width of the display screen of the image processing device and the width of the to-be-displayed image is less than a third threshold.

12. The image processing apparatus of any one of claims 7 to 10, wherein The operating system is an Android operating system, and the image processing module is a bitmap factory class in a framework layer of the Android operating system. The processor is further configured to: decode the to-be-displayed image.

13. An image processing apparatus characterized by comprising: The image processing device belongs to a framework layer of an operating system of an image processing device, an application layer of the operating system comprises a first application program, and the image processing device comprises: A receiving unit configured to receive an instruction of a first application program calling an image processing module of an operating system, the instruction carrying a to-be-displayed image; A processing unit configured to perform image optimization processing on the to-be-displayed image and display the image after the image optimization processing. The processing unit is specifically configured to: when the resolution of the image to be displayed is lower than a first threshold, performing super-resolution processing on the image to be displayed. The determining unit is configured to determine, according to the identifier of the first application program, that the first application program has the permission of super-resolution processing. The determining unit is specifically configured to: determine, according to the identifier of the first application program, that the first application program has the permission of image optimization.

14. The image processing apparatus of claim 13, wherein The processing unit is specifically configured to: add the image to be displayed as a task object into a task queue; determine, according to the resolution of the image to be displayed, an image optimization algorithm corresponding to the image to be displayed; perform super-resolution processing on the task object corresponding to the image to be displayed in the task queue by using the corresponding image optimization algorithm.

15. The image processing apparatus of claim 14, wherein The processing unit is further configured to: release the memory space corresponding to the task object after the task object corresponding to the image to be displayed completes the super-resolution processing.

16. The image processing apparatus of claim 14, wherein The first task in the task queue is assigned to a first processor to perform super-resolution processing, and the second task in the task queue is assigned to a second processor to perform super-resolution processing.

17. The image processing apparatus of any of claims 13 to 16, characterized by The instructions further include the width and height of the image to be displayed. The processing unit is further configured to: determine that the width and height of the image to be displayed satisfy a set condition, the set condition being that a difference between the height of the display screen of the terminal device and the height of the image to be displayed is less than a second threshold, and / or a difference between the width of the display screen of the terminal device and the width of the image to be displayed is less than a third threshold.

18. The image processing apparatus of any of claims 13 to 16, characterized by The processing unit is further configured to: decode the image to be displayed.

19. A computer storage medium, comprising, The computer storage medium stores computer executable instructions, and the computer executable instructions are used to make the computer execute the method in any one of claims 1-6.

20. A computer program product, characterised in that, When the computer program product is executed by the computer, the computer executes the method in any one of claims 1-6.

Images