Image processing method, device, terminal device and storage medium

Abstract

The present application discloses an image processing method, an image processing device, a terminal device and a readable storage medium. The image processing method includes: obtaining an image captured by an image sensor and a current filter usage scenario; rendering the captured image according to the LUT table data file in the current filter usage scenario to obtain a target display image. The present application can reduce the runtime memory during filter calculation, improve the filter calculation performance, and reduce the filter calculation power consumption without affecting the filter effect by loading different filter LUT table data file tables according to different usage scenarios. In scenarios with high requirements for filter effects, the filter conversion effect can be effectively improved.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to an image processing method, an image processing apparatus, a terminal device and a computer-readable storage medium. Background Art

[0002] When using a terminal device to shoot, the user can use color filters to adjust the display effect of the shot image according to his or her preferences and usage habits to improve the user experience.

[0003] Filters are generally divided into two categories: code implementation and display lookup table (LUT) implementation. Code implementation refers to the realization of filter effects by describing calculations through code, while LUT implementation is to realize filter effects by looking up tables. Generally, filters implemented by code can be called ordinary filters, or filters for short, and filters implemented by LUT are called LUT filters. Among them, LUT filters can enumerate the possible value ranges of red, green, and blue (RGB) pixel values, pre-calculate the results after filtering for each possible value range, and then save the results. In this way, when a certain image needs to be processed by LUT filter later, the result after filtering can be determined by looking up the table without performing filter calculations. Therefore, compared with ordinary filters, LUT filters can be processed faster.

[0004] However, since the LUT filter needs to load additional filter resources such as material images before use, LUT filter processing can be performed, and loading the filter resources of the LUT filter takes a long time, it will cause the image processing time to be longer during the shooting process, resulting in lower image processing efficiency. Summary of the invention

[0005] The present application provides an image processing method, which includes:

[0006] Obtain the captured image of the image sensor and the current filter usage scenario; and

[0007] The captured image is rendered according to the LUT table data file in the current filter usage scenario to obtain a target display image.

[0008] The present application also provides an image processing device, comprising:

[0009] An acquisition module, used to acquire the captured image of the image sensor and the current filter usage scenario; and

[0010] A rendering module is used to render the captured image according to the LUT table data file in the current filter usage scenario to obtain a target display image.

[0011] The present application also provides a terminal device, including a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor, and the computer program includes instructions for executing the image processing method. The image processing method includes: acquiring an image captured by an image sensor and a current filter usage scenario; and rendering the captured image according to a LUT table data file under the current filter usage scenario to obtain a target display image.

[0012] The present application provides a non-volatile computer-readable storage medium containing a computer program, and when the computer program is executed by a processor, the processor executes the image processing method. The image processing method includes: acquiring an image captured by an image sensor and a current filter usage scenario; and rendering the captured image according to a LUT table data file under the current filter usage scenario to obtain a target display image.

[0013] In the image processing method, image processing apparatus, terminal device and computer-readable storage medium of the embodiments of the present application, by loading different filter LUT table data files according to different usage scenarios, it is possible to reduce the runtime memory during filter calculation, improve the filter calculation performance, and reduce the filter calculation power consumption without affecting the filter effect. In usage scenarios with high requirements for filter effects, the filter conversion effect can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 It is a flowchart of an image processing method of certain embodiments of the present application.

[0016] Figure 2 It is a module schematic diagram of an image processing device in certain embodiments of the present application.

[0017] Figure 3 It is a module schematic diagram of a terminal device of certain embodiments of the present application.

[0018] Figure 4 It is a scene schematic diagram of the image processing method of certain embodiments of the present application.

[0019] Figure 5 It is a flowchart of an image processing method of certain embodiments of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0021] See also Figure 1 , the present application embodiment provides an image processing method, including:

[0022] 01, obtain the captured image of the image sensor and the current filter usage scenario; and

[0023] 03. Render the captured image according to the LUT table data file in the current filter usage scenario to obtain the target display image.

[0024] Please refer to Figure 2 The embodiment of the present application provides an image processing device 10. The image processing device 10 includes an acquisition module 11 and a rendering module 12. Among them, step 01 can be implemented by the acquisition module 11, and step 03 can be implemented by the rendering module 12.

[0025] In other words, the acquisition module 11 can be used to acquire the captured image of the image sensor and the current filter usage scenario. The rendering module 12 can be used to render the captured image according to the LUT table data file in the current filter usage scenario to obtain the target display image.

[0026] Please combine Figure 3 The embodiment of the present application provides a terminal device 100, including a processor 20, an image sensor 30 and a memory 40, the memory 40 stores a computer program, the computer program includes instructions for executing the image processing method, when the computer program is executed by the processor 20, the processor 20 executes the above-mentioned transplantation method, that is, the processor 20 can be used to obtain the captured image of the image sensor and the current filter usage scenario, the processor 20 can also be used to render the captured image according to the LUT table data file in the current filter usage scenario to obtain the target display image.

[0027] In the image processing method, image processing apparatus 10 and terminal device 100 of these embodiments, by loading different filter LUT table data file tables according to different usage scenarios, it is possible to reduce the runtime memory during filter calculation, improve the filter calculation performance, and reduce the filter calculation power consumption without affecting the filter effect. In usage scenarios with higher requirements for filter effects, the filter conversion effect can be effectively improved.

[0028] In some implementations, the image processing apparatus 10 may be a part of the terminal device 100 . In other words, the terminal device 100 includes the image processing apparatus 10 .

[0029] In some implementations where the image processing apparatus 10 is a part of the terminal device 100, as software, the code segments corresponding to the image processing apparatus 10 may be stored in the memory 40 and executed by the processor 20 to implement the aforementioned functions. In other words, the image processing apparatus 10 includes the aforementioned computer program, or the aforementioned computer program includes the image processing apparatus 10.

[0030] In some embodiments where the image processing apparatus 10 is a part of the terminal device 100, the evaluation device may be a hardware device that is independently installed or added as an additional peripheral component to the terminal device 100. For example, when the image processing apparatus 10 is a part of the terminal device 100, the image processing apparatus 10 may be integrated into the processor 20.

[0031] In some implementations, the terminal device 100 may be a mobile phone, a tablet computer, a laptop computer, a drone, a robot, a smart wearable device (smart helmet, smart glasses, etc.), a virtual reality device, etc. For example, in this application, the terminal device 100 is described by taking a mobile phone as an example, the image processing device 10 and the image processing method are applied to the mobile phone, and the image sensor 30 may be a camera of the mobile phone.

[0032] It should be noted that the filter usage scenario refers to the scenario in which the filter is used in the terminal device 100. It can be understood that the current filter usage scenario refers to the scenario in which the current filter is in the terminal device 100. The filter usage scenarios include preview scenarios, filter menu scenarios, photo shooting scenarios, and video recording scenarios.

[0033] In the preview scene, the target display image generated by the collected image after the filter rendering is displayed as the preview image. In the photo scene, the target display image generated by the collected image after the filter rendering is displayed as the photo. In the filter menu scene, the target display image generated by the collected image after the filter rendering is displayed as the filter effect image. In the video recording scene, the target display image generated by the collected image after the filter rendering is displayed as each frame image in the video.

[0034] The terminal device 100 may include multiple filters, each filter includes multiple LUT table data files, and the multiple LUT table data files are respectively applied to different filter usage scenarios. For example, the terminal device 100 may include filter A, and filter A may include LUT table data file 1, LUT table data file 2, and LUT table data file 3. Among them, LUT table data file 1 is used for preview scene, LUT table data file 2 is used for filter menu scene, and LUT table data file 3 is used for photo shooting scene.

[0035] It should also be noted that different LUT table data files will result in different quality effects of the generated target display images. For example, the quality effect of the target display image rendered according to the LUT table data file of the filter menu scene is worse than the quality effect of the target display image rendered according to the LUT table data file of the preview scene, and the quality effect of the target display image rendered according to the LUT table data file of the preview scene is worse than the quality effect of the target display image rendered according to the LUT table data file of the photo scene or the video recording scene.

[0036] It can be understood that the better the quality of the target display image, the greater the data processing volume of the terminal device 100 during the rendering process, the higher the power consumption and the lower the efficiency. Conversely, the worse the quality of the target display image, the smaller the data processing volume of the terminal device 100 during the rendering process, the lower the power consumption and the higher the efficiency. Figure 4 , and the filter menu may contain a variety of filter images, which are displayed in square or circular small images for users to choose. The images are all displayed in small images, which mainly serve as a sliding selection for users, and the image quality requirements are the lowest. In the preview scene, usually one of the filters is selected in the filter menu for display, which will be displayed in full screen or occupy a large screen area. The image quality requirements are higher than the filter images in the filter menu. In the photo scene, the photos taken can be posted in Moments or sent to friends, can be displayed on various mobile phones or computers, and can also be printed out for display in reality. In the video recording scene, the captured image is rendered by the filter to generate a video. Because in the photo scene and the video recording scene, in order to improve the image or video effect, the required image quality requirements are usually high, and the preview image is only to show the filter effect to the user before taking a photo or recording a video. In this way, the runtime memory during filter calculation can be reduced without affecting the filter effect, the filter calculation performance can be improved, and the filter calculation power consumption can be reduced. In the use scenario with high requirements for filter effects, the filter conversion effect can also be effectively improved.

[0037] The current filter usage scenario may include one or more of a preview scene, a filter menu scene, a photo-taking scene, and a video recording scene. For example, when the current filter usage scenario is one, the current filter usage scenario may be a photo-taking scene; when the current filter usage scenario is two, the current usage scenario may be a preview scene and a filter menu scene.

[0038] In addition, in some embodiments, after the target display image is generated, the target display image may be sent to a corresponding location according to the filter usage scenario. For example, the terminal device 100 may also include a display screen, and the target display image generated according to the LUT table data file of the preview scene and the filter menu scene may be sent to the preview area and the filter menu area of ​​the display screen for display (e.g., Figure 4 ).

[0039] Please combine Figure 5 In some embodiments, step 03 includes:

[0040] 031, determine the size and data accuracy of the LUT table data file in the current filter usage scenario;

[0041] 033, rendering the captured image according to the target LUT table data file whose size is the target size and data precision is the target data precision to obtain the target display image.

[0042] In some embodiments, sub-steps 031 and 033 can be implemented by the rendering module 12, or in other words, the rendering module 12 can be used to determine the size and data accuracy of the LUT table data file in the current filter usage scenario, and render the captured image according to the target LUT table data file having a target size and a target data accuracy to obtain a target display image.

[0043] In some embodiments, the processor 20 is further configured to determine the size and data precision of the LUT table data file, and to render the captured image according to a target LUT table data file having a target size and a target data precision to obtain a target display image.

[0044] It should be noted that the size and data accuracy of the LUT table data file determine the quality of the target display image. The larger the size of the LUT table data file and the higher the data accuracy, the higher the quality of the generated target display image.

[0045] The size and data accuracy of the LUT table data file of the filter menu scene are lower than those of the preview scene, and the size and data accuracy of the LUT table data file of the preview scene are lower than those of the photo scene or the video recording scene. It can be understood that in the filter menu scene, because the display item of the target display image in the filter menu is very small, filter data with a smaller size and lower accuracy can be used, which will not affect the user's perception, and can effectively improve rendering performance and reduce power consumption. In the preview scene, because the area to be displayed is relatively large, filter data with medium size and data accuracy can be used, so that performance and power consumption can be taken into account while maintaining good filter effects. In the photo scene or the video recording scene, filter data with a larger size and higher data accuracy can be used, so that a target display image with finer filter effect details can be generated.

[0046] Furthermore, in step 033, one-dimensional or three-dimensional interpolation calculation is performed on the RGB pixel values ​​of the captured image according to the dimension, size and data size of the filter data in the LUT table data file to generate the color after the filter conversion, and then generate the target display image.

[0047] In the present application, the size range of the LUT table data file is 8-128, and the data precision includes one of 8-bit, 16-bit half float and 32-bit float. For example, the data precision of the filter data in the LUT table data file in the preview scene is 8-bit integer, and the size range is 8-32. The data precision of the filter data in the LUT table data file in the filter menu scene is 8-bit integer / 16-bit half float, and the size range is 16-55. The data precision of the filter data in the LUT table data file in the photo scene or video recording scene is 32-bit floating point, and the size range is 32-64.

[0048] The present application provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by a processor, the processor executes the above-mentioned image processing method.

[0049] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, It is characterized in that The image processing method comprises: Obtain the captured image of the image sensor and the current filter usage scenario; and Rendering the captured image according to the LUT table data file of the filter in the current filter usage scenario to obtain a target display image, each of the filters includes a plurality of LUT table data files, and the plurality of LUT table data files are respectively applied to different filter usage scenarios; The current filter usage scenario includes at least one of a preview scenario, a filter menu scenario, a photo taking scenario, and a video recording scenario.

2. The image processing method according to claim 1, It is characterized in that The step of rendering the captured image according to the LUT table data file of the filter in the current filter usage scenario to obtain a target display image includes: Determine the size and data accuracy of the LUT table data file in the current filter usage scenario; The target display image is obtained by rendering the acquired image according to a target LUT table data file whose size is the target size and whose data precision is the target data precision.

3. The image processing method according to claim 2, It is characterized in that The size range of the LUT table data file is 8-12, and the data precision includes one of 8-bit, 16-bit half float and 32-bit float.

4. The image processing method according to claim 3, It is characterized in that The data precision of the filter data in the LUT table data file in the preview scene is 8-bit integer, and the size range is 8-32.

5. The image processing method according to claim 3, It is characterized in that The data precision of the filter data in the LUT table data file under the filter menu scene is 8-bit integer or 16-bit half floating point, and the size range is 16-55.

6. The image processing method according to claim 3, It is characterized in that The data precision of the filter data in the LUT table data file in the photo-taking scene or the video-recording scene is 32-bit floating point type, and the size range is 32-64.

7. An image processing device, It is characterized in that The image processing device comprises: An acquisition module, used to acquire the captured image of the image sensor and the current filter usage scenario; and A rendering module is used to render the captured image according to the LUT table data file of the filter in the current filter usage scenario to obtain a target display image, each of the filters includes multiple LUT table data files, and the multiple LUT table data files are respectively applied to different filter usage scenarios; the current filter usage scenario includes at least one of a preview scene, a filter menu scene, a photo shooting scene and a video recording scene.

8. The image processing device according to claim 7, It is characterized in that The rendering module is also used to determine the size and data accuracy of the LUT table data file in the current filter usage scenario, and to render the captured image according to the target LUT table data file whose size is the target size and whose data accuracy is the target data accuracy to obtain the target display image.

9. A terminal device, It is characterized in that The system comprises a processor and a memory, wherein the memory stores a computer program and is executed by the processor, wherein the computer program comprises instructions for executing the image processing method according to any one of claims 1 to 6.

10. A non-volatile computer-readable storage medium containing a computer program, It is characterized in that When the computer program is executed by a processor, the processor is caused to execute the image processing method according to any one of claims 1 to 6.

Images