Image processing method, terminal equipment and computer readable storage medium

Abstract

The invention provides an image processing method, terminal equipment and a computer readable storage medium, and is applied to the technical field of terminals. The method comprises the following steps: starting a camera; acquiring a first original image; setting backward dependence on the first original image, that is, the first original image depends backward on the second original image; based on the first original image, a first preview interface is displayed, and the first preview interface comprises the first preview image. Therefore, by setting backward dependence on a certain frame of original image (such as the first original image), in the process of previewing the frame of original image, the first original image does not need to be copied from the first memory segment compared with the existing operation of copying the image for multiple times, so that the operation efficiency is improved. According to the embodiment of the invention, the first preview image can be displayed under the condition that a frame of original image (such as a second original image) after the frame of original image is obtained, so that the power consumption of the terminal equipment can be saved, the jamming phenomenon of the preview image can be reduced, and the preview processing time can be shortened.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular, to an image processing method, a terminal device, and a computer-readable storage medium. Background Art

[0002] With the continuous advancement of terminal device technology, terminal camera applications are becoming increasingly versatile. Camera applications can provide users with real-time visual feedback and perform corresponding preview processing (e.g., beautification, blurring, watermarking, etc.) on images captured by the camera. However, the preview processing time required by terminal devices for camera-captured images is long, resulting in increased power consumption and a poor user experience. Summary of the Invention

[0003] The present application provides an image processing method, a terminal device, and a computer-readable storage medium, which can save the power consumption of the terminal device, shorten the time the terminal device takes to preview images, and improve the user experience.

[0004] In a first aspect, the present application provides an image processing method, which is applied to a terminal device and includes:

[0005] Start the camera;

[0006] Acquire a first original image; store the first original image in a first memory segment;

[0007] Based on the first original image, a first preview interface is displayed, the first preview interface including a first preview image; the first preview image is an image obtained by processing the first original image using a first preset algorithm; the first preview image is stored in a first memory segment;

[0008] Among them, there is a first dependency relationship between the first original image and the second original image; the first dependency relationship includes: the first preview image is displayed when the terminal device obtains the first original image and the second original image; the second original image is any frame image obtained by the terminal device after the first original image; the second original image is stored in the second memory segment, and the second memory segment and the first memory segment are different cache spaces.

[0009] The present application starts the camera, and after starting the camera, it can obtain the first original image through the camera, as well as the second original image, that is, any frame image after the first original image. Furthermore, based on the obtained first original image and the second original image, the first dependency relationship between the first original image and the second original image can be determined, and then the second original image can be obtained without releasing the first original image. Furthermore, based on the first original image and the first dependency relationship, a first preview interface can be displayed, and the first preview interface includes a first preview image, wherein the first preview image is an image obtained by processing the first original image using a first preset algorithm, and the first preview image is displayed by the terminal device when the second original image is obtained.

[0010] Thus, by generating a dependency relationship (e.g., a first dependency relationship) between a certain frame of original image (e.g., a first original image) and any frame of image obtained after the certain frame of original image (e.g., a second original image), it is possible to achieve, during the preview processing of a certain frame of original image, that any frame of image after the certain frame of original image (e.g., a second original image) can be obtained without copying the certain frame of original image (e.g., a first original image) or releasing the certain frame of original image (e.g., a first original image), compared to the existing operation of requiring multiple copies of the image. Furthermore, when any frame of image after the certain frame of original image (e.g., a second original image) is obtained, a preview image (e.g., a first preview image) processed from the certain frame of original image (e.g., a first original image) can be output and displayed, which can reduce the freeze phenomenon of the preview screen, save the power consumption of the terminal device, shorten the time it takes for the terminal device to process the original image (e.g., a first original image) using a preset algorithm (e.g., a first preset algorithm), and improve the user's experience of previewing images.

[0011] In a possible implementation of the first aspect, the first preview interface further includes a photo taking control, and the method further includes:

[0012] In response to a user clicking a photo control, a third original image is acquired; the exposure value of the third original image is the first exposure value; based on the exposure value of the third original image, a second preview interface is displayed, the second preview interface including a second preview image; the second preview image is the result of processing the third original image using the first preset algorithm. Therefore, the exposure value of the third original image can be used to determine whether the third original image was taken with a variable exposure. Based on the determination of whether the third original image was taken with a variable exposure, a second preview image corresponding to the third original image is obtained through different processing methods.

[0013] In a possible implementation of the first aspect, when the first exposure value is less than a preset exposure parameter, the second preview image is displayed when the terminal device acquires a third original image and a fourth original image; the fourth original image is any frame image acquired by the terminal device after the third original image; and a second dependency relationship exists between the third original image and the fourth original image;

[0014] Before displaying the second preview image, the method further includes: processing the third original image using a first preset algorithm in a first thread, where the first thread is any thread other than the main thread in the terminal device.

[0015] In a possible implementation of the first aspect, when the first exposure value is greater than or equal to a preset exposure parameter, the second preview image is displayed when the terminal device acquires the third original image;

[0016] Before displaying the second preview image, the method further includes:

[0017] The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process refers to pausing the first task in the second thread and processing the third original image using the first preset algorithm; wherein, after the third original image is processed using the first preset algorithm, the first task is continued to be executed until the first task processing is completed; the first task is the task in the second thread other than the processing of the third original image using the first preset algorithm.

[0018] Based on the above description of displaying the second preview image under different exposure conditions, it can be seen that when the first exposure value is less than the preset exposure parameter, the third original image has image data with a backward dependency, namely, the fourth original image. In this case, the second preview image is derived based on the third original image and the second dependency relationship, and the second preview image can only be displayed when the terminal device has acquired the third and fourth original images. However, when the first exposure value is greater than or equal to the preset exposure parameter, the third original image does not have image data with a backward dependency, and the processing of the third original image cannot be completed using the backward dependency relationship. In this case, the second thread can be blocked, that is, the first task in the second thread is paused, so that the second thread processes the third original image using the first preset algorithm. After the processing of the third original image using the first preset algorithm is completed, the first task is resumed until the first task is completed, and the second preview image, which is the result of processing the third original image, is returned by the second thread. This ensures that the processing of the third original image can be completed and the second preview image can be displayed even when the third original image does not have image data with a backward dependency.

[0019] In a possible implementation of the first aspect, the first preview interface includes controls for the first shooting mode and controls for the second shooting mode; the first preview interface is an interface in the first shooting mode; and the method further includes:

[0020] In response to a user clicking a control for the second shooting mode, the first dependency is released and a third preview interface is displayed, the third preview interface including the first control; the second shooting mode is different from the first shooting mode; upon receiving the user clicking the first control, a third dependency is generated; the third dependency is a dependency between any two frames of original images acquired by the terminal device in the second shooting mode; based on the third dependency, a fourth preview interface is displayed, the fourth preview interface being an interface with the second preset algorithm enabled; the fourth preview interface including a fourth preview image, and the fourth preview image is displayed when the terminal device acquires the sixth and seventh original images; the second preset algorithm is the same as or different from the first preset algorithm. Therefore, when the shooting mode is switched from the first shooting mode to the second shooting mode, the first dependency in the first shooting mode can be released, quickly returning to the camera application, thereby preparing for the generation of the third dependency in the second shooting mode and ensuring the smoothness and consistency of the preview image in the second shooting mode.

[0021] In a possible implementation of the first aspect, the second shooting mode is a video recording mode; the third preview interface further includes a video recording control; and the method further includes:

[0022] In response to a user clicking a recording control, a recording interface is displayed; the recording interface includes an end recording control and a pause recording control; in response to a user clicking the end recording control or the pause recording control, a fifth preview interface is displayed; the fifth preview interface includes a third preview image, which is displayed when the terminal device captures a fifth original image, the fifth original image being the image captured by the terminal device when the recording is ended or paused;

[0023] Before displaying the third preview image, the method further includes:

[0024] The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process refers to pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; wherein, after the fifth original image is processed using the first preset algorithm, the second task is continued to be executed until the second task processing is completed; the second task is the task in the second thread other than the processing of the fifth original image using the first preset algorithm.

[0025] In summary, when the user clicks the End Recording control or Pause Recording control, the resulting fifth original image does not have the image data for which a backward dependency is set, and the processing of the fifth original image cannot be completed by setting a backward dependency. In this case, the second thread can be blocked, that is, the second task in the second thread is paused, so that the fifth original image is processed using the first preset algorithm in the second thread. After the fifth original image is processed using the first preset algorithm, the second task is continued until the second task is completed. The third preview image of the processed fifth original image is returned through the second thread. This ensures that the processing of the fifth original image can be completed and the third preview image can be displayed when the fifth original image does not have the image data for which a backward dependency is set.

[0026] In a possible implementation of the first aspect, displaying the first preview interface includes:

[0027] In response to the user's operation of starting the first preset algorithm, a first preview interface is displayed. By starting the first preset algorithm, an algorithm basis can be provided for obtaining the first preview image.

[0028] In a possible implementation of the first aspect, after displaying the first preview interface, the method further includes:

[0029] In response to the user's operation of closing the first preset algorithm, the first dependency is released. By releasing the first dependency, preparations can be made for generating other dependencies for the terminal device (for example, the dependency between the second original image and a frame of image acquired after the second original image).

[0030] In a possible implementation of the first aspect, before displaying the first preview interface, the method further includes:

[0031] Obtain identification information; the identification information includes a first identification, a second identification, and a third identification; the first identification is used to indicate the dependency direction of the first original image; the second identification is used to indicate that the first original image depends on the second original image; the third identification is used to indicate that the first preset algorithm is in an enabled state;

[0032] A first dependency relationship is generated based on the first identifier, the second identifier, and the third identifier.

[0033] Based on the above description of generating the first dependency relationship, by generating the first dependency relationship between the first original image and the second original image based on different identification information, it is possible to ensure that, based on the pre-set first dependency relationship, the terminal device will only output and display the first preview image, which is the result of processing the first original image using the first preset algorithm, when the second original image is obtained. This eliminates the need to copy or release the first original image before obtaining the second original image, and can save power consumption on the terminal device compared to current operations that require copying the original image.

[0034] In a possible implementation of the first aspect, before displaying the first preview image, the method further includes:

[0035] In the third thread, the first original image is processed using the first preset algorithm; the third thread is any thread in the terminal device other than the main thread. By starting a thread other than the main thread, that is, the third thread, and processing the first original image using the first preset algorithm in the third thread, the main thread and the third thread can be executed in parallel, thereby shortening the time to display the first preview image and improving the user's experience of previewing the image.

[0036] In a possible implementation of the first aspect, the first preset algorithm is any one of the following algorithms: beautification algorithm, blurring algorithm, sharpening algorithm, filter algorithm, sticker algorithm, watermark algorithm, neon algorithm and slimming algorithm. Based on different preset algorithms, the terminal device can perform different processing on the original image to obtain multiple preview effects.

[0037] In a second aspect, the present application provides a terminal device comprising: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, and the one or more processors calling the computer instructions to enable the terminal device to execute the image processing method in the first aspect and any possible design of the first aspect.

[0038] In a third aspect, the present application provides a chip system, which is applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to call computer instructions so that the terminal device executes the image processing method in the first aspect and any possible design of the first aspect.

[0039] Among them, the chip system may include one chip or multiple chips; when the chip system includes multiple chips, this application does not limit parameters such as the type and quantity of the chips.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a terminal device, enable the terminal device to execute the image processing method in the first aspect and any possible design of the first aspect.

[0041] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the image processing method in the first aspect and any possible design of the first aspect.

[0042] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 A schematic diagram of an application scenario of an image processing method provided in one embodiment of the present application;

[0045] Figure 2 This is a schematic block diagram of the structure of a current terminal device;

[0046] Figure 3 This is a schematic diagram of the principle of preview processing of original images by current terminal devices;

[0047] Figure 4 A flowchart of an image processing method provided in one embodiment of the present application;

[0048] Figure 5 A schematic diagram of an interface display effect of an image processing method provided in one embodiment of the present application;

[0049] Figure 6 A schematic block diagram of the structure of a terminal device provided in one embodiment of the present application;

[0050] Figure 7 A schematic diagram illustrating the principle of previewing an original image by a terminal device provided in one embodiment of the present application;

[0051] Figure 8 A flowchart of an image processing method provided in one embodiment of the present application;

[0052] Figure 9A schematic diagram of a display interface of a terminal device provided in one embodiment of the present application;

[0053] Figure 10 A schematic diagram illustrating the principle of previewing an original image by a terminal device provided in one embodiment of the present application;

[0054] Figure 11 A flowchart of another image processing method provided in one embodiment of the present application;

[0055] Figure 12 A schematic diagram of a software system of a terminal device provided in one embodiment of the present application;

[0056] Figure 13 An interactive diagram of an image processing method provided in one embodiment of the present application;

[0057] Figure 14 A hardware system diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0059] The terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated, nor do they indicate a sequential order among technical features. Therefore, a feature designated as "first," "second," or "third" may explicitly or implicitly include one or more of such features.

[0060] The image processing method provided in the embodiment of the present application can be applied to scenes where image preview processing is required. In some embodiments, the image processing method provided in the embodiment of the present application can be applied to scenes where image preview processing is required, such as shooting, recording, video calling, video conferencing, and screenshots.

[0061] The image processing method provided in the embodiment of the present application can be applied to a terminal device. The terminal device can be a terminal device with display hardware and corresponding software support. For example, the terminal device can be a mobile phone, a folding screen, a smart screen, a tablet computer, a wearable terminal device, a vehicle-mounted terminal device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a home appliance, a projector, etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0062] Take the photo scene as an example below, combined with Figure 1 , an application scenario of the image processing method of an embodiment of the present application is introduced in detail.

[0063] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of an image processing method provided in an embodiment of the present application is shown.

[0064] When the terminal device 100 is unlocked, the desktop of the terminal device 100 may be displayed as follows: Figure 1 The interface 11 shown in FIG. (1) is shown in FIG. Figure 1 As shown in FIG. 1 , multiple applications are installed in the terminal device 100. The multiple applications may include a camera 101, address book, phone, message, setting, and photo album.

[0065] like Figure 1 As shown in FIG. (1), the terminal device 100 responds to the user's triggering operation on the camera 101 in the interface 11 (for example, a single click, multiple clicks or a long press operation), and displays the following Figure 1 The interface 12 shown in FIG. (2) is shown in FIG.

[0066] like Figure 1 As shown in Figure (2), the interface 12 displayed by the terminal device 100 is an internal interface of the camera 101. The interface 12 may include multiple shooting modes of the camera 101. The multiple shooting modes may include aperture mode, night scene mode, photo mode, portrait mode, video mode, and professional mode. Among them, the portrait mode defaults to a mode that adds a preview algorithm (for example, a beauty algorithm and a blur algorithm).

[0067] like Figure 1As shown in FIG. (2), interface 12 is the interface displayed when terminal device 100 is in camera mode. Interface 12 may include viewfinder 102, control 103, control 104, control 105, and control 106. In response to the user triggering operations on different controls in interface 12, terminal device 100 may perform different operations.

[0068] Optionally, the viewfinder 102 is used to display the image to be captured (eg Figure 1 For example, in the photo mode of the camera 101, the viewfinder 102 is used to display the image of the person to be photographed. In the video mode of the camera 101, the viewfinder 102 is used to display the video of the person to be recorded.

[0069] Optionally, the control 103 is used to trigger the terminal device 100 to start a photographing mode in the camera 101. In response to the user's triggering operation on the control 103, the terminal device 100 can capture an image in the photographing mode.

[0070] Optionally, the control 104 is used to trigger the terminal device 100 to start the portrait mode in the camera 101. In response to the user's triggering operation on the control 104, the terminal device 100 can capture an image in the portrait mode.

[0071] Optionally, the control 105 is used to display the captured image (e.g. Figure 1 In response to the user's triggering operation on the control 105, the terminal device 100 can display the captured image or the recorded video.

[0072] Optionally, the control 106 is used to trigger the start of shooting. In response to the user's triggering operation on the control 106, the terminal device 100 can shoot an image.

[0073] It should be noted that if Figure 1 As shown in FIG. (2) of FIG. 1 , the interface 12 may also display the optical magnification of the lens in the terminal device 100. The optical magnification of a lens refers to the ability of the lens to magnify an observed object on an imaging sensor or film. The optical magnification of a lens is a fixed value determined by the physical properties of the lens and cannot be changed by the user.

[0074] Optionally, the optical magnification of the lens may include the optical magnification of a short-focus lens and the optical magnification of a long-focus lens. Figure 1 As shown in Figure (2), the optical magnification of the short-focus lens is 1, and the magnification of the long-focus lens is 3.

[0075] Taking into account that different shooting modes produce different image effects, in actual use, users can obtain different image effects by manually switching between different shooting modes.

[0076] In the case where the user needs to perform preview processing on the image (for example, beautification processing or blurring processing), in response to the user's Figure 1 Slide right in the interface 12 shown in FIG. (2) (as shown in FIG. Figure 1 According to the operation in the direction of the dotted arrow in (2) of the figure, the terminal device 100 can display interface 13. Interface 13 is the interface displayed when the terminal device 100 is in portrait mode. That is, at this time, the terminal device 100 has switched from the photo mode to the portrait mode.

[0077] It should be noted that the above-mentioned method of switching the shooting mode of the terminal device 100 is only for illustrative purposes and is not limited in the embodiments of the present application. For example, when the terminal device 100 is in the photo mode, in response to the user triggering the control 104 (for example, a click operation), the terminal device 100 can also switch to the portrait mode.

[0078] like Figure 1 As shown in FIG. (3), the interface 13 may include a control 107 and a control 108. Optionally, the control 107 is used to trigger the terminal device 100 to turn on or off the beauty algorithm in the camera 101. The control 108 is used to trigger the terminal device 100 to turn on or off the blur algorithm in the camera 101.

[0079] When the beauty algorithm is not turned on, Figure 1 As shown in FIG. (3), the character image displayed in the viewfinder 102 is an image that has not been beautified. In response to the user's triggering operation on the control 107 (for example, the user clicks the control 107 for the first time), the terminal device 100 can start the beautification algorithm. At this time, the terminal device 100 can display the following Figure 1 The interface 14 shown in FIG. (4) .

[0080] like Figure 1 As shown in FIG. (4), the interface 14 may include a control 109. The control 109 is used to prompt the user that the beautification algorithm is turned on. When the beautification algorithm is turned on, Figure 1 As shown in FIG. 4 , the character image displayed in the viewfinder 102 is an image that has been beautified.

[0081] Optionally, in some embodiments, when the user does not need to beautify the image to be captured, the terminal device 100 can also turn off the beautification algorithm in response to the user triggering the control 107 (for example, the user clicks the control 107 for the second time).

[0082] It should be noted that the above interface is described using the beautification algorithm in the preview algorithm as an example, and the embodiments of the present application are not limited to this. For example, when the user needs to blur the image, the user can also manually turn on the blurring algorithm. Among them, the specific process of the user turning on the blurring algorithm is similar to the specific process of the user turning on the beautification algorithm. Please refer to the above-mentioned process of turning on the beautification algorithm, which will not be repeated here. In addition, in addition to the beautification algorithm and the blurring algorithm, the turning on and off process of other algorithms in the preview algorithm (such as the filter algorithm) can also refer to the relevant description of the beauty algorithm, which will not be repeated here.

[0083] It should also be noted that the above interfaces are all illustrated with an optical magnification of 1, and the embodiments of the present application are not limited to this. In actual applications, users can enlarge or reduce the optical magnification according to actual shooting needs.

[0084] It should be understood that the above-mentioned interfaces and controls in each interface are only examples and do not constitute a limitation on the interfaces in the embodiments of this application. In another embodiment of this application, based on different application scenarios, each interface may include more or fewer controls than those in the above examples, and the positions of each control may also be adjusted accordingly, and this application does not impose any specific limitations on them. For example, Figure 1 In addition to being displayed at the bottom of the viewfinder 102, the controls 107 and 108 shown in FIG. (3) can also be displayed on the right or left side of the viewfinder 102. For example, Figure 1 The interface 13 shown in FIG. (3) may also include controls corresponding to other preview algorithms (eg, filter algorithms) in addition to the beauty algorithm and the blur algorithm, so that the user can turn on or off the corresponding preview algorithm through the controls.

[0085] It should also be understood that the above descriptions related to directions (such as right, left and bottom, etc.) are based on Figure 1 The direction shown is described above. When the direction of the terminal device 100 changes, the direction and position of each control can be automatically adjusted accordingly, which is not limited in the embodiments of the present application.

[0086] The following combination Figure 2 and Figure 3 , introduces the specific implementation process of the current terminal device 100 previewing the original image.

[0087] like Figure 2 As shown, the terminal device 100 can implement the preview processing of the original image through multiple software modules. Figure 2As shown, the multiple software modules may include an acquisition module 10, a preview algorithm processing module 11 and a display module 12. The preview algorithm processing module 11 may include an algorithm node module 111 and a preview algorithm module 112.

[0088] Optionally, when the preview algorithm is a beautification algorithm, the algorithm node module 111 is the algorithm node module corresponding to the beautification algorithm, and the preview algorithm module 112 is the beautification algorithm module. It should be noted that different preview algorithms correspond to different algorithm node modules 111, and different preview algorithms correspond to different preview algorithm modules 112.

[0089] The terminal device 100 can obtain the original image through the acquisition module 10. The original image is the image to be previewed that has not undergone any image processing.

[0090] Exemplarily, the acquisition module 10 can be a camera (e.g., a front camera or a rear camera) installed on the terminal device 100. The camera converts optical signals into electrical signals and transmits the content to be captured in the form of digital images to the subsequent preview algorithm processing module 11, providing a data basis for subsequent preview processing.

[0091] After obtaining the original image, the preview algorithm processing module 11 can perform preview processing on the original image. Optionally, the preview algorithm processing module 11 can receive the original image through the algorithm node module 111. The algorithm node module 111 can transmit the received original image to the preview algorithm module 112, which can perform preview processing on the original image to obtain a preview-processed image, which can provide image data to the display module 12.

[0092] It should be understood that the algorithm node module 111 is equivalent to a pipeline for transmitting image data, but does not perform any conversion or analysis on the image data.

[0093] If the preview algorithm processing module 11 is a software module based on the Qualcomm platform software architecture, it can only preview one frame of image at a time. In other words, if the preview algorithm processing module 11 has not completed preview processing of the current frame of image, it cannot receive the next frame of image after the current frame of image, nor can it perform parallel preview processing on the next frame of image after the current frame of image.

[0094] The terminal device 100 can process multiple original images (for example, Figure 3The preview algorithm module 112 may include at least memory segments 4 and 5. Memory segment 4 is used to store the copied original image that has not been previewed (e.g., Figure 3 Memory segment 5 is used to store preview-processed images (e.g., Figure 3 F1' or F2' shown).

[0095] Optionally, memory segment 1, memory segment 2, memory segment 3, memory segment 4, and memory segment 5 are five buffer spaces (buffers) for temporarily storing images and having different addresses. It should be noted that memory segment 1, memory segment 2, memory segment 3, memory segment 4, and memory segment 5 can be five different buffer spaces divided in the memory of the terminal device 100, or can also be five different hardware memories, and this application does not limit this.

[0096] like Figure 3 As shown, after receiving F1 through the algorithm node module 111, the terminal device 100 can copy F1 from memory segment 1 to memory segment 4. At this time, the algorithm node module 111 can release F1 to provide an idle channel for receiving F2. The terminal device 100 uses the preview algorithm module 112 to perform preview processing on F1 in memory segment 4, obtaining the preview-processed F1'. Furthermore, the terminal device 100 can use the preview algorithm module 112 to copy F1' in memory segment 4 to memory segment 5, leaving memory segment 4 in an idle state. Therefore, when the algorithm node module 111 receives F2, the terminal device 100 can use the preview algorithm module 112 to copy F2 from memory segment 2 to the idle memory segment 4, thereby performing preview processing on F2. In addition, the terminal device 100 can use the preview algorithm module 112 to copy F1' in memory segment 5 to memory segment 1, completing the preview processing of F1.

[0097] It should be noted that the specific implementation process of the terminal device 100 for previewing each frame of the original image is the same, such as Figure 3 As shown, the specific implementation process of the terminal device 100 performing preview processing on F2 or F3 can refer to the specific implementation process of the terminal device 100 performing preview processing on F1, which will not be repeated here.

[0098] It should also be noted that the above description is based on an example in which the preview algorithm module 112 includes two memory segments, memory segment 4 and memory segment 5, but the embodiments of the present application are not limited thereto.

[0099] Optionally, the preview algorithm module 112 may include more than two memory segments to enable copying of different original images. For example, the preview algorithm module 112 may include memory segments 4, 5, 6, and 7. When previewing F1 through the preview algorithm module 112, the terminal device 100 performs the corresponding copy operation through memory segments 4 and 5. When previewing F2 through the preview algorithm module 112, the terminal device 100 performs the corresponding copy operation through memory segments 6 and 7. By configuring the preview algorithm module 112 to include multiple memory segments, the terminal device 100 can avoid the situation where the preview algorithm module 112 is unable to preview F2 if memory segments 4 and 5 are occupied, resulting in freezes or flickering in the preview image.

[0100] At present, the terminal device 100 ensures that the algorithm node module 111 can continuously receive the original image (for example, continuously receive F2 and F3 after F1) by copying the original image (for example, F1) and previewing the copied original image through the preview algorithm module 112, thereby reducing the phenomenon of image preview jamming. By copying F1 for preview processing, it can be ensured that the algorithm node module 111 can release F1 while the terminal device 100 is previewing F1 through the preview algorithm module 112. After the algorithm node module 111 releases F1, the algorithm node module 111 is in an idle state. At this time, the algorithm node module 111 can receive F2 and preview F2 through the preview algorithm module 112.

[0101] After getting the preview processed image (for example, Figure 3 In the case of F1' or F2' shown in FIG, the terminal device 100 can use the preview algorithm processing module 11 (or the preview algorithm module 112 in the preview algorithm processing module 11) to convert the image after preview processing (for example, Figure 3 F1' or F2' shown in FIG. 1 is transmitted to the display module 12. Thus, the display module 12 can display the image after preview processing to the user (for example, Figure 3 For example, when the preview algorithm is a beauty algorithm, the image after preview processing may be as follows: Figure 1 The image of a person that has been beautified is displayed in the viewfinder 102 shown in Figure (4).

[0102] However, currently, when the terminal device 100 previews the original image, it needs to perform multiple copy operations (for example, copy F1, copy F1', copy F2, and copy F2', etc.), which will cause the power consumption of the terminal device 100 to increase. In particular, when the preview processing time for each frame of the original image is relatively long (for example, 30ms), multiple copy operations will also cause the terminal device 100 to take a longer time to preview the original image, which in turn causes the user to wait for a longer time, resulting in a poor user experience.

[0103] In view of this, an embodiment of the present application proposes an image processing method, which ensures that the terminal device will not release the current frame original image or output a preview image (for example, the first preview image) after preview processing of the current frame original image before obtaining the dependent image frame (for example, the second original image) by setting a backward frame dependency for the acquired current frame original image (for example, the first original image) (for example, there is a first dependency relationship between the first original image and the second original image, and the second original image is any frame image acquired after the first original image is acquired). Thus, by generating a dependency relationship (for example, a first dependency relationship) between the current frame original image and any frame image obtained after the current frame original image, during the preview processing of the current frame original image (for example, the first original image), compared with the existing operation that requires multiple image copying (for example, copying the current frame original image and copying the preview image after the current frame original image is previewed, etc.), there is no need to copy the first original image or release the current frame original image (for example, the first original image), and any frame image after the current frame original image (for example, the second original image) can be obtained. Furthermore, when any frame image after the current frame original image (for example, the second original image) is obtained, the preview image after the current frame original image is previewed (for example, the first preview image) is displayed.

[0104] It can be seen that in the image processing method provided in the embodiment of the present application, the terminal device generates a dependency relationship (for example, a first dependency relationship) between the current frame original image and any frame image obtained after the current frame original image. In the process of previewing the frame original image, compared with the existing operation of needing to perform multiple copying of the image, there is no need to perform multiple copying operations on the image (for example, copying the current frame original image and the image after previewing the current frame original image), which can save the power consumption of the terminal device, and can reduce the freeze phenomenon of the preview screen, shorten the time for previewing the original image, and improve the user's experience of previewing the image.

[0105] Combined with the above Figure 1 This paper introduces an application scenario of the embodiment of the present application, and Figure 2 and Figure 3 This paper introduces the implementation process of the current terminal device to preview the original image. Figures 4 to 7 , which details the specific implementation process of an image processing method provided in an embodiment of the present application.

[0106] See also Figure 4 , Figure 4 A flowchart of an image processing method provided in an embodiment of the present application is shown.

[0107] like Figure 4 As shown, the image processing method in the embodiment of the present application includes the following steps:

[0108] S101, start the camera.

[0109] Optionally, in some embodiments, a camera is used to obtain the original image. The camera may be a front camera or a rear camera of the terminal device.

[0110] It should be noted that, in actual applications, the terminal device may determine to start the front camera or the rear camera according to the actual usage scenario, and the embodiments of the present application do not limit this. For example, in the scenario of shooting or recording, the default setting of the terminal device starting the camera is to start the rear camera of the terminal device. At this time, in response to the user's operation of switching the camera, the terminal device can switch from the rear camera to the front camera. For another example, in the scenario of video calls or video conferences, the default setting of the terminal device starting the camera is to start the front camera of the terminal device. At this time, in response to the user's operation of switching the camera, the terminal device can switch from the front camera to the rear camera.

[0111] Optionally, in some embodiments, the terminal device responds to the user's operation of starting the camera application (for example, Figure 1 The camera can be started by the user triggering the camera 101 as shown in (1) in the figure.

[0112] It should be noted that the above-mentioned method of starting the camera is only for illustrative purposes, and the embodiments of the present application are not limited thereto. Those skilled in the art may start the camera in other ways. For example, the terminal device receives a voice command input by the user to start the camera. For another example, the terminal device starts the camera in response to the user calling the camera function in a third-party application (such as WeChat, Weibo, Taobao, or Meitu Xiuxiu).

[0113] S102: Acquire a first original image.

[0114] Optionally, the first original image is a frame of image obtained by the terminal device through a camera and not processed by an image algorithm.

[0115] Optionally, the first original image is stored in the first memory segment. The first memory segment is a cache space for temporarily storing the first original image. For example, the first original image can be as follows: Figure 3 As shown in F1, at this time, the first memory segment is as follows Figure 3 Memory segment 1 is shown.

[0116] It should be noted that the first memory segment can be a cache space divided in the memory of the terminal device 100, or it can be a hardware memory, and this application does not limit this.

[0117] After executing S101, the terminal device may obtain the first original image through the camera. By obtaining the first original image, the terminal device can provide a data basis for subsequent preview processing.

[0118] It should be noted that the terminal device may include multiple shooting modes (for example, aperture mode, night scene mode, photo mode, portrait mode, video mode and professional mode, etc.). In different shooting modes, the first original image obtained by the terminal device through the camera may be the same or different. The embodiments of the present application do not limit this.

[0119] S103: Display a first preview interface based on the first original image, where the first preview interface includes a first preview image.

[0120] Optionally, the first preview interface is an interface for displaying the first preview image on the terminal device. The first preview interface may also include multiple controls. The terminal device can perform corresponding processing in response to the user's triggering operation on different controls in the first preview interface. For example, the first preview interface may be as follows: Figure 1 The interface 14 shown in FIG. (4) .

[0121] Optionally, the first preview image is an image obtained by processing the first original image using a first preset algorithm. Figure 1 In the case of the interface 14 shown in FIG. (4), the first preview image is as follows Figure 1 The image of the person is displayed in the viewfinder 102 shown in Figure (4).

[0122] Optionally, the first preview image is stored in a first memory segment. The first memory segment is a cache space for temporarily storing the first preview image.

[0123] It should be noted that after obtaining the first preview image, the terminal device may delete the first original image already stored in the first memory segment and store the first preview image instead. Alternatively, after obtaining the first preview image, the terminal device may directly store the first preview image in the first memory segment. In this case, the first memory segment stores both the first original image and the first preview image.

[0124] Optionally, the first preset algorithm is any one of a plurality of preset algorithms (equivalent to the preview algorithm mentioned above) pre-set in the terminal device. The plurality of preset algorithms may include: a beauty algorithm, a blur algorithm, a sharpening algorithm, a filter algorithm, a sticker algorithm, a watermark algorithm, a neon algorithm, and a slimming algorithm.

[0125] For example, the first preset algorithm may be Figure 1 The beauty algorithm corresponding to the control 107 shown in (3) or the Figure 1 The blurring algorithm corresponding to the control 108 shown in FIG (3) is shown in FIG. In the case where the first preset algorithm is a beauty algorithm, the first preview image is an image obtained by beauty processing the first original image using the beauty algorithm (e.g., Figure 1 (4) The character image displayed in the viewfinder 102 shown in FIG.

[0126] It should be noted that the multiple preset algorithms in the terminal device can set corresponding controls in the camera 101, so that the user can turn on or off the corresponding preset algorithms through the corresponding controls. Those skilled in the art can set controls corresponding to at least one preset algorithm in at least one shooting mode of the camera 101 according to actual product design requirements. The embodiments of the present application are not limited to this. For example, in a shooting mode of the camera 101, a control corresponding to each of the multiple preset algorithms is set. For another example, in a shooting mode of the camera 101 (for example, Figure 1 (3) in the portrait mode), set the controls corresponding to some of the preset algorithms (for example, Figure 1 (3) in FIG. 1 shows a control 107 corresponding to the beautification algorithm and a control 108 corresponding to the blurring algorithm. For example, in each shooting mode of the camera 101, a control corresponding to each of the plurality of preset algorithms is set.

[0127] It should also be noted that the multiple preset algorithms in the terminal device may not need to set corresponding controls in the camera 101. In actual applications, the terminal device automatically starts one or more of the multiple preset algorithms based on the actual needs of taking images or recording videos. The embodiments of the present application are not limited to this.

[0128] Optionally, there is a first dependency relationship between the first original image and the second original image. The second original image is any frame image acquired by the terminal device after the first original image.

[0129] Optionally, the second original image is stored in a second memory segment. The second memory segment is a cache space for temporarily storing the second original image. The second memory segment and the first memory segment are different cache spaces. For example, the storage address of the first memory segment is different from the storage address of the second memory segment.

[0130] For example, the second original image may be: Figure 3 The second original image is as shown in F2 or F3. Figure 3 When F2 is shown, the first memory segment is as follows Figure 3 Memory segment 2 shown; or, in the second original image as shown Figure 3 When F3 is shown, the first memory segment is as follows Figure 3 Memory segment 3 is shown.

[0131] It should be noted that the second memory segment can be a cache space divided in the memory of the terminal device 100, or it can be a hardware memory, and this application does not limit this.

[0132] Optionally, the first dependency relationship includes: the first preview image is displayed when the terminal device acquires the second original image. In other words, the terminal device outputs and displays the first preview image only when the second original image is acquired.

[0133] After acquiring the first original image, the terminal device can continue to acquire the second original image through the camera. By acquiring the second original image, the terminal device can provide a backward dependency data basis for the first original image, and prepare for the subsequent output and display of the first preview image. Backward dependency means that the output of the processing result of the current frame image (for example, the first original image) depends on the input of a frame image after the current frame image (for example, the second original image). By setting the backward dependency, the terminal device can display the first preview image after processing the first original image using the first preset algorithm without performing multiple copy operations on the image. While saving the power consumption of the terminal device, it can also ensure that the preview screen of the image has good fluency and continuity.

[0134] Before displaying the first preview image, the terminal device also needs to generate a first dependency relationship between the first original image and the second original image to provide a dependency condition for the terminal device to display the first preview image. A possible implementation method for the terminal device to generate the first dependency relationship is described below.

[0135] Optionally, in some embodiments, after acquiring the first original image, the terminal device may acquire identification information.

[0136] Optionally, the identification information is information pre-stored in the terminal device. The identification information may include a first identification, a second identification, and a third identification.

[0137] Optionally, the first identifier is used to indicate a dependency direction of the first original image. For example, the first identifier may be a "+" symbol or a "-" symbol. The "+" symbol indicates that the dependency direction of the first original image is forward dependency. The "-" symbol indicates that the dependency direction of the first original image is backward dependency.

[0138] Optionally, the second identifier is used to indicate that the first original image is dependent on the second original image. Exemplarily, the second identifier may be the absolute value of the difference between the frame number of the first original image (or the frame number of the dependent frame) and the frame number of the second original image (or the frame number of the dependent frame).

[0139] Based on the description of the first and second identifiers above, when the frame number of the first original image is 1 and the frame number of the second original image is 2, the absolute value of the difference between the frame numbers of the first and second original images is 1. In this case, the identifier of the first original image's backward dependence on the second original image is "-1." When the frame number of the first original image is 1 and the frame number of the second original image is 3, the absolute value of the difference between the frame numbers of the first and second original images is 2. In this case, the identifier of the first original image's backward dependence on the second original image is "-2." Compared to the first original image's backward dependence on the second original image with frame number 3, the first original image's backward dependence on the second original image with frame number 2 enables faster output and display of the first preview image after processing the first original image, reducing user waiting time and enhancing the user experience.

[0140] Optionally, in another embodiment, the second identifier may also be the time difference between the time of acquiring the dependent frame (eg, the first original image) and the time of the dependent frame (eg, the second original image), which is not limited in this embodiment of the present application.

[0141] Optionally, the third identifier is used to indicate that the first preset algorithm is in an enabled state. Exemplarily, the third identifier may be a character string.

[0142] After obtaining the identification information, the terminal device may generate a first dependency relationship based on the first identifier, the second identifier, and the third identifier in the identification information. By generating the first dependency relationship, the terminal device can ensure that the first original image is not released if the terminal device has not obtained the second original image, and the first preview image processed using the first preset algorithm is not output and displayed, thereby providing the necessary dependency conditions for the terminal device to subsequently display the first preview image.

[0143] Optionally, in some embodiments, the terminal device can display the first preview interface in response to the user's operation of starting the first preset algorithm. By starting the first preset algorithm, the terminal device can provide an algorithm basis for the terminal device to successfully obtain and display the first preview image.

[0144] For example, the terminal device responds to the user clicking on the Figure 1 The operation of the control 107 in the interface 13 shown in FIG (3) can start the beauty algorithm corresponding to the control 107 and display the following Figure 1 The interface 14 shown in FIG. (4) .

[0145] It should be noted that the above-mentioned terminal device starts the first preset algorithm in response to the user clicking the relevant control operation is only for exemplary description, and the embodiments of the present application are not limited to this.

[0146] In actual applications, the terminal device can also automatically start the first preset algorithm according to the specific usage scenario. For example, in a video call scenario, in response to the user's operation of connecting the video call, the terminal device 100 can automatically start the first preset algorithm (for example, a beauty algorithm) while starting the front camera. By automatically starting the first preset algorithm, the terminal device can shorten the time it takes to preview the original image in the video call scenario, thereby improving the user's experience of previewing images in the video call scenario. Figure 5 As shown, after the user connects the video call, the terminal device 100 may display the following Figure 5 The interface A shown in FIG. The interface A includes a character image processed by the beautification algorithm. At this time, the first preview interface is the interface A, and the first preview image is the character image in the interface A processed by the beautification algorithm.

[0147] It should be noted that if Figure 5 The illustrated interface A may also include controls for hanging up a video call, converting a video call to a voice call, and switching between the front and rear cameras. In an actual video call scenario, the terminal device 100 may execute the processing corresponding to each control in response to a user triggering operation on the different controls in interface A, and this embodiment of the present application is not limited to this.

[0148] Optionally, in some embodiments, before displaying the first preview image, the terminal device may start a third thread, and process the first original image using a first preset algorithm in the third thread.

[0149] Optionally, a thread refers to the smallest unit of execution of a program. The third thread is any thread in the terminal device except the main thread (ie, the second thread below).

[0150] Based on the above description, the terminal device starts a third thread and processes the first original image using the first preset algorithm in the third thread. This enables the terminal device to perform the preview processing of the first original image (for example, preview processing of the first original image by a third thread other than the main thread) and the process of obtaining the second original image (for example, obtaining the second original image by the main thread) in parallel when executing the task in the main thread (for example, generating the first dependency relationship). This enables the terminal device to output and display the first preview image after obtaining the second original image. Compared with the method of completing the preview processing of the first original image in the main thread (i.e., the second thread below), there is no need to wait for the completion of other tasks in the main thread (i.e., tasks other than preview processing of the first original image) before outputting and displaying the first preview image. This means that the first preview image is displayed faster and the user experience is better.

[0151] It should be noted that there is no logical connection between the terminal device processing the first original image using the first preset algorithm and the terminal device acquiring the second image. In other words, after acquiring the first original image, the terminal device can process the first original image using the first preset algorithm without waiting for the acquisition of the second original image.

[0152] Optionally, in some embodiments, when the terminal device includes multiple software modules, the image processing method of the embodiment of the present application can be implemented by multiple software modules. Figure 6 As shown, the terminal device 100 may include an acquisition module 20, a preset algorithm processing module 21 and a display module 22. Among them, the preset algorithm processing module 21 may include an algorithm node module 211, a preset algorithm module 212 and a management module 213. The preset algorithm module 212 does not include any memory segment. The management module 213 is used to control the original image input algorithm node module 211, and control the preset algorithm module 212 to process the original image, and control the display module 22 to display a preview image after the original image is processed. The management module 213 is also used to generate or release the dependency relationship between any two frames of original images (for example, the first dependency relationship between the first original image and the second original image).

[0153] Optionally, when the preset algorithm is a first preset algorithm (eg, a beautification algorithm), the algorithm node module 211 is an algorithm node module corresponding to the first preset algorithm, and the preset algorithm module 212 is the first preset algorithm module.

[0154] It should be noted that different preset algorithms correspond to different algorithm node modules 211, and different preset algorithms also correspond to different preset algorithm modules 212. By setting different algorithm node modules and different preset algorithm modules for different preset algorithms, the terminal device 100 can ensure that when multiple preset algorithms are included in the terminal device 100, the multiple preset algorithms are decoupled, that is, the processing processes of the multiple preset algorithms do not affect each other, which facilitates the management and maintenance of the software programs of the multiple preset algorithms.

[0155] like Figure 6 As shown, the terminal device 100 can obtain the original image (eg, the first original image, the second original image, etc.) through the acquisition module 20. The acquisition module 20 can transmit the original image to the preset algorithm processing module 21 for processing.

[0156] It should be noted that Figure 6 The acquisition module 20 shown can refer to the above description of Figure 2 The description related to the acquisition module 10 shown is not repeated here.

[0157] After obtaining the original image, the terminal device 100 may process the original image using a preset algorithm (eg, a first preset algorithm) in the preset algorithm processing module 21 .

[0158] It should be noted that Figure 6 In the example, the algorithm node module 211 and the preset algorithm module 212 are independently configured. The embodiments of the present application are not limited thereto. For example, Figure 6 The algorithm node module 211 and the preset algorithm module 212 shown may also be integrated into one software module to implement processing of the original image.

[0159] like Figure 7 As shown, the terminal device 100 processes multiple original images (for example, Figure 7 The specific implementation process of preview processing of F1, F2, and F3 shown in the figure includes: after the terminal device 100 receives F1 through the algorithm node module 211, the terminal device 100 can start a third thread and process F1 using the preset algorithm in the preset algorithm module 212 in the third thread. At the same time, the terminal device 100 can generate the dependency relationship of F1 received by the algorithm node module 211 through the management module 213. For example, in the case where F1 is the first original image, the dependency relationship of F1 is the first dependency relationship. Figure 7 As shown, the dependency relationship of F1 generated by the management module 213 is the dependency relationship between F1 and F2. That is, F1' is output and displayed by the terminal device 100 when the algorithm node module 211 receives F2.

[0160] It should be noted that the dependency relationship of F1 generated by the management module 213 can be generated in the main thread of the terminal device, that is, the second thread, or in other threads started by the terminal device except the second thread and the third thread. This embodiment of the present application does not limit this.

[0161] like Figure 7 As shown, when the algorithm node module 211 receives F2, the terminal device 100 performs non-blocking processing on the second thread. Non-blocking processing means that the second thread continues to execute the corresponding task (for example, generating the dependency of F2), and the terminal device 100 can start a fourth thread in addition to the main thread and the third thread, and use the preset algorithm in the preset algorithm module 212 in the fourth thread to preview F2. At the same time, the terminal device 100 can notify the management module 213 through the preset algorithm module 212 that F1 has been processed.

[0162] It should be noted that the preset algorithm module 212 can directly send a notification signal to the management module 213 to notify the management module 213 that F1 has been processed. Alternatively, the preset algorithm module 212 can also send a notification signal to the management module 213 through the algorithm node module 211 to notify the management module 213 that F1 has been processed (e.g., Figure 7 This embodiment of the present application does not limit this.

[0163] It should also be noted that the above non-blocking processing can start the thread processing algorithm (for example, the third thread) when the terminal device 100 receives each frame of the original image through the algorithm node module 211. The frame of the original image (for example, the first original image) can be directly returned to the algorithm node module 211, and the management module 213 controls the execution of subsequent operations on the frame of the original image (for example, the execution of subsequent operations in the main thread). In addition, the processing result of the frame of the original image (for example, the first preview image) is returned to the next frame of the frame of the original image (for example, the second original image). Figure 7 As shown, when the algorithm node module 211 receives F2, it returns the processing result F1' of F1. Thus, through multi-threaded parallel processing (for example, parallel processing of the main thread and the third thread), the time it takes for the terminal device to process the original image is shortened, and the user's experience of previewing the image is improved.

[0164] When the management module 213 determines that F1 has been processed, it can remove the dependency of F1, generate the dependency of F2, and continue to perform subsequent operations on the processed F1'. By removing the dependency of F1, the management module 213 can ensure that the management module 213 can successfully continue to perform subsequent operations on the processed F1'.

[0165] It should be noted that the above-mentioned F1 is stored in memory segment 1. Therefore, after obtaining F1', the management module 213 can overwrite F1 already stored in memory segment 1 with F1'. That is, the management module 213 can delete F1 already stored in memory segment 1 and store F1'. This allows the management module 213 to continue to perform subsequent operations based on F1' stored in memory segment 1.

[0166] Optionally, in some embodiments, when the terminal device includes a preset algorithm (eg, a first preset algorithm) and the preset algorithm is enabled, the management module 213 continues to perform subsequent operations on the processed F1', which means inputting F1' into the display module 22 for display.

[0167] Optionally, in another embodiment, when the terminal device includes multiple preset algorithms (for example, a first preset algorithm and a second preset algorithm, etc.), and all of the preset algorithms are turned on, the management module 213 continues to perform subsequent operations on the processed F1', which means inputting F1' into the algorithm node module and preset algorithm module corresponding to other preset algorithms (for example, the second preset algorithm) other than the current preset algorithm (for example, the first preset algorithm) for corresponding processing, until the processing corresponding to each of the multiple preset algorithms is completed on F1, and the image after multiple processing of F1 is displayed through the display module 22.

[0168] It should be noted that when the preset algorithm is turned on, the specific processing process of F2 and F3 by the terminal device 100 is similar to the processing process of F1. Please refer to the processing process of F1 by the terminal device 100 above, which will not be repeated here.

[0169] It should also be noted that if Figure 7 As shown, when the terminal device initiates the third thread and processes F1 in the third thread, the terminal device 100 can receive F2 via the algorithm node module 211 and initiate the fourth thread to process F2 in the fourth thread. While the terminal device is processing F2 in the fourth thread, the terminal device 100 can receive F3 via the algorithm node module 211. At this point, the terminal device can initiate the fifth thread and process F3 in the fifth thread. By initiating different threads to process different original images, the terminal device 100 can ensure that the processing of different original images does not conflict, thereby speeding up the display of preview images on the terminal device. In special circumstances, the terminal device can also initiate the same thread and sequentially perform preview processing on F1, F2, and F3 in the same thread. For example, when the preview processing of F1 in the third thread is completed, the terminal device 100 receives F2 via the algorithm node module 211. At this time, the terminal device 100 can initiate the third thread and perform preview processing on F2 in the third thread.

[0170] By comparison Figure 7 and Figure 3 It can be seen that the image processing method of the embodiment of the present application sets a backward dependency on a certain frame of original image (for example, F1) when previewing the frame of original image. For example, F1 is backward dependent on F2, so that the terminal device will not release the frame of original image during the preview processing of the frame of original image. Moreover, without releasing the frame of original image, the image frame (for example, F2) on which the frame of original image is backward dependent can still be obtained. Furthermore, when the image frame (for example, F1') on which the frame of original image is backward dependent is obtained, the image after preview processing of the frame of original image is output and displayed. Therefore, there is no need to copy F1 from memory segment 1 and then process F1. Compared with the existing method that requires multiple copies of the image (such as Figure 3 As for the operations of copying F1 and copying F1' shown in the figure, it can save the power consumption of the terminal device, shorten the time for processing the original image, reduce the freeze phenomenon of the preview screen, and thus improve the user's preview experience.

[0171] In summary, when the first preset algorithm is enabled, the terminal device can display a first preview interface based on the obtained first original image, the second original image, and the first dependency relationship. The first preview interface includes the first preview image after the first original image is processed using the first preset algorithm. This ensures that, even when the terminal device processes the first original image using the first preset algorithm, based on the first dependency relationship, the terminal device can still obtain the second original image subsequent to the first original image and output and display the first preview image.

[0172] When the original image does not need to be previewed, the user can manually turn off the first preset algorithm. By turning off the first preset algorithm, the power consumption of the terminal device can be saved.

[0173] Optionally, after displaying the first preview interface, the terminal device responds to the user's operation of closing the first preset algorithm (for example, the user clicks on the Figure 1 (3) The operation of the control 107 in the interface 13 shown in Figure 13) can release the first dependency relationship.

[0174] When the second original image is acquired, the terminal device disables the first preset algorithm. That is, the first original image acquired before the second original image satisfies the backward dependency, and when the second original image is acquired, the terminal device can successfully display the first preview image. At this point, the first dependency relationship between the first and second original images generated by the terminal device has been fulfilled. Therefore, the terminal device can release the first dependency relationship, preparing for the generation of other dependencies (e.g., the dependency relationship between the second original image and an image frame acquired after the second original image).

[0175] Given that the second original image is the last original image frame captured before the terminal device deactivated the first preset algorithm, after the first preset algorithm is deactivated, the terminal device cannot capture original images subsequent to the second original image. In other words, the second original image lacks the image data foundation for establishing a backward dependency. Therefore, the terminal device cannot establish a backward dependency for the second original image, i.e., it cannot establish a dependency relationship between the second original image and an image frame captured subsequent to the second original image.

[0176] Furthermore, since the terminal device needs to go through a series of processing from acquiring the second original image to displaying the preview image after processing the second original image using the first preset algorithm, when the terminal device turns off the first preset algorithm, the terminal device will automatically discard the second original image regardless of whether the second original image is being processed, that is, the preview image processed by the first preset algorithm will not be displayed. Therefore, the terminal device does not need to set a backward dependency on the second original image.

[0177] It should be noted that the specific implementation process of the above-mentioned image processing method is explained by taking the terminal device using the first preset algorithm to process the original image as an example. The terminal device uses each preset algorithm among multiple preset algorithms to process the original image and display the corresponding preview image. The implementation principle is the same. Please refer to the above terminal device using the first preset algorithm to process the original image and display the corresponding preview image. The specific implementation process will not be repeated here.

[0178] The image processing method provided in the embodiment of the present application starts the camera, and after starting the camera, it can obtain the first original image and the second original image, that is, any frame image after the first original image, through the camera. Furthermore, based on the obtained first original image and the second original image, the first dependency relationship between the first original image and the second original image can be determined, and then the second original image can be obtained without releasing the first original image. Furthermore, based on the first original image and the first dependency relationship, a first preview interface can be displayed, and the first preview interface includes a first preview image, wherein the first preview image is an image obtained by processing the first original image using a first preset algorithm, and the first preview image is displayed by the terminal device when the second original image is obtained. Therefore, by generating a dependency relationship (for example, the first dependency relationship) between a certain frame of original image (for example, the first original image) and any frame of image (for example, the second original image) obtained after the certain frame of original image, it can be achieved that in the process of preview processing of a certain frame of original image, compared with the existing method of needing to perform multiple copies of images (for example, Figure 3 As for the operations of copy F1 and copy F2, etc. shown in the figure, there is no need to copy a certain frame of original image (for example, the first original image) or release a certain frame of original image (for example, the first original image), and any frame of image after a certain frame of original image (for example, the second original image) can be obtained. Furthermore, when any frame of image after a certain frame of original image (for example, the second original image) is obtained, a preview image (for example, the first preview image) processed from a certain frame of original image (for example, the first original image) can be output and displayed, which can reduce the freeze phenomenon of the preview screen, save the power consumption of the terminal device, shorten the time for the terminal device to process the original image (for example, the first original image) using a preset algorithm (for example, the first preset algorithm), and enhance the user's experience of previewing images.

[0179] In addition, especially when the preview processing time for each frame of the original image is relatively long (for example, 30ms), by dynamically setting the backward dependency of each frame of the original image, it can be ensured to a certain extent that the preview of the image of the terminal device will not be stuck, without the need to perform the operation of copying the image multiple times, which can save the power consumption of the terminal device to a greater extent and further shorten the time for the terminal device to process each frame of the original image using a preset algorithm (for example, the first preset algorithm).

[0180] Taking into account that the frame dependency in the image processing method of the embodiment of the present application is set backward (for example, the first dependency relationship mentioned above, that is, the first original image is backward dependent on the second original image), if the terminal device does not obtain the dependent frame (for example, the second original image), it is impossible to generate the dependency relationship between the two original images (for example, the first dependency relationship), which will cause the preview of the terminal device to be stuck. Therefore, under some edge conditions (for example, switching shooting modes) and discontinuous preview frames (for example, changing exposure to take pictures, ending recording or pausing recording), the terminal device can determine whether to generate a dependency relationship between the two original images based on the actual situation, or determine whether to release the dependency relationship between the two original images, thereby completing the process of dynamically setting the dependency relationship between the two original images.

[0181] The following combination Figure 8 and Figure 9 , which details the specific implementation process of an image processing method in an embodiment of the present application.

[0182] Optionally, the first preview interface may include controls for the first shooting mode and controls for the second shooting mode. The first preview interface is the interface in the first shooting mode. For example, Figure 9 As shown in Figure (1), when the first shooting mode is the portrait mode, the first preview interface can be as follows Figure 9 In the interface 14 in the portrait mode shown in FIG. (1), the control for the first shooting mode may be control 104 , and the control for the second shooting mode may be control 113 .

[0183] Optionally, in some embodiments, the first shooting mode is any shooting mode pre-set in the terminal device. In other words, the first shooting mode can be aperture mode, night scene mode, photo mode, portrait mode, video mode or professional mode.

[0184] Optionally, the second shooting mode is a different shooting mode from the first shooting mode. Figure 9 As shown in Figure (1), the first shooting mode is portrait mode and the second shooting mode is video mode.

[0185] like Figure 8 As shown, the image processing method in the embodiment of the present application specifically includes the following steps:

[0186] S201: In response to a user clicking on a control of a second shooting mode, releasing a first dependency relationship and displaying a third preview interface.

[0187] Optionally, the third preview interface is a display interface in the second shooting mode. The third preview interface includes a first control. The first control is used to trigger the terminal device to turn on or off the second preset algorithm. For example, the third preview interface can be as follows: Figure 9 In the interface 15 in the video recording mode shown in FIG. (2), the first control may be as follows Figure 9 In the control 107 or control 108 in the interface 15 shown in Figure (2), when the second preset algorithm is the beauty algorithm, the corresponding first control is control 107, and when the second preset algorithm is the blurring algorithm, the corresponding first control is control 108.

[0188] It should be noted that the second preset algorithm and the first preset algorithm can be the same preset algorithm, for example, the first preset algorithm and the second preset algorithm are both beautification algorithms. Alternatively, the second preset algorithm and the first preset algorithm can also be different preset algorithms, for example, the first preset algorithm is a beautification algorithm, and the second preset algorithm is a blurring algorithm. Optionally, in some embodiments, the first dependency relationship is a dependency relationship between the first original image and the second original image in the first shooting mode, and correspondingly, the first original image and the second original image are images acquired by the terminal device in the first shooting mode.

[0189] It should be noted that, in actual applications, the terminal device generally sets the shooting mode (e.g., the first shooting mode) after the camera is activated as the photo mode by default. Those skilled in the art may also set the shooting mode of the terminal device after the camera is activated to other shooting modes (e.g., portrait mode) in addition to the photo mode according to actual usage requirements, and the embodiments of the present application are not limited to this.

[0190] It should also be noted that the above-mentioned switching between different shooting modes by the user clicking the corresponding shooting mode control is only for exemplary purposes, and the embodiments of the present application are not limited thereto. In actual applications, the terminal device can also switch between different shooting modes in response to the user's sliding operation. For example, the terminal device can switch from portrait mode to video mode in response to the user's continuous right swipe operation.

[0191] Optionally, after the terminal device executes S103, the user can determine whether to switch the shooting mode based on the current shooting needs. If the user needs to switch the shooting mode, the user can click the control of the second shooting mode in the first preview interface (for example, Figure 9As shown in Figure (1), the user clicks on the control 113 in the interface 14). In response to the user clicking on the control for the second shooting mode, the terminal device can release the first dependency in the first shooting mode. By releasing the first dependency in the first shooting mode and quickly returning to the camera application, the terminal device can prepare for the normal generation of the dependency between any two frames of original images in the second shooting mode by the terminal device. In addition, when the user switches the shooting mode, the first preview image is the last preview image displayed by the terminal device in the first shooting mode.

[0192] In addition, in response to a user clicking a control in the second shooting mode, the terminal device can, while releasing the first dependency, switch from the first shooting mode to the second shooting mode, i.e., switch from displaying the first preview interface to displaying the third preview interface. By displaying the third preview interface, the terminal device can enable the user to trigger operations on related controls in the third preview interface, thereby causing the terminal device to perform corresponding processing in response to the user's operation. S202: Upon receiving a user click on the first control, generate a third dependency relationship.

[0193] Optionally, the third dependency relationship is a dependency relationship between any two original images captured by the terminal device in the second shooting mode. For example, the third dependency relationship may be a dependency relationship between a sixth original image and a seventh original image. The seventh original image is any image captured by the terminal device after the sixth original image. The third dependency relationship includes the following: the fourth preview image is displayed when the terminal device captures both the sixth and seventh original images.

[0194] It should be noted that the specific implementation process of generating the third dependency relationship in S202 is similar to the specific implementation process of generating the first dependency relationship in S103. Please refer to the relevant description in S103 and will not be repeated here.

[0195] When the terminal device switches from the first shooting mode to the second shooting mode, the terminal device will restart the configuration of multiple preset algorithms (for example, the first preset algorithm and the second preset algorithm). In other words, in the first shooting mode, the first preset algorithm in the terminal device is in the on state (for example, Figure 9 As shown in Figure (1), the beauty algorithm in the portrait mode is in the on state). After the terminal device switches from the first shooting mode to the second shooting mode, in the second shooting mode, the second preset algorithm in the terminal device is in the off state (for example, Figure 9 As shown in Figure (2), the beauty algorithm in video recording mode is turned off).

[0196] Therefore, after executing S201, the terminal device can start the second preset algorithm in the second shooting mode when receiving the user's first click on the first control. Figure 9 As shown in FIG. (2), in response to the user's first click operation on the control 107 in the interface 15, the terminal device can activate the beautification algorithm corresponding to the control 107. By activating the second preset algorithm in the second shooting mode, the terminal device can process the original image obtained in the second shooting mode using the second preset algorithm in the second shooting mode and display the processed preview image, thereby enhancing the user's visual experience in the second shooting mode.

[0197] It should be noted that the above Figure 9 In the exemplary description of the interface, the first preset algorithm and the second preset algorithm are the same preset algorithms, and both are beautification algorithms, but the embodiments of the present application are not limited thereto.

[0198] S203: Display a fourth preview interface based on the third dependency relationship.

[0199] Optionally, the fourth preview interface is an interface in which the second preset algorithm is enabled. Figure 9 The interface 16 shown in (3) in FIG.

[0200] After executing S202, the terminal device may display a fourth preview interface based on the generated third dependency relationship, where the fourth preview interface includes a fourth preview image, and the fourth preview image is displayed when the terminal device obtains the sixth original image and the seventh original image.

[0201] Optionally, the fourth preview image is an image obtained by processing the sixth original image using the first preset algorithm. It should be noted that the specific implementation process of displaying the fourth preview interface in S203 is similar to the specific implementation process of displaying the first preview interface in S103. Please refer to the relevant description of S103 and will not be repeated here. It should also be noted that when the terminal device switches from the first shooting mode to the second shooting mode, the terminal device will also restart the numbering of the original images. In other words, the frame number of the original image obtained by the terminal device in the second shooting mode may be the same as or different from the frame number of the original image obtained by the terminal device in the first shooting mode. This is not limited in this embodiment of the present application. For example, in the first shooting mode, the terminal device may number the frame numbers of the obtained original images as 1, 2, 3, etc. in the order of acquisition. In the second shooting mode, the terminal device may number the frame numbers of the obtained original images as 1, 2, 3, etc. or as 0, 1, 2, etc. in the order of acquisition.

[0202] Optionally, in some embodiments, when the second shooting mode is a video mode and the first preset algorithm in the video mode is turned on, the terminal device may display a video interface in response to the user clicking the video control. Figure 9 As shown in FIG. (3), the video recording control is control 114. The terminal device responds to the user in the following manner: Figure 9 Clicking the control 114 in the interface 16 shown in FIG. (3) can display the following Figure 9 The interface 17 shown in FIG. (4) .

[0203] Optionally, the recording interface may include an end recording control and a pause recording control. Figure 9 As shown in FIG. 4 , the control for ending recording is control 115 in interface 17 , and the control for pausing recording is control 116 in interface 17 .

[0204] After displaying the recording interface, the terminal device may display a fifth preview interface ( Figure 9 not shown).

[0205] It should be noted that when the terminal device responds to the user clicking the "End Recording" control, it indicates that the terminal device has completed a recording session. When the terminal device responds to the user clicking the "Pause Recording" control for the first time, it indicates that the terminal device has paused a recording session. When the user clicks the "Pause Recording" control again, the terminal device responds to the user clicking the "Pause Recording" control again and can resume recording from the paused position until the user clicks the "End Recording" control, completing the recording session.

[0206] Optionally, the fifth preview interface may include a third preview image. The third preview image is displayed when the terminal device acquires a fifth original image, which is an image acquired by the terminal device when the video recording is ended or paused.

[0207] For example, Figure 9 As shown in FIG. 4 , in response to the user clicking the control 115 in the interface 17 , the terminal device can obtain the fifth original image when the user ends the recording.

[0208] It should be noted that Figure 9 Other controls in the four interfaces shown can be found in Figure 1 The relevant instructions in are not repeated here.

[0209] Considering the high real-time nature of terminal device recording, when the terminal device stops recording (including ending or pausing recording), the fifth original image captured by the terminal device has not yet been processed by the first preset algorithm. In this case, the terminal device can block the second thread to complete the processing of the fifth original image, thereby ensuring that every frame of the video captured by the terminal device in recording mode is processed by the first preset algorithm, thereby improving the user experience in recording mode.

[0210] Optionally, the second thread is a main thread in the terminal device. Blocking processing refers to pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; after the fifth original image is processed using the first preset algorithm, the second task is continued until the second task is processed. The second task is the task in the second thread other than processing the fifth original image using the first preset algorithm.

[0211] Alternatively, as Figure 10 As shown, after receiving the fifth original image through the algorithm node module 211, the terminal device 100 can block the second thread, that is, suspend the second task in the second thread; and use the preset algorithm (e.g., the first preset algorithm) in the preset algorithm module 212 to process the fifth original image in the second thread. After the fifth original image is processed using the preset algorithm (e.g., the first preset algorithm) in the preset algorithm module 212, the second task in the second thread continues to be executed. After the second task in the second thread is executed, the preset algorithm module 212 can return the processed third preview image to the management module 213. The management module 213 can transmit the third preview image to the display module 22 and control the display module 22 to display the third preview image.

[0212] like Figure 10 As shown, when the fifth original image is represented as F1, the third preview image is correspondingly represented as F1'. When the second thread is blocked, there is no lag between inputting the original image and obtaining the corresponding preview image. That is, the terminal device can obtain the preview image (for example, the third preview image) after processing the current frame original image using the preset algorithm after obtaining the current frame original image (for example, the fifth original image). Figure 10 As shown, the input of the algorithm node module 211 is F1, and the output is F1'. Figure 7 As shown in the non-blocking processing of the second thread (ie, outputting the processing result of the previous frame in the next frame, that is, outputting the processing result F1' of F1 when F2 is input), Figure 10As shown, it takes longer to obtain the preview image (e.g., the third preview image) corresponding to the original image of the current frame (e.g., the fifth original image). Therefore, in the embodiment of the present application, the terminal device blocks the second thread to obtain the preview image corresponding to the original image of the current frame (e.g., the fifth original image) only under marginal conditions (e.g., the original image of the current frame does not have an image that can be set as a dependency). This further shortens the time it takes for the terminal device to preview the original image while ensuring that the terminal device can successfully display the preview image.

[0213] Combined with the above Figures 8 to 10 The specific implementation process of an image processing method in the embodiment of the present application is described in detail when the terminal device switches the shooting mode and the terminal device ends or pauses the recording. Figure 11 , details the specific implementation process of another image processing method in the embodiment of this application. Figure 11 , Figure 11 A flowchart of another image processing method in an embodiment of the present application is shown.

[0214] like Figure 11 As shown, when the first preview interface also includes a photo taking control, another image processing method in an embodiment of the present application specifically includes the following steps:

[0215] S301: In response to a user clicking a photo control in a first preview interface, outputting a third original image.

[0216] Optionally, in some embodiments, the exposure value of the third original image is a first exposure value. The first exposure value may be a dynamic range of the third original image. The dynamic range of the third original image is used to describe the visible range from the brightest portion to the darkest portion of the third original image. Exemplarily, the dynamic range of the third original image refers to the difference between the maximum brightness value of the third original image and the minimum brightness value of the third original image. Exemplarily, when the maximum brightness value of the third original image is 255 pixels and the minimum brightness value of the third original image is 0 pixels, the first exposure value is 255 pixels.

[0217] The terminal device responds to the user clicking on the first preview interface (for example, Figure 1 (4) in the interface 14) shown in FIG) in the photo control (for example, Figure 1 By operating the control 106 in the interface 14 shown in FIG. (4), a third original image can be obtained. By obtaining the third original image, preparation can be made for the subsequent display of the second preview interface, and a data basis for image algorithm processing can be provided for displaying the second preview image.

[0218] S302: Display a second preview interface according to the exposure value of the third original image.

[0219] Optionally, the second preview interface includes a second preview image. The second preview image is an image obtained by processing the third original image using the first preset algorithm.

[0220] After executing S302, the terminal device can determine the maximum brightness value and the minimum brightness value of the third original image. Furthermore, the terminal device can calculate an exposure value for the third original image based on the maximum brightness value and the minimum brightness value of the third original image. Furthermore, the terminal device can display a second preview interface based on the exposure value of the third original image. By displaying the second preview interface, the user can be presented with an image processed from the third original image, i.e., the second preview image.

[0221] like Figure 11 As shown, S302 specifically includes the following steps:

[0222] S3020: Obtain a first exposure value.

[0223] It should be understood that the specific implementation process of S3030 can refer to the specific process of calculating the first exposure value in S302, which will not be repeated here.

[0224] S3021: Determine whether the first exposure value is less than a preset exposure parameter.

[0225] Optionally, the preset exposure parameter is an exposure parameter pre-stored in the terminal device. Exemplarily, the preset exposure parameter may be 150 pixels.

[0226] After executing S3020, the terminal device may determine whether the exposure value of the third original image, that is, the first exposure value, is less than the preset exposure parameter. If the first exposure value is less than the preset exposure parameter, the terminal device may execute S3022. If the first exposure value is greater than or equal to the preset exposure parameter, the terminal device may execute S3023.

[0227] S3022. When the first exposure value is less than the preset exposure parameter, in the first thread, the third original image is processed using the first preset algorithm, and a second preview interface is displayed, where the second preview interface includes the second preview image; wherein the second preview image is displayed when the terminal device obtains the third original image and the fourth original image, and the fourth original image is any frame image obtained by the terminal device after the third original image, and there is a second dependency relationship between the third original image and the fourth original image.

[0228] Optionally, the first thread is any thread other than the main thread (ie, the second thread) in the terminal device. It should be noted that the first thread and the third thread mentioned above can be the same thread or different threads, and this embodiment of the application does not limit this.

[0229] It should be noted that the specific implementation process of S3022 is similar to that of S103. Please refer to the relevant description of S103. The generation process of the second dependency relationship can also refer to the relevant description of generating the first dependency relationship above, which will not be repeated here.

[0230] When the first exposure value is greater than or equal to the preset exposure parameter, the third original image is an image captured using variable exposure. In this case, to avoid white or dark flashes on the terminal device, the terminal device may discard one or more frames acquired after the third original image. In other words, after acquiring the third original image, the terminal device experiences image frame discontinuity, and the third original image has no image frames to rely on. Therefore, the terminal device may execute S3023 to successfully display the second preview image corresponding to the third original image.

[0231] It should be noted that, under the above-mentioned variable exposure photography, the number of image frames discarded by the terminal device can be set by a person skilled in the art, and this application does not limit this. For example, under the variable exposure photography, the number of image frames discarded by the terminal device is 3 frames.

[0232] It should also be noted that in response to the user clicking the photo control, the terminal device 100 can directly display the image after preview processing, that is, the second preview image, without displaying the third original image, thereby giving the user a better experience in previewing the image.

[0233] S3023. When the first exposure value is greater than or equal to the preset exposure parameter, the second thread is blocked and a second preview interface is displayed. The second preview interface includes a second preview image. The second preview image is displayed when the terminal device obtains the third original image.

[0234] Optionally, the second thread is a main thread in the terminal device. Blocking processing refers to pausing the first task in the second thread and processing the third original image using the first preset algorithm; after the third original image is processed using the first preset algorithm, the first task is continued until the first task is processed. The first task is the task in the second thread other than processing the third original image using the first preset algorithm.

[0235] It should be noted that the specific implementation process of S3023 is the same as Figure 10 The blocking process is similar to that shown in Figure 10 The relevant description is not repeated here.

[0236] The image processing method of the embodiment of the present application is described in detail above. In practical applications, the image processing method of the embodiment of the present application can be implemented by relying on the hardware system and software system of the terminal device. Figure 12 and Figure 14 , respectively introduce the software system and hardware system of a terminal device in an embodiment of the present application.

[0237] In the embodiment of the present application, the software system of the terminal device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture or a cloud architecture. The embodiment of the present application takes the layered architecture as an example to exemplarily describe the software system of the terminal device 100.

[0238] See also Figure 12 , Figure 12 A software system diagram of a terminal device provided in an embodiment of the present application is shown.

[0239] like Figure 12 As shown, the software system in the terminal device 100 can be divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, Figure 12 As shown, the software system of the terminal device 100 is composed of an application layer (APP), an application framework layer (application framework), a hardware abstraction layer (HAL) and a driver layer (driver) from top to bottom.

[0240] The application layer may include multiple applications. For example, the multiple applications may include a camera application, a settings application, an album application, a calendar application, and a music application. It is understood that the multiple applications in the application layer may be third-party applications installed by the user or system applications.

[0241] The application framework layer is a framework layer used to support the operation of multiple applications in the application layer. The application framework layer includes standard Android application programming interfaces (APIs), and multiple interfaces can be called by multiple applications in the application layer.

[0242] The application framework layer may also include core services, etc. Core services are used to provide core functions of the software system (eg, data storage, etc.).

[0243] The language used in the Hardware Abstraction Layer (HAL) is the HAL interface definition language (HIDL). This language is used to describe the interfaces and functionality of hardware components. The HIDL framework is the HAL framework provided by native Android.

[0244] The hardware abstraction layer is used to access cameras, sensors, and displays. Optionally, in some embodiments, the hardware abstraction layer may include an acquisition module, a preset algorithm processing module, and a display module. The preset algorithm processing module may include an algorithm node module, a management module, and a preset algorithm module.

[0245] It should be noted that the detailed description of the acquisition module, algorithm node module, management module, preset algorithm module and display module can be found in Figure 6 The relevant description will not be repeated here.

[0246] Optionally, in another embodiment, the hardware abstraction layer may further include an image sensor, an image front-end engine, and an image processing engine. The image sensor is used to acquire raw image data. The image front-end engine is used to perform preliminary processing on the raw image data transmitted by the image sensor, preparing for subsequent image preview processing. The image processing engine is used to further process and optimize the raw image data transmitted by the image front-end engine, further preparing for subsequent image preview processing. The preset algorithm processing module is used to perform preview processing on the raw image data transmitted by the image processing engine.

[0247] It should be understood that the image sensor, image front-end engine and image processing engine have the same function as the acquisition module.

[0248] like Figure 12 As shown in Figure 1, the driver layer is an abstract layer between hardware and software. The driver layer includes Figure 12 The display driver, Bluetooth driver, hardware interface driver, audio driver and sensor driver shown.

[0249] Optionally, the display driver is used to ensure that the terminal device 100 can display a preview image (eg, a first preview image or a second preview image, etc.) obtained by processing the original image using a preset algorithm.

[0250] Optionally, the Bluetooth driver is used to ensure that the terminal device 100 can identify and use the Bluetooth chip.

[0251] Optionally, the hardware interface driver includes a hardware connection driver between the Bluetooth chip and the processor. Specifically, the hardware interface driver may be a universal serial bus (USB) interface driver. Alternatively, the hardware interface driver may be a universal asynchronous receiver / transmitter (UART) interface driver. This enables the terminal device 100 to connect to corresponding hardware through different hardware driver interfaces.

[0252] It should be understood that Figure 12 The hierarchical structure shown does not constitute a specific limitation on the software system of the terminal device 100. In other embodiments of the present application, the software system of the terminal device 100 may include Figure 12 The layered architecture shown may be more or less than the architecture shown. For example, the software system of the terminal device 100 may also include a hardware layer (hardware), etc. The hardware layer is used to define the connection hardware between the Bluetooth chip and the Bluetooth and processor. Alternatively, each layer of the software system of the terminal device 100 may include more than Figure 12 The components shown may have more or fewer structures, and the embodiments of the present application are not limited thereto.

[0253] The image processing method of the embodiment of the present application can be executed in the hardware abstraction layer. Figure 13 , which details the specific implementation process of the image processing method of the embodiment of the present application executed in the hardware abstraction layer.

[0254] See also Figure 13 , Figure 13 An interactive diagram of an image processing method in an embodiment of the present application is shown. Figure 13 As shown, the specific implementation process of an image processing method in an embodiment of the present application includes:

[0255] Step 11: In response to the user's operation of starting the camera application (for example, the user clicks Figure 1 (1) The control 101 in the interface 11 shown in FIG), the terminal device can display the interface in the camera mode (for example, Figure 1 The interface 12 shown in (2) in FIG.

[0256] Step 12: In response to the user clicking on the relevant control in the interface in the photo mode (for example, the user clicks on Figure 1 (2) in the interface 12 shown in the control 104), the terminal device can display the interface in the portrait mode (for example, Figure 1 The interface 13 shown in (3) in FIG.

[0257] Step 13, in response to the user clicking on the relevant control in the interface in the portrait mode (for example, the user clicks on the Figure 1 (3) in the interface 13 shown in the control 107), the terminal device can start the first preset algorithm (for example, Figure 1 (3) The beautification algorithm corresponding to the control 107 in the interface 13 shown in Figure 13).

[0258] In step 14, when the first preset algorithm is enabled, the terminal device may obtain the first original image through the obtaining module.

[0259] It should be understood that the specific implementation process of step 14 is the same as the specific implementation process of S102. Please refer to the relevant description of S102 and will not be repeated here.

[0260] In step 15, when the acquisition module acquires the first original image, the terminal device can transmit the first original image to the preset algorithm processing module through the acquisition module. Furthermore, the terminal device can generate a first dependency relationship based on the first original image through the preset algorithm processing module, and use the first preset algorithm to process the first original image to obtain a first preview image.

[0261] In step 16, the terminal device may obtain the second original image through the obtaining module.

[0262] It should be understood that there is no sequence or order between step 15 and step 16. That is, step 15 and step 16 can be executed simultaneously, or they can be executed in the order of step 15 first and step 16 later, or they can be executed in the order of step 15 later and step 16 first.

[0263] Step 17: When the acquisition module acquires the second original image, the terminal device may transmit the second original image to the preset algorithm processing module through the acquisition module. Further, when the preset algorithm processing module obtains the second original image, the terminal device may send the first preview image to the display module through the preset algorithm processing module. Further, the terminal device may display the first preview interface (for example, Figure 1 (4) in the interface 14), and displaying the first preview image (for example, Figure 1 The interface 14 shown in Figure (4) is a character image processed by the beauty algorithm).

[0264] It should be understood that the specific implementation process of steps 15 to 17 is the same as the specific implementation process of S103. Please refer to the relevant description of S103 and will not be repeated here.

[0265] It can be seen that by setting a backward dependency on the first original image in steps 15 to 17, it can be ensured that the terminal device can still obtain the second original image after the first original image when previewing the first original image, thereby ensuring that the preview of the terminal device will not be stuck. In addition, the terminal device displays the first preview image when the terminal device obtains the second original image. It can be seen that the terminal device starts a thread (for example, a third thread) other than the main thread (or the second thread) when previewing the first original image, and does not block the main thread. This can speed up the terminal device's preview processing of the first original image, shorten the time the terminal device takes to preview the first original image, and thus improve the user experience.

[0266] Step 18, in response to the user clicking on the relevant control in the interface in the portrait mode (for example, the user clicks on the Figure 1 (3) In the control 107 in the interface 13 shown in Figure 1), the terminal device can turn off the first preset algorithm.

[0267] In step 19, when the first preset algorithm is disabled, the terminal device can release the first dependency through the preset algorithm processing module. By releasing the first dependency, the terminal device can prepare to generate other dependencies (for example, the dependency between the second original image and a frame of image acquired after the second original image) and prepare the preset algorithm processing module to input the first preview image into other preset algorithm processing modules (the preset algorithms corresponding to the other preset algorithm processing modules are any preset algorithms other than the first preset algorithm) for corresponding processing.

[0268] It should be understood that Figure 13 The interactive diagram shown is only an interactive diagram corresponding to an image processing method in the embodiment of the present application, and its purpose is to fully reflect an interactive process in the embodiment of the present application. Other implementation methods with interactive processes in the embodiment of the present application can also be adopted. Figure 13 The interactive process is completed by using the relevant software modules shown in the figure. The implementation principle of the interactive process in each implementation method is similar and will not be repeated here.

[0269] The above describes in detail a software system of a terminal device in an embodiment of the present application. Figure 14 , introduces the hardware system of a terminal device in an embodiment of the present application.

[0270] See also Figure 14 , Figure 14 A hardware system diagram of a terminal device provided in an embodiment of the present application is shown.

[0271] like Figure 14As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L and a bone conduction sensor 180M, etc.

[0272] It should be noted that Figure 14 The structure shown does not constitute a specific limitation on the hardware system of the terminal device 100. In other embodiments of the present application, the hardware system of the terminal device 100 may include Figure 14 The components shown are more or less than the components shown, or the hardware system of the terminal device 100 may include Figure 14 A combination of some of the components shown, or a hardware system of the terminal device 100 may include Figure 14 Subcomponents of some of the components shown. For example, Figure 14 The illustrated proximity light sensor 180G may be optional. Figure 14 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0273] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices.

[0274] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0275] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0276] Figure 14 The connection relationship between the modules shown is only for illustrative purposes and does not limit the connection relationship between the modules of the terminal device 100. Optionally, the modules of the terminal device 100 may also adopt a combination of the multiple connection modes in the above embodiments.

[0277] The terminal device 100 can implement display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0278] The display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light emitting diode (QLED). In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0279] In some embodiments, the expanded state of the display screen 194 can be as follows: Figure 1 In another embodiment, the unfolded state of the display screen 194 can be as follows: Figure 9 Any of the four states shown in .

[0280] The terminal device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0281] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0282] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0283] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0284] The terminal device 100 can implement audio functions, such as music playback and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0285] The touch sensor 180K is also referred to as a touch-sensitive device. The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a touch screen. The touch sensor 180K is configured to detect touch operations applied thereto or in the vicinity thereof. The touch sensor 180K can transmit the detected touch operations to the application processor to determine the type of touch event. In other embodiments, the touch sensor 180K can also be disposed on the surface of the terminal device 100, in a location different from the display screen 194.

[0286] Optionally, in some embodiments, the processor 110 activates a camera, thereby acquiring a first original image through the camera; then, sets a backward dependency on the first original image, i.e., a first dependency relationship exists between the first original image and a second original image acquired after the first original image; further, based on the first original image, displays a first preview interface, wherein an image processed by the first original image using a first preset algorithm, i.e., a first preview image, is displayed in the first preview interface, and the first preview image is displayed when the terminal device acquires the second original image. Thus, by setting a backward dependency on a frame of original image (e.g., the first original image), it is possible to ensure that a copy operation is not required on the frame of original image (e.g., the first original image) during the preview processing of the frame of original image, thereby saving power consumption of the terminal device. When an original image after the frame of original image (e.g., the second original image) is acquired, displaying the image processed by the frame of original image (e.g., the first preview image) can reduce preview screen freezes, shorten the time it takes to process the original image using the preset algorithm (e.g., the first preset algorithm), and enhance the user's preview experience.

[0287] Exemplarily, the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a terminal device, the terminal device executes the following Figure 4 The image processing method shown in .

[0288] For example, the present application provides a chip system, which is applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to call computer instructions to enable the terminal device to execute the following Figure 4 The image processing method shown in .

[0289] Exemplarily, the present application provides a computer program product, which, when executed on a computer, enables a terminal device to execute the following Figure 4 The image processing method shown in .

[0290] In the above embodiments, all or part of the functions can be implemented by software, hardware, or a combination of software and hardware. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer codes or instructions. When the computer program code or instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer code or instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0291] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by a computer program instructing related hardware to perform the processes. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image processing method, characterized in that: The method is applied to a terminal device, and the method includes: Start the camera; Acquire a first original image; the first original image is stored in a first memory segment; Based on the first original image, displaying a first preview interface, the first preview interface including a first preview image; the first preview image is an image obtained by processing the first original image using a first preset algorithm; the first preview image is stored in the first memory segment; Among them, there is a first dependency relationship between the first original image and the second original image; the first dependency relationship includes: the first preview image is displayed when the terminal device obtains the first original image and the second original image; the second original image is any frame image obtained by the terminal device after the first original image; the second original image is stored in a second memory segment, and the second memory segment and the first memory segment are different cache spaces.

2. The method according to claim 1, characterized in that The first preview interface further includes a photo taking control, and the method further includes: In response to a user clicking the photo control, acquiring a third original image; the exposure value of the third original image is the first exposure value; A second preview interface is displayed according to the exposure value of the third original image, where the second preview interface includes a second preview image; the second preview image is an image obtained by processing the third original image using the first preset algorithm.

3. The method according to claim 2, characterized in that When the first exposure value is less than a preset exposure parameter, the second preview image is displayed when the terminal device acquires the third original image and the fourth original image; the fourth original image is any frame image acquired by the terminal device after the third original image; There is a second dependency relationship between the third original image and the fourth original image; Before displaying the second preview image, the method further includes: In a first thread, the third original image is processed by using the first preset algorithm. The first thread is any thread in the terminal device except the main thread.

4. The method according to claim 2, characterized in that When the first exposure value is greater than or equal to a preset exposure parameter, the second preview image is displayed when the terminal device acquires the third original image; Before displaying the second preview image, the method further includes: Blocking processing is performed on the second thread; the second thread is the main thread in the terminal device; the blocking processing refers to pausing the first task in the second thread and processing the third original image using the first preset algorithm; wherein, after the processing of the third original image using the first preset algorithm is completed, the first task is continued to be executed until the processing of the first task is completed; the first task is the task in the second thread other than processing the third original image using the first preset algorithm.

5. The method according to claim 1, wherein The first preview interface includes controls for the first shooting mode and controls for the second shooting mode; the first preview interface is an interface in the first shooting mode; the method further includes: In response to a user clicking on a control for the second shooting mode, the first dependency is released and a third preview interface is displayed, the third preview interface including the first control; the second shooting mode is different from the first shooting mode; When receiving a user click on the first control, generating a third dependency relationship; the third dependency relationship is a dependency relationship between any two frames of original images acquired by the terminal device in the second shooting mode; Based on the third dependency, a fourth preview interface is displayed, where the fourth preview interface is an interface with a second preset algorithm enabled; the second preset algorithm is the same as or different from the first preset algorithm.

6. The method according to claim 5, characterized in that The second shooting mode is a video recording mode; the third preview interface further includes a video recording control; and the method further includes: In response to the user clicking the recording control, a recording interface is displayed; the recording interface includes an end recording control and a pause recording control; In response to a user clicking the end recording control or the pause recording control, a fifth preview interface is displayed; the fifth preview interface includes a third preview image, the third preview image is displayed when the terminal device acquires a fifth original image, the fifth original image being the image acquired by the terminal device when the recording is ended or paused; Before displaying the third preview image, the method further includes: Blocking processing is performed on the second thread; the second thread is the main thread in the terminal device; the blocking processing refers to pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; wherein, after the processing of the fifth original image using the first preset algorithm is completed, the second task continues to be executed until the processing of the second task is completed; the second task is the task in the second thread other than processing the fifth original image using the first preset algorithm.

7. The method according to any one of claims 1 to 6, characterized in that Displaying the first preview interface includes: In response to a user operation of starting the first preset algorithm, the first preview interface is displayed.

8. The method according to any one of claims 1 to 7, characterized in that After displaying the first preview interface, the method further includes: In response to a user operation of closing the first preset algorithm, the first dependency relationship is released.

9. The method according to any one of claims 1 to 8, characterized in that Before displaying the first preview interface, the method further includes: Obtain identification information; the identification information includes a first identification, a second identification, and a third identification; the first identification is used to indicate the dependency direction of the first original image; the second identification is used to indicate that the first original image depends on the second original image; the third identification is used to indicate that the first preset algorithm is in an enabled state; The first dependency relationship is generated based on the first identifier, the second identifier, and the third identifier.

10. The method according to any one of claims 1 to 9, characterized in that Before displaying the first preview image, the method further includes: In a third thread, the first original image is processed using a first preset algorithm; the third thread is any thread in the terminal device except the main thread.

11. The method according to any one of claims 1 to 10, characterized in that The first preset algorithm is any one of the following algorithms: beautification algorithm, blurring algorithm, sharpening algorithm, filter algorithm, sticker algorithm, watermark algorithm, neon algorithm and slimming algorithm.

12. A terminal device, characterized in that: The terminal device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the terminal device to execute the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions, and when the instructions are executed on a terminal device, the terminal device is caused to perform the method according to any one of claims 1 to 11.

14. A chip system, characterized in that: The chip system is applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to call computer instructions so that the terminal device executes the method as described in any one of claims 1 to 11.

Images