An image processing method, a terminal device, and a computer-readable storage medium

Abstract

The application provides an image processing method, a terminal device and a computer readable storage medium, and applies to the terminal technical field. The method comprises the following steps: starting a camera; acquiring a first original image; setting backward dependence for the first original image, that is, the first original image depends backward on a second original image; displaying a first preview interface based on the first original image, and the first preview interface comprises a first preview image. Thus, by setting backward dependence for a certain frame of original image (for example, the first original image), in the process of previewing the frame of original image, compared with the existing operation of performing multiple image copying operations, the first original image does not need to be copied from the first memory segment, the first preview image can be displayed in the case that a frame of original image (for example, the second original image) is acquired after the frame of original image, the power consumption of the terminal device can be saved, the phenomenon of preview picture lag can be reduced, and the preview processing time can be shortened.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and more specifically, to an image processing method, a terminal device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of terminal device technology, the functions of camera applications on terminal devices are also increasing. Camera applications can provide users with real-time visual feedback and perform corresponding preview processing on the images captured by the camera (such as beautification, blurring, watermarking, etc.). However, the time required for terminal devices to preview and process images captured by the camera is relatively long, resulting in increased power consumption of the terminal devices and consequently a poor user experience. Summary of the Invention

[0003] This application provides an image processing method, a terminal device, and a computer-readable storage medium, which can save power consumption of the terminal device, shorten the time for the terminal device to preview images, and improve the user experience.

[0004] In a first aspect, this application provides an image processing method applied to a terminal device, the method comprising:

[0005] Turn on the camera;

[0006] The first original image is acquired; the first original image is stored in the first memory segment;

[0007] Based on the first original image, a first preview interface is displayed, the first preview interface including the first preview image; the first preview image is an image after processing the first original image using a first preset algorithm; the first preview image is stored in a first memory segment;

[0008] The first original image and the second original image have a first dependency relationship; the first dependency relationship includes: the first preview image is displayed when the terminal device has acquired the first original image and the second original image; the second original image is any frame image acquired by the terminal device after the first original image; the second original image is stored in a second memory segment, and the second memory segment and the first memory segment are different cache spaces.

[0009] This application enables the acquisition of a first original image and a second original image (i.e., any frame image following the first original image) by activating the camera. Furthermore, based on the acquired first and second original images, a first dependency relationship between them can be determined, allowing the acquisition of the second original image without releasing the first original image. Even further, based on the first original image and the first dependency relationship, a first preview interface can be displayed, including a first preview image. This first preview image is a processed image of the first original image using a first preset algorithm, and it is displayed by the terminal device after acquiring the second original image.

[0010] By generating a dependency relationship (e.g., a first dependency relationship) between a certain original image frame (e.g., a first original image) and any image frame acquired after that original image frame (e.g., a second original image), it is possible to acquire any image frame (e.g., a second original image) after a certain original image frame during the preview processing of that original image frame, without copying or releasing the original image frame (e.g., a first original image) compared to existing operations that require multiple image copying. Furthermore, after acquiring any image frame (e.g., a second original image) after a certain original image frame, a preview image (e.g., a first preview image) processed from the original image frame (e.g., a first original image) can be output and displayed. This reduces preview stuttering, saves power consumption of the terminal device, shortens the time the terminal device spends processing the original image (e.g., a first preset algorithm) using a preset algorithm, and improves the user's preview experience.

[0011] In one possible implementation of the first aspect, the first preview interface further includes a camera control, and the method further includes:

[0012] In response to the user clicking the camera control, a third original image is acquired; the exposure value of the third original image is a first exposure value; based on the exposure value of the third original image, a second preview interface is displayed, which includes a second preview image; the second preview image is an image processed by a first preset algorithm on the third original image. Therefore, it is possible to determine whether the third original image is a variable exposure photograph by using its exposure value, and based on the determination result of whether the third original image is a variable exposure photograph, a second preview image corresponding to the third original image can be obtained through different processing methods.

[0013] In one possible implementation of the first aspect, when the first exposure value is less than the preset exposure parameter, the second preview image is displayed after the terminal device has acquired the third original image and the fourth original image; the fourth original image is any frame image acquired by the terminal device after the third original image; there is a second dependency relationship between the third original image and the fourth original image.

[0014] Before displaying the second preview image, the method further includes: processing the third original image using a first preset algorithm in a first thread, wherein the first thread is any thread in the terminal device other than the main thread.

[0015] In one possible implementation of the first aspect, if the first exposure value is greater than or equal to a preset exposure parameter, the second preview image is displayed after the terminal device has acquired the third original image;

[0016] Before displaying the second preview image, the method also includes:

[0017] The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process means pausing the first task in the second thread and using the first preset algorithm to process the third original image; after the processing of the third original image using the first preset algorithm is completed, the first task continues to be executed until the processing of the first task is completed; the first task is the task in the second thread other than the processing of the third original image using the first preset algorithm.

[0018] Based on the above description of displaying the second preview image under different exposure conditions, it can be seen that when the first exposure value is less than the preset exposure parameter, the third original image has image data with backward dependencies, i.e., the fourth original image. In this case, the second preview image is obtained based on the third original image and the second dependency relationship, and can only be displayed when the terminal device obtains both the third and fourth original images. However, when the first exposure value is greater than or equal to the preset exposure parameter, the third original image does not have image data with backward dependencies, and the processing of the third original image cannot be completed by setting backward dependencies. In this case, the second thread can be blocked, i.e., the first task in the second thread can be paused, so that the third original image is processed using the first preset algorithm in the second thread. After the processing of the third original image using the first preset algorithm is completed, the first task continues to be executed until the processing of the first task is completed. The second thread then returns the second preview image after processing the third original image, ensuring that the processing of the third original image can be completed when the third original image does not have the image data basis for setting backward dependencies, thus ensuring that the second preview image can be displayed.

[0019] In one possible implementation of the first aspect, the first preview interface includes controls for a first shooting mode and controls for a second shooting mode; the first preview interface is the interface in the first shooting mode; the method further includes:

[0020] In response to the user's click on the control for the second shooting mode, the first dependency is removed and a third preview interface is displayed. The third preview interface includes the first control. The second shooting mode is a different shooting mode from the first shooting mode. Upon receiving the user's click on the first control, a third dependency is generated. The third dependency is the dependency between any two original images acquired by the terminal device in the second shooting mode. Based on the third dependency, a fourth preview interface is displayed. The fourth preview interface is the interface with the second preset algorithm enabled. The fourth preview interface includes a fourth preview image, which is displayed after the terminal device has acquired the sixth and seventh original images. The second preset algorithm may be the same as or different from the first preset algorithm. Therefore, when switching from the first shooting mode to the second shooting mode, the first dependency in the first shooting mode can be removed, and the camera application can be quickly returned to, thus preparing for the generation of the third dependency in the second shooting mode and ensuring the smoothness and continuity of the preview screen in the second shooting mode.

[0021] In one possible implementation of the first aspect, the second shooting mode is a video recording mode; the third preview interface also includes video recording controls; the method further includes:

[0022] In response to the user clicking the recording control, a recording interface is displayed; the recording interface includes an end recording control and a pause recording control; in response to the user clicking the end recording control or the pause recording control, a fifth preview interface is displayed; the fifth preview interface includes a third preview image, which is displayed when the terminal device acquires the fifth original image, which is the image acquired by the terminal device when recording ends or pauses;

[0023] Before displaying the third preview image, the method also includes:

[0024] The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process means pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; after the processing of the fifth original image using the first preset algorithm is completed, the second task continues to be executed until the processing of the second task is completed; the second task is the task in the second thread other than the processing of the fifth original image using the first preset algorithm.

[0025] In summary, when the user clicks the end recording control or pause recording control, the obtained fifth original image lacks image data for setting backward dependencies, making it impossible to complete the processing of the fifth original image through setting backward dependencies. At this point, by blocking the second thread—that is, pausing the second task within the second thread—the fifth original image can be processed using the first preset algorithm within the second thread. After processing the fifth original image using the first preset algorithm is complete, the second task continues until it finishes. The second thread then returns the third preview image after processing the fifth original image. This ensures that even when the fifth original image lacks the image data foundation for setting backward dependencies, the processing of the fifth original image can be completed, ensuring the display of the third preview image.

[0026] In one possible implementation of the first aspect, displaying the first preview interface includes:

[0027] In response to the user's operation of activating the first preset algorithm, the first preview interface is displayed. Activating the first preset algorithm provides the algorithmic basis for obtaining the first preview image.

[0028] In one possible implementation of the first aspect, after displaying the first preview interface, the method further includes:

[0029] In response to the user's operation of closing the first preset algorithm, the first dependency is removed. By removing the first dependency, preparation can be made for the terminal device to generate other dependencies (such as the dependency between the second original image and a frame image obtained after the second original image).

[0030] In one possible implementation of the first aspect, the method further includes, before displaying the first preview interface:

[0031] Obtain identification information; the identification information includes a first identifier, a second identifier, and a third identifier; the first identifier is used to indicate the dependency direction of the first original image; the second identifier is used to indicate that the first original image depends on the second original image; the third identifier is used to indicate that the first preset algorithm is in the enabled state;

[0032] The first dependency relationship is generated based on the first identifier, the second identifier, and the third identifier.

[0033] Based on the above description of generating the first dependency relationship, by generating the first dependency relationship between the first original image and the second original image based on different identifier information, it can be ensured that the terminal device will only output and display the first preview image after processing the first original image using the first preset algorithm, based on the pre-set first dependency relationship and after acquiring the second original image. Therefore, there is no need to copy the first original image or release the first original image in order to acquire the second original image, which saves power consumption of the terminal device compared to the current operation that requires copying the original image.

[0034] In one possible implementation of the first aspect, the method further includes, before displaying the first preview image:

[0035] In the third thread, the first original image is processed using the first preset algorithm. The third thread is any thread in the terminal device other than the main thread. By starting a thread other than the main thread, i.e. the third thread, and processing the first original image using the first preset algorithm in the third thread, it is possible to ensure that the main thread and the third thread execute in parallel, thereby shortening the time to display the first preview image and improving the user's preview image experience.

[0036] In one possible implementation of the first aspect, the first preset algorithm is any one of the following algorithms: beautification algorithm, blurring algorithm, sharpening algorithm, filter algorithm, sticker algorithm, watermark algorithm, neon algorithm, and slimming algorithm. Based on different preset algorithms, the terminal device can perform different processing on the original image to obtain a variety of preview effects.

[0037] In a second aspect, this application provides a terminal device, including: one or more processors, and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the terminal device to execute the image processing method in the first aspect and any possible design of the first aspect.

[0038] Thirdly, this application provides a chip system applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to invoke computer instructions to cause the terminal device to execute the image processing method in the first aspect and any possible design of the first aspect.

[0039] The chip system may include one chip or multiple chips; when the chip system includes multiple chips, this application does not limit the type and number of chips.

[0040] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a terminal device, cause the terminal device to perform the image processing method of the first aspect and any possible design of the first aspect.

[0041] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to execute the image processing method in the first aspect and any possible design of the first aspect.

[0042] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram illustrating an application scenario of an image processing method provided in an embodiment of this application;

[0045] Figure 2 This is a schematic block diagram of the structure of a current terminal device;

[0046] Figure 3 This is a schematic diagram illustrating the current principle of how terminal devices preview and process raw images.

[0047] Figure 4 A flowchart illustrating an image processing method provided in one embodiment of this application;

[0048] Figure 5 A schematic diagram illustrating the interface display effect of an image processing method provided in an embodiment of this application;

[0049] Figure 6 A schematic block diagram of a terminal device provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram illustrating the principle of a terminal device performing preview processing on an original image, as provided in an embodiment of this application.

[0051] Figure 8 A flowchart illustrating an image processing method provided in one embodiment of this application;

[0052] Figure 9This is a schematic diagram of the display interface of a terminal device provided in an embodiment of this application;

[0053] Figure 10 This is a schematic diagram illustrating the principle of a terminal device performing preview processing on an original image, as provided in an embodiment of this application.

[0054] Figure 11 A flowchart illustrating another image processing method provided in an embodiment of this application;

[0055] Figure 12 This application provides a schematic diagram of the software system of a terminal device according to an embodiment of the present application;

[0056] Figure 13 An interactive diagram illustrating an image processing method provided in an embodiment of this application;

[0057] Figure 14 This is a schematic diagram of the hardware system of a terminal device provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0059] In the following text, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated, nor do they indicate a sequential order among the technical features. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0060] The image processing method provided in this application can be applied to scenarios where image preview processing is required. In some embodiments, the image processing method provided in this application can be applied to scenarios requiring image preview processing, such as shooting, recording, video calling, video conferencing, and screenshotting.

[0061] The image processing method provided in this application can be applied to terminal devices. The terminal device can be any device with display hardware and corresponding software support. For example, the terminal device can be a mobile phone, foldable screen, smart screen, tablet computer, wearable terminal device, in-vehicle terminal device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), home appliance, projector, etc. This application does not impose any restrictions on the specific type of terminal device.

[0062] The following example uses a photography scenario, combined with... Figure 1 This paper details an application scenario of the image processing method according to embodiments of this application.

[0063] Please see Figure 1 , Figure 1 The illustration shows an application scenario diagram of an image processing method provided in an embodiment of this application.

[0064] When terminal device 100 is unlocked, the desktop of terminal device 100 can display as follows: Figure 1 Interface 11 is shown in Figure (1) of the document. (See Figure 1) Figure 1 As shown in Figure (1), the terminal device 100 has multiple applications installed. These applications may include camera 101, contacts, phone, messages, settings, and photo album, etc.

[0065] like Figure 1 As shown in Figure (1), the terminal device 100 responds to the user's trigger operation on the camera 101 in the interface 11 (e.g., single click, multiple clicks, or long press), and displays as shown in Figure (1). Figure 1 The interface 12 is shown in Figure (2) of the document.

[0066] like Figure 1 As shown in Figure (2), the interface 12 displayed by the terminal device 100 is an internal interface of the camera 101. The interface 12 may include various shooting modes of the camera 101. The various shooting modes may include aperture mode, night scene mode, photo mode, portrait mode, video recording mode, and professional mode, etc. Among them, the portrait mode is a mode that adds preview algorithms (such as beautification algorithms and blurring algorithms) by default.

[0067] like Figure 1As shown in Figure (2), interface 12 is the interface displayed by terminal device 100 when it is in camera mode. Interface 12 may include viewfinder 102, control 103, control 104, control 105, and control 106. In response to the user's triggering operation on different controls in interface 12, terminal device 100 can perform different operations.

[0068] Optionally, the viewfinder 102 is used to display the image to be captured (e.g., ...). Figure 1 (2) The image of the person shown in Figure 101 or the video to be recorded. For example, in the photo mode of camera 101, the viewfinder 102 is used to display the image of the person to be photographed. In the video recording mode of camera 101, the viewfinder 102 is used to display the video of the person to be recorded.

[0069] Optionally, control 103 is used to trigger the terminal device 100 to activate the photo-taking mode in camera 101. In response to the user's triggering operation on control 103, the terminal device 100 can take an image in photo-taking mode.

[0070] Optionally, control 104 is used to trigger terminal device 100 to activate portrait mode in camera 101. In response to the user's triggering operation on control 104, terminal device 100 can capture an image in portrait mode.

[0071] Optionally, control 105 is used to display the captured image (e.g., Figure 1 (2) The figure shown in the figure) or the recorded video. In response to the user's trigger operation on the control 105, the terminal device 100 can display the captured image or the recorded video.

[0072] Optionally, control 106 is used to trigger the start of shooting. In response to the user's triggering operation on control 106, terminal device 100 can capture an image.

[0073] It should be noted that, as Figure 1 As shown in Figure (2), the interface 12 can also display the optical magnification of the lens in the terminal device 100. The optical magnification of a lens refers to the ability of the lens to magnify the observed object on the imaging sensor or film. The optical magnification of a lens is a fixed value, which is determined by the physical characteristics of the lens and cannot be changed by the user.

[0074] Optionally, the optical magnification of a lens can include the optical magnification of a short-focal-length lens and the optical magnification of a telephoto lens. For example... Figure 1 As shown in Figure (2), the optical magnification of the short focal length lens is 1, and the magnification of the long focal length lens is 3.

[0075] Considering that different shooting modes produce different image effects, users can manually switch between different shooting modes to obtain different image effects during actual use.

[0076] When a user needs to preview an image (e.g., apply beauty filters or blur effects), the system responds to the user's input, such as... Figure 1 In the interface 12 shown in Figure (2), slide to the right (as shown in Figure 12). Figure 1 According to the direction of the dashed arrow shown in Figure (2), the terminal device 100 can display interface 13. Interface 13 is the interface displayed when the terminal device 100 is in portrait mode. That is to say, at this time, the terminal device 100 has switched from photo mode to portrait mode.

[0077] It should be noted that the above-described method of switching shooting modes by the terminal device 100 is only an example and is not limited in this application embodiment. For example, when the terminal device 100 is in photo mode, in response to the user's trigger operation on the control 104 (e.g., click operation), the terminal device 100 can also switch to portrait mode.

[0078] like Figure 1 As shown in Figure (3), interface 13 may include control 107 and control 108. Optionally, control 107 is used to trigger the terminal device 100 to turn on or off the beautification algorithm in camera 101. Control 108 is used to trigger the terminal device 100 to turn on or off the blurring algorithm in camera 101.

[0079] Without beauty filters enabled, such as Figure 1 As shown in Figure (3), the image of the person displayed in viewfinder 102 is an image without beautification processing. In response to a user's trigger operation on control 107 (e.g., the user's first click on control 107), terminal device 100 can activate the beautification algorithm. At this time, terminal device 100 can display as shown in Figure (3). Figure 1 The interface 14 is shown in Figure (4) of the document.

[0080] like Figure 1 As shown in Figure (4), interface 14 may include control 109. Control 109 is used to prompt the user that the beauty algorithm is enabled. When the beauty algorithm is enabled, as shown in Figure (4), interface 14 may include control 109. Figure 1 As shown in Figure (4), the image of the person displayed in viewfinder 102 is an image that has been beautified.

[0081] Optionally, in some embodiments, if the user does not need to perform beautification processing on the image to be captured, the terminal device 100 may also turn off the beautification algorithm in response to the user's triggering operation on the control 107 (e.g., the user's second click on the control 107).

[0082] It should be noted that the above interface uses the beautification algorithm in the preview algorithm as an example for illustration, and the embodiments of this application are not limited to this. For example, if the user needs to blur the image, the user can also manually enable the blurring algorithm. The specific process of enabling the blurring algorithm is similar to the specific process of enabling the beautification algorithm, and can be referred to the above description of the beautification algorithm's enabling process, which will not be repeated here. In addition, besides the beautification algorithm and the blurring algorithm, the enabling and disabling processes of other algorithms in the preview algorithm (such as filter algorithms) can also be referred to the relevant description of the beautification algorithm, which will not be repeated here.

[0083] It should also be noted that the above interfaces are illustrated with an optical magnification of 1. The embodiments of this application are not limited to this. In practical applications, users can increase or decrease the optical magnification according to their actual shooting needs.

[0084] It should be understood that the interfaces and controls described above are merely illustrative examples and do not constitute a limitation on the interfaces in the embodiments of this application. In another embodiment of this application, based on different application scenarios, each interface may include more or fewer controls than those in the examples above, and the positions of each control may also be adjusted accordingly; this application does not impose specific limitations on this. For example, Figure 1 In Figure (3), controls 107 and 108 can be displayed not only at the bottom of the viewfinder 102, but also on the right or left side of the viewfinder 102. For example, Figure 1 The interface 13 shown in Figure (3) may also include controls corresponding to other preview algorithms (such as filter algorithms) besides beautification and blurring algorithms, so that users can turn the corresponding preview algorithm on or off through the controls.

[0085] It should also be understood that the above descriptions related to direction (e.g., right, left, and bottom) are based on Figure 1 The directions shown are described in the diagram. When the orientation of the terminal device 100 changes, the orientation and position of each control can be automatically adjusted accordingly. This application embodiment does not limit this.

[0086] The following is combined with Figure 2 and Figure 3 This paper introduces the specific implementation process of the current terminal device 100 for previewing original images.

[0087] like Figure 2 As shown, the terminal device 100 can perform preview processing of the original image through multiple software modules. For example... Figure 2As shown, the multiple software modules may include an acquisition module 10, a preview algorithm processing module 11, and a display module 12. The preview algorithm processing module 11 may include an algorithm node module 111 and a preview algorithm module 112.

[0088] Optionally, when the preview algorithm is a beautification algorithm, algorithm node module 111 is the algorithm node module corresponding to the beautification algorithm, and preview algorithm module 112 is the beautification algorithm module. It should be noted that different preview algorithms correspond to different algorithm node modules 111, and different preview algorithm modules 112 also correspond to different preview algorithms.

[0089] Terminal device 100 can acquire the original image through acquisition module 10. The original image is the image to be previewed without any image processing.

[0090] For example, the acquisition module 10 can be a camera (e.g., a front-facing camera or a rear-facing camera) installed on the terminal device 100. The camera converts light signals into electrical signals, enabling it to transmit the content to be captured as a digital image to the subsequent preview algorithm processing module 11, providing a data basis for subsequent preview processing.

[0091] After obtaining the original image, the preview algorithm processing module 11 can perform preview processing on the original image. Optionally, the preview algorithm processing module 11 can receive the original image through the algorithm node module 111. The algorithm node module 111 can transmit the received original image to the preview algorithm module 112, and the preview algorithm module 112 can perform preview processing on the original image to obtain a preview-processed image, which can provide image data to the display module 12.

[0092] It should be understood that the algorithm node module 111 is equivalent to a pipe for transmitting image data, but does not perform any transformation or analysis on the image data.

[0093] When the preview algorithm processing module 11 is a software module based on the Qualcomm platform software architecture, the preview algorithm processing module 11 can only perform preview processing on one frame of image at a time. That is to say, if the preview algorithm processing module 11 has not completed the preview processing of the current frame of image, the preview algorithm processing module 11 cannot receive the next frame of image of the current frame of image, nor can it perform parallel preview processing on the next frame of image of the current frame of image.

[0094] Terminal device 100 can use the preview algorithm module 112 in the preview algorithm processing module 11 to preview multiple original images (e.g., such as...) Figure 3Preview processing is performed on the images (F1, F2, and F3, etc.) shown. F1 is stored in memory segment 1, F2 in memory segment 2, and F3 in memory segment 3. The preview algorithm module 112 may include at least memory segment 4 and memory segment 5. Memory segment 4 is used to store the copied, unpreviewed original image (e.g., ...). Figure 3 Memory segment 5 (as shown in F1 or F2, etc.) is used to store the image after preview processing (e.g., ...). Figure 3 (as shown in F1' or F2').

[0095] Optionally, memory segment 1, memory segment 2, memory segment 3, memory segment 4, and memory segment 5 are five buffer spaces with different addresses for temporary storage of images. It should be noted that memory segment 1, memory segment 2, memory segment 3, memory segment 4, and memory segment 5 can be five different buffer spaces partitioned from the memory of the terminal device 100, or they can be five different hardware memories; this application does not limit them in this regard.

[0096] like Figure 3 As shown, after receiving F1 through the algorithm node module 111, the terminal device 100 can copy F1 from memory segment 1 to memory segment 4. At this time, the algorithm node module 111 can release F1 to provide an idle channel for receiving F2. The terminal device 100 performs preview processing on F1 in memory segment 4 through the preview algorithm module 112 to obtain the previewed F1'. Further, the terminal device 100 can copy F1' from memory segment 4 to memory segment 5 through the preview algorithm module 112, making memory segment 4 idle. Thus, when the algorithm node module 111 receives F2, the terminal device 100 can copy F2 from memory segment 2 to the idle memory segment 4 through the preview algorithm module 112 to perform preview processing on F2. In addition, the terminal device 100 can copy F1' from memory segment 5 to memory segment 1 through the preview algorithm module 112 to complete the preview processing of F1.

[0097] It should be noted that the specific implementation process of previewing each frame of the original image on the terminal device 100 is the same, such as... Figure 3 As shown, the specific implementation process of the terminal device 100 previewing F2 or F3 can be found in the above-described specific implementation process of the terminal device 100 previewing F1, and will not be repeated here.

[0098] It should also be noted that the above description uses memory segment 4 and memory segment 5 as examples of the preview algorithm module 112, and the embodiments of this application are not limited to this.

[0099] Optionally, the preview algorithm module 112 may also include two or more memory segments to enable copying different original images. For example, the preview algorithm module 112 may include memory segments 4, 5, 6, and 7. When previewing F1 using the preview algorithm module 112, the terminal device 100 completes the corresponding copying operation using memory segments 4 and 5. When previewing F2 using the preview algorithm module 112, the terminal device 100 completes the corresponding copying operation using memory segments 6 and 7. By setting the preview algorithm module 112 to include multiple memory segments, the terminal device 100 can avoid the preview algorithm module 112 being unable to preview F2 when memory segments 4 and 5 are occupied, thus preventing stuttering or flickering in the preview image.

[0100] Currently, the terminal device 100 ensures that the algorithm node module 111 can continuously receive original images (e.g., continuously receive F2 and F3 after F1) by copying the original image (e.g., F1) and performing preview processing on the copied original image through the preview algorithm module 112, thus reducing the occurrence of image preview stuttering. By copying F1 for preview processing, the algorithm node module 111 can release F1 while the terminal device 100 is performing preview processing on F1 through the preview algorithm module 112. After releasing F1, the algorithm node module 111 is in an idle state, at which point it can receive F2 and perform preview processing on F2 through the preview algorithm module 112.

[0101] After obtaining the previewed image (e.g., ... Figure 3 In the case of F1' or F2' shown, the terminal device 100 can use the preview algorithm processing module 11 (or the preview algorithm module 112 in the preview algorithm processing module 11) to preview the image after preview processing (e.g., as shown). Figure 3 The image (F1' or F2' shown) is transmitted to the display module 12. This allows the display module 12 to display the previewed image (e.g., as shown in the image) to the user. Figure 3 (as shown in F1' or F2'). For example, when the preview algorithm is a beautification algorithm, the image after preview processing can be as follows: Figure 1 The beautified image of a person is shown in viewfinder 102 as shown in Figure (4).

[0102] However, when the terminal device 100 performs preview processing on the original image, it needs to perform multiple copy operations (e.g., copy F1, copy F1', copy F2, and copy F2', etc.), which will increase the power consumption of the terminal device 100. Especially when the preview processing time for each frame of the original image is relatively long (e.g., 30ms), multiple copy operations will also make the terminal device 100 take longer to preview the original image, resulting in a longer waiting time for the user and a poor user experience.

[0103] In view of this, embodiments of this application propose an image processing method that, by setting a backward frame dependency on the acquired current frame original image (e.g., a first original image) (e.g., a first dependency relationship exists between the first original image and a second original image, and the second original image is any frame image acquired after the first original image), ensures that the terminal device will not release the current frame original image or output a preview image (e.g., a first preview image) after preview processing of the current frame original image before acquiring the dependent image frame (e.g., the second original image). Therefore, by generating a dependency relationship (e.g., a first dependency relationship) between the original image of the current frame and any frame image obtained after the original image of the current frame, it is possible to obtain any frame image (e.g., a second original image) after the original image of the current frame (e.g., a first original image) without copying or releasing the original image of the current frame (e.g., a first original image) during the preview processing of the original image of the current frame (e.g., a first original image) compared to the existing operation that requires multiple image copying (e.g., copying the original image of the current frame and copying the preview image after preview processing of the original image of the current frame). Furthermore, when any frame image (e.g., a second original image) after the original image of the current frame is obtained, the preview image after preview processing of the original image of the current frame (e.g., a first preview image) is displayed.

[0104] Therefore, in the image processing method provided in this application embodiment, the terminal device generates a dependency relationship (e.g., a first dependency relationship) between the original image of the current frame and any frame image obtained after the original image of the current frame. In the process of previewing the original image of the frame, compared with the existing operation that requires multiple image copying, there is no need to perform multiple image copying operations (e.g., copying the original image of the current frame and the image after previewing the original image of the current frame). This can save the power consumption of the terminal device, reduce the lag phenomenon of the preview screen, shorten the time for previewing the original image, and improve the user's preview image experience.

[0105] The above text combined Figure 1 This application describes an application scenario based on an embodiment of the present application, and combines it with... Figure 2 and Figure 3 This paper introduces the current implementation process of previewing raw images on terminal devices. The following section combines... Figures 4 to 7 This application provides a detailed description of the specific implementation process of an image processing method provided in its embodiments.

[0106] Please see Figure 4 , Figure 4 A flowchart of an image processing method provided in an embodiment of this application is shown.

[0107] like Figure 4 As shown, the image processing method in this embodiment includes the following steps:

[0108] S101, turn on the camera.

[0109] Optionally, in some embodiments, the camera is used to acquire raw images. The camera can be a front-facing camera or a rear-facing camera of the terminal device.

[0110] It should be noted that in practical applications, the terminal device can determine whether to activate the front-facing camera or the rear-facing camera based on the actual usage scenario, and this application embodiment does not limit this. For example, in a shooting or recording scenario, the terminal device defaults to activating the rear-facing camera. In this case, in response to the user's camera switching operation, the terminal device can switch from the rear-facing camera to the front-facing camera. As another example, in a video call or video conferencing scenario, the terminal device defaults to activating the front-facing camera. In this case, in response to the user's camera switching operation, the terminal device can switch from the front-facing camera to the rear-facing camera.

[0111] Optionally, in some embodiments, the terminal device responds to the user's action of launching a camera application (e.g., ...). Figure 1 The user's trigger operation on camera 101 shown in Figure (1) can start the camera.

[0112] It should be noted that the above-described methods for activating the camera are merely illustrative examples, and the embodiments of this application are not limited thereto. Those skilled in the art can activate the camera in other ways. For example, the terminal device can activate the camera by receiving a voice command input by the user. Or, the terminal device can activate the camera in response to a user's operation of calling the photo function in a third-party application (such as WeChat, Weibo, Taobao, or Meitu Xiu Xiu).

[0113] S102, Obtain the first raw image.

[0114] Optionally, the first original image is a frame of image acquired by the terminal device through the camera without undergoing image algorithm processing.

[0115] Optionally, the first original image is stored in a first memory segment. The first memory segment is a cache space for temporarily storing the first original image. For example, the first original image can be as follows: Figure 3 As shown in the F1 example, at this time, the first memory segment is as follows: Figure 3 The memory segment shown is 1.

[0116] It should be noted that the first memory segment can be a cache space allocated in the memory of the terminal device 100, or it can be a hardware memory; this application does not limit it in this regard.

[0117] After executing S101, the terminal device can acquire a first raw image through the camera. By acquiring the first raw image, the terminal device can provide a data basis for subsequent preview processing.

[0118] It should be noted that the terminal device may include a variety of shooting modes (such as aperture mode, night scene mode, photo mode, portrait mode, video mode, and professional mode). In different shooting modes, the first original image obtained by the terminal device through the camera may be the same or different. This application embodiment does not limit this.

[0119] S103, based on the first original image, display a first preview interface, the first preview interface including the first preview image.

[0120] Optionally, the first preview interface is an interface on the terminal device that displays the first preview image. The first preview interface may also include multiple controls. The terminal device, in response to user actions on different controls in the first preview interface, can perform corresponding processing. For example, the first preview interface may be as follows: Figure 1 The interface 14 is shown in Figure (4) of the document.

[0121] Optionally, the first preview image is an image obtained by processing the first original image using a first preset algorithm. For example, the first preview interface is as follows: Figure 1 In the case of interface 14 shown in Figure (4), the first preview image is as follows: Figure 1 The image of the person shown in viewfinder 102 as shown in Figure (4) is a person image.

[0122] Optionally, the first preview image is stored in a first memory segment. The first memory segment is a cache space for temporarily storing the first preview image.

[0123] It should be noted that after obtaining the first preview image, the terminal device can delete the first original image already stored in the first memory segment and store the first preview image. Alternatively, after obtaining the first preview image, the terminal device can also directly store the first preview image in the first memory segment. In this case, the first memory segment stores both the first original image and the first preview image.

[0124] Optionally, the first preset algorithm is any one of a plurality of preset algorithms (equivalent to the preview algorithm mentioned above) pre-set in the terminal device. The plurality of preset algorithms may include: beautification algorithm, blurring algorithm, sharpening algorithm, filter algorithm, sticker algorithm, watermark algorithm, neon algorithm, and slimming algorithm, etc.

[0125] For example, the first preset algorithm may be similar to... Figure 1 The beautification algorithm corresponding to control 107 shown in Figure (3) is similar to that in other figures. Figure 1 The blurring algorithm corresponding to control 108 shown in Figure (3) is as follows. When the first preset algorithm is a beautification algorithm, the first preview image is the image after beautifying the first original image using the beautification algorithm (e.g., ...). Figure 1 The image of the person shown in viewfinder 102 as shown in Figure (4).

[0126] It should be noted that multiple preset algorithms in the terminal device can have corresponding controls set in the camera 101, allowing users to enable or disable the corresponding preset algorithms through these controls. Those skilled in the art can, according to actual product design requirements, set controls corresponding to at least one preset algorithm in at least one shooting mode of the camera 101. This application embodiment does not limit this. For example, in one shooting mode of the camera 101, controls corresponding to each of the multiple preset algorithms can be set. Or, for example, in one shooting mode of the camera 101 (e.g., as shown in the image),... Figure 1 (3) The portrait mode shown in the figure) sets up controls corresponding to a few preset algorithms among multiple preset algorithms (for example, such as Figure 1 (See Figure (3) for the control 107 corresponding to the beautification algorithm and the control 108 corresponding to the blurring algorithm). For example, in each shooting mode of the camera 101, a control corresponding to each of the multiple preset algorithms is set.

[0127] It should also be noted that the multiple preset algorithms in the terminal device may not require corresponding controls to be set in the camera 101. In practical applications, the terminal device may automatically activate one or more of the multiple preset algorithms according to the actual needs of capturing images or recording videos. This application embodiment does not limit this.

[0128] Optionally, there is a first dependency relationship between the first original image and the second original image. The second original image is any frame image acquired by the terminal device after the first original image.

[0129] Optionally, the second original image is stored in a second memory segment. The second memory segment is a cache space for temporarily storing the second original image. The second memory segment and the first memory segment are different cache spaces. For example, the storage address of the first memory segment is different from the storage address of the second memory segment.

[0130] For example, the second original image can be as follows: Figure 3 As shown in F2 or F3. The second original image is as follows: Figure 3 When F2 is shown, the first memory segment is as follows: Figure 3 Memory segment 2 as shown; or, in the second original image as shown Figure 3 When F3 is activated, the first memory segment is as shown. Figure 3 Memory segment 3 is shown.

[0131] It should be noted that the second memory segment can be a cache space allocated in the memory of the terminal device 100, or it can be a hardware memory; this application does not limit this.

[0132] Optionally, the first dependency relationship includes: the first preview image is displayed only after the terminal device has acquired the second original image. In other words, the terminal device will only output and display the first preview image if it has acquired the second original image.

[0133] After acquiring the first original image, the terminal device can continue to acquire a second original image via its camera. Acquiring the second original image provides the data foundation for backward dependency on the first original image and prepares for the subsequent output and display of the first preview image. Backward dependency means that the output of the processing result of the current frame image (e.g., the first original image) depends on the input of the next frame image (e.g., the second original image). By setting backward dependency, the terminal device can display the first preview image processed by the first preset algorithm without performing multiple image copying operations. This saves power consumption while ensuring good smoothness and continuity in the image preview.

[0134] Before displaying the first preview image, the terminal device also needs to generate a first dependency relationship between the first original image and the second original image to provide the dependency conditions for the terminal device to display the first preview image. The following describes one possible implementation method for the terminal device to generate the first dependency relationship.

[0135] Optionally, in some embodiments, after acquiring the first original image, the terminal device may acquire identification information.

[0136] Optionally, the identification information is information pre-stored in the terminal device. The identification information may include a first identifier, a second identifier, and a third identifier.

[0137] Optionally, the first identifier is used to indicate the dependency direction of the first original image. For example, the first identifier can be a "+" symbol or a "-" symbol. A "+" symbol indicates that the dependency direction of the first original image is forward dependency. A "-" symbol indicates that the dependency direction of the first original image is backward dependency.

[0138] Optionally, the second identifier is used to indicate that the first original image depends on the second original image. For example, the second identifier can be the absolute value of the difference between the frame number of the first original image (or the frame number of the dependent frame) and the frame number of the second original image (or the frame number of the dependent frame).

[0139] Based on the aforementioned descriptions of the first and second identifiers, when the frame number of the first original image is 1 and the frame number of the second original image is 2, the absolute value of the difference between the frame numbers of the first and second original images is 1. In this case, the identifier of the first original image being dependent on the second original image is "-1". When the frame number of the first original image is 1 and the frame number of the second original image is 3, the absolute value of the difference between the frame numbers of the first and second original images is 2. In this case, the identifier of the first original image being dependent on the second original image is "-2". Compared to the first original image being dependent on the second original image with frame number 3, the first original image being dependent on the second original image with frame number 2 allows for faster output and display of the first preview image after processing the first original image, reducing user waiting time and enhancing user experience.

[0140] Alternatively, in another embodiment, the second identifier may also be the time difference between the time of the acquired dependent frame (e.g., the first original image) and the time of the dependent frame (e.g., the second original image), and this application embodiment does not limit this.

[0141] Optionally, the third identifier is used to indicate that the first preset algorithm is enabled. For example, the third identifier can be a string.

[0142] After obtaining the identification information, the terminal device can generate a first dependency relationship based on the first identifier, second identifier, and third identifier in the identification information. By generating the first dependency relationship, the terminal device can ensure that it will not release the first original image or output and display the first preview image after processing the first original image using the first preset algorithm if it has not obtained the second original image. This provides the necessary dependency conditions for the terminal device to subsequently display the first preview image.

[0143] Optionally, in some embodiments, the terminal device can display a first preview interface in response to a user's operation of activating a first preset algorithm. By activating the first preset algorithm, the terminal device can provide an algorithmic basis for successfully obtaining and displaying the first preview image.

[0144] For example, the terminal device responds to the user's first click as follows: Figure 1 The operation of control 107 in interface 13 shown in Figure (3) can activate the beautification algorithm corresponding to control 107 and display as shown in Figure (3). Figure 1 The interface 14 is shown in Figure (4) of the document.

[0145] It should be noted that the above-mentioned terminal device responding to the user's click on the relevant control and activating the first preset algorithm is only an illustrative example, and the embodiments of this application are not limited thereto.

[0146] In practical applications, terminal devices can automatically activate a first preset algorithm based on the specific usage scenario. For example, in a video call scenario, in response to the user's operation to connect to the video call, terminal device 100 can automatically activate the first preset algorithm (such as a beautification algorithm) while activating the front-facing camera. By automatically activating the first preset algorithm, the terminal device can shorten the time spent previewing the original image in video call scenarios, thereby improving the user's experience of previewing images in video call scenarios. Figure 5 As shown, after a user connects to a video call, the terminal device 100 can display the following: Figure 5 The interface shown is A. Interface A includes a person image processed by a beautification algorithm. At this time, the first preview interface is interface A, and the first preview image is the person image in interface A processed by the beautification algorithm.

[0147] It should be noted that, as Figure 5 The interface A shown may also include controls for ending a video call, converting a video call to an audio call, and switching between the front and rear cameras. In actual video call scenarios, the terminal device 100 responds to user actions on different controls in interface A and can perform processing corresponding to each control; this embodiment does not limit this.

[0148] Optionally, in some embodiments, before displaying the first preview image, the terminal device may start a third thread and process the first original image using a first preset algorithm in the third thread.

[0149] Optionally, a thread refers to the smallest unit of execution of a program. The third thread is any thread in the terminal device other than the main thread (i.e., the second thread below).

[0150] Based on the above description, by starting a third thread and processing the first original image using a first preset algorithm within that third thread, the terminal device can simultaneously execute the preview processing of the first original image (e.g., preview processing of the first original image via a third thread outside the main thread) and the acquisition of the second original image (e.g., acquiring the second original image via the main thread) while performing tasks in the main thread (e.g., generating the first dependency relationship). This allows the terminal device to output and display the first preview image after acquiring the second original image. Compared to the method of completing the preview processing of the first original image in the main thread (i.e., the second thread below), this eliminates the need to wait for other tasks in the main thread (i.e., tasks other than preview processing of the first original image) to finish before outputting and displaying the first preview image, resulting in faster display speed and a better user experience.

[0151] It should be noted that there is no logical connection between the terminal device processing the first original image using the first preset algorithm and the terminal device acquiring the second image. In other words, after acquiring the first original image, the terminal device can process the first original image using the first preset algorithm without waiting for the acquisition of the second original image.

[0152] Optionally, in some embodiments, when the terminal device includes multiple software modules, the image processing method of this application embodiment can be implemented by multiple software modules. For example... Figure 6 As shown, the terminal device 100 may include an acquisition module 20, a preset algorithm processing module 21, and a display module 22. The preset algorithm processing module 21 may include an algorithm node module 211, a preset algorithm module 212, and a management module 213. The preset algorithm module 212 does not include any memory segment. The management module 213 controls the input of the original image to the algorithm node module 211, controls the preset algorithm module 212 to process the original image, and controls the display module 22 to display a preview image after processing the original image. The management module 213 is also used to generate or remove dependencies between any two frames of original images (e.g., a first dependency between a first original image and a second original image).

[0153] Optionally, when the preset algorithm is a first preset algorithm (e.g., a beautification algorithm), the algorithm node module 211 is the algorithm node module corresponding to the first preset algorithm, and the preset algorithm module 212 is the first preset algorithm module.

[0154] It should be noted that different preset algorithms correspond to different algorithm node modules 211, and different preset algorithm modules 212. By setting different algorithm node modules and different preset algorithm modules for different preset algorithms, the terminal device 100 can ensure that when multiple preset algorithms are included in the terminal device 100, the multiple preset algorithms are decoupled, that is, the processing of multiple preset algorithms will not affect each other, which facilitates the management and maintenance of the software programs of multiple preset algorithms.

[0155] like Figure 6 As shown, the terminal device 100 can acquire the original image (e.g., a first original image, a second original image, etc.) through the acquisition module 20. The acquisition module 20 can then transmit the original image to the preset algorithm processing module 21 for processing.

[0156] It should be noted that, Figure 6 The acquisition module 20 shown above can be found in the text above. Figure 2 The description of the acquisition module 10 shown is not repeated here.

[0157] After obtaining the original image, the terminal device 100 can process the original image using a preset algorithm (e.g., the first preset algorithm) in the preset algorithm processing module 21.

[0158] It should be noted that, Figure 6 The example described uses the independent configuration of algorithm node module 211 and preset algorithm module 212. However, the embodiments in this application are not limited to this. For example, Figure 6 The algorithm node module 211 and the preset algorithm module 212 shown can also be integrated into a software module to process the original image.

[0159] like Figure 7 As shown, the terminal device 100 processes multiple original images (e.g., such as...) through the preset algorithm processing module 21. Figure 7 The specific implementation process for previewing F1, F2, and F3 (as shown) includes: After receiving F1 through the algorithm node module 211, the terminal device 100 can start a third thread, in which it processes F1 using the preset algorithm in the preset algorithm module 212. Simultaneously, the terminal device 100 can generate the dependency relationship of F1 received by the algorithm node module 211 through the management module 213. For example, when F1 is the first original image, the dependency relationship of F1 is the first dependency relationship. Figure 7 As shown, the dependency relationship of F1 generated by the management module 213 is the dependency relationship between F1 and F2. That is to say, F1' is output and displayed by the terminal device 100 when the algorithm node module 211 receives F2.

[0160] It should be noted that the dependency relationship of F1 generated by the management module 213 can be generated in the main thread of the terminal device, i.e., the second thread, or it can be generated in other threads started by the terminal device besides the second and third threads. This application embodiment does not limit this.

[0161] like Figure 7 As shown, when the algorithm node module 211 receives F2, the terminal device 100 performs non-blocking processing on the second thread. Non-blocking processing means that the second thread continues to execute the corresponding task (e.g., generating the dependency relationship of F2). The terminal device 100 can start a fourth thread in addition to the main thread and the third thread. In the fourth thread, the preset algorithm in the preset algorithm module 212 is used to preview the processing of F2. At the same time, the terminal device 100 can notify the management module 213 through the preset algorithm module 212 that the processing of F1 has been completed.

[0162] It should be noted that the preset algorithm module 212 can directly send a notification signal to the management module 213 to notify the management module 213 that F1 has completed processing. Alternatively, the preset algorithm module 212 can also send a notification signal to the management module 213 through the algorithm node module 211 to notify the management module 213 that F1 has completed processing (e.g., ...). Figure 7 As shown in the embodiments, this application does not limit this.

[0163] It should also be noted that the aforementioned non-blocking processing enables the terminal device 100 to initiate a thread processing algorithm (e.g., a third thread) when it receives each frame of original image through the algorithm node module 211. The original image frame (e.g., the first original image) can then be directly returned to the algorithm node module 211, where the management module 213 controls the subsequent operations performed on that frame (e.g., in the main thread). Furthermore, the processing result (e.g., a first preview image) of the next frame (e.g., the second original image) is returned. Figure 7 As shown, when the algorithm node module 211 receives F2, it returns the processing result F1' of F1. This multi-threaded parallel processing (e.g., the main thread and a third thread processing in parallel) shortens the processing time of the original image on the terminal device, improving the user's image preview experience.

[0164] Once the management module 213 determines that F1 has been processed, it can remove the dependency of F1, generate the dependency of F2, and continue to perform subsequent operations on the processed F1'. By removing the dependency of F1, the management module 213 can ensure that it can successfully continue to perform subsequent operations on the processed F1'.

[0165] It should be noted that F1 is stored in memory segment 1. Therefore, after obtaining F1', the management module 213 can overwrite the F1 already stored in memory segment 1 with F1'. That is, the management module 213 can delete the F1 already stored in memory segment 1 and store F1'. This allows the management module 213 to continue performing subsequent operations based on the F1' stored in memory segment 1.

[0166] Optionally, in some embodiments, when the terminal device includes a preset algorithm (e.g., a first preset algorithm) and the preset algorithm is enabled, the management module 213 continues to perform subsequent operations on the processed F1' by inputting F1' into the display module 22 for display.

[0167] Optionally, in another embodiment, when the terminal device includes multiple preset algorithms (e.g., a first preset algorithm and a second preset algorithm), and all preset algorithms are enabled, the management module 213 continues to perform subsequent operations on the processed F1' by inputting F1' into the algorithm node module and preset algorithm module corresponding to other preset algorithms (e.g., the second preset algorithm) besides the current preset algorithm (e.g., the first preset algorithm) for corresponding processing, until the processing corresponding to each of the multiple preset algorithms is completed for F1, and then the image after multiple processing of F1 is displayed by the display module 22.

[0168] It should be noted that when the preset algorithm is enabled, the specific processing of F2 and F3 by the terminal device 100 is similar to that of F1. Please refer to the processing of F1 by the terminal device 100 above, which will not be repeated here.

[0169] It should also be noted that, such as Figure 7 As shown, when the terminal device starts the third thread and processes F1 in the third thread, the terminal device 100 can receive F2 through the algorithm node module 211, start the fourth thread, and process F2 in the fourth thread. While the terminal device is processing F2 in the fourth thread, the terminal device 100 can receive F3 through the algorithm node module 211. At this time, the terminal device can start the fifth thread and process F3 in the fifth thread. By starting different threads to process different original images, the terminal device 100 can ensure that the processing of different original images will not conflict, thereby speeding up the display of preview images. In special cases, the terminal device can also start the same thread and sequentially perform preview processing of F1, F2, and F3 in the same thread. For example, if the preview processing of F1 in the third thread has finished, and the terminal device 100 receives F2 through the algorithm node module 211, the terminal device 100 can start the third thread and perform preview processing of F2 in the third thread.

[0170] By comparison Figure 7 and Figure 3 As can be seen, the image processing method of this application embodiment, when previewing a certain frame of original image (e.g., F1), sets a backward dependency on the certain frame of original image (e.g., F1 depends on F2), so that the terminal device does not release the original frame of original image during the previewing process. Furthermore, even without releasing the original frame of original image, the image frame that the original frame depends on (e.g., F2) can still be obtained. Further, when the image frame that the original frame depends on is obtained, the image after previewing the original frame of original image (e.g., F1') is output and displayed. Therefore, it is not necessary to copy F1 from memory segment 1 and then process F1, compared to the existing method that requires multiple image copying (e.g., ...). Figure 3 The operations shown (such as copy F1 and copy F1') can save power consumption of terminal devices, shorten the time for processing the original image, reduce the phenomenon of lag in the preview screen, and thus improve the user's preview experience.

[0171] In summary, when the first preset algorithm is enabled, the terminal device can display a first preview interface based on the obtained first original image, second original image, and first dependency relationship. The first preview interface includes a first preview image after processing the first original image using the first preset algorithm. This ensures that even when the terminal device processes the first original image using the first preset algorithm, it can still obtain the second original image based on the first dependency relationship, and output and display the first preview image.

[0172] If previewing the original image is not required, the user can manually turn off the first preset algorithm. Turning off the first preset algorithm can save power consumption of the terminal device.

[0173] Optionally, after displaying the first preview interface, the terminal device responds to the user's operation of closing the first preset algorithm (e.g., the user clicks again...). Figure 1 The operation of control 107 in interface 13 shown in Figure (3) can remove the first dependency relationship.

[0174] Upon acquiring the second original image, the terminal device disables the first preset algorithm. That is, the first original image acquired before the second original image satisfies a backward dependency, and the terminal device can successfully display the first preview image upon acquiring the second original image. At this point, the first dependency relationship between the first and second original images generated by the terminal device has been fulfilled. Therefore, the terminal device can release the first dependency relationship, preparing for the generation of other dependencies (e.g., the dependency relationship between the second original image and a frame acquired after the second original image).

[0175] Considering that the second original image is the last original image acquired by the terminal device before the first preset algorithm is turned off, the terminal device cannot acquire the original images following the second original image after the first preset algorithm is turned off. In other words, the second original image does not have the image data basis for setting backward dependencies. Therefore, the terminal device cannot set backward dependencies on the second original image, that is, it cannot generate a dependency relationship between the second original image and the image acquired after the second original image.

[0176] Furthermore, since the terminal device needs to go through a series of processes from acquiring the second original image to displaying the preview image after processing the second original image using the first preset algorithm, when the terminal device closes the first preset algorithm, regardless of whether the second original image is being processed, the terminal device will automatically discard the second original image, that is, it will not display the preview image processed by the first preset algorithm. Therefore, the terminal device does not need to set backward dependency on the second original image.

[0177] It should be noted that the specific implementation process of the above image processing method is illustrated by taking the terminal device using the first preset algorithm to process the original image as an example. The implementation principle of the terminal device using each of the multiple preset algorithms to process the original image and display the corresponding preview image is the same. Please refer to the above description of the specific implementation process of the terminal device processing the original image using the first preset algorithm and displaying the corresponding preview image. It will not be repeated here.

[0178] The image processing method provided in this application embodiment, by activating a camera, can acquire a first original image and a second original image (i.e., any frame image after the first original image) through the camera. Further, based on the acquired first and second original images, a first dependency relationship between the first and second original images can be determined, thereby enabling the acquisition of the second original image without releasing the first original image. Even further, based on the first original image and the first dependency relationship, a first preview interface can be displayed, and the first preview interface includes a first preview image, wherein the first preview image is an image processed using a first preset algorithm, and the first preview image is displayed by the terminal device after acquiring the second original image. Thus, by generating a dependency relationship (e.g., the first dependency relationship) between a certain frame of original image (e.g., the first original image) and any frame image acquired after that frame (e.g., the second original image), it is possible to achieve a faster preview processing of a certain frame of original image compared to existing methods that require multiple image copies (e.g., ...). Figure 3 Regarding the operations of copying F1 and copying F2, etc., it is possible to obtain any frame of image (e.g., the second original image) after a given original image without copying or releasing it. Furthermore, when any frame of image (e.g., the second original image) after a given original image is obtained, a preview image (e.g., the first preview image) after processing the given original image (e.g., the first original image) can be output and displayed. This reduces lag in the preview, saves power consumption of the terminal device, shortens the time the terminal device spends processing the original image (e.g., the first original image) using a preset algorithm (e.g., the first preset algorithm), and improves the user's preview experience.

[0179] In addition, especially when the preview processing time for each original image frame is relatively long (e.g., 30ms), by dynamically setting the backward dependency of each original image frame, it can be ensured to a certain extent that the preview of the image on the terminal device will not be stuttered, without having to perform the image copying operation multiple times. This can save the power consumption of the terminal device to a greater extent and further shorten the time for the terminal device to process each original image frame using a preset algorithm (e.g., the first preset algorithm).

[0180] Considering that the frame dependency in the image processing method of this application embodiment is set backward (for example, the first dependency relationship mentioned above, i.e., the first original image depends on the second original image), if the terminal device does not obtain the dependent frame (for example, the second original image), it is impossible to generate the dependency relationship between the two original images (for example, the first dependency relationship), which will cause the terminal device to freeze during preview. Therefore, under some edge conditions (for example, switching shooting modes) and discontinuous preview frames (for example, taking a photo with varying exposure, ending recording, or pausing recording), the terminal device can determine whether to generate the dependency relationship between the two original images, or whether to remove the dependency relationship between the two original images, based on the actual situation, thereby completing the process of dynamically setting the dependency relationship between the two original images.

[0181] The following is combined with Figure 8 and Figure 9 This paper details the specific implementation process of an image processing method in the embodiments of this application.

[0182] Optionally, the first preview interface may include controls for a first shooting mode and controls for a second shooting mode. The first preview interface is the interface in the first shooting mode. For example, as shown... Figure 9 As shown in Figure (1), when the first shooting mode is portrait mode, the first preview interface can be as follows: Figure 9 In the portrait mode interface 14 shown in Figure (1), the control for the first shooting mode can be control 104, and the control for the second shooting mode can be control 113.

[0183] Optionally, in some embodiments, the first shooting mode is any shooting mode preset in the terminal device. That is, the first shooting mode can be aperture mode, night scene mode, photo mode, portrait mode, video mode, or professional mode.

[0184] Optionally, the second shooting mode is a different shooting mode from the first shooting mode. For example, such as... Figure 9 As shown in Figure (1), the first shooting mode is portrait mode and the second shooting mode is video mode.

[0185] like Figure 8 As shown, the image processing method in this embodiment of the application specifically includes the following steps:

[0186] S201, in response to the user's click on the control of the second shooting mode, removes the first dependency and displays the third preview interface.

[0187] Optionally, the third preview interface is the display interface in the second shooting mode. The third preview interface includes a first control. The first control is used to trigger the terminal device to enable or disable the second preset algorithm. For example, the third preview interface can be as follows: Figure 9 In the recording mode interface 15 shown in Figure (2), the first control can be as follows: Figure 9 In the interface 15 shown in Figure (2), when the second preset algorithm is the beautification algorithm, the first control is control 107, and when the second preset algorithm is the blurring algorithm, the first control is control 108.

[0188] It should be noted that the second preset algorithm and the first preset algorithm can be the same preset algorithm; for example, both the first and second preset algorithms can be beautification algorithms. Alternatively, the second preset algorithm and the first preset algorithm can be different preset algorithms; for example, the first preset algorithm is a beautification algorithm, and the second preset algorithm is a blurring algorithm. Optionally, in some embodiments, the first dependency relationship is the dependency relationship between the first original image and the second original image in the first shooting mode, corresponding to the first original image and the second original image being images acquired by the terminal device in the first shooting mode.

[0189] It should be noted that in practical applications, terminal devices typically set the default shooting mode after activating the camera (e.g., the first shooting mode) to photo mode. Those skilled in the art can also set the shooting mode after activating the camera to other shooting modes besides photo mode (e.g., portrait mode) according to actual usage needs; this application does not limit this.

[0190] It should also be noted that the above-described switching between different shooting modes by the user clicking the corresponding shooting mode control is only an illustrative example, and the embodiments of this application are not limited thereto. In practical applications, the terminal device can also switch between different shooting modes in response to the user's swipe operation. For example, the terminal device can switch from portrait mode to video recording mode in response to the user's continuous swipe to the right.

[0191] Optionally, after the terminal device executes S103, the user can determine whether to switch shooting modes based on current shooting needs. If the user needs to switch shooting modes, the user can click the control for the second shooting mode in the first preview interface (e.g., ...). Figure 9As shown in Figure (1), the user clicks control 113 in interface 14. In response to the user's click on the control for the second shooting mode, the terminal device can release the first dependency relationship in the first shooting mode. By releasing the first dependency relationship in the first shooting mode and quickly returning to the camera application, the terminal device can prepare for the normal generation of the dependency relationship between any two original frames in the second shooting mode. Furthermore, when the user switches shooting modes, the first preview image is the last preview image displayed by the terminal device in the first shooting mode.

[0192] In addition, in response to the user's click on the control in the second shooting mode, the terminal device can switch from the first shooting mode to the second shooting mode, i.e., switch from displaying the first preview interface to displaying the third preview interface. By displaying the third preview interface, the terminal device enables the user to trigger operations on the relevant controls in the third preview interface, thereby causing the terminal device to respond to the user's operation and perform corresponding processing. S202, upon receiving the user's click on the first control, a third dependency relationship is generated.

[0193] Optionally, the third dependency is the dependency between any two original images acquired by the terminal device in the second shooting mode. For example, the third dependency could be the dependency between the sixth and seventh original images. The seventh original image is any frame acquired by the terminal device after the sixth original image. The third dependency includes: the fourth preview image is displayed after the terminal device has acquired the sixth and seventh original images.

[0194] It should be noted that the specific implementation process of generating the third dependency in S202 is similar to the specific implementation process of generating the first dependency in S103. Please refer to the relevant description in S103, which will not be repeated here.

[0195] When a terminal device switches from the first shooting mode to the second shooting mode, it restarts the configuration of multiple preset algorithms (e.g., the first preset algorithm and the second preset algorithm). In other words, in the first shooting mode, the first preset algorithm in the terminal device is enabled (e.g., ...). Figure 9 As shown in Figure (1), the beautification algorithm in portrait mode is enabled. After the terminal device switches from the first shooting mode to the second shooting mode, the second preset algorithm in the terminal device is disabled in the second shooting mode (e.g., as shown in Figure (1)). Figure 9 As shown in Figure (2), the beautification algorithm in video recording mode is turned off.

[0196] Therefore, after executing S201, the terminal device, upon receiving the user's first click on the first control, can activate the second preset algorithm in the second shooting mode. For example, as... Figure 9 As shown in Figure (2), in response to the user's first click on the control 107 in the interface 15, the terminal device can activate the beautification algorithm corresponding to the control 107. By activating the second preset algorithm in the second shooting mode, the terminal device can process the original image obtained in the second shooting mode through the second preset algorithm and display the processed preview image, thereby enhancing the user's visual experience in the second shooting mode.

[0197] It should be noted that the above refers to... Figure 9 In the exemplary description of the interface, the first preset algorithm and the second preset algorithm are the same preset algorithm, and both are beautification algorithms. The embodiments of this application are not limited thereto.

[0198] S203, based on the third dependency relationship, displays the fourth preview interface.

[0199] Optionally, the fourth preview interface is the interface with the second preset algorithm enabled. For example, the fourth preview interface can be as follows: Figure 9 Interface 16 is shown in (3) of the document.

[0200] After executing S202, the terminal device can display a fourth preview interface based on the generated third dependency relationship. The fourth preview interface includes a fourth preview image, and the fourth preview image is displayed after the terminal device has obtained the sixth original image and the seventh original image.

[0201] Optionally, the fourth preview image is the image obtained by processing the sixth original image using the first preset algorithm. It should be noted that the specific implementation process of displaying the fourth preview interface in S203 is similar to the specific implementation process of displaying the first preview interface in S103; please refer to the relevant description of S103, which will not be repeated here. It should also be noted that when the terminal device switches from the first shooting mode to the second shooting mode, the terminal device will restart the numbering of the original images. That is, the frame number of the original image obtained by the terminal device in the second shooting mode can be the same as or different from the frame number of the original image obtained by the terminal device in the first shooting mode; this application embodiment does not limit this. For example, in the first shooting mode, the terminal device can number the frame numbers of the obtained original images sequentially as 1, 2, 3, etc., according to the order of acquisition. In the second shooting mode, the terminal device can number the frame numbers of the obtained original images sequentially as 1, 2, 3, etc., or sequentially as 0, 1, 2, etc., according to the order of acquisition.

[0202] Optionally, in some embodiments, when the second shooting mode is video recording mode and the first preset algorithm in video recording mode is enabled, the terminal device can display a video recording interface in response to the user clicking the video recording control. For example, as... Figure 9 As shown in Figure (3), the recording control is control 114. The terminal device responds to the user's input as follows: Figure 9 Clicking control 114 in interface 16 (Figure 3) can display the following: Figure 9 The interface shown in Figure (4) is 17.

[0203] Optionally, the recording interface may include an end recording control and a pause recording control. For example, such as... Figure 9 As shown in Figure (4), the control to end recording is control 115 in interface 17, and the control to pause recording is control 116 in interface 17.

[0204] After displaying the recording interface, the terminal device can display the fifth preview interface in response to the user's click of the end recording control or pause recording control. Figure 9 (Not shown in the image).

[0205] It should be noted that the aforementioned terminal device's response to the user clicking the end recording control indicates that the terminal device has completed a recording session. The terminal device's response to the user's first click of the pause recording control indicates that the recording session is paused. If the user clicks the pause recording control again, the terminal device can resume recording from the paused position until the user clicks the end recording control, thus completing the recording session.

[0206] Optionally, the fifth preview interface may include a third preview image. The third preview image is displayed when the terminal device acquires the fifth original image, which is the image acquired by the terminal device when recording ends or pauses.

[0207] For example, such as Figure 9 As shown in Figure (4), in response to the user clicking the control 115 in the interface 17, the terminal device can obtain the fifth original image when the user ends the recording.

[0208] It should be noted that, Figure 9 Other controls in the four interfaces shown can be found in [reference]. Figure 1 The relevant explanations in the document will not be repeated here.

[0209] Considering the high real-time nature of video recording on terminal devices, when the terminal device stops recording (including ending or pausing recording), the fifth original image acquired by the terminal device has not yet been processed by the first preset algorithm. At this time, the terminal device can block the second thread to complete the processing of the fifth original image, thereby ensuring that every frame in the video obtained by the terminal device in recording mode is an image processed by the first preset algorithm, which can improve the user experience in recording mode.

[0210] Optionally, the second thread is the main thread in the terminal device. Blocking processing refers to pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; wherein, after processing the fifth original image using the first preset algorithm is completed, the second task continues execution until its processing is finished. The second task is any task in the second thread other than processing the fifth original image using the first preset algorithm.

[0211] Optionally, such as Figure 10 As shown, after receiving the fifth original image through the algorithm node module 211, the terminal device 100 can block the second thread, i.e., pause the second task in the second thread; and process the fifth original image using the preset algorithm (e.g., the first preset algorithm) in the preset algorithm module 212 within the second thread. After processing the fifth original image using the preset algorithm (e.g., the first preset algorithm) in the preset algorithm module 212 is completed, the second task in the second thread continues to execute. After the second task in the second thread is completed, the preset algorithm module 212 can return the processed third preview image to the management module 213. The management module 213 can transmit the third preview image to the display module 22 and control the display module 22 to display the third preview image.

[0212] like Figure 10 As shown, when the fifth original image is represented as F1, the third preview image is correspondingly represented as F1'. When the second thread is blocked, there is no lag between the input original image and the obtained corresponding preview image. That is, the terminal device can obtain a preview image (e.g., the third preview image) after processing the current frame's original image using a preset algorithm, once it has acquired the current frame's original image (e.g., the fifth original image). Figure 10 As shown, the input of algorithm node module 211 is F1, and the output is F1'. Compared to... Figure 7 The method shown involves non-blocking processing of the second thread (i.e., outputting the processing result of the previous frame in the next frame, that is, outputting the processing result F1' of F1 when F2 is input). Figure 10The time required to obtain the preview image (e.g., the third preview image) corresponding to the original image of the current frame (e.g., the fifth original image) is longer. Therefore, in this embodiment, the terminal device blocks the second thread to ensure that the preview image corresponding to the original image of the current frame (e.g., the fifth original image) is only used under edge conditions (e.g., the original image of the current frame does not have an image that can be set to depend on later). This further shortens the time for the terminal device to preview the original image while ensuring that the terminal device can successfully display the preview image.

[0213] The above text combined Figures 8 to 10 This paper details the specific implementation process of an image processing method in this application embodiment when the terminal device switches shooting modes or stops or pauses recording. The following section combines... Figure 11 This section details the specific implementation process of another image processing method described in the embodiments of this application. Please refer to [link / reference]. Figure 11 , Figure 11 A flowchart of another image processing method according to an embodiment of this application is shown.

[0214] like Figure 11 As shown, when the first preview interface also includes a camera control, another image processing method in this embodiment specifically includes the following steps:

[0215] S301, in response to the user clicking the camera control in the first preview interface, outputs a third original image.

[0216] Optionally, in some embodiments, the exposure value of the third original image is a first exposure value. The first exposure value can be the dynamic range of the third original image. The dynamic range of the third original image describes the visible range from the brightest to the darkest part of the third original image. For example, the dynamic range of the third original image refers to the difference between the maximum brightness value and the minimum brightness value of the third original image. For example, when the maximum brightness value of the third original image is 255 pixels and the minimum brightness value of the third original image is 0 pixels, the first exposure value is 255 pixels.

[0217] The terminal device responds to the user clicking on the first preview screen (e.g., ... Figure 1 The camera control (e.g., as shown in Figure 14) of the interface (4) is shown in Figure 14. Figure 1 The operation of control 106 in interface 14 shown in Figure (4) can obtain the third original image. By obtaining the third original image, preparation can be made for the subsequent display of the second preview interface, and the data basis for image algorithm processing can be provided for the display of the second preview image.

[0218] S302, based on the exposure value of the third original image, display the second preview interface.

[0219] Optionally, the second preview interface includes a second preview image. The second preview image is an image obtained by processing the third original image using a first preset algorithm.

[0220] After executing S302, the terminal device can determine the maximum brightness value and the minimum brightness value of the third original image. Further, the terminal device can calculate the exposure value of the third original image based on the maximum and minimum brightness values. Even further, the terminal device can display a second preview interface based on the exposure value of the third original image. By displaying the second preview interface, the user can be shown the image after processing the third original image, i.e., the second preview image.

[0221] like Figure 11 As shown, S302 specifically includes the following steps:

[0222] S3020, obtain the first exposure value.

[0223] It should be understood that the specific implementation process of S3030 can be found in the specific process of calculating the first exposure value in S302, and will not be repeated here.

[0224] S3021, Determine whether the first exposure value is less than the preset exposure parameter.

[0225] Optionally, the preset exposure parameters are exposure parameters pre-stored in the terminal device. For example, the preset exposure parameters can be 150 pixels.

[0226] After executing S3020, the terminal device can determine whether the exposure value of the third original image, i.e., the first exposure value, is less than the preset exposure parameter. If the first exposure value is less than the preset exposure parameter, the terminal device can execute S3022. If the first exposure value is greater than or equal to the preset exposure parameter, the terminal device can execute S3023.

[0227] S3022, when the first exposure value is less than the preset exposure parameter, in the first thread, the third original image is processed using the first preset algorithm to display the second preview interface, which includes the second preview image; wherein, the second preview image is displayed when the terminal device acquires the third original image and the fourth original image, the fourth original image is any frame image acquired by the terminal device after the third original image, and there is a second dependency relationship between the third original image and the fourth original image.

[0228] Optionally, the first thread can be any thread in the terminal device other than the main thread (i.e., the second thread). It should be noted that the first thread and the third thread mentioned above can be the same thread or different threads; this embodiment does not limit this.

[0229] It should be noted that the specific implementation process of S3022 is similar to that of S103. Please refer to the relevant description of S103. The process of generating the second dependency can also refer to the relevant description of generating the first dependency above, and will not be repeated here.

[0230] When the first exposure value is greater than or equal to the preset exposure parameter, it indicates that the third original image is an image captured under variable exposure. At this time, to avoid flashing white or dark images on the terminal device, the terminal device can discard one or more frames acquired after the third original image. In other words, after acquiring the third original image, the terminal device encounters a situation where image frames are discontinuous, and there are no subsequent image frames that the third original image can rely on. Therefore, the terminal device can execute S3023, enabling it to successfully display the second preview image corresponding to the third original image.

[0231] It should be noted that the number of image frames discarded by the terminal device under the above-mentioned variable exposure photography can be set by those skilled in the art, and this application does not limit this. For example, under variable exposure photography, the number of image frames discarded by the terminal device is 3 frames.

[0232] It should also be noted that, in response to the user's click on the camera control, the terminal device 100 can directly display the image after preview processing, i.e., the second preview image, instead of displaying the third original image, thus providing a better preview experience for the user.

[0233] S3023, if the first exposure value is greater than or equal to the preset exposure parameter, the second thread is blocked and a second preview interface is displayed. The second preview interface includes a second preview image. The second preview image is displayed when the terminal device acquires the third original image.

[0234] Optionally, the second thread is the main thread in the terminal device. Blocking processing refers to pausing the first task in the second thread and processing the third original image using a first preset algorithm; wherein, after processing the third original image using the first preset algorithm is completed, the first task continues execution until the first task is completed. The first task is the task in the second thread other than processing the third original image using the first preset algorithm.

[0235] It should be noted that the specific implementation process of S3023 is different from... Figure 10 The blocking process shown is similar; see [link to relevant documentation]. Figure 10 The relevant descriptions are not elaborated here.

[0236] The image processing method of the embodiments of this application has been described in detail above. In practical applications, the image processing method of the embodiments of this application can be implemented based on the hardware system and software system of the terminal device. The following section combines... Figure 12 and Figure 14 The software system and hardware system of a terminal device in the embodiments of this application are described respectively.

[0237] The software system of the terminal device 100 in this application embodiment can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture or cloud architecture. This application embodiment takes a layered architecture as an example to exemplarily describe the software system of the terminal device 100.

[0238] Please see Figure 12 , Figure 12 A schematic diagram of the software system of a terminal device provided in an embodiment of this application is shown.

[0239] like Figure 12 As shown, the software system in terminal device 100 can be divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, such as... Figure 12 As shown, the software system of the terminal device 100 consists of, from top to bottom, the application layer (APP), the application framework layer (APP), the hardware abstraction layer (HAL), and the driver layer (driver).

[0240] The application layer can include multiple applications. For example, these applications could be a camera app, settings app, photo album app, calendar app, and music app. It's understood that the applications in the application layer can be third-party applications installed by the user, or they can be system applications.

[0241] The application framework layer is a framework layer used to support the operation of multiple applications within the application layer. The application framework layer includes standard Android application programming interfaces (APIs), and these interfaces can be called by multiple applications within the application layer.

[0242] The application framework layer may also include core services. Core services provide the core functionalities of the software system (e.g., data storage).

[0243] The language used in the Hardware Abstraction Layer (HAL) is the Interface Definition Language (HIDL). This language describes the interfaces and functions of the hardware components. The HIDL framework is the native HAL framework provided by Android.

[0244] The hardware abstraction layer is used to access devices such as cameras, sensors, and displays. Optionally, in some embodiments, the hardware abstraction layer may include an acquisition module, a preset algorithm processing module, and a display module. The preset algorithm processing module may include an algorithm node module, a management module, and a preset algorithm module.

[0245] It should be noted that detailed descriptions of the acquisition module, algorithm node module, management module, preset algorithm module, and display module can be found in [link to relevant documentation]. Figure 6 The relevant descriptions will not be repeated here.

[0246] Optionally, in another embodiment, the hardware abstraction layer may further include an image sensor, an image front end, and an image processing engine. The image sensor is used to acquire raw image data. The image front end is used to perform preliminary processing on the raw image data transmitted from the image sensor, preparing for subsequent image preview processing. The image processing engine is used to further process and optimize the raw image data transmitted from the image front end, further preparing for subsequent image preview processing. A preset algorithm processing module is used to perform preview processing on the raw image data transmitted from the image processing engine.

[0247] It should be understood that the image sensor, image front-end engine, and image processing engine serve the same purpose as the acquisition module.

[0248] like Figure 12 As shown, the driver layer is an abstraction layer between hardware and software. The driver layer includes, for example,... Figure 12 The drivers shown include display drivers, Bluetooth drivers, hardware interface drivers, audio drivers, and sensor drivers.

[0249] Optionally, the display driver is used to ensure that the terminal device 100 can display a preview image (e.g., a first preview image or a second preview image, etc.) after the original image has been processed using a preset algorithm.

[0250] Optionally, the Bluetooth driver is used to ensure that the terminal device 100 can recognize and use the Bluetooth chip.

[0251] Optionally, the hardware interface driver includes a hardware connection driver between the Bluetooth chip and the processor. Specifically, the hardware interface driver can be a Universal Serial Bus (USB) interface driver. Alternatively, the hardware interface driver can also be a Universal Asynchronous Receiver / Transmitter (UART) interface driver. This enables the terminal device 100 to connect to the corresponding hardware through different hardware driver interfaces.

[0252] It should be understood that Figure 12 The layered structure shown does not constitute a specific limitation on the software system of terminal device 100. In other embodiments of this application, the software system of terminal device 100 may include more than Figure 12 The layered architecture shown may have more or fewer layers. For example, the software system of terminal device 100 may also include a hardware layer, which defines the connection hardware between the Bluetooth chip and the Bluetooth and processor. Alternatively, each layer of the software system of terminal device 100 may include more than [previous layers]. Figure 12 The embodiments shown may have more or fewer constituent structures, and the present application is not limited to these.

[0253] The image processing method of this application embodiment can be executed in the hardware abstraction layer. The following is in conjunction with... Figure 13 This paper details the specific implementation process of the image processing method in the hardware abstraction layer according to the embodiments of this application.

[0254] Please see Figure 13 , Figure 13 An interactive diagram of an image processing method according to an embodiment of this application is shown. For example... Figure 13 As shown, the specific implementation process of an image processing method in this application embodiment includes:

[0255] Step 11, in response to the user's action of launching the camera application (e.g., the user clicks on...). Figure 1 (1) As shown in Figure 11, the control 101 in the interface allows the terminal device to display the interface in the camera mode (e.g., as shown in Figure 1). Figure 1 Interface 12 is shown in Figure (2).

[0256] Step 12, responding to the user's action of clicking on the relevant controls in the interface during camera mode (e.g., the user clicks on...). Figure 1 (2) As shown in the figure, the interface 12 (control 104) allows the terminal device to display the interface in portrait mode (e.g., as shown in Figure 12). Figure 1 Interface 13 is shown in Figure (3).

[0257] Step 13, responding to the user's click on the relevant control in the interface of portrait mode (e.g., the user's first click on such...). Figure 1 (3) As shown in the interface 13 (control 107), the terminal device can enable the first preset algorithm (for example, with...). Figure 1 (3) The beautification algorithm corresponding to control 107 in interface 13 shown in Figure 13).

[0258] Step 14: When the first preset algorithm is enabled, the terminal device can acquire the first original image through the acquisition module.

[0259] It should be understood that the specific implementation process of step 14 is the same as that of S102, and can be found in the relevant description of S102, which will not be repeated here.

[0260] Step 15: When the acquisition module acquires the first original image, the terminal device can transmit the first original image to the preset algorithm processing module through the acquisition module. Further, the terminal device can generate a first dependency relationship based on the first original image through the preset algorithm processing module, and process the first original image using the first preset algorithm to obtain the first preview image.

[0261] Step 16: The terminal device can acquire the second original image through the acquisition module.

[0262] It should be understood that steps 15 and 16 are not sequential in terms of timing or order. That is to say, steps 15 and 16 can be executed simultaneously, or in the order of step 15 first and step 16 second, or in the order of step 15 second and step 16 first.

[0263] Step 17: When the acquisition module acquires the second original image, the terminal device can transmit the second original image to the preset algorithm processing module through the acquisition module. Further, when the preset algorithm processing module obtains the second original image, the terminal device can send the first preview image to the display module through the preset algorithm processing module. Even further, the terminal device can display the first preview interface (e.g., as shown in the image) through the display module. Figure 1 The interface 14 shown in Figure (4) and the first preview image (e.g., as shown in Figure 4) are displayed. Figure 1 (4) The image of the person in the interface 14 shown in Figure 14 after the beautification algorithm.

[0264] It should be understood that the specific implementation process of steps 15 to 17 is the same as that of S103. Please refer to the relevant description of S103. It will not be repeated here.

[0265] Therefore, by setting a backward dependency on the first original image in steps 15 to 17, it is ensured that the terminal device can still obtain the second original image after the first original image while previewing the first original image, thus ensuring that the preview on the terminal device will not lag. Furthermore, the terminal device displays the first preview image only after it has obtained the second original image. This shows that the terminal device starts a thread other than the main thread (or the second thread) (e.g., a third thread) while previewing the first original image, without blocking the main thread. This speeds up the previewing process of the first original image, shortens the previewing time, and improves the user experience.

[0266] Step 18, responding to the user's click on the relevant control in the interface of portrait mode (for example, the user clicks again such as...). Figure 1 In the interface 13 shown in Figure (3), the terminal device can turn off the first preset algorithm (control 107).

[0267] Step 19: When the first preset algorithm is turned off, the terminal device can remove the first dependency through the preset algorithm processing module. By removing the first dependency, the terminal device can prepare for generating other dependencies (e.g., the dependency between the second original image and a frame image acquired after the second original image), and can also prepare for the preset algorithm processing module to input the first preview image to other preset algorithm processing modules (the preset algorithms corresponding to other preset algorithm processing modules are any preset algorithms other than the first preset algorithm) for corresponding processing.

[0268] It should be understood that Figure 13 The interactive diagram shown is only an interactive diagram corresponding to one image processing method in the embodiments of this application. Its purpose is to fully illustrate one interactive process in the embodiments of this application. Other implementation methods with interactive processes in the embodiments of this application can also be adopted. Figure 13 The relevant software modules shown in the figure complete the interaction process. The implementation principle of the interaction process in each implementation method is similar, and will not be described again here.

[0269] The software system of a terminal device in an embodiment of this application has been described in detail above. The following section, in conjunction with... Figure 14 This application describes the hardware system of a terminal device in an embodiment.

[0270] Please see Figure 14 , Figure 14 A schematic diagram of the hardware system of a terminal device provided in an embodiment of this application is shown.

[0271] like Figure 14As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, and a bone conduction sensor 180M, etc.

[0272] It should be noted that, Figure 14 The structure shown does not constitute a specific limitation on the hardware system of terminal device 100. In other embodiments of this application, the hardware system of terminal device 100 may include more than Figure 14 The components shown may include more or fewer components, or the hardware system of terminal device 100 may include... Figure 14 The components shown may be a combination of certain components, or the hardware system of terminal device 100 may include... Figure 14 The components shown are sub-components of certain components. For example, Figure 14 The proximity sensor 180G shown is optional. Figure 14 The components shown can be implemented in hardware, software, or a combination of both.

[0273] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0274] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0275] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0276] Figure 14 The connection relationships between the modules shown are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the terminal device 100. Optionally, the modules of the terminal device 100 may also adopt a combination of various connection methods described in the above embodiments.

[0277] Terminal device 100 can implement display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0278] The display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, the terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0279] In some embodiments, the unfolded state of the display screen 194 can be as follows: Figure 1 Any one of the four states shown. In another embodiment, the unfolded state of the display screen 194 can be as follows: Figure 9 Any one of the four states shown.

[0280] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0281] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0282] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0283] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0284] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0285] Touch sensor 180K, also known as a touch device, can be disposed on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a touch display. Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. In other embodiments, touch sensor 180K may also be disposed on the surface of terminal device 100, and in a different location from display screen 194.

[0286] Optionally, in some embodiments, the processor 110 activates the camera; based on this, it can acquire a first original image through the camera; then, it sets a backward dependency on the first original image, that is, there is a first dependency relationship between the first original image and a second original image acquired after the first original image; further, based on the first original image, it displays a first preview interface, which can display an image processed by a first preset algorithm on the first preview interface, i.e., the first preview image, and the first preview image is displayed when the terminal device acquires the second original image. Thus, by setting a backward dependency on a certain frame of original image (e.g., the first original image), it can be ensured that during the preview processing of that frame of original image, there is no need to copy the frame of original image (e.g., the first original image), which can save the power consumption of the terminal device. When the next frame of original image (e.g., the second original image) is acquired, displaying the image processed by that frame of original image (e.g., the first preview image) can reduce the lag in the preview screen, shorten the processing time of the original image using the preset algorithm (e.g., the first preset algorithm), and improve the user's preview experience.

[0287] For example, this application provides a computer-readable storage medium including instructions that, when executed on a terminal device, cause the terminal device to perform actions such as... Figure 4 The image processing method shown.

[0288] For example, this application provides a chip system applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to invoke computer instructions to cause the terminal device to perform, for example... Figure 4 The image processing method shown.

[0289] For example, this application provides a computer program product that, when run on a computer, causes a terminal device to perform actions such as... Figure 4 The image processing method shown.

[0290] In the above embodiments, all or part of the functionality can be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it can be implemented wholly or partially in the form of a computer program product. A computer program product includes one or more computer codes or instructions. When the computer program code or instructions are loaded and executed on a computer, all or part of the flow or functionality according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer code or instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0291] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image processing method, characterized by, The method is applied to a terminal device, and the method includes: Turn on the camera; Obtain the first original image; store the first original image in the first memory segment; Based on the first original image, a first preview interface is displayed, the first preview interface including a first preview image; the first preview image is an image after processing the first original image using a first preset algorithm; the first preview image is stored in the first memory segment; Wherein, there is a first dependency relationship between the first original image and the second original image; the first dependency relationship includes: the first preview image is displayed when the terminal device has acquired the first original image and the second original image; the second original image is any frame image acquired by the terminal device after the first original image; the second original image is stored in a second memory segment, and the second memory segment and the first memory segment are different cache spaces.

2. The method of claim 1, wherein, The first preview interface also includes a camera control, and the method further includes: In response to the user clicking the camera control, a third raw image is acquired; the exposure value of the third raw image is the first exposure value. Based on the exposure value of the third original image, a second preview interface is displayed, which includes a second preview image; the second preview image is an image processed by the first preset algorithm on the third original image.

3. The method of claim 2, wherein, When the first exposure value is less than the preset exposure parameter, the second preview image is displayed when the terminal device has acquired the third original image and the fourth original image; the fourth original image is any frame image acquired by the terminal device after the third original image; There is a second dependency relationship between the third original image and the fourth original image; Before displaying the second preview image, the method further includes: In the first thread, the third original image is processed using the first preset algorithm. The first thread is any thread in the terminal device other than the main thread.

4. The method of claim 2, wherein, When the first exposure value is greater than or equal to the preset exposure parameter, the second preview image is displayed when the terminal device acquires the third original image; Before displaying the second preview image, the method further includes: The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process refers to pausing the first task in the second thread and processing the third original image using the first preset algorithm; wherein, after the processing of the third original image using the first preset algorithm is completed, the first task continues to be executed until the processing of the first task is completed; the first task is the task in the second thread other than the processing of the third original image using the first preset algorithm.

5. The method of claim 1, wherein, The first preview interface includes controls for a first shooting mode and controls for a second shooting mode; the first preview interface is the interface in the first shooting mode; the method further includes: In response to the user clicking the control of the second shooting mode, the first dependency relationship is removed and a third preview interface is displayed, the third preview interface including the first control; the second shooting mode is a different shooting mode from the first shooting mode; Upon receiving a user's click on the first control, a third dependency relationship is generated; the third dependency relationship is the dependency relationship between any two original images acquired by the terminal device in the second shooting mode; Based on the third dependency relationship, a fourth preview interface is displayed, which is an interface with the second preset algorithm enabled; the second preset algorithm may be the same as or different from the first preset algorithm.

6. The method of claim 5, wherein, The second shooting mode is a video recording mode; the third preview interface also includes video recording controls; the method further includes: In response to a user clicking the recording control, a recording interface is displayed; the recording interface includes an end recording control and a pause recording control. In response to the user clicking the end recording control or the pause recording control, a fifth preview interface is displayed; the fifth preview interface includes a third preview image, which is displayed when the terminal device acquires a fifth original image, which is the image acquired by the terminal device when ending or pausing recording; Before displaying the third preview image, the method further includes: The second thread is blocked; the second thread is the main thread in the terminal device; the blocking process refers to pausing the second task in the second thread and processing the fifth original image using the first preset algorithm; wherein, after the processing of the fifth original image using the first preset algorithm is completed, the second task continues to be executed until the processing of the second task is completed; the second task is the task in the second thread other than the processing of the fifth original image using the first preset algorithm.

7. The method according to any one of claims 1 to 6, characterized in that, The first preview interface includes: In response to the user's operation of activating the first preset algorithm, the first preview interface is displayed.

8. The method according to any one of claims 1 to 7, characterized in that, After displaying the first preview interface, the method further includes: In response to the user's action of disabling the first preset algorithm, the first dependency relationship is terminated.

9. The method according to any one of claims 1 to 8, characterized in that, Before displaying the first preview interface, the method further includes: Obtain identification information; the identification information includes a first identifier, a second identifier, and a third identifier; the first identifier is used to indicate the dependency direction of the first original image; the second identifier is used to indicate that the first original image depends on the second original image; the third identifier is used to indicate that the first preset algorithm is in the enabled state; The first dependency relationship is generated based on the first identifier, the second identifier, and the third identifier.

10. The method according to any one of claims 1 to 9, characterized in that, Before displaying the first preview image, the method further includes: In the third thread, the first original image is processed using the first preset algorithm; the third thread is any thread in the terminal device other than the main thread.

11. The method according to any one of claims 1 to 10, characterized in that, The first preset algorithm is any one of the following algorithms: beautification algorithm, blurring algorithm, sharpening algorithm, filter algorithm, sticker algorithm, watermark algorithm, neon algorithm, and slimming algorithm.

12. A terminal device, comprising: The terminal device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the terminal device to perform the method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1 to 11.

14. A chip system, characterized by The chip system is applied to a terminal device; the chip system includes one or more processors; the one or more processors are used to invoke computer instructions to cause the terminal device to perform the method as described in any one of claims 1 to 11.

Images