Image display method and related device

Abstract

The present application provides an image display method and related devices. In the camera preview scenario, when the electronic device detects that the time it takes to process the original image through an algorithm exceeds a time threshold, the electronic device can obtain an intermediate frame image based on the current original image and the previous preview image. In this way, the intermediate frame image can be displayed when the next vertical synchronization signal arrives. The method provided by the present application can reserve sufficient time for algorithmic processing of the original image, thereby reducing frame loss and, in turn, reducing the phenomenon of screen freezes observed by users. In this way, the smoothness of the image displayed by the electronic device can be guaranteed to a certain extent, and the user's visual experience can be good.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image display method and related devices. Background Art

[0002] Electronic devices can display preview images. Electronic devices can process raw images captured by cameras using various camera algorithms to produce more detailed preview images. However, this process takes a considerable amount of time, resulting in the electronic device being unable to display the preview image coherently, i.e., frame dropout. Enabling electronic devices to display preview images coherently to a certain extent is an urgent problem in the field. Summary of the Invention

[0003] The present application provides an image display method and related devices, which can ensure the continuity of images displayed by electronic devices to a certain extent.

[0004] In a first aspect, the present application provides an image display method, which is applied to an electronic device, the method comprising: at a first moment, displaying a first preview image; the electronic device capturing a second original image; the electronic device processing the second original image through at least one algorithm, the at least one algorithm including a first algorithm, the processing duration of the first algorithm exceeding a first duration threshold; the electronic device obtaining a first intermediate frame image based on the first preview image and the second original image; at a second moment, displaying the first intermediate frame image, the second moment being a moment separated by a single signal cycle after the first moment; the electronic device obtaining a second preview image after the second moment based on the second original image processed by the at least one algorithm; and at a third moment, displaying the second preview image, the third moment being a moment separated by a single signal cycle after the second moment.

[0005] By implementing the above method, the electronic device can reserve sufficient time to perform algorithmic processing on the original image, thereby reducing frame drops and, in turn, reducing the screen freezes observed by the user. This can ensure, to a certain extent, the smoothness of the electronic device's displayed image and provide a good visual experience for the user.

[0006] In conjunction with the first aspect, in some implementations, the first duration threshold is determined by one or more of the following: the single signal period, the historical processing duration of the first algorithm, the historical duration of acquiring original images, and the historical duration of synthesizing preview images. In this way, the electronic device can set a response duration threshold for each camera algorithm.

[0007] In conjunction with the first aspect, in some implementations, at the third moment, before displaying the second preview image, the method further includes:

[0008] The electronic device collects a third original image;

[0009] The electronic device processes the second original image using the at least one algorithm, where the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold;

[0010] The electronic device obtains a second intermediate frame image based on the second preview image and the third original image;

[0011] At the third moment, after displaying the second preview image, the method further includes:

[0012] The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm;

[0013] The electronic device generates a fourth preview image or a third intermediate frame image;

[0014] The electronic device deletes the second intermediate frame image;

[0015] At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment;

[0016] At a fifth moment, the fourth preview image or the third intermediate frame image is displayed. The fifth moment is a moment separated by a single signal cycle from the fourth moment.

[0017] In this way, the electronic device can read a newly generated image each time it detects a vertical synchronization signal, preventing the same image from being displayed for two consecutive frames. Furthermore, the number of images stored by the electronic device remains within a certain range, preventing frame accumulation and reducing storage resource consumption.

[0018] In conjunction with the first aspect, in some implementations, when the electronic device processes the second original image using at least one algorithm, the method further includes:

[0019] The electronic device determines that no preview image is currently stored;

[0020] At the third moment, before displaying the second preview image, the method further includes:

[0021] The electronic device collects a third original image;

[0022] The electronic device processes the second original image using the at least one algorithm, where the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold;

[0023] The electronic device determines that a preview image is currently stored;

[0024] At the third moment, after displaying the second preview image, the method further includes:

[0025] The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm;

[0026] a fourth preview image of the electronic device;

[0027] At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment;

[0028] At a fifth moment, the fourth preview image is displayed. The fifth moment is a moment separated by a single signal cycle from the fourth moment.

[0029] In this way, the electronic device can also ensure that the number of stored preview images is within a certain range and can also avoid the occurrence of frame accumulation.

[0030] In conjunction with the first aspect, in some implementations, the electronic device includes a cache queue, the cache queue being used to store preview images and intermediate frame images, and the maximum storage capacity of the cache queue is two frames of images. In an embodiment of the present application, the SurfaceFallingerBuffer in the electronic device stores at least 0 frames of images and can store at most 2 frames of images.

[0031] In conjunction with the first aspect, in some implementations, the at least one algorithm includes any one or more of the following: a face detection algorithm, a motion detection algorithm, an automatic exposure algorithm, a gesture detection algorithm, an object detection algorithm, and a scene detection algorithm. In embodiments of the present application, more camera algorithms may also be included, without limitation.

[0032] In combination with the first aspect, in some implementations, the electronic device further includes: a camera, a hardware abstraction layer, a surface synthesizer, and a display screen, wherein:

[0033] At a first moment, the display screen is used to display a first preview image;

[0034] The camera is used to collect a second original image;

[0035] The hardware abstraction layer is used to process the second original image by using at least one algorithm, where the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold;

[0036] The hardware abstraction layer is used to obtain a first intermediate frame image based on the first preview image and the second original image;

[0037] At a second moment, the display screen is used to display the first intermediate frame image, and the second moment is a moment separated by a single signal cycle after the first moment;

[0038] The surface synthesizer obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm;

[0039] At a third moment, the display screen is used to display the second preview image. The third moment is a moment separated by a single signal cycle from the second moment.

[0040] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, wherein the computer program code comprises computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the method of the first aspect or any embodiment of the first aspect.

[0041] In a third aspect, an embodiment of the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The user interface of the electronic device in the preview scenario provided by the embodiment of the present application;

[0043] Figure 2A A schematic diagram of the principle of an electronic device from capturing an original image to displaying a preview image provided by an embodiment of the present application;

[0044] Figure 2B A schematic diagram of the principle of another electronic device provided in an embodiment of the present application from capturing an original image to displaying a preview image;

[0045] Figure 3 A flow chart of an image display method provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the processing time of an electronic device monitoring algorithm provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram showing the principle of another electronic device using an image display method provided in an embodiment of the present application;

[0048] Figure 6 A flow chart of another image display method provided in an embodiment of the present application;

[0049] Figure 7 A schematic diagram showing the principle of another electronic device using an image display method provided in an embodiment of the present application;

[0050] Figure 8 An internal module interaction diagram of an electronic device using an image display method provided by an embodiment of the present application;

[0051] Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0052] Figure 10 A software structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following is a clear and detailed description of the technical solutions in the embodiments of the present application, with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of "or." For example, A / B can represent A or B. "and / or" in the text is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0054] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0055] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0056] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0057] First, the application scenarios involved in the embodiments of the present application are introduced.

[0058] Figure 1 The following example illustrates a scenario in which an electronic device provided by an embodiment of the present application displays a preview image.

[0059] like Figure 1 As shown, after detecting the operation of starting the camera application, the electronic device can turn on the camera (for example, the rear camera) and display the user interface 200. The user interface 200 may include but is not limited to: area 201, area 202, and area 203.

[0060] Among them, area 201 (also called preview area) occupies most of the area in user interface 200 and is mainly used to display a preview image. The preview image may refer to an image generated by the electronic device during the preview phase of capturing an image, and the preview image may change as the electronic device moves.

[0061] Region 202 is fixedly displayed below region 201 and is mainly used to display various shooting modes provided by the camera application, such as professional mode, screen recording mode, photo mode, portrait mode, and night mode, etc. For example, in user interface 200, the electronic device uses the photo mode to display the preview image.

[0062] Region 203 is permanently displayed below region 202 and is primarily used to display multiple function options, such as a "shoot" option, an "image" option, and a "switch camera" option. For example, when the electronic device is moved to a suitable shooting angle, it can respond to a user's click on the "shoot" option to capture an image.

[0063] In some implementations, during the process of displaying the preview image, the camera application may drive the image sensor (ie, Senor) in the camera of the electronic device to start.

[0064] Then, after the Senor in the electronic device is started, it can receive periodic acquisition instructions sent by the camera application (i.e., Cameraapk), acquire the original image used to generate the preview image, and send the acquired original image to the hardware abstraction layer (i.e., HAL layer).

[0065] Next, the hardware abstraction layer in the electronic device can process the original image with relevant algorithms.

[0066] In embodiments of the present application, algorithms may include camera algorithms. Camera algorithms may refer to a series of computer science techniques and mathematical methods used to process and optimize raw images captured by a camera. Camera algorithms can improve the quality of preview images and enhance camera performance through various methods, providing users with a better user experience.

[0067] Finally, the surface compositor (Surface Flinger) in the electronic device synthesizes the processed original image and stores the synthesized preview image in the Surface Flinger consumer queue. When the vertical synchronization signal arrives, the display driver of the electronic device reads the synthesized preview image from the Surface Flinger buffer and displays it on the display.

[0068] The vertical synchronization signal is used to trigger the hardware to refresh the image displayed on the display screen. Every time an electronic device has a signal cycle, a vertical synchronization signal is triggered by the hardware driver.

[0069] In some implementations, the period for generating the vertical synchronization signal can be determined by the frame rate of the display screen of the electronic device. For example, the period can be the inverse of the frame rate of the display screen. Exemplarily, the frame rate of the display screen of the electronic device can be any value such as 30 Hz, 60 Hz, 70 Hz, 75 Hz, 80 Hz, 90 Hz or 120 Hz. Taking the frame rate of the display screen as 60 Hz as an example, the period for generating the vertical synchronization signal is 1 / 60 = 0.01667 seconds (s) = 16.667 milliseconds (ms). That is, the electronic device displays a preview image for 16.667 ms, and then displays the next preview image, also for 16.667 ms.

[0070] In one possible implementation, the electronic device may support multiple different frame rates. The frame rate of the electronic device can switch between the above different frame rates. In the embodiment of the present application, the frame rate is the frame rate currently used by the electronic device. That is, the period for generating the vertical synchronization signal is the inverse of the frame rate currently used by the electronic device.

[0071] In some implementations, the capture instructions generated by the camera application of the electronic device in the preview scene can also be obtained based on the above-mentioned vertical synchronization signal. For example, in order to better synchronize with the display refresh, the period of the capture instructions is the same as the period of generating the vertical synchronization signal.

[0072] The following is a schematic diagram of the principle of an electronic device from capturing an original image to displaying a preview image.

[0073] For example, Figure 2A As shown, at time t1, the electronic device generates a capture instruction through the camera application. In response to the capture instruction at time t1, the electronic device captures the original image a. After capturing the original image a, the electronic device can perform algorithmic processing on the original image a through the HAL layer.

[0074] Then, the electronic device can use Surface Flinger to synthesize image A from the original image a processed by the algorithm and store the synthesized image A in the Surface Flinger buffer. At time t11, the electronic device detects the vertical synchronization signal, reads image A from the Surface Flinger buffer, and displays image A.

[0075] During the time period T before time t11, the electronic device displays image M. Image M is an image stored in the Surface Flinger buffer before image A.

[0076] Similarly, at time t2, the electronic device generates a capture command through the camera application. In response to the capture command at time t2, the electronic device captures the original image b. After capturing the original image b, the electronic device performs algorithmic processing on the original image b through the HAL layer. The electronic device then uses Surface Flinger to synthesize the processed original image b into image B, storing the synthesized image B in the Surface Flinger buffer. At time t21, the electronic device detects the vertical sync signal, reads image B from the Surface Flinger buffer, and displays image B.

[0077] Similarly, at time t3, the electronic device generates a capture command through the camera application. In response to the capture command at time t3, the electronic device captures the original image c. After capturing the original image c, the electronic device performs algorithmic processing on the original image c through the HAL layer. The electronic device then uses Surface Flinger to synthesize the processed original image c into image C, storing the synthesized image C in the Surface Flinger buffer. At time t31, the electronic device detects the vertical sync signal, reads image C from the Surface Flinger buffer, and displays image C.

[0078] There is a single signal cycle between time t1 and time t2, a single signal cycle between time t2 and time t3, a single signal cycle between time t11 and time t21, and a single signal cycle between time t21 and time t31. In one possible implementation, time t2 and time t11 may be the same time, and time t3 and time t21 may be the same time.

[0079] As can be seen from the above, if the electronic device stores image A at time t11, image B at time t21, and image C at time t31, the electronic device can display image A from time t11 to time t21, display image B from time t21 to time t31, and begin displaying image C at time t31. In other words, each time the electronic device detects a vertical synchronization signal and stores a newly synthesized image, the electronic device can continuously display images, and will not display the same frame of image for two signal cycles.

[0080] However, there are some situations, for example: if the electronic device takes too long to perform algorithm processing on the original image, frame loss may occur.

[0081] In the embodiment of the present application, frame loss may refer to the phenomenon that the screen displayed by the electronic device becomes stuck or discontinuous due to various reasons during the process of the electronic device displaying a preview image.

[0082] The following is based on Figure 2B The schematic diagram of the principle of the electronic device from capturing the original image to displaying the preview image is used to introduce the scenario of frame loss.

[0083] For example, Figure 2B As shown, at time t1, the electronic device generates a capture instruction through the camera application. In response to the capture instruction at time t1, the electronic device captures the original image a. After capturing the original image a, the electronic device can perform algorithmic processing on the original image a through the HAL layer.

[0084] Then, the electronic device can use Surface Flinger to synthesize image A with the original image a processed by the algorithm, and store the synthesized image A in the Surface Flinger buffer.

[0085] At time t11, the electronic device detects the vertical synchronization signal, reads image A from the Surface Flinger buffer, and displays image A. During the duration T before time t11, the electronic device displays image M. Image M is the image stored in the Surface Flinger buffer before image A.

[0086] Similarly, at time t2, the electronic device generates a capture command through the camera application. In response to the capture command at time t2, the electronic device captures the original image b. After capturing the original image b, the electronic device performs algorithmic processing on the original image b through the HAL layer. The electronic device then uses Surface Flinger to synthesize the processed original image b into image B, and stores the synthesized image B in the Surface Flinger buffer.

[0087] However, because the electronic device takes a long time to perform algorithmic processing on the original image b, it is unable to synthesize image B before the next vertical synchronization signal arrives. Therefore, at time t21, although the electronic device detects the vertical synchronization signal, because image B has not yet been synthesized, the electronic device cannot read image B from the Surface Flinger buffer. After time t21, the electronic device again displays image A for a duration of T. Optionally, the electronic device does not display any image for a duration of T after time t21.

[0088] Similarly, at time t3, the electronic device generates an acquisition instruction through the camera application. In response to the acquisition instruction at time t3, the electronic device captures the original image c. After capturing the original image c, the electronic device performs algorithmic processing on the original image c through the HAL layer. The electronic device can then use Surface Flinger to synthesize the algorithmically processed original image c into image C and store the synthesized image C in the Surface Flinger buffer. At this point, the Surface Flinger buffer stores two frames of images, namely image B and image C. At time t31, the electronic device detects the vertical synchronization signal, reads image B from the Surface Flinger buffer, and displays image B.

[0089] As can be seen from the above, if the electronic device stores image A at time t11, does not store image B at time t21, and stores both image B and image C at time t31, the electronic device displays image A from time t11 to time t21, continues to display image A from time t21 to time t31, and begins displaying image B at time t31. In other words, if the electronic device does not store a newly synthesized image when it detects a vertical sync signal, frame dropouts may occur when the electronic device displays the image, meaning that the electronic device displays the same frame for two consecutive signal cycles.

[0090] Optionally, if the electronic device is currently in a low frame rate or variable frame rate scenario, the above frame loss phenomenon is more obvious.

[0091] To reduce the occurrence of the above situation, an embodiment of the present application provides an image display method in which the electronic device can monitor the time it takes for the original image captured by the sensor to be processed by an algorithm (i.e., the processing duration of the algorithm).

[0092] When the processing time of the algorithm exceeds the time threshold, the electronic device can generate an intermediate frame image based on the original image captured by the current sensor and the image synthesized by the previous frame Surface Flinger, and store the intermediate frame image in the Surface Flinger buffer.

[0093] When the electronic device detects the vertical synchronization signal, the electronic device reads the intermediate frame image from the Surface Flinger buffer and displays the intermediate frame image.

[0094] By implementing the above method, the electronic device can reserve sufficient time to perform algorithmic processing on the original image, thereby reducing frame drops and, in turn, reducing the screen freezes observed by the user. This can ensure, to a certain extent, the smoothness of the electronic device's displayed image and provide a good visual experience for the user.

[0095] The following combination Figure 3 A flowchart illustrating an image display method according to an embodiment of the present application is provided.

[0096] The method flow includes:

[0097] S301: The electronic device collects a first original image.

[0098] In some implementations, before capturing the first original image, the electronic device has already started a camera application and the camera.

[0099] Specifically, the camera of the electronic device may respond to a capture instruction sent by a camera application to capture the first original image. The camera of the electronic device may include, but is not limited to, a front-facing camera and a rear-facing camera. The rear-facing camera may further include a main camera, a telephoto camera, a wide-angle camera, and the like.

[0100] In the embodiment of the present application, the above-mentioned first original image may refer to any frame of original image captured after the electronic device starts a camera application.

[0101] S302: The electronic device processes the first original image using an algorithm and monitors a first processing time.

[0102] After executing S301, the electronic device may execute S302.

[0103] In some implementations, the electronic device includes one or more camera algorithms configured to process the captured raw image.

[0104] Exemplarily, camera algorithms may include detection-based perception algorithms and image effects-based algorithms, among other types. Detection-based perception algorithms may include, but are not limited to, face detection algorithms, motion detection algorithms, automatic exposure algorithms, gesture detection algorithms, object detection algorithms, and scene detection algorithms. Image effects algorithms may include, but are not limited to, autofocus algorithms, white balance algorithms, pixel correction algorithms, noise reduction algorithms, shadow correction algorithms, and color enhancement algorithms. In the embodiments of this application, the aforementioned camera algorithms are merely illustrative and are not intended to be limiting.

[0105] In the embodiment of the present application, in S302, the algorithms used by the electronic device may be all camera algorithms built into the electronic device, or may be some camera algorithms, without limitation. For example, the shooting mode used by the camera application may affect the number and type of algorithms currently used.

[0106] In some implementations, the first processing duration has a one-to-many relationship with the processing time of the camera algorithm employed by the electronic device. That is, the first processing duration includes: multiple durations. The multiple durations include: multiple processing durations corresponding to each of the multiple non-selectable (i.e., serial) camera algorithms during processing, and multiple processing durations corresponding to each of one or more of the multiple optional (i.e., parallel) camera algorithms.

[0107] For example, the electronic device applies camera algorithm 1, camera algorithm 2, and camera algorithm 3 to the first original image. The three camera algorithms are all in a serial relationship. The first processing duration may include three processing durations: the processing duration of camera algorithm 1, the processing duration of camera algorithm 2, and the processing duration of camera algorithm 3.

[0108] For another example, the electronic device processes a first original image using camera algorithm 1, camera algorithm 2, and camera algorithm 3. Camera algorithm 1 and the other camera algorithms are processed in series, while camera algorithm 2 and camera algorithm 3 are processed in parallel. The first processing duration may include two processing durations: the processing duration of camera algorithm 1 and the processing duration of camera algorithm 2 or camera algorithm 3.

[0109] Because the time between capturing the original image and displaying the synthesized image is limited, the electronic device can set a processing time threshold for each camera algorithm based on the complexity of each camera algorithm and the processing time reserved for the algorithm, and then monitor the processing time of each camera algorithm. If the processing time of a camera algorithm is detected to exceed the corresponding time threshold, it means that even if the processing time of other camera algorithms does not exceed the corresponding time threshold, frame drop will occur. Therefore, the electronic device needs to perform frame interpolation processing, that is, generate intermediate frame images.

[0110] In an embodiment of the present application, the duration threshold of a camera algorithm can be determined based on one or more of the following: the vertical signal period used by the current electronic device, the historical processing duration of a camera algorithm, the historical duration of the electronic device's camera capturing original images, and the historical duration of the electronic device's Surface Flingeer synthesized preview image.

[0111] Optionally, the electronic device may also monitor the time for acquiring the original image and the pre-processing.

[0112] For example, Figure 4 As shown, the duration threshold of image pre-processing is duration Ta, the duration threshold of algorithm 1 is duration Tb, the duration threshold of algorithm 2 is duration Tc, the duration threshold of algorithm 3 is duration Tc, and the duration threshold of algorithm 4 is duration Td.

[0113] Among them, Algorithm 1, Algorithm 2, Algorithm 3, Algorithm 4, and even more algorithms are camera algorithms. For example, after an electronic device processes the original image using Algorithm 1, it generally selects one of Algorithm 2 and Algorithm 3 to process the original image, and then processes the original image using Algorithm 4.

[0114] In the embodiments of this application, Figure 4 The number of algorithms is only for illustrative purposes, and a greater or lesser number of algorithms may be included.

[0115] Exemplarily, if the time for capturing the original image and pre-processing exceeds the time Ta, the electronic device generates an intermediate frame image while continuing to perform subsequent algorithm processing on the original image and synthesize a preview image;

[0116] If the time for acquiring the original image and pre-processing does not exceed the time Ta, but the time for processing by Algorithm 1 exceeds the time Tb, the electronic device generates an intermediate frame image while continuing to perform subsequent algorithm processing on the original image and synthesize a preview image;

[0117] If the time for acquiring the original image and pre-processing does not exceed the time Ta and the time for processing by Algorithm 1 does not exceed the time Tb, but the time for processing by Algorithm 2 exceeds the time Tc or the time for processing by Algorithm 3 exceeds the time Tc, the electronic device generates an intermediate frame image while continuing to perform subsequent algorithm processing on the original image and synthesizes a preview image;

[0118] If the time for acquiring the original image and pre-processing does not exceed the time Ta, the time for processing by Algorithm 1 does not exceed the time Tb, and the time for processing by Algorithm 2 or Algorithm 3 does not exceed the time Tc, but the time for processing by Algorithm 4 exceeds the time Td, the electronic device generates an intermediate frame image and continues to synthesize the preview image based on the original image;

[0119] If the time for acquiring the original image and pre-processing does not exceed the time Ta, the time for processing by algorithm 1 does not exceed the time Tb, the time for processing by algorithm 2 or algorithm 3 does not exceed the time Tc, and the time for processing by algorithm 4 does not exceed the time Td, the electronic device does not need to generate an intermediate frame image and can directly synthesize a preview image based on the original image.

[0120] The monitoring of the first processing duration may include monitoring the duration of processing the first original image by each camera algorithm.

[0121] S303: When the first processing time does not exceed the time threshold, the electronic device synthesizes the first original image processed by the algorithm to obtain a first preview image.

[0122] The electronic device may execute S303 while executing S302 or after executing S303 .

[0123] In some implementations, the first processing duration not exceeding the duration threshold may include: multiple durations for the electronic device to process the first original image using each camera algorithm do not exceed the corresponding duration threshold.

[0124] In the embodiment of the present application, the reasons why the processing time of the algorithm exceeds the time threshold include: the current electronic device is in a high-load scenario, resulting in insufficient supply of computing resources, thereby causing timeout.

[0125] After synthesizing the first preview image, the electronic device may store the first preview image.

[0126] S304: The electronic device detects a vertical synchronization signal at a first moment and displays a first preview image.

[0127] After executing S303 , the electronic device may execute S304 .

[0128] In some implementations, the electronic device detects a vertical synchronization signal at a first moment, indicating that content displayed on the display screen needs to be refreshed at the first moment. The electronic device can read a first preview image and display the first preview image on the display screen. At the first moment, the image stored oldest in the electronic device is the first preview image.

[0129] S305: The electronic device collects a second original image.

[0130] The electronic device may execute S305 after executing S301. That is, the electronic device may execute S305 while executing S302, S303, or S304, or in an interval between executing S304 and S305.

[0131] In some implementations, in a preview scenario, a camera application of the electronic device may periodically generate an acquisition instruction, for example, generating an acquisition instruction at intervals of a single vertical signal period.

[0132] The second original image may refer to the next frame of original image of the first original image captured by the electronic device.

[0133] S306: The electronic device processes the second original image using an algorithm and monitors a second processing time.

[0134] After executing S305 , the electronic device may execute S306 .

[0135] In some implementations, the electronic device may include one or more camera algorithms configured to process the captured original image. The second processing duration may include multiple durations for each camera algorithm used by the electronic device to process the second original image.

[0136] For the relevant description of S306 , please refer to the detailed description of S302 above, which will not be repeated here.

[0137] S307: When the second processing time exceeds the time threshold, the electronic device generates a first intermediate frame image based on the second original image and the first preview image.

[0138] The electronic device can execute S307 after executing S306. However, since the electronic device needs to use the first preview image, the electronic device needs to execute S307 after S303.

[0139] The second processing duration exceeding the duration threshold may mean that the processing duration of a camera algorithm among multiple camera algorithms used by the electronic device to process the second original image exceeds the corresponding duration threshold. For a description of exceeding the duration threshold, please refer to the detailed description of S302 above.

[0140] When the electronic device detects that the second processing duration exceeds the duration threshold, it may have already performed some algorithmic processing on the second original image, but the processing may not be complete. Therefore, the effect of the unprocessed original image is not ideal. Therefore, the electronic device cannot generate an intermediate frame image based on the unprocessed original image. The electronic device can read the second original layer from a zero-shutter lag (ZSL) buffer. The ZSL buffer can be used to store data for several frames of original images.

[0141] S308: The electronic device detects a vertical synchronization signal at a second moment and displays a first intermediate frame image, wherein the first moment and the second moment are separated by a single signal cycle.

[0142] The electronic device may execute S308 after executing S307.

[0143] In some implementations, the electronic device detects a vertical synchronization signal at the second moment, indicating that the content displayed on the display screen needs to be refreshed at the second moment. The electronic device can read the first intermediate frame image and display the first intermediate frame image on the display screen. At the second moment, the image stored oldest in the electronic device is the first intermediate frame image.

[0144] In this way, since the first intermediate frame is obtained based on the first preview image and the second original image, the first intermediate frame displayed by the electronic device has good consistency with the first preview image displayed in the previous frame.

[0145] S309: The electronic device synthesizes the second original image processed by the algorithm to obtain a second preview image after the second moment.

[0146] After processing the second original image through the algorithm, the electronic device may execute S309. In other words, the electronic device may synthesize the second original image simultaneously with or between S307 and S308, and execute S309 after completing S308.

[0147] In some implementations, because the second processing duration exceeds the duration threshold, the electronic device cannot synthesize the second preview image before the first vertical synchronization signal detected after capturing the second original image, as described in S302 above. Therefore, the electronic device can obtain the second preview image after the second moment.

[0148] After synthesizing the second preview image, the electronic device may store the second preview image.

[0149] S310: The electronic device detects a vertical synchronization signal at a third moment and displays a second preview image, wherein the second moment and the third moment are separated by a single signal cycle.

[0150] After executing S309 , the electronic device may execute S310 .

[0151] In some implementations, the electronic device detects a vertical synchronization signal at the third moment, indicating that the content displayed on the display screen needs to be refreshed at the third moment. The electronic device can read the second preview image and display the second preview image on the display screen. At the third moment, the image stored oldest in the electronic device is the second preview image.

[0152] By implementing the above method, the electronic device can generate intermediate frames in the event of an anomaly (e.g., algorithm processing timeout) during the process from capturing the original image to synthesizing the preview image. Furthermore, the electronic device can reserve sufficient time for algorithm processing of the original image. This allows the electronic device to read a newly generated image each time it detects a vertical sync signal, preventing the display of the same image for two consecutive frames.

[0153] To better understand the above Figure 3 The image display method shown below is combined with Figure 5 Let me introduce it in detail.

[0154] Figure 5 The schematic diagram of the principle of the image display method used in the electronic device provided in the embodiment of the present application. Figure 3 The image display method shown is described as an example.

[0155] The first moment may refer to time t11, the second moment may refer to time t21, and the third moment may refer to time t31. The first original image may refer to an image captured by the electronic device at time t1, and the second original image may refer to an image captured by the electronic device at time t2. The first image may refer to image A synthesized and stored by the electronic device before time t11. The second image may refer to image B synthesized by the electronic device after time t21. The first intermediate frame image may refer to image B1 synthesized by the electronic device before time t21.

[0156] like Figure 5 As shown, at time t1, the electronic device generates an acquisition instruction. The electronic device responds to the acquisition instruction at time t1 and acquires the original image a. After acquiring the original image a, the electronic device can perform algorithmic processing on the original image a. Then, the electronic device can synthesize the algorithmically processed original image a into image A and store image A. At time t11, the electronic device detects the vertical synchronization signal, reads image A, and displays image A. In this embodiment of the present application, since the duration of the algorithmic processing of the original image a does not exceed the duration threshold, the electronic device is able to synthesize image A before time t11. Therefore, the electronic device is able to display image A in response to the vertical synchronization signal at time t11.

[0157] At time t2, the electronic device generates an acquisition instruction. In response to the acquisition instruction at time t2, the electronic device acquires original image b. After acquiring original image b, the electronic device may perform algorithmic processing on original image b. In this embodiment of the present application, because the duration of algorithmic processing of original image b exceeds the duration threshold, the electronic device cannot synthesize image B before time t21. Therefore, it is necessary to generate a new image before the next vertical sync signal arrives to avoid frame dropout.

[0158] At time t22, the electronic device detects that the algorithm processing duration of original image b exceeds a corresponding duration threshold. The electronic device generates image B1 based on original image b and image A and stores image B1. At time t21, the electronic device detects a vertical synchronization signal, reads image B1, and displays image B1. After time t22, the electronic device synthesizes image B with the algorithm-processed original image b and stores image B.

[0159] At time t3, the electronic device generates an acquisition instruction. The electronic device responds to the acquisition instruction at time t3 and acquires the original image c. After acquiring the original image c, the electronic device can perform algorithmic processing on the original image c. Then, the electronic device can synthesize the algorithmically processed original image c into image C and store image C. At time t31, the electronic device detects the vertical synchronization signal, reads image B, and displays image B. In the embodiment of the present application, since the above-mentioned image B is not consumed, two frames of images are stored in the Surface Flinger buffer after the synthesis of image C, and image B is stored before image C. Therefore, when the next vertical synchronization signal arrives, image B is consumed first.

[0160] In some implementations, if an electronic device processes multiple frames of collected original images through an algorithm, and the processing time exceeds a time threshold, it will cause the synthesised images stored in the electronic device to accumulate too much. In response to the above problem, in an embodiment of the present application, the electronic device monitors the number of images stored in the Surface Flinger buffer in real time. When the number of stored images exceeds a certain number (for example, two frames), the electronic device deletes the first stored frame of the image. In this way, it can ensure that the number of images stored in the Surface Flinger buffer remains within a certain range, avoid the phenomenon of frame accumulation, and reduce the consumption of storage resources.

[0161] The following combination Figure 6 The following is a flow chart of another image display method provided by an embodiment of the present application. Figure 6 The method flow chart shown is Figure 5 The method flow shown differs in that: Figure 6 In the method flowchart shown, the SurfaceFlinger buffer contains two frames of images and the electronic device executes an operation of deleting the frame of image with the earliest storage time.

[0162] like Figure 6 As shown, the method flow includes:

[0163] S601: The electronic device collects a first original image.

[0164] S602: The electronic device processes the first original image using an algorithm and monitors a first processing time.

[0165] S603: When the first processing time does not exceed the time threshold, the electronic device synthesizes the first original image processed by the algorithm to obtain a first image.

[0166] S604: The electronic device detects a vertical synchronization signal at a first moment and displays a first preview image.

[0167] S605: The electronic device collects a second original image.

[0168] S606: The electronic device processes the second original image using an algorithm and monitors a second processing time.

[0169] S607: When the second processing time exceeds the time threshold, the electronic device generates a first intermediate frame image based on the second original image and the first preview image.

[0170] S608: The electronic device detects a vertical synchronization signal at a second moment and displays a first intermediate frame image, wherein the first moment and the second moment are separated by a single signal cycle.

[0171] S609: The electronic device synthesizes the second original image processed by the algorithm to obtain a second preview image after the second moment.

[0172] For the related description of the above S601-S609, please refer to the detailed introduction of the above S301-S309, which will not be repeated here.

[0173] S610: The electronic device collects a third original image.

[0174] The electronic device may execute S610 after executing S605. That is, the electronic device may execute S610 at any time between executing S606 and executing S609.

[0175] Furthermore, the time interval between when the electronic device collects the third original image and when it collects the second original image is consistent with the time interval between when the electronic device collects the second original image and when it collects the first original image.

[0176] The third original image may refer to the next frame of original image of the second original image captured by the electronic device.

[0177] S611: The electronic device processes the third original image using an algorithm and monitors a third processing time.

[0178] After executing S610 , the electronic device may execute S611 .

[0179] In some implementations, the electronic device may process the captured raw image using one or more camera algorithms. The third processing duration may include multiple durations for each camera algorithm used by the electronic device to process the third raw image.

[0180] S612: When the third processing time exceeds the time threshold, the electronic device generates a second intermediate frame image based on the third original image and the second preview image.

[0181] The electronic device can execute S612 after executing S611. However, since the electronic device needs to use the second preview image, the electronic device needs to execute S612 after S609.

[0182] The third processing duration exceeding the duration threshold may mean that the processing duration of a camera algorithm among multiple camera algorithms used by the electronic device to process the third original image exceeds the corresponding duration threshold. For a description of exceeding the duration threshold, please refer to the detailed description of S302 above.

[0183] S613: The electronic device detects a vertical synchronization signal at a third moment and displays a second preview image, wherein the second moment and the third moment are separated by a single signal cycle.

[0184] After executing S612 , the electronic device may execute S613 .

[0185] In some implementations, the electronic device detects a vertical synchronization signal at the third moment, indicating that the content displayed on the display screen needs to be refreshed at the third moment. The electronic device can read the second preview image and display the second preview image on the display screen. At the third moment, the image stored oldest in the electronic device is the second preview image.

[0186] S614: The electronic device synthesizes the third original image processed by the algorithm to obtain a third preview image after a third moment.

[0187] After processing the third original image through the algorithm, the electronic device may execute S614. That is, the electronic device may synthesize the third original image synchronously while executing S612 to S613 or in the interval between executing S612 and S613 and complete S614 after executing S613.

[0188] S615: The electronic device collects a fourth original image.

[0189] The electronic device may execute S615 after executing S610. That is, the electronic device may execute S615 at any time between executing S610 and executing S614.

[0190] Furthermore, the time interval between the electronic device capturing the fourth original image and capturing the third original image is consistent with the time interval between the electronic device capturing the third original image and capturing the second original image.

[0191] The fourth original image may refer to the next frame of original image of the third original image captured by the electronic device.

[0192] S616: The electronic device processes the fourth original image using an algorithm and monitors a fourth processing time.

[0193] After executing S615 , the electronic device may execute S616 .

[0194] In some implementations, the electronic device may process the captured raw image using one or more camera algorithms. The fourth processing duration may include multiple durations for each camera algorithm used by the electronic device to process the fourth raw image.

[0195] S617: When the fourth processing time does not exceed the time threshold, the electronic device synthesizes the fourth original image processed by the algorithm to obtain a fourth preview image.

[0196] The electronic device may execute S617 simultaneously with or after executing S616.

[0197] In some implementations, the fourth processing duration not exceeding the duration threshold may include: multiple durations for the electronic device to process the fourth original image through each camera algorithm do not exceed the corresponding duration threshold.

[0198] After synthesizing the fourth preview image, the electronic device may store the fourth preview image.

[0199] S618: The electronic device detects that the number of stored images exceeds two frames, and deletes the second intermediate frame image with the earliest current storage time.

[0200] After synthesizing the fourth preview image, the electronic device stores three frame images, ie, the second intermediate frame image, the third preview image, and the fourth preview image.

[0201] At this time, the electronic device detects that the number of stored images exceeds two frames, and deletes the image frame with the earliest current storage time, that is, the second intermediate frame image.

[0202] This can avoid the situation where the electronic device reads and displays a preview image generated too early, causing the user to see a delayed preview image. It can also avoid frame accumulation and waste of storage resources while reducing frame loss.

[0203] S619: The electronic device detects a vertical synchronization signal at a fourth moment and displays a third preview image, wherein the third moment and the fourth moment are separated by a single signal cycle.

[0204] After executing S618 , the electronic device may execute S619 .

[0205] In some implementations, the electronic device detects a vertical synchronization signal at the fourth moment, indicating that the content displayed on the display screen needs to be refreshed at the fourth moment. The electronic device can read the third preview image and display the first preview image on the display screen. At the fourth moment, the image stored oldest in the electronic device is the third preview image.

[0206] By implementing the above method, the electronic device can generate intermediate frames in the event of an anomaly (e.g., algorithm processing timeout) during the process from capturing the original image to synthesizing the preview image. Furthermore, the electronic device can reserve sufficient time for algorithm processing of the original image. Furthermore, the electronic device can display a fresher preview image.

[0207] In this way, the electronic device can read a newly generated image each time it detects a vertical synchronization signal, preventing the same image from being displayed for two consecutive frames. Furthermore, the number of images stored by the electronic device remains within a certain range, preventing frame accumulation and reducing storage resource consumption.

[0208] In a possible implementation, the reasons for frame accumulation are in addition to the above Figure 6 In addition to the method flow shown, the method may also include: the current electronic device performance deterioration triggers the generation of an intermediate frame. In this case, the electronic device may also consider deleting the earliest stored image frame.

[0209] To better understand the above Figure 6 The image display method shown below is combined with Figure 7 Let me introduce it in detail.

[0210] Figure 7 The schematic diagram of the principle of the image display method used in the electronic device provided in the embodiment of the present application. Figure 6 The image display method shown is described as an example.

[0211] The first time may refer to time t11, the second time may refer to time t21, the third time may refer to time t31, and the fourth time may refer to time t41. The first original image may refer to an image captured by the electronic device at time t1, the second original image may refer to an image captured by the electronic device at time t2, the third original image may refer to an image captured by the electronic device at time t3, and the fourth original image may refer to an image captured by the electronic device at time t4. The first image may refer to image A synthesized and stored by the electronic device before time t11. The second image may refer to image B synthesized by the electronic device after time t21, and the third image may refer to image C synthesized by the electronic device after time t31. The fourth image may refer to image D synthesized and stored by the electronic device before time t41. The first intermediate frame image may refer to image B1 synthesized by the electronic device before time t21, and the second intermediate frame image may refer to image C1 synthesized by the electronic device before time t31.

[0212] like Figure 7 As shown, the electronic device responds to the capture instruction at time t1 and captures the original image a. Thereafter, the electronic device can perform algorithmic processing on the original image a. The electronic device can then synthesize the algorithmically processed original image a into image A and store image A. At time t11, the electronic device detects the vertical synchronization signal, reads image A, and displays image A. In this embodiment of the present application, since the duration of the algorithmic processing of the original image a does not exceed the duration threshold, the electronic device is able to synthesize image A before time t11. Therefore, the electronic device is able to display image A in response to the vertical synchronization signal at time t11.

[0213] The electronic device responds to the capture instruction at time t2 and captures the original image b. The electronic device can then perform algorithmic processing on the original image b. In this embodiment of the present application, because the duration of the algorithmic processing of the original image b exceeds the duration threshold, the electronic device cannot synthesize image B before time t21. Therefore, a new image must be generated before the next vertical sync signal arrives to avoid frame loss.

[0214] At time t22, the electronic device detects that the duration of processing original image b using a certain algorithm exceeds a corresponding duration threshold. The electronic device may generate image B1 based on original image b and image A, and store image B1. At time t21, the electronic device detects a vertical synchronization signal, reads image B1, and displays image B1. After time t21, the electronic device may synthesize image B with the original image b processed by the algorithm, and store image B.

[0215] The electronic device responds to the capture instruction at time t3 and captures original image c. The electronic device can then perform algorithmic processing on original image c. In this embodiment of the present application, because the duration of algorithmic processing on original image c also exceeds the duration threshold, the electronic device cannot synthesize image C before time t31. Therefore, a new image must be generated before the next vertical synchronization signal arrives to avoid frame loss.

[0216] At time t32, the electronic device detects that the duration of processing original image c using a certain algorithm exceeds a corresponding duration threshold. The electronic device then generates image C1 based on original image c and image B and stores image C1. At this point, the electronic device stores two image frames: image B and image C1. At time t31, the electronic device detects a vertical synchronization signal, reads image B, and displays it. After time t31, the electronic device can synthesize image C from the processed original image c and store it.

[0217] The electronic device responds to the capture instruction at time t4 and captures the original image d. The electronic device may then perform algorithmic processing on the original image d. The electronic device then synthesizes the processed original image d into image D and stores image D. At this point, the Surface Flinger buffer stores three frames: image C1, image C, and image D. Because the electronic device has implemented measures to address frame accumulation, once it detects that the Surface Flinger buffer stores three frames, that is, when the synthesis of image D is complete, the Surface Flinger buffer deletes image C1.

[0218] At time t41, the electronic device detects the vertical synchronization signal, reads image C, and displays image C. In this embodiment of the present application, because the duration of the algorithmic processing of original image d does not exceed the duration threshold, the electronic device is able to synthesize image D before time t41. However, at time t41, the electronic device stores two image frames: image C and image D. Image C is stored before image D. Therefore, the electronic device is able to display image C in response to the vertical synchronization signal at time t41.

[0219] Next, combine Figure 8 An internal module interaction diagram of an electronic device's operating system corresponding to the image display method process provided in an embodiment of the present application is introduced.

[0220] Figure 8The user operation module, camera, camera algorithm module, timeout judgment module, interpolation algorithm module, frame stacking processing module, Surface Flinger and display driver data storage module shown in are modules included in the electronic device for managing the process from acquiring original images to displaying preview images. For the definitions of these modules, please refer to the detailed description of the electronic device software architecture later. The specific functions of these modules have been recorded in detail in the following process and will not be repeated here.

[0221] S801: A user operation module detects an operation of starting a first application.

[0222] In some implementations, the first application may include a camera application. The operation may include, but is not limited to, a click operation, a gesture command, a voice command, etc. on an icon of the camera application.

[0223] S801-1. The frame accumulation processing module sends an instruction to Surface Flinger. The instruction is used to instruct Surface Flinger to send the number of stored images to the frame accumulation processing module when updating the Surface Flinger buffer storage information.

[0224] In some implementations, S801-1 can be executed after the electronic device acquires the frame stacking processing module. For example, S801-1 can be executed after the electronic device starts the first application.

[0225] S802: The user operation module sends a collection instruction to the camera.

[0226] In some implementations, the user operation module may be located in the application framework layer. The user operation module may obtain the acquisition instruction sent by the first application program and send the acquisition instruction to the camera.

[0227] Illustratively, S802 may be executed before the electronic device captures the first original image, the second original image, the third original image, and the fourth original image.

[0228] S803: The camera collects the original image.

[0229] For example, the camera may capture a first original image, a second original image, a third original image, and a fourth original image. In some implementations, the electronic device may temporarily store the captured original images in a ZSL buffer. S804: The camera sends the original images to the camera algorithm module.

[0230] S805: The camera algorithm module performs algorithm processing on the original image.

[0231] In some implementations, the number of algorithms used by the camera algorithm module can be one or more. Figure 5 S806: The timeout determination module obtains the processing time of the camera algorithm module when executing S805.

[0232] In some implementations, the timeout determination module may obtain multiple times when each of the multiple algorithms processes the original image. For the introduction of S806, please refer to the detailed description of S302 above, which will not be repeated here.

[0233] Optionally, the timeout determination module may also obtain the time it takes to capture the original image and the time it takes to pre-process the original image. Exemplarily, the timeout determination module may also obtain the time it takes for the original image to be processed by the image frontend (IFE) and the image processing engine (IPE). S807: The timeout determination module determines whether the processing duration exceeds a duration threshold.

[0234] The timeout judgment module stores the duration thresholds set for the multiple camera algorithms in the camera algorithm module.

[0235] If the timeout determination module determines that the processing time of a certain camera algorithm exceeds the corresponding time threshold during the execution of S805, S808 is executed;

[0236] If the timeout determination module determines that the processing time of all camera algorithms does not exceed the corresponding time threshold during the execution of S805, S808 is not executed and the process directly jumps to S814.

[0237] S808. The timeout determination module sends a frame insertion instruction to the frame insertion algorithm module.

[0238] S809 , the frame interpolation algorithm module generates an intermediate frame image based on the original image and the previous synthesized image.

[0239] In some implementations, after receiving the frame insertion instruction, the frame insertion algorithm module can read the original image of the current frame from the ZSL buffer and generate an intermediate frame image by combining it with the synthesized preview image of the previous frame.

[0240] S810: The interpolation algorithm module sends the intermediate frame image to Surface Flinger.

[0241] S811. Surface Flinger stores the intermediate frame image in the Surface Flinger buffer.

[0242] S812: Surface Flinger sends the number of currently stored images to the frame accumulation processing module. S813: The frame accumulation processing module determines whether to delete the first stored image.

[0243] Since the number of images stored in the Surface Flinger buffer changes after S811 is executed, the frame accumulation processing module needs to determine whether the number of images currently stored is greater than two frames. If so, the oldest image needs to be deleted; otherwise, the stored images do not need to be deleted.

[0244] S814. The camera algorithm module sends the processed original image to Surface Flinger.

[0245] S815. Surface Flinger synthesizes a preview image with the processed original image and stores the preview image.

[0246] S816. Surface Flinger sends the number of currently stored images to the frame accumulation processing module.

[0247] S817: The frame accumulation processing module determines whether to delete the first stored image.

[0248] Similarly, since the number of images stored in the Surface Flinger buffer has changed since S816 was executed, the frame accumulation processing module needs to determine whether the number of images currently stored is greater than two frames. If so, the oldest image needs to be deleted; otherwise, the stored images do not need to be deleted.

[0249] S818: The display driver detects a vertical synchronization signal.

[0250] S819. The display driver sends a read instruction to Surface Flinger.

[0251] S820. Surface Flinger sends the earliest image stored in the Surface Flinger buffer to the display driver.

[0252] In a possible implementation, the electronic device may further determine whether the Surface Flingerbuffer stores a preview image before generating an intermediate frame image each time, that is, after determining that the duration of processing a certain frame of the original image exceeds a duration threshold.

[0253] If the Surface Flingerbuffer stores a preview image, the electronic device will no longer generate an intermediate frame image. This is because the electronic device generates an intermediate frame image only when it determines that the processing time for a frame of the original image exceeds the time threshold and the vertical synchronization signal arrives, and there is an image that can be displayed on the display. However, at this time, the electronic device has already stored a preview image that was displayed on the display when the vertical synchronization signal arrived.

[0254] If the Surface Flingerbuffer stores a preview image, the electronic device generates an intermediate frame image. For details, please refer to the above description of generating the intermediate frame image, which will not be repeated here.

[0255] In this way, the electronic device can also ensure that the number of stored preview images is within a certain range and can also avoid the occurrence of frame accumulation.

[0256] In one possible implementation, if the electronic device's performance deteriorates, resulting in the synthesis of multiple preview images within a certain signal cycle or a short period of time, the electronic device may not delete a frame of preview image, but instead reduce the current signal cycle so that the electronic device can quickly display the synthesized preview image in a short period of time. For example, due to performance deterioration, the electronic device generates three preview images within a 66ms signal cycle. The electronic device can adjust the 66ms signal cycle to a 22ms signal cycle. The electronic device will have three 22ms signal cycles in the next 66ms. In this way, the electronic device can receive three vertical synchronization signals in the next 66ms, consume the three preview images, and display the three preview images. The electronic device then adjusts the signal cycle back to 66ms. In this way, the electronic device will neither arbitrarily lose the preview images obtained by actually acquiring the original images nor cause long-term storage resource usage.

[0257] Figure 9 Schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0258] The electronic device 100 may be equipped with Or a portable terminal device with other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, an in-vehicle device, a smart home device and / or a smart city device. Without limitation, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or touch panel, a desktop computer with a touch-sensitive surface or touch panel, and other non-portable terminal devices. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0259] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0260] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0261] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0262] In an embodiment of the present application, the processor 110 can be configured to monitor the time it takes to process each frame of the original image using the algorithm. The processor 110 can also be configured to, in the event of a timeout, generate an intermediate frame image based on the original image of the current frame and the preview image of the previous frame. The processor 110 can also be configured to delete the preview image with the oldest storage time if the number of stored images exceeds two.

[0263] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0264] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0265] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0266] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0267] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0268] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0269] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0270] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0271] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0272] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0273] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0274] In the embodiment of the present application, the display screen 194 can be used to display a preview image read out when a vertical synchronization signal is detected. The preview image is a preview image frame with the earliest storage time when the preview image is read out.

[0275] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0276] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0277] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0278] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0279] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0280] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0281] In the embodiment of the present application, the electronic device 100 implements the image display method provided in the embodiment of the present application, which first relies on the ISP and the original image captured by the camera 193, and secondly relies on the image computing and processing capabilities provided by the video codec and the GPU. Specifically, the electronic device 100 can implement algorithms such as face recognition, body recognition, and re-identification through the computing and processing capabilities provided by the NPU.

[0282] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0283] In the embodiment of the present application, the internal memory 121 can store preview images and intermediate frame images.

[0284] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0285] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage cell potential level; universal flash storage (UFS) and embedded multi media card (eMMC) can be divided into UFS and embedded multi media card according to the storage specification.

[0286] The random access memory can be directly read and written by the processor 110 and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data. Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into the random access memory for direct reading and writing by the processor 110.

[0287] In the embodiment of the present application, the code for implementing the image display described in the embodiment of the present application may be stored in a non-volatile memory. When running the camera application, the electronic device 100 may load the executable code stored in the non-volatile memory into the random access memory.

[0288] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0289] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0290] The audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio input into digital audio signals.

[0291] Speaker 170A, also known as a "horn," is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music or make hands-free calls through speaker 170A. Receiver 170B, also known as a "handset," is used to convert audio electrical signals into sound signals. When electronic device 100 receives a call or voice message, the voice can be heard by placing receiver 170B close to the human ear. Microphone 170C, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. Headphone jack 170D is used to connect wired headphones.

[0292] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be set on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes. The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use a magnetic sensor 180D to detect the opening and closing of a flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be set. The accelerometer 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the electronic device's posture, for applications such as switching between landscape and portrait modes and pedometers. The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser technology. In some embodiments, when photographing a scene, the electronic device 100 can use the distance sensor 180F to measure distance for rapid focusing. The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches. The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. The touch sensor 180K is also called a "touch device."The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen." The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human vocal cords. The buttons 190 include a power button, a volume button, etc. The buttons 190 can be mechanical buttons. They can also be touch buttons. The electronic device 100 can receive key inputs and generate key signal inputs related to the user settings and function control of the electronic device 100. The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts or for touch vibration feedback. The indicator 192 can be an indicator light that can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0293] Figure 10 1 is a schematic diagram of the system structure of the electronic device 100 provided in an embodiment of the present application.

[0294] A layered architecture divides the system into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system is divided into five layers: application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer, from top to bottom.

[0295] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include a camera, a gallery, etc.

[0296] The application framework layer provides an application programming interface (API) and programming framework for the application layer's applications. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, where the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and programming framework for camera applications.

[0297] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system.

[0298] In an embodiment of the present application, the hardware abstraction layer may include but is not limited to: a camera hardware abstraction layer, a camera algorithm library, a timeout judgment module, an interpolation algorithm module, a frame stacking processing module and Surface Flinger.

[0299] The camera hardware abstraction layer may provide virtual hardware of camera device 1, camera device 2, or more camera devices.

[0300] For example, the camera device 1 may correspond to the front camera in the electronic device 100, and the camera device 2 may correspond to the rear camera in the electronic device 100. In the embodiment of the present application, the camera device 1 and the camera device 2 are only used for exemplary description and are not limited thereto.

[0301] The camera algorithm library may include operating code and data for processing the original images captured by the camera in the embodiments of the present application.

[0302] Exemplarily, the camera algorithm library may include, but is not limited to: a face detection algorithm, a motion detection algorithm, an automatic exposure algorithm, a gesture detection algorithm, an object detection algorithm, a scene detection algorithm, and the like.

[0303] The timeout determination module can be used to detect the time it takes for the electronic device 100 to process the original image using multiple algorithms in the camera algorithm library. The timeout determination module can also store multiple time thresholds for the processing of the original image by multiple algorithms in the camera algorithm library, and can also be used to determine whether the processing time of each algorithm exceeds the time threshold.

[0304] The interpolation algorithm module can be used to generate an intermediate frame image based on the current frame original image and the previous frame synthesized image when the processing time of a certain algorithm exceeds the corresponding time threshold.

[0305] For example, the interpolation algorithm can read the original image of the current frame from the ZSL buffer. The interpolation algorithm module can also store multiple algorithms for generating intermediate frame images.

[0306] The frame accumulation processing module can be used to determine in real time whether the preview image stored in the Surface Flingerbuffer is greater than the storage threshold. The frame accumulation processing module can also be used to instruct the Surface Flingerbuffer to delete the image with the oldest storage time.

[0307] Surface Flinger can include Surface Flingerbuffer. Surface Flinger is used to synthesize the original image processed by the algorithm to obtain a preview image.

[0308] Surface Flingerbuffer can be used to store the preview image after synthesis, and can also be used to store the intermediate frame images generated by the interpolation algorithm module.

[0309] The driver layer is the layer between hardware and software. It includes drivers for various hardware components. These drivers can include camera device drivers, digital signal processor drivers, image processor drivers, and display drivers.

[0310] The camera device driver drives the camera's sensor to capture images and the image signal processor to pre-process them. The digital signal processor driver drives the digital signal processor to process images. The image processor driver drives the graphics processor to process images. The display driver drives the display screen to update displayed content.

[0311] The image display method in the embodiment of the present application is described in detail below in conjunction with the above system structure:

[0312] In response to the operation of opening the camera application, such as the user clicking the camera application icon, the camera application calls the camera access interface of the application framework layer, starts the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer.

[0313] The camera hardware abstraction layer sends a camera start instruction to the camera device driver in the driver layer. This camera device driver activates the corresponding camera sensor and uses it to collect image light signals. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0314] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0315] The camera hardware abstraction layer sends raw images to the camera algorithm library. The library stores the program code for various camera algorithms. It executes this code based on a digital signal processor and image processor.

[0316] The timeout judgment module can detect the corresponding processing time during the processing of different camera algorithms.

[0317] If the timeout determination module detects that the processing time of a camera algorithm exceeds the corresponding processing threshold, it sends an interpolation instruction to the interpolation algorithm module. The interpolation algorithm module responds to the interpolation instruction by generating an intermediate frame image based on the currently acquired original image and the previously synthesized preview image, and sends the intermediate frame image to Surface Flinger.

[0318] Surface Flingerbuffer then stores the intermediate frame image. While the above steps are being executed, the camera algorithm library continues to process the currently acquired raw image and sends the processed raw image to SurfaceFlinger. Surface Flinger then synthesizes it into a preview image and stores it in SurfaceFlingerbuffer.

[0319] If the timeout detection module detects that the processing time of all executed camera algorithms exceeds the corresponding processing threshold, it will not send the interpolation instruction to the interpolation algorithm module. The camera algorithm library directly sends the processed raw image to Surface Flinger. Surface Flinger then synthesizes it into a preview image and stores it in the Surface Flinger buffer.

[0320] When the display driver detects a vertical sync signal, it reads instructions from Surface Flinger. Surface Flinger reads the oldest frame of image stored in Surface Flingerbuffer and sends it to the display driver. It's worth noting that each time the display driver detects a vertical sync signal and reads instructions from Surface Flinger, Surface Flingerbuffer stores the preview image to be consumed.

[0321] Finally, the display driver can drive the display screen to display the read frame of image.

[0322] The various implementation modes of this application can be combined arbitrarily to achieve different technical effects.

[0323] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0324] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0325] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image display method, characterized in that: The method is applied to an electronic device, and includes: At a first moment, displaying a first preview image; The electronic device collects a second original image; The electronic device processes the second original image using at least one algorithm, where the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold; The electronic device obtains a first intermediate frame image based on the first preview image and the second original image; At a second moment, displaying the first intermediate frame image, the second moment being a moment separated by a single signal cycle after the first moment; The electronic device obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm; At a third moment, the second preview image is displayed. The third moment is a moment separated from the second moment by a single signal cycle.

2. The method according to claim 1, characterized in that The first duration threshold is determined by one or more of the following: the single signal cycle, the historical processing duration of the first algorithm, the historical duration of acquiring original images, and the historical duration of synthesizing preview images.

3. The method according to claim 1 or 2, characterized in that At a third moment, before displaying the second preview image, the method further includes: The electronic device collects a third original image; The electronic device processes the second original image using the at least one algorithm, where the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold; The electronic device obtains a second intermediate frame image based on the second preview image and the third original image; At a third moment, after displaying the second preview image, the method further includes: The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm; The electronic device generates a fourth preview image or a third intermediate frame image; The electronic device deletes the second intermediate frame image; At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment; At a fifth moment, the fourth preview image or the third intermediate frame image is displayed. The fifth moment is a moment separated from the fourth moment by a single signal cycle.

4. The method according to claim 1 or 2, characterized in that When the electronic device processes the second original image using at least one algorithm, the method further includes: The electronic device determines that no preview image is currently stored; At a third moment, before displaying the second preview image, the method further includes: The electronic device collects a third original image; The electronic device processes the second original image using the at least one algorithm, where the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold; The electronic device determines that a preview image is currently stored; At a third moment, after displaying the second preview image, the method further includes: The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm; a fourth preview image of the electronic device; At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment; At a fifth moment, the fourth preview image is displayed. The fifth moment is a moment separated from the fourth moment by a single signal cycle.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device includes a buffer queue, which is used to store preview images and intermediate frame images, and the maximum storage capacity of the buffer queue is two frames of images.

6. The method according to any one of claims 1 to 5, characterized in that The at least one algorithm includes any one or more of the following: face detection algorithm, motion detection algorithm, automatic exposure algorithm, gesture detection algorithm, object detection algorithm, scene detection algorithm, autofocus algorithm, white balance algorithm, bad pixel correction algorithm, noise reduction algorithm, shadow correction algorithm, and color enhancement algorithm.

7. The method according to any one of claims 1 to 6, characterized in that The electronic device further includes: a camera, a hardware abstraction layer, a surface synthesizer, and a display screen, wherein: At a first moment, the display screen is used to display a first preview image; The camera is used to collect a second original image; The hardware abstraction layer is configured to process the second original image using at least one algorithm, where the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold; The hardware abstraction layer is used to obtain a first intermediate frame image based on the first preview image and the second original image; At a second moment, the display screen is used to display the first intermediate frame image, and the second moment is a moment separated by a single signal cycle after the first moment; The surface synthesizer obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm; At a third moment, the display screen is used to display the second preview image. The third moment is a moment separated by a single signal cycle from the second moment.

8. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images