Image display method and related device

Abstract

The invention provides an image display method and a related device. In a scene of camera preview, when the electronic equipment detects that the duration of processing an original image through an algorithm exceeds a duration threshold, the electronic equipment can obtain a middle frame image based on a current original image and a previous frame preview image. In this way, the intermediate frame image can be displayed when the next vertical synchronization signal arrives. According to the method provided by the invention, sufficient time can be reserved to carry out algorithm processing on the original image, so that the frame loss phenomenon is reduced, and the phenomenon that a user observes that a screen is stuck is reduced. Therefore, the smoothness of image display of the electronic equipment can be ensured to a certain extent, and the visual experience of a user is good.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image display method and related devices. Background Art

[0002] The electronic device can display a preview image. The electronic device can process the original image captured by the camera through a variety of camera algorithms to obtain a more detailed preview image. In this way, it takes a long time, resulting in the electronic device being unable to display the preview image coherently, that is, frame loss occurs. How to make the electronic device display the preview image coherently to a certain extent is a problem that needs to be solved in the art. Summary of the invention

[0003] The present application provides an image display method and related devices, which can ensure the continuity of images displayed by electronic devices to a certain extent.

[0004] In a first aspect, the present application provides an image display method, which is applied to an electronic device, and the method includes: at a first moment, displaying a first preview image; the electronic device capturing a second original image; the electronic device processing the second original image through at least one algorithm, the at least one algorithm including a first algorithm, the processing time of the first algorithm exceeding a first time threshold; the electronic device obtaining a first intermediate frame image based on the first preview image and the second original image; at a second moment, displaying the first intermediate frame image, the second moment being a moment after the first moment with a single signal cycle interval; the electronic device obtaining a second preview image after the second moment based on the second original image processed by the at least one algorithm; at a third moment, displaying the second preview image, the third moment being a moment after the second moment with a single signal cycle interval.

[0005] By implementing the above method, the electronic device can reserve sufficient time to perform algorithm processing on the original image, thereby reducing frame loss, and further reducing the phenomenon of screen freeze observed by the user. In this way, the smoothness of the image displayed by the electronic device can be guaranteed to a certain extent, and the user's visual experience is good.

[0006] In conjunction with the first aspect, in some implementations, the first duration threshold is determined by one or more of the following: the single signal cycle, the historical processing duration of the first algorithm, the historical duration of acquiring the original image, and the historical duration of synthesizing the preview image. In this way, the electronic device can set a response duration threshold for each camera algorithm.

[0007] In combination with the first aspect, in some implementations, at the third moment, before displaying the second preview image, the method further includes:

[0008] The electronic device collects a third original image;

[0009] The electronic device processes the second original image by using the at least one algorithm, wherein the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold;

[0010] The electronic device obtains a second intermediate frame image based on the second preview image and the third original image;

[0011] At the third moment, after displaying the second preview image, the method further includes:

[0012] The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm;

[0013] The electronic device generates a fourth preview image or a third intermediate frame image;

[0014] The electronic device deletes the second intermediate frame image;

[0015] At a fourth moment, displaying the third preview image, the fourth moment being a moment after the third moment and spaced apart by a single signal cycle;

[0016] At a fifth moment, the fourth preview image or the third intermediate frame image is displayed. The fifth moment is a moment separated by a single signal cycle from the fourth moment.

[0017] In this way, the electronic device can read a newly generated image each time it detects a vertical synchronization signal, and will not display the same image for two consecutive frames. In addition, the number of images stored in the electronic device is kept within a certain range, avoiding frame accumulation and reducing the consumption of storage resources.

[0018] In conjunction with the first aspect, in some implementations, when the electronic device processes the second original image using at least one algorithm, the method further includes:

[0019] The electronic device determines that no preview image is currently stored;

[0020] At the third moment, before displaying the second preview image, the method further includes:

[0021] The electronic device collects a third original image;

[0022] The electronic device processes the second original image by using the at least one algorithm, wherein the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold;

[0023] The electronic device determines that a preview image is currently stored;

[0024] At the third moment, after displaying the second preview image, the method further includes:

[0025] The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm;

[0026] a fourth preview image of the electronic device;

[0027] At a fourth moment, displaying the third preview image, the fourth moment being a moment after the third moment and spaced apart by a single signal cycle;

[0028] At a fifth moment, the fourth preview image is displayed. The fifth moment is a moment separated from the fourth moment by a single signal cycle.

[0029] In this way, the electronic device can also ensure that the number of stored preview images is within a certain range and can also avoid the occurrence of frame accumulation.

[0030] In conjunction with the first aspect, in some implementations, the electronic device includes a cache queue, the cache queue is used to store preview images and intermediate frame images, and the maximum storage capacity of the cache queue is two frames of images. In an embodiment of the present application, the Surface fader buffer in the electronic device stores at least 0 frames of images and can store at most 2 frames of images.

[0031] In conjunction with the first aspect, in some implementations, the at least one algorithm includes any one or more of the following: a face detection algorithm, a motion detection algorithm, an automatic exposure algorithm, a gesture detection algorithm, an object detection algorithm, and a scene detection algorithm. In the embodiments of the present application, more camera algorithms may also be included, which is not limited.

[0032] In combination with the first aspect, in some implementations, the electronic device further includes: a camera, a hardware abstraction layer, a surface synthesizer, and a display screen, wherein:

[0033] At a first moment, the display screen is used to display a first preview image;

[0034] The camera is used to collect a second original image;

[0035] The hardware abstraction layer is used to process the second original image by using at least one algorithm, wherein the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold;

[0036] The hardware abstraction layer is used to obtain a first intermediate frame image based on the first preview image and the second original image;

[0037] At a second moment, the display screen is used to display the first intermediate frame image, and the second moment is a moment separated by a single signal cycle after the first moment;

[0038] The surface synthesizer obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm;

[0039] At a third moment, the display screen is used to display the second preview image. The third moment is a moment separated by a single signal cycle from the second moment.

[0040] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method of the first aspect or any implementation of the first aspect.

[0041] In a third aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A user interface of an electronic device in a preview scenario provided by an embodiment of the present application;

[0043] Figure 2A A schematic diagram of the principle of an electronic device from capturing an original image to displaying a preview image provided by an embodiment of the present application;

[0044] Figure 2B A schematic diagram of the principle of another electronic device from capturing an original image to displaying a preview image provided by an embodiment of the present application;

[0045] Figure 3 A flow chart of an image display method provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the processing time of an electronic device monitoring algorithm provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the principle of another electronic device using an image display method provided in an embodiment of the present application;

[0048] Figure 6 A flowchart of another image display method provided in an embodiment of the present application;

[0049] Figure 7 A schematic diagram of the principle of another electronic device using an image display method provided in an embodiment of the present application;

[0050] Figure 8 An internal module interaction diagram of an electronic device using an image display method provided by an embodiment of the present application;

[0051] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0052] Fig.10 A software structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0054] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0055] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0056] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0057] First, the application scenarios involved in the embodiments of the present application are introduced.

[0058] Figure 1 The following example illustrates a scenario in which an electronic device provided by an embodiment of the present application displays a preview image.

[0059] like Figure 1 As shown, after detecting the operation of starting the camera application, the electronic device can turn on the camera (eg, the rear camera) and display the user interface 200. The user interface 200 may include, but is not limited to: area 201, area 202, and area 203.

[0060] Among them, area 201 (also called preview area) occupies most of the area in the user interface 200, and is mainly used to display a preview image. The preview image may refer to an image generated by the electronic device during the preview stage of capturing an image, and the preview image may change as the electronic device moves.

[0061] Area 202 is fixedly displayed below area 201, and is mainly used to display various shooting modes provided by the camera application, such as professional mode, screen recording mode, photo mode, portrait mode, and night scene mode, etc. Exemplarily, in user interface 200, the electronic device uses the photo mode to display the preview image.

[0062] Area 203 is fixedly displayed below area 202, and is mainly used to display multiple function options, such as a "shoot" option, an "image" option, and a "switch camera" option. For example, when the electronic device moves to a suitable shooting angle, it can respond to a user's click operation on the "shoot" option to implement the function of shooting an image.

[0063] In some implementations, during the process of displaying the preview image, the camera application may drive the image sensor (ie, Senor) in the camera of the electronic device to start.

[0064] Then, after the Senor in the electronic device is started, it can receive the periodic acquisition instructions sent by the camera application (ie, Cameraapk), acquire the original image used to generate the preview image, and send the acquired original image to the hardware abstraction layer (ie, HAL layer).

[0065] Next, the hardware abstraction layer in the electronic device can process the original image with relevant algorithms.

[0066] In the embodiments of the present application, the algorithm may include a camera algorithm. The camera algorithm may refer to a series of computer science and mathematical methods used to process and optimize the original images captured by the camera. The camera algorithm can improve the quality of the preview image and enhance the camera performance through a variety of methods to provide users with a better user experience.

[0067] Finally, the surface synthesizer (Surface Flinger) in the electronic device synthesizes the processed original image and stores the synthesized preview image in the Surface Flinger consumer queue. When the vertical synchronization signal arrives, the display driver of the electronic device reads the synthesized preview image from the Surface Flinger buffer and displays the preview image on the display screen.

[0068] The vertical synchronization signal is used to trigger the hardware to refresh the image displayed on the display screen. The electronic device will have a vertical synchronization signal triggered by the hardware driver every time a signal cycle is interrupted.

[0069] In some implementations, the period for generating a vertical synchronization signal can be determined by the frame rate of a display screen of the electronic device. For example, the period can be the inverse of the frame rate of the display screen. Exemplarily, the frame rate of the display screen of the electronic device can be any value such as 30 Hz, 60 Hz, 70 Hz, 75 Hz, 80 Hz, 90 Hz or 120 Hz. Taking the frame rate of the display screen as 60 Hz as an example, the period for generating the vertical synchronization signal is 1 / 60 = 0.01667 seconds (s) = 16.667 milliseconds (ms). In other words, the electronic device displays a preview image for 16.667 ms, and then displays the next preview image, also for 16.667 ms.

[0070] In a possible implementation, the electronic device may support multiple different frame rates. The frame rate of the electronic device can be switched between the above different frame rates. In the embodiment of the present application, the frame rate is the frame rate currently used by the electronic device. That is, the period of generating the vertical synchronization signal is the inverse of the frame rate currently used by the electronic device.

[0071] In some implementations, the acquisition instructions generated by the camera application of the electronic device in the preview scene can also be obtained based on the above vertical synchronization signal. For example, in order to better synchronize with the display refresh, the cycle of the acquisition instructions is the same as the cycle of generating the vertical synchronization signal.

[0072] The following is a schematic diagram of the principle of an electronic device from capturing an original image to displaying a preview image.

[0073] For example, Figure 2A As shown, at time t1, the electronic device generates a capture instruction through the camera application. The electronic device responds to the capture instruction at time t1 and captures the original image a. After capturing the original image a, the electronic device can perform algorithm processing on the original image a through the HAL layer.

[0074] Then, the electronic device can synthesize image A from the original image a processed by the algorithm through Surface Flinger, and store the synthesized image A in the Surface Flinger buffer. At time t11, the electronic device detects the vertical synchronization signal, reads image A from the Surface Flinger buffer, and displays image A.

[0075] During the time period T before time t11, the electronic device displays image M. Image M is an image stored in the Surface Flinger buffer before image A.

[0076] Similarly, at time t2, the electronic device generates an acquisition instruction through the camera application. The electronic device responds to the acquisition instruction at time t2 and acquires the original image b. After acquiring the original image b, the electronic device can perform algorithmic processing on the original image b through the HAL layer. Then, the electronic device can synthesize the original image b processed by the algorithm into image B through Surface Flinger, and store the synthesized image B in the Surface Flinger buffer. At time t21, the electronic device detects the vertical synchronization signal, reads image B from the Surface Flinger buffer, and displays image B.

[0077] Similarly, at time t3, the electronic device generates an acquisition instruction through the camera application. The electronic device responds to the acquisition instruction at time t3 and acquires the original image c. After acquiring the original image c, the electronic device can perform algorithmic processing on the original image c through the HAL layer. Then, the electronic device can synthesize the original image c processed by the algorithm into image C through Surface Flinger, and store the synthesized image C in the Surface Flinger buffer. At time t31, the electronic device detects the vertical synchronization signal, reads image C from the Surface Flinger buffer, and displays image C.

[0078] There is a single signal cycle between time t1 and time t2, and a single signal cycle between time t2 and time t3. There is a single signal cycle between time t11 and time t21, and a single signal cycle between time t21 and time t31. In a possible implementation, time t2 and time t11 may refer to the same time, and time t3 and time t21 may refer to the same time.

[0079] From the above content, it can be known that when the electronic device stores image A at time t11, image B at time t21, and image C at time t31, the electronic device can display image A from time t11 to time t21, display image B from time t21 to time t31, and start to display image C at time t31. In other words, whenever the electronic device detects a vertical synchronization signal and stores a newly synthesized image, the electronic device can continuously display images, and the same frame of image will not be displayed in two signal cycles.

[0080] However, there are some situations, for example: if the electronic device takes too long to perform algorithm processing on the original image, frame loss may occur.

[0081] In the embodiment of the present application, frame loss may refer to the phenomenon that during the process of the electronic device displaying a preview image, the picture displayed by the electronic device becomes stuck or discontinuous due to various reasons.

[0082] The following is based on Figure 2B The schematic diagram of the principle of the electronic device from capturing the original image to displaying the preview image is used to introduce the scenario of frame loss.

[0083] For example, Figure 2B As shown, at time t1, the electronic device generates a capture instruction through the camera application. The electronic device responds to the capture instruction at time t1 and captures the original image a. After capturing the original image a, the electronic device can perform algorithm processing on the original image a through the HAL layer.

[0084] Then, the electronic device can use Surface Flinger to synthesize image A with the original image a processed by the algorithm, and store the synthesized image A in the Surface Flinger buffer.

[0085] At time t11, the electronic device detects the vertical synchronization signal, reads image A from the Surface Flinger buffer, and displays image A. During the time T before time t11, the electronic device displays image M. Image M is an image stored in the Surface Flinger buffer before image A.

[0086] Similarly, at time t2, the electronic device generates an acquisition instruction through the camera application. The electronic device responds to the acquisition instruction at time t2 and acquires the original image b. After acquiring the original image b, the electronic device can perform algorithm processing on the original image b through the HAL layer. Then, the electronic device can synthesize the original image b processed by the algorithm into image B through Surface Flinger, and store the synthesized image B in the Surface Flinger buffer.

[0087] However, since the electronic device takes a long time to perform algorithm processing on the original image b, the electronic device cannot synthesize image B before the next vertical synchronization signal arrives. Therefore, at time t21, although the electronic device detects the vertical synchronization signal, since image B has not been synthesized at this moment, the electronic device cannot read image B from the Surface Flinger buffer, and the electronic device displays image A for a duration of T again after time t21. Optionally, the electronic device does not display any image for a duration of T after time t21.

[0088] Similarly, at time t3, the electronic device generates an acquisition instruction through the camera application. The electronic device responds to the acquisition instruction at time t3 and acquires the original image c. After acquiring the original image c, the electronic device can perform algorithmic processing on the original image c through the HAL layer. Then, the electronic device can synthesize the original image c processed by the algorithm into image C through Surface Flinger, and store the synthesized image C in the Surface Flinger buffer. At this time, the Surface Flinger buffer stores two frames of images, namely image B and image C. At time t31, the electronic device detects the vertical synchronization signal, reads image B from the Surface Flinger buffer, and displays image B.

[0089] From the above content, it can be known that when the electronic device stores image A at time t11, does not store image B at time t21, and stores image B and image C at time t31, the electronic device displays image A from time t11 to time t21, still displays image A from time t21 to time t31, and starts to display image B at time t31. That is to say, when the electronic device detects the vertical synchronization signal, if the electronic device does not store the newly synthesized image, the electronic device will display the image with frame loss, that is, the electronic device displays the same frame image in two consecutive signal cycles.

[0090] Optionally, if the electronic device is currently in a low frame rate or variable frame rate scenario, the above frame loss phenomenon is more obvious.

[0091] In order to reduce the occurrence of the above situation, an embodiment of the present application provides an image display method, in which the electronic device can monitor the time (i.e., the processing time of the algorithm) for performing algorithm processing on the original image collected by the sensor.

[0092] When the processing time of the algorithm exceeds the time threshold, the electronic device can generate an intermediate frame image based on the original image captured by the current sensor and the image synthesized by the previous frame Surface Flinger, and store the intermediate frame image in the Surface Flinger buffer.

[0093] When the electronic device detects the vertical synchronization signal, the electronic device reads the intermediate frame image from the Surface Flinger buffer and displays the intermediate frame image.

[0094] By implementing the above method, the electronic device can reserve sufficient time to perform algorithm processing on the original image, thereby reducing frame loss, and further reducing the phenomenon of screen freeze observed by the user. In this way, the smoothness of the image displayed by the electronic device can be guaranteed to a certain extent, and the user's visual experience is good.

[0095] Combine the following Figure 3 A flowchart of an image display method provided in an embodiment of the present application is introduced.

[0096] The method flow includes:

[0097] S301: An electronic device collects a first original image.

[0098] In some implementations, before capturing the first original image, the electronic device has already started a camera application and has already started the camera.

[0099] Specifically, the camera of the electronic device may respond to the acquisition instruction sent by the camera application to acquire the first original image. The camera of the electronic device may include but is not limited to: a front camera and a rear camera. The rear camera may include a main camera, a telephoto camera, a wide-angle camera, etc.

[0100] In the embodiment of the present application, the above-mentioned first original image may refer to any frame of original image captured after the electronic device starts a camera application.

[0101] S302: The electronic device processes the first original image through an algorithm and monitors a first processing time.

[0102] After executing S301, the electronic device may execute S302.

[0103] In some implementations, the electronic device includes one or more camera algorithms configured to process the captured raw images.

[0104] Exemplarily, the camera algorithm may include detection perception algorithms and image effect algorithms and other types of algorithms. Among them, the detection perception algorithms may include but are not limited to: face detection algorithm, motion detection algorithm, automatic exposure algorithm, gesture detection algorithm, object detection algorithm, scene detection algorithm, etc., and the image effect algorithm may include but is not limited to: autofocus algorithm, white balance algorithm, bad pixel correction algorithm, noise reduction algorithm, shadow correction algorithm, color enhancement algorithm. In the embodiments of the present application, the above-mentioned camera algorithms are only exemplified and not limited to this.

[0105] In the embodiment of the present application, in S302, the algorithms used by the electronic device may be all camera algorithms built into the electronic device, or may be some camera algorithms, without limitation. For example, the shooting mode used by the camera application may affect the number and type of algorithms currently used.

[0106] In some implementations, the first processing duration has a one-to-many relationship with the processing time of the camera algorithm used by the electronic device. That is, the first processing duration includes: multiple durations. Among them, the above multiple durations include: during the processing process, multiple processing durations corresponding to each of the multiple non-optional (i.e., serial) camera algorithms and multiple processing durations corresponding to each of one or more of the multiple optional (i.e., parallel) camera algorithms.

[0107] For example, the electronic device uses camera algorithm 1, camera algorithm 2, and camera algorithm 3 for the first original image. The above three camera algorithms are all in a serial relationship. The first processing duration may include three processing durations, namely, the processing duration of camera algorithm 1, the processing duration of camera algorithm 2, and the processing duration of camera algorithm 3.

[0108] For another example, the electronic device uses camera algorithm 1, camera algorithm 2, and camera algorithm 3 for the first original image. Camera algorithm 1 and other camera algorithms are in a serial relationship, and camera algorithm 2 and camera algorithm 3 are in a parallel relationship. The first processing duration may include two processing durations, namely, the processing duration of camera algorithm 1 and the processing duration of camera algorithm 2 or camera algorithm 3.

[0109] Since the time between the electronic device capturing the original image and displaying the synthesized image is limited, the electronic device can set a time threshold for the processing time of each camera algorithm according to the complexity of each camera algorithm and the processing time reserved for the algorithm, and then monitor the processing time of each camera algorithm. Once it is detected that the processing time of a camera algorithm exceeds the corresponding time threshold, it means that even if the processing time of other camera algorithms does not exceed the corresponding time threshold, it will cause frame loss. For this reason, the electronic device needs to perform frame insertion processing, that is, generate an intermediate frame image.

[0110] In an embodiment of the present application, the duration threshold of a camera algorithm can be determined based on one or more of the following: the vertical signal period used by the current electronic device, the historical processing duration of a camera algorithm, the historical duration of the electronic device's camera capturing original images, and the historical duration of the electronic device's Surface Flingeer synthesized preview image.

[0111] Optionally, the electronic device may also monitor the time for acquiring the original image and the pre-processing.

[0112] For example, Figure 4 As shown, the duration threshold of image pre-processing is duration Ta, the duration threshold of algorithm 1 is duration Tb, the duration threshold of algorithm 2 is duration Tc, the duration threshold of algorithm 3 is duration Tc, and the duration threshold of algorithm 4 is duration Td.

[0113] Among them, Algorithm 1, Algorithm 2, Algorithm 3, Algorithm 4 and even more algorithms are camera algorithms. For example, after the electronic device processes the original image through Algorithm 1, it generally selects one from Algorithm 2 and Algorithm 3 to process the original image, and then processes the original image through Algorithm 4.

[0114] In the embodiments of the present application, Figure 4 The number of algorithms is only for illustrative purposes, and a greater or lesser number of algorithms may be included.

[0115] Exemplarily, if the time for collecting the original image and pre-processing exceeds the time length Ta, the electronic device generates an intermediate frame image, while continuing to perform subsequent algorithm processing on the original image and synthesizing a preview image;

[0116] If the time for collecting the original image and pre-processing does not exceed the time length Ta, but the time for processing by Algorithm 1 exceeds the time length Tb, the electronic device generates an intermediate frame image, while continuing to perform subsequent algorithm processing on the original image and synthesizing a preview image;

[0117] If the time for acquiring the original image and pre-processing does not exceed the time length Ta and the time for processing by algorithm 1 does not exceed the time length Tb, but the time for processing by algorithm 2 exceeds the time length Tc or the time for processing by algorithm 3 exceeds the time length Tc, the electronic device generates an intermediate frame image, while continuing to perform subsequent algorithm processing on the original image, and synthesizing a preview image;

[0118] If the time for acquiring the original image and pre-processing does not exceed the time length Ta, the time for processing by algorithm 1 does not exceed the time length Tb, and the time for processing by algorithm 2 or algorithm 3 does not exceed the time length Tc, but the time for processing by algorithm 4 exceeds the time length Td, the electronic device generates an intermediate frame image and continues to synthesize a preview image based on the original image;

[0119] If the time for acquiring the original image and pre-processing does not exceed the time Ta, the time for processing by algorithm 1 does not exceed the time Tb, the time for processing by algorithm 2 or algorithm 3 does not exceed the time Tc, and the time for processing by algorithm 4 does not exceed the time Td, the electronic device does not need to generate an intermediate frame image and can directly synthesize a preview image based on the original image.

[0120] The monitoring of the first processing duration may include: monitoring the duration of processing the first original image by each camera algorithm.

[0121] S303: When the first processing time does not exceed the time threshold, the electronic device synthesizes the first original image processed by the algorithm to obtain a first preview image.

[0122] The electronic device may execute S303 while or after executing S302.

[0123] In some implementations, the first processing duration not exceeding the duration threshold may include: multiple durations for the electronic device to process the first original image through each camera algorithm do not exceed the corresponding duration threshold.

[0124] In the embodiment of the present application, the reasons why the processing time of the algorithm exceeds the time threshold include: the current electronic device is in a high-load scenario, resulting in insufficient supply of computing resources, thereby causing a timeout.

[0125] After synthesizing the first preview image, the electronic device may store the first preview image.

[0126] S304: The electronic device detects a vertical synchronization signal at a first moment and displays a first preview image.

[0127] After executing S303, the electronic device may execute S304.

[0128] In some implementations, the electronic device detects a vertical synchronization signal at a first moment, indicating that the content displayed on the display screen needs to be refreshed at the first moment. The electronic device can read out a first preview image and display the first preview image on the display screen. At the first moment, the image stored earliest in the electronic device is the first preview image.

[0129] S305: The electronic device collects a second original image.

[0130] The electronic device may execute S305 after executing S301. That is, the electronic device may execute S305 while executing S302, S303, or S304, or in the interval between executing S304 and S305.

[0131] In some implementations, in a preview scenario, a camera application of the electronic device may periodically generate acquisition instructions, for example, generating an acquisition instruction at intervals of a single vertical signal period.

[0132] The second original image may refer to the next frame of the original image of the first original image captured by the electronic device.

[0133] S306: The electronic device processes the second original image through an algorithm and monitors a second processing time.

[0134] After executing S305, the electronic device may execute S306.

[0135] In some implementations, the electronic device may use one or more camera algorithms to process the captured original image. The second processing duration may include: multiple durations for each camera algorithm used by the electronic device to process the second original image.

[0136] For the relevant description of S306, reference may be made to the above detailed description of S302, which will not be repeated here.

[0137] S307: When the second processing time exceeds the time threshold, the electronic device generates a first intermediate frame image based on the second original image and the first preview image.

[0138] After the electronic device executes S306, S307 can be executed. However, since the electronic device needs to use the first preview image, the electronic device needs to execute S307 after S303.

[0139] The second processing time exceeding the time threshold may mean that the processing time of one of the multiple camera algorithms used by the electronic device to process the second original image exceeds the corresponding time threshold. For the description of exceeding the time threshold, please refer to the detailed description of S302.

[0140] When the electronic device detects that the second processing duration exceeds the duration threshold, the electronic device may have performed a portion of algorithm processing on the second original image, but the processing may not be completed. At this time, the effect of the unprocessed original image is not ideal. Therefore, the electronic device cannot generate an intermediate frame image based on the original image that has not been processed at this time. The electronic device can read the second original layer from the zero shutter lag (ZSL) buffer. Among them, the ZSL buffer can be used to store data of several frames of original images.

[0141] S308: The electronic device detects a vertical synchronization signal at a second moment and displays a first intermediate frame image, wherein the first moment and the second moment are separated by a single signal cycle.

[0142] The electronic device may execute S308 after executing S307.

[0143] In some implementations, the electronic device detects a vertical synchronization signal at the second moment, indicating that the content displayed on the display needs to be refreshed at the second moment. The electronic device can read out the first intermediate frame image and display the first intermediate frame image on the display. At the second moment, the image stored earliest in the electronic device is the first intermediate frame image.

[0144] In this way, since the first intermediate frame is obtained based on the first preview image and the second original image, the first intermediate frame displayed by the electronic device has good consistency with the first preview image displayed in the previous frame.

[0145] S309: The electronic device synthesizes the second original image processed by the algorithm to obtain a second preview image after the second moment.

[0146] After processing the second original image through the algorithm, the electronic device may execute S309. That is, the electronic device may synthesize the second original image synchronously while executing S307 to S308 or in the interval between S307 and S308, and complete S309 after completing S308.

[0147] In some implementations, since the second processing duration exceeds the duration threshold, it can be seen from the above description in S302 that the electronic device cannot synthesize the second preview image before the first vertical synchronization signal detected after the second original image is captured. Therefore, the electronic device can obtain the second preview image after the second moment.

[0148] After synthesizing the second preview image, the electronic device may store the second preview image.

[0149] S310: The electronic device detects a vertical synchronization signal at a third moment and displays a second preview image, wherein the second moment and the third moment are separated by a single signal cycle.

[0150] After executing S309 , the electronic device may execute S310 .

[0151] In some implementations, the electronic device detects a vertical synchronization signal at the third moment, indicating that the content displayed on the display needs to be refreshed at the third moment. The electronic device can read out a second preview image and display the second preview image on the display. At the third moment, the image stored earliest in the electronic device is the second preview image.

[0152] By implementing the above method, in the process from capturing the original image to synthesizing the preview image, the electronic device can generate an intermediate frame image in the event of an abnormality (such as algorithm processing timeout). In addition, the electronic device can reserve sufficient time to perform algorithm processing on the original image. In this way, the electronic device can read the newly generated image every time it detects a vertical synchronization signal, and the same image will not be displayed for two consecutive frames.

[0153] To better understand the above Figure 3 The image display method shown below is combined with Figure 5 Let me introduce it in detail.

[0154] Figure 5 The schematic diagram of the principle of the image display method used in the electronic device provided in the embodiment of the present application. Figure 3 The image display method shown is described as an example.

[0155] The first moment may refer to moment t11, the second moment may refer to moment t21, and the third moment may refer to moment t31. The first original image may refer to an image captured by the electronic device at moment t1, and the second original image may refer to an image captured by the electronic device at moment t2. The first image may refer to image A synthesized and stored by the electronic device before moment t11. The second image may refer to image B synthesized by the electronic device after moment t21. The first intermediate frame image may refer to image B1 synthesized by the electronic device before moment t21.

[0156] like Figure 5 As shown, at time t1, the electronic device generates an acquisition instruction. The electronic device responds to the acquisition instruction at time t1 and acquires the original image a. After acquiring the original image a, the electronic device can perform algorithmic processing on the original image a. Then, the electronic device can synthesize the original image a after the algorithmic processing into image A, and store image A. At time t11, the electronic device detects the vertical synchronization signal, reads image A, and displays image A. In an embodiment of the present application, since the duration of the algorithmic processing of the original image a does not exceed the duration threshold, the electronic device is able to synthesize image A before time t11. Therefore, the electronic device is able to display image A in response to the vertical synchronization signal at time t11.

[0157] At time t2, the electronic device generates an acquisition instruction. The electronic device responds to the acquisition instruction at time t2 and acquires the original image b. After acquiring the original image b, the electronic device can perform algorithm processing on the original image b. In the embodiment of the present application, since the duration of the algorithm processing on the original image b exceeds the duration threshold, the electronic device cannot synthesize the image B before time t21. Therefore, it is necessary to generate a new image before the next vertical synchronization signal arrives, so that frame loss does not occur.

[0158] At time t22, the electronic device detects that the duration of processing the original image b through the algorithm exceeds the corresponding duration threshold, and the electronic device can generate image B1 based on the original image b and image A, and store image B1. At time t21, the electronic device detects the vertical synchronization signal, reads image B1, and displays image B1. After time t22, the electronic device can synthesize image B with the original image b processed by the algorithm, and store image B.

[0159] At time t3, the electronic device generates an acquisition instruction. In response to the acquisition instruction at time t3, the electronic device acquires the original image c. After acquiring the original image c, the electronic device may perform algorithmic processing on the original image c. Then, the electronic device may synthesize the original image c processed by the algorithm into image C, and store image C. At time t31, the electronic device detects the vertical synchronization signal, reads image B, and displays image B. In an embodiment of the present application, since the above-mentioned image B is not consumed, two frames of images are stored in the Surface Flinger buffer after synthesizing image C, and image B is stored before image C. Therefore, when the next vertical synchronization signal arrives, image B is consumed first.

[0160] In some implementations, if an electronic device processes multiple frames of collected original images through an algorithm, and the processing time exceeds the time threshold, it will cause the phenomenon of excessive accumulation of synthesized images stored in the electronic device. In response to the above problem, in an embodiment of the present application, the electronic device monitors the number of images stored in the Surface Flinger buffer in real time. When the number of stored images exceeds a certain number (for example, two frames), the electronic device deletes the first frame of the image stored. In this way, it can be ensured that the number of images stored in the Surface Flinger buffer remains within a certain range, avoiding frame accumulation and reducing the consumption of storage resources.

[0161] Combine the following Figure 6 A schematic flow chart of another image display method provided in an embodiment of the present application is introduced. Figure 6 The method flow chart shown is Figure 5 The method flow shown differs in that: Figure 6 In the method flow chart shown, the SurfaceFlinger buffer contains two frames of images and the electronic device executes an operation of deleting the frame of image with the earliest storage time.

[0162] like Figure 6 As shown, the method flow includes:

[0163] S601: The electronic device collects a first original image.

[0164] S602: The electronic device processes the first original image using an algorithm and monitors a first processing time.

[0165] S603: When the first processing time does not exceed the time threshold, the electronic device synthesizes the first original image processed by the algorithm to obtain a first image.

[0166] S604: The electronic device detects a vertical synchronization signal at a first moment and displays a first preview image.

[0167] S605: The electronic device collects a second original image.

[0168] S606: The electronic device processes the second original image using an algorithm and monitors a second processing time.

[0169] S607: When the second processing time exceeds the time threshold, the electronic device generates a first intermediate frame image based on the second original image and the first preview image.

[0170] S608: The electronic device detects a vertical synchronization signal at a second moment and displays a first intermediate frame image, wherein the first moment and the second moment are separated by a single signal cycle.

[0171] S609: The electronic device synthesizes the second original image processed by the algorithm to obtain a second preview image after the second moment.

[0172] For the related description of the above S601-S609, please refer to the detailed introduction of the above S301-S309, which will not be repeated here.

[0173] S610: The electronic device collects a third original image.

[0174] The electronic device may execute S610 after executing S605. That is, the electronic device may execute S610 at any time between executing S606 and executing S609.

[0175] Furthermore, the time interval between when the electronic device collects the third original image and when it collects the second original image is consistent with the time interval between when it collects the second original image and when it collects the first original image.

[0176] The third original image mentioned above may refer to the next frame of original image of the second original image captured by the electronic device.

[0177] S611: The electronic device processes the third original image through an algorithm and monitors a third processing time.

[0178] After executing S610, the electronic device may execute S611.

[0179] In some implementations, the electronic device may use one or more camera algorithms to process the captured raw image. The third processing duration may include: multiple durations for each camera algorithm used by the electronic device to process the third raw image.

[0180] S612: When the third processing time exceeds the time threshold, the electronic device generates a second intermediate frame image based on the third original image and the second preview image.

[0181] After the electronic device executes S611, S612 can be executed. However, since the electronic device needs to use the second preview image, the electronic device needs to execute S612 after S609.

[0182] The third processing time exceeding the time threshold may mean that the processing time of one of the multiple camera algorithms used by the electronic device to process the third original image exceeds the corresponding time threshold. For the description of exceeding the time threshold, please refer to the detailed description of S302.

[0183] S613: The electronic device detects a vertical synchronization signal at a third moment and displays a second preview image, wherein the second moment and the third moment are separated by a single signal cycle.

[0184] The electronic device may execute S613 after executing S612.

[0185] In some implementations, the electronic device detects a vertical synchronization signal at the third moment, indicating that the content displayed on the display needs to be refreshed at the third moment. The electronic device can read out a second preview image and display the second preview image on the display. At the third moment, the image stored earliest in the electronic device is the second preview image.

[0186] S614: The electronic device synthesizes the third original image processed by the algorithm to obtain a third preview image after a third moment.

[0187] After processing the third original image through the algorithm, the electronic device may execute S614. That is, the electronic device may synthesize the third original image synchronously while executing S612 to S613 or in the interval between S612 and S613 and complete S614 after completing S613.

[0188] S615: The electronic device collects a fourth original image.

[0189] After executing S610, the electronic device may execute S615. That is, the electronic device may execute S615 at any time between executing S610 and executing S614.

[0190] Furthermore, the time interval between when the electronic device collects the fourth original image and when it collects the third original image is consistent with the time interval between when it collects the third original image and when it collects the second original image.

[0191] The fourth original image may refer to the next frame of the original image of the third original image captured by the electronic device.

[0192] S616: The electronic device processes the fourth original image through an algorithm and monitors a fourth processing time.

[0193] After executing S615, the electronic device may execute S616.

[0194] In some implementations, the electronic device may use one or more camera algorithms to process the captured raw image. The fourth processing duration may include: multiple durations for each camera algorithm used by the electronic device to process the fourth raw image.

[0195] S617: When the fourth processing time does not exceed the time threshold, the electronic device synthesizes the fourth original image processed by the algorithm to obtain a fourth preview image.

[0196] The electronic device may execute S617 while or after executing S616.

[0197] In some implementations, the fourth processing duration not exceeding the duration threshold may include: multiple durations for the electronic device to process the fourth original image through each camera algorithm do not exceed the corresponding duration threshold.

[0198] After synthesizing the fourth preview image, the electronic device may store the fourth preview image.

[0199] S618: The electronic device detects that the number of stored images exceeds two frames, and deletes the second intermediate frame image with the earliest current storage time.

[0200] After synthesizing the fourth preview image, the electronic device stores three frame images, namely, the second intermediate frame image, the third preview image and the fourth preview image.

[0201] At this time, the electronic device detects that the number of stored images exceeds two frames, and deletes the image frame with the earliest current storage time, that is, the second intermediate frame image.

[0202] In this way, it is possible to avoid the situation where the electronic device reads and displays a preview image generated too early, causing the user to see a delayed preview image. It is also possible to avoid the phenomenon of frame accumulation and consumption of storage resources on the basis of reducing the phenomenon of frame loss.

[0203] S619: The electronic device detects a vertical synchronization signal at a fourth moment and displays a third preview image, wherein the third moment and the fourth moment are separated by a single signal cycle.

[0204] After executing S618, the electronic device may execute S619.

[0205] In some implementations, the electronic device detects a vertical synchronization signal at the fourth moment, indicating that the content displayed on the display needs to be refreshed at the fourth moment. The electronic device can read the third preview image and display the first preview image on the display. At the fourth moment, the image stored earliest in the electronic device is the third preview image.

[0206] By implementing the above method, in the process from capturing the original image to synthesizing the preview image, the electronic device can generate an intermediate frame image in the event of an abnormality (such as algorithm processing timeout). In addition, the electronic device can reserve sufficient time to perform algorithm processing on the original image. The electronic device can also display a fresher preview image.

[0207] In this way, the electronic device can read a newly generated image each time it detects a vertical synchronization signal, and will not display the same image for two consecutive frames. In addition, the number of images stored in the electronic device is kept within a certain range, avoiding frame accumulation and reducing the consumption of storage resources.

[0208] In a possible implementation, the reasons for frame accumulation are in addition to the above Figure 6 In addition to the method flow shown, the method may also include: the current electronic device performance deterioration triggers the generation of an intermediate frame. In this case, the electronic device may also consider deleting the earliest frame of image stored.

[0209] To better understand the above Figure 6 The image display method shown below is combined with Figure 7 Let me introduce it in detail.

[0210] Figure 7 The schematic diagram of the principle of the image display method used in the electronic device provided in the embodiment of the present application. Figure 6 The image display method shown is described as an example.

[0211] The first moment may refer to moment t11, the second moment may refer to moment t21, the third moment may refer to moment t31, and the fourth moment may refer to moment t41. The first original image may refer to an image captured by the electronic device at moment t1, the second original image may refer to an image captured by the electronic device at moment t2, the third original image may refer to an image captured by the electronic device at moment t3, and the fourth original image may refer to an image captured by the electronic device at moment t4. The first image may refer to image A synthesized and stored by the electronic device before moment t11. The second image may refer to image B synthesized by the electronic device after moment t21, and the third image may refer to image C synthesized by the electronic device after moment t31. The fourth image may refer to image D synthesized and stored by the electronic device before moment t41. The first intermediate frame image may refer to image B1 synthesized by the electronic device before moment t21, and the second intermediate frame image may refer to image C1 synthesized by the electronic device before moment t31.

[0212] like Figure 7 As shown, the electronic device responds to the capture instruction at time t1 and captures the original image a. Afterwards, the electronic device can perform algorithmic processing on the original image a. Then, the electronic device can synthesize the original image a processed by the algorithm into image A, and store image A. At time t11, the electronic device detects the vertical synchronization signal, reads image A, and displays image A. In the embodiment of the present application, since the duration of the algorithmic processing of the original image a does not exceed the duration threshold, the electronic device can synthesize image A before time t11. Therefore, the electronic device can display image A in response to the vertical synchronization signal at time t11.

[0213] The electronic device responds to the acquisition instruction at time t2 and acquires the original image b. Afterwards, the electronic device can perform algorithm processing on the original image b. In the embodiment of the present application, since the duration of the algorithm processing on the original image b exceeds the duration threshold, the electronic device cannot synthesize the image B before time t21. Therefore, it is necessary to generate a new image before the next vertical synchronization signal arrives, so that frame loss will not occur.

[0214] At time t22, the electronic device detects that the duration of processing the original image b by a certain algorithm exceeds the corresponding duration threshold, and the electronic device can generate image B1 based on the original image b and image A, and store image B1. At time t21, the electronic device detects the vertical synchronization signal, reads image B1, and displays image B1. After time t21, the electronic device can synthesize image B with the original image b processed by the algorithm, and store image B.

[0215] The electronic device responds to the acquisition instruction at time t3 and acquires the original image c. Afterwards, the electronic device can perform algorithm processing on the original image c. In the embodiment of the present application, since the duration of the algorithm processing on the original image c also exceeds the duration threshold, the electronic device cannot synthesize the image C before time t31. Therefore, it is necessary to generate a new image before the next vertical synchronization signal arrives, so that frame loss will not occur.

[0216] At time t32, the electronic device detects that the duration of processing the original image c by a certain algorithm exceeds the corresponding duration threshold, and the electronic device can generate image C1 based on the original image c and image B, and store image C1. At this time, the electronic device stores two frames of images, namely image B and image C1. At time t31, the electronic device detects the vertical synchronization signal, reads image B, and displays image B. After time t31, the electronic device can synthesize image C with the original image c processed by the algorithm, and store image C.

[0217] The electronic device responds to the acquisition instruction at time t4 and acquires the original image d. Afterwards, the electronic device can perform algorithmic processing on the original image d. Then, the electronic device can synthesize the original image d processed by the algorithm into image D, and store image D. At this moment, the Surface Flinger buffer stores three frames of images, namely, image C1, image C, and image D. Since the electronic device has taken measures to solve the frame accumulation, once it is detected that the Surface Flinger buffer stores three frames of images, that is, when the synthesis of image D is completed, the image C1 stored in the Surface Flinger buffer is deleted.

[0218] At time t41, the electronic device detects the vertical synchronization signal, reads image C, and displays image C. In the embodiment of the present application, since the duration of the algorithm processing of the original image d does not exceed the duration threshold, the electronic device can synthesize image D before time t41. However, the electronic device stores two frames of images at time t41, namely, image C and image D. And image C is stored before image D. Therefore, the electronic device can display image C in response to the vertical synchronization signal at time t41.

[0219] Next, combine Figure 8 An internal module interaction diagram of an operating system of an electronic device corresponding to the image display method process provided in an embodiment of the present application is introduced.

[0220] Figure 8The user operation module, camera, camera algorithm module, timeout judgment module, interpolation algorithm module, frame stacking processing module, Surface Flinger and display driver data storage module shown in are modules included in the electronic device for managing the process from acquiring original images to displaying preview images. For the definitions of these modules, please refer to the detailed description of the electronic device software architecture later. The specific functions of these modules have been recorded in detail in the following process and will not be repeated here.

[0221] S801: A user operation module detects an operation of starting a first application program.

[0222] In some implementations, the first application may include a camera application. The operation may include, but is not limited to: a click operation, a gesture command, a voice command, etc. on an icon of the camera application.

[0223] S801-1. The frame accumulation processing module sends an instruction to Surface Flinger. The instruction is used to instruct Surface Flinger to send the number of stored images to the frame accumulation processing module when updating Surface Flinger buffer storage information.

[0224] In some implementations, S801-1 can be executed after the electronic device acquires the frame stacking processing module. For example, S801-1 can be executed after the electronic device starts the first application.

[0225] S802: The user operation module sends a collection instruction to the camera.

[0226] In some implementations, the user operation module may be located in the application framework layer. The user operation module may obtain the acquisition instruction sent by the first application program and send it to the camera.

[0227] Exemplarily, S802 may be executed before the electronic device captures the first original image, the second original image, the third original image, and the fourth original image.

[0228] S803: The camera collects original images.

[0229] Exemplarily, the camera may collect a first original image, a second original image, a third original image, and a fourth original image. In some implementations, the electronic device may temporarily store the collected original images in a ZSL buffer. S804: The camera sends the original image to the camera algorithm module.

[0230] S805: The camera algorithm module performs algorithm processing on the original image.

[0231] In some implementations, the number of algorithms used by the camera algorithm module may be one or more. Figure 5 S806: The timeout determination module obtains the processing time of the camera algorithm module when executing S805.

[0232] In some implementations, the timeout determination module may obtain multiple times when the multiple algorithms process the original image. For the introduction of S806, please refer to the detailed description of S302 above, which will not be repeated here.

[0233] Optionally, the timeout judgment module may also obtain the time of capturing the original image and the time of pre-processing the original image. Exemplarily, the timeout judgment module may also obtain the time of processing the original image through the image frontend (IFE) and the image processing engine (IPE). S807, the timeout judgment module determines whether the processing time exceeds the time threshold.

[0234] The timeout determination module stores the duration thresholds set for the multiple camera algorithms in the camera algorithm module.

[0235] If the timeout determination module determines that during the execution of S805, the processing time of a certain camera algorithm exceeds the corresponding time threshold, S808 is executed;

[0236] If the timeout determination module determines that during the execution of S805, the processing time of all camera algorithms does not exceed the corresponding time threshold, S808 is not executed, and the process directly jumps to S814.

[0237] S808. The timeout determination module sends a frame insertion instruction to the frame insertion algorithm module.

[0238] S809: The interpolation algorithm module generates an intermediate frame image based on the original image and the previous synthesized image.

[0239] In some implementations, after receiving the frame insertion instruction, the frame insertion algorithm module can read the original image of the current frame from the ZSL buffer, and generate an intermediate frame image by combining it with the synthesized preview image of the previous frame.

[0240] S810. The interpolation algorithm module sends an intermediate frame image to Surface Flinger.

[0241] S811. Surface Flinger stores the intermediate frame image in the Surface Flinger buffer.

[0242] S812, Surface Flinger sends the number of currently stored images to the frame accumulation processing module. S813, the frame accumulation processing module determines whether to delete the first stored image.

[0243] Since the number of images stored in the Surface Flinger buffer changes after S811 is executed, the frame accumulation processing module needs to determine whether the number of images currently stored is greater than two frames. If so, the image with the earliest storage time needs to be deleted; if not, the stored image does not need to be deleted.

[0244] S814. The camera algorithm module sends the processed original image to Surface Flinger.

[0245] S815. Surface Flinger synthesizes a preview image with the processed original image and stores the preview image.

[0246] S816. Surface Flinger sends the number of currently stored images to the frame accumulation processing module.

[0247] S817: The frame accumulation processing module determines whether to delete the first stored image.

[0248] Similarly, since the number of images stored in the Surface Flinger buffer changes after S816 is executed, the frame accumulation processing module needs to determine whether the number of images currently stored is greater than two frames. If so, the image with the earliest storage time needs to be deleted; if not, the stored image does not need to be deleted.

[0249] S818: The display driver detects a vertical synchronization signal.

[0250] S819. The display driver sends a read instruction to Surface Flinger.

[0251] S820. Surface Flinger sends the image with the earliest storage time in Surface Flinger buffer to the display driver.

[0252] In a possible implementation, the electronic device may further determine whether the Surface Flingerbuffer stores a preview image before generating an intermediate frame image each time, that is, after determining that the duration of processing a certain frame of the original image exceeds a duration threshold.

[0253] If the Surface Flingerbuffer stores a preview image, the electronic device will no longer generate an intermediate frame image. This is because the electronic device generates an intermediate frame image only when it determines that there is an image that can be displayed on the display screen when the vertical synchronization signal arrives after the processing time for a certain frame of the original image exceeds the time threshold. However, at this time, the electronic device has already stored a preview image that is displayed on the display screen when the vertical synchronization signal arrives.

[0254] If the Surface Flingerbuffer stores a preview image, the electronic device generates an intermediate frame image. For details, please refer to the above description of generating the intermediate frame image, which will not be repeated here.

[0255] In this way, the electronic device can also ensure that the number of stored preview images is within a certain range, and can also avoid the occurrence of frame accumulation.

[0256] In a possible implementation, if the electronic device deteriorates in performance, resulting in the synthesis of multiple preview images within a certain signal cycle or in a short time, the electronic device may not delete a certain frame of preview image, but reduce the current signal cycle, so that the electronic device can quickly send the synthesized preview image for display in a short time. For example, due to performance deterioration, the electronic device generates three frames of preview images within a signal cycle of 66ms. The electronic device can adjust the signal cycle of 66ms to a signal cycle of 22ms. The electronic device has three 22ms signal cycles in the next 66ms. In this way, the electronic device can receive three vertical synchronization signals in the next 66ms, and can consume the above three frames of preview images, that is, display the above three frames of preview images. Then the electronic device adjusts the signal cycle back to 66ms. In this way, the electronic device will neither arbitrarily lose the preview image obtained by the original image obtained by real acquisition, nor cause long-term storage resource occupation.

[0257] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0258] The electronic device 100 may be equipped with Or a portable terminal device of other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, a vehicle-mounted device, a smart home device and / or a smart city device, but not limited thereto, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or a touch panel, a desktop computer with a touch-sensitive surface or a touch panel, and other non-portable terminal devices, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0259] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0260] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0261] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0262] In the embodiment of the present application, the processor 110 can be used to monitor the time for processing each frame of the original image by the algorithm. The processor 110 can also be used to generate an intermediate frame image based on the original image of the current frame and the preview image of the previous frame in the case of timeout. The processor 110 can also be used to delete the preview image with the earliest storage time when the number of stored images exceeds two frames.

[0263] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0264] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0265] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0266] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0267] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0268] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0269] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0270] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0271] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0272] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0273] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0274] In the embodiment of the present application, the display screen 194 can be used to display the preview image read out when the vertical synchronization signal is detected. The preview image is a preview image frame with the earliest storage time when the preview image is read.

[0275] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0276] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0277] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0278] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0279] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0280] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0281] In the embodiment of the present application, the electronic device 100 implements the image display method provided in the embodiment of the present application, which first relies on the ISP and the original image collected by the camera 193, and secondly relies on the image calculation and processing capabilities provided by the video codec and the GPU. Among them, the electronic device 100 can implement algorithms such as face recognition, body recognition, and re-recognition through the computing and processing capabilities provided by the NPU.

[0282] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0283] In the embodiment of the present application, the internal memory 121 can store preview images and intermediate frame images.

[0284] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0285] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0286] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data, etc. The non-volatile memory can also store executable programs and store user and application data, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.

[0287] In the embodiment of the present application, the code for implementing the image display described in the embodiment of the present application may be stored in a non-volatile memory. When running the camera application, the electronic device 100 may load the executable code stored in the non-volatile memory into the random access memory.

[0288] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0289] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0290] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0291] Speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music or listen to hands-free calls through speaker 170A. Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When electronic device 100 answers a call or voice message, the voice can be heard by placing receiver 170B close to the human ear. Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. Headphone interface 170D is used to connect wired headphones.

[0292] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes. The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover is automatically unlocked and other features are set. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device, and is applied to horizontal and vertical screen switching, pedometers and other applications. Distance sensor 180F is used to measure distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focus. The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch. The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. The touch sensor 180K is also called a "touch device."The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of vibrating bones of the human body. The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to the user settings and function control of the electronic device 100. The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0293] Fig.10 It is a schematic diagram of the system structure of the electronic device 100 provided in an embodiment of the present application.

[0294] The layered architecture divides the system into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom: application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer.

[0295] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include a camera, a gallery, etc.

[0296] The application framework layer provides an application programming interface (API) and a programming framework for the application of the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, wherein the camera access interface may include camera management and camera devices. The camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0297] The hardware abstraction layer is an interface layer between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system.

[0298] In an embodiment of the present application, the hardware abstraction layer may include but is not limited to: a camera hardware abstraction layer, a camera algorithm library, a timeout judgment module, an interpolation algorithm module, a frame accumulation processing module and a Surface Flinger.

[0299] The camera hardware abstraction layer may provide virtual hardware of camera device 1, camera device 2 or more camera devices.

[0300] Exemplarily, the camera device 1 may correspond to the front camera in the electronic device 100, and the camera device 2 may correspond to the rear camera in the electronic device 100. In the embodiment of the present application, the camera device 1 and the camera device 2 are only used for exemplary description and are not limited thereto.

[0301] The camera algorithm library may include operating code and data for implementing the processing of the original image captured by the camera in the embodiments of the present application.

[0302] Exemplarily, the camera algorithm library may include, but is not limited to: a face detection algorithm, a motion detection algorithm, an automatic exposure algorithm, a gesture detection algorithm, an object detection algorithm, a scene detection algorithm, and the like.

[0303] The timeout determination module can be used to detect the time taken by the electronic device 100 to process the original image through multiple algorithms in the camera algorithm library. The timeout determination module can also store multiple time thresholds for the multiple algorithms in the camera algorithm library to process the original image, and can also be used to determine whether the processing time of each algorithm exceeds the time threshold.

[0304] The interpolation algorithm module can be used to generate an intermediate frame image based on the current frame original image and the previous frame synthesized image when the processing time of a certain algorithm exceeds the corresponding time threshold.

[0305] Exemplarily, the interpolation algorithm can read the original image of the current frame from the ZSL buffer. The interpolation algorithm module can also store a variety of algorithms for generating intermediate frame images.

[0306] The frame accumulation processing module can be used to determine in real time whether the preview image stored in the Surface Flingerbuffer is greater than the storage threshold. The frame accumulation processing module can also be used to instruct the Surface Flingerbuffer to delete the image with the earliest storage time.

[0307] Surface Flinger may include Surface Flingerbuffer. Surface Flinger is used to synthesize the original image processed by the algorithm to obtain a preview image.

[0308] Surface Flingerbuffer can be used to store the preview image after synthesis, and can also be used to store the intermediate frame images generated by the interpolation algorithm module.

[0309] The driver layer is a layer between hardware and software. The driver layer includes drivers for various hardware. The driver layer may include camera device drivers, digital signal processor drivers, image processor drivers, and display drivers.

[0310] Among them, the camera device driver is used to drive the camera sensor to collect images and drive the image signal processor to pre-process the image. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images. The display driver is used to drive the display screen to update the displayed content.

[0311] The following is a detailed description of the image display method in the embodiment of the present application in combination with the above system structure:

[0312] In response to the operation of opening the camera application, such as the user clicking the camera application icon, the camera application calls the camera access interface of the application framework layer, starts the camera application, and then sends an instruction to start the camera by calling the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer.

[0313] The camera hardware abstraction layer sends the camera start instruction to the camera device driver in the driver layer. The camera device driver can start the corresponding camera sensor and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0314] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0315] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores the program code of various camera algorithms. Based on the digital signal processor and image processor, the camera algorithm library can execute the above code.

[0316] The timeout judgment module can detect the corresponding processing time during the processing of different camera algorithms.

[0317] If the timeout judgment module detects that the processing time of a camera algorithm exceeds the corresponding processing threshold, it sends an interpolation instruction to the interpolation algorithm module. The interpolation algorithm module responds to the interpolation instruction, generates an intermediate frame image based on the currently acquired original image and the last synthesized preview image, and sends the intermediate frame image to Surface Flinger.

[0318] The intermediate frame image is then stored in Surface Flingerbuffer. While the above steps are being executed, the camera algorithm library continues to process the currently acquired original image and sends the processed original image to SurfaceFlinger. Surface Flinger then synthesizes it into a preview image and stores it in SurfaceFlingerbuffer.

[0319] If the timeout judgment module detects that the processing time of all executed camera algorithms exceeds the corresponding processing threshold, it will not send the interpolation instruction to the interpolation algorithm module. The camera algorithm library directly sends the processed original image to Surface Flinger. Surface Flinger then synthesizes it into a preview image and stores the preview image in Surface Flingerbuffer.

[0320] When the display driver detects a vertical synchronization signal, it reads instructions from Surface Flinger. Surface Flinger reads the oldest frame of image stored in Surface Flingerbuffer and sends it to the display driver. It is worth noting that every time the display driver detects a vertical synchronization signal and reads instructions from Surface Flinger, Surface Flingerbuffer stores the preview image to be consumed.

[0321] Finally, the display driver can drive the display screen to display the frame of image read out above.

[0322] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0323] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0324] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0325] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image display method, characterized in that: The method is applied to an electronic device, and the method comprises: At a first moment, displaying a first preview image; The electronic device collects a second original image; The electronic device processes the second original image by using at least one algorithm, wherein the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold; The electronic device obtains a first intermediate frame image based on the first preview image and the second original image; At a second moment, displaying the first intermediate frame image, the second moment being a moment separated by a single signal cycle after the first moment; The electronic device obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm; At a third moment, the second preview image is displayed, and the third moment is a moment separated by a single signal cycle from the second moment.

2. The method according to claim 1, characterized in that The first duration threshold is determined by one or more of the following: the single signal cycle, the historical processing duration of the first algorithm, the historical duration of acquiring the original image, and the historical duration of synthesizing the preview image.

3. The method according to claim 1 or 2, characterized in that: At a third moment, before displaying the second preview image, the method further includes: The electronic device collects a third original image; The electronic device processes the second original image by using the at least one algorithm, the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold; The electronic device obtains a second intermediate frame image based on the second preview image and the third original image; At a third moment, after displaying the second preview image, the method further includes: The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm; The electronic device generates a fourth preview image or a third intermediate frame image; The electronic device deletes the second intermediate frame image; At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment; At a fifth moment, the fourth preview image or the third intermediate frame image is displayed. The fifth moment is a moment separated from the fourth moment by a single signal cycle.

4. The method according to claim 1 or 2, characterized in that: When the electronic device processes the second original image by using at least one algorithm, the method further includes: The electronic device determines that no preview image is currently stored; At a third moment, before displaying the second preview image, the method further includes: The electronic device collects a third original image; The electronic device processes the second original image by using the at least one algorithm, the at least one algorithm includes a second algorithm, and a processing time of the second algorithm exceeds a second time threshold; The electronic device determines that a preview image is currently stored; At a third moment, after displaying the second preview image, the method further includes: The electronic device obtains a third preview image after the third moment based on the third original image processed by the at least one algorithm; a fourth preview image of the electronic device; At a fourth moment, displaying the third preview image, the fourth moment being a moment separated by a single signal cycle after the third moment; At a fifth moment, the fourth preview image is displayed. The fifth moment is a moment separated from the fourth moment by a single signal cycle.

5. The method according to any one of claims 1 to 4, characterized in that The electronic device comprises a cache queue, wherein the cache queue is used to store preview images and intermediate frame images, and a maximum storage capacity of the cache queue is two frame images.

6. The method according to any one of claims 1 to 5, characterized in that The at least one algorithm includes any one or more of the following: face detection algorithm, motion detection algorithm, automatic exposure algorithm, gesture detection algorithm, object detection algorithm, scene detection algorithm, auto focus algorithm, white balance algorithm, bad pixel correction algorithm, noise reduction algorithm, shadow correction algorithm, color enhancement algorithm.

7. The method according to any one of claims 1 to 6, characterized in that The electronic device further includes: a camera, a hardware abstraction layer, a surface synthesizer, and a display screen, wherein: At a first moment, the display screen is used to display a first preview image; The camera is used to collect a second original image; The hardware abstraction layer is used to process the second original image by using at least one algorithm, wherein the at least one algorithm includes a first algorithm, and a processing time of the first algorithm exceeds a first time threshold; The hardware abstraction layer is used to obtain a first intermediate frame image based on the first preview image and the second original image; At a second moment, the display screen is used to display the first intermediate frame image, and the second moment is a moment separated by a single signal cycle after the first moment; The surface synthesizer obtains a second preview image after the second moment based on the second original image processed by the at least one algorithm; At a third moment, the display screen is used to display the second preview image, and the third moment is a moment separated by a single signal cycle from the second moment.

8. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images