Continuous shooting method and electronic equipment

By processing preview requests and photo requests separately in the continuous shooting mode of electronic devices, the problem that the preview image output speed depends on the speed of taking pictures, and the preview fluency and user experience during continuous shooting are improved.

CN117729420BActive Publication Date: 2024-11-29HONOR DEVICE CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310553397.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-29
Estimated Expiration
2043-05-16

AI Technical Summary

Technical Problem

In the continuous shooting mode of electronic devices, the output speed of the preview image depends on the output speed of the photographic image, resulting in a stuttering preview.

Method used

By sending the preview request and the photo request separately to the hardware abstraction layer when the target application is in continuous shooting mode, the preview image output speed is avoided from relying on the photo output speed of the photo taken.

Benefits of technology

It achieves smoother preview during continuous shooting, improves the user's photography experience, and avoids the problem of preview lag.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117729420B_ABST
    Figure CN117729420B_ABST
Patent Text Reader

Abstract

The present application proposes a continuous shooting method and electronic device, which relates to the field of terminal technology and is used to solve the problem of preview freeze during the continuous acquisition of multiple frames of images in the continuous shooting mode of the electronic device. The continuous shooting method may include: after the electronic device detects the operation of the user for instructing to start the target application, the target application is started, the electronic device displays a first preview image based on a first request including a preview request issued by the target application, after the electronic device detects the first operation of the user for indicating the continuous shooting mode of the target application, in response to the first operation, the electronic device removes the preview request in the first request through the target application, adds at least one photo request to the first request, obtains a second request, and the electronic device saves the photo image based on the second request issued by the target application.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a continuous shooting method and electronic device. Background Art

[0002] At present, the camera function is a common function provided by electronic devices. Users can use electronic devices to make video calls, take photos or record videos. Users usually use the continuous shooting function of the camera to record the different postures of moving objects at different times. However, when the electronic device continuously takes photos, the camera application of the electronic device continuously collects images, and there is a phenomenon of preview freeze. Summary of the invention

[0003] The embodiments of the present application provide a continuous shooting method and an electronic device, which can reduce the preview freeze phenomenon when the electronic device is capturing images in continuous shooting mode, and improve the user experience when using the electronic device for continuous shooting.

[0004] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0005] In a first aspect, a continuous shooting method is provided, which is applied to an electronic device. The method may include: the electronic device starts a target application in response to a user operation, the electronic device displays a first preview image based on a first request sent by the target application, after the electronic device detects a first operation by the user for indicating a continuous shooting mode of the target application, in response to the first operation, the preview request in the first request is removed through the target application, at least one photo request is added to the first request to obtain a second request, and the electronic device saves the photo image based on the second request sent by the target application.

[0006] In this solution, when the target application is in continuous shooting mode, the target application sends a preview request and then sends a photo request, that is, the target application sends a preview request and a photo request respectively, and the bottom layer of the system processes the preview request and the photo request in parallel, so that the output speed of the preview image and the output speed of the photo image do not affect each other, thereby avoiding the phenomenon in the related art that the output speed of the preview image depends on the output speed of the photo image, resulting in a freeze in the preview during continuous shooting.

[0007] In a possible implementation manner of the first aspect, the target application removes the preview request in the first request, adds at least one photo request to the first request, and obtains the second request, which may include:

[0008] The target application calls the remove function to remove the preview request in the first request; the target application calls the add function to add at least one photo request to the first request to obtain the second request.

[0009] For example, the target application can call the request.removeTarget(previewSurface) function to remove the preview request in the first request, and the target application can call the request.addTarget(captureSurface) function to add the capture request to the first request.

[0010] In another possible implementation manner of the first aspect, the continuous shooting method may further include:

[0011] The electronic device detects a second operation of the user for instructing the target application to exit the continuous shooting mode; in response to the second operation, the target application removes the photo request in the second request and adds at least one preview request to the second request, thereby obtaining a third request;

[0012] The electronic device displays the second preview image based on the third request sent by the target application.

[0013] For example, when the electronic device detects that the user stops pressing the shooting control, the target application removes the shooting request in the second request, adds at least one preview request to the second request, and obtains a third request. The electronic device displays the second preview image based on the third request sent by the target application. Thus, the process of the target application exiting the continuous shooting mode and re-entering the preview mode is realized.

[0014] In addition, the second operation may also be an operation of exiting the continuous shooting mode by the user's voice instruction.

[0015] In another possible implementation manner of the first aspect, the electronic device saves the photographed image based on the second request, including:

[0016] The electronic device sends a photographing request to the application framework layer based on the target application, the target application receives the first photographing image sent by the application framework layer, and the electronic device saves the first photographing image;

[0017] The electronic device sends a photographing request for collecting a next frame of photographic image to the application framework layer based on the target application; after the target application receives the second photographic image sent by the application framework layer, the electronic device saves the second photographic image.

[0018] That is, in the process of the target application continuously capturing multiple frames of images, after the target application receives a frame of photographic image, the target application sends a photographic request for capturing the next frame of photographic image to capture the next frame of photographic image.

[0019] In another possible implementation manner of the first aspect, the electronic device displays a first preview image based on a first request sent by a target application, including:

[0020] The electronic device sends a preview request to the application framework layer based on the target application, and the target application receives a first preview image sent by the application framework layer; the electronic device displays the first preview image.

[0021] In another possible implementation of the first aspect, after the electronic device sends the preview request to the application framework layer based on the target application, the continuous shooting method may further include:

[0022] The application framework layer stores the preview requests in the first queue.

[0023] In another possible implementation of the first aspect, after the electronic device sends a photo request to the application framework layer based on the target application, the continuous shooting method may further include:

[0024] The application framework layer stores the photo-taking request in the second queue.

[0025] In another possible implementation manner of the first aspect, the target application is an application with a shooting function. For example, the target application may be a camera application.

[0026] In a second aspect, the present application provides an electronic device comprising: a display screen; one or more processors; and a memory; wherein the memory stores one or more computer programs, and the one or more computer programs include instructions, which, when executed by the electronic device, enable the electronic device to perform a continuous shooting method as described in any one of the above-mentioned first aspects.

[0027] In a third aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the continuous shooting method as described in any one of the first aspects.

[0028] In a fourth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the continuous shooting method as described in any one of the first aspects.

[0029] It can be understood that the electronic device described in the second aspect, the computer storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 An example diagram of an image captured by a camera provided in an embodiment of the present application;

[0031] Figure 2An interactive diagram of a camera capturing a single frame image provided for related technology;

[0032] Figure 3 An interactive diagram of a camera continuously capturing multiple frames of images provided for related technology;

[0033] Figure 4 The timing diagram of the hardware abstraction layer processing preview request and photo request in the related technology;

[0034] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0035] Figure 6 A software structure diagram of an electronic device provided in an embodiment of the present application;

[0036] Figure 7 An interface diagram of a continuous shooting method provided in an embodiment of the present application;

[0037] Figure 8 An example diagram of starting a continuous shooting mode provided in an embodiment of the present application;

[0038] Fig. 9 Another example diagram of starting the continuous shooting mode provided in an embodiment of the present application;

[0039] Fig.10 An interactive diagram of a continuous shooting method provided in an embodiment of the present application;

[0040] Fig.11 A timing diagram of the hardware abstraction layer processing preview requests and photo request provided in an embodiment of the present application;

[0041] Fig.12 A time sequence diagram of image frame generation provided in an embodiment of the present application;

[0042] Fig.13 A flowchart of a continuous shooting method provided in an embodiment of the present application;

[0043] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solution in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. In the description of the embodiment of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0045] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0046] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0047] The electronic device in the embodiment of the present application has a shooting function, and the electronic device can use an application with a shooting function (such as a camera application) to acquire a frame of image, or use a continuous shooting mode of the application with a shooting function to acquire multiple frames of image. The following is an exemplary description using the application with a shooting function as a camera application as an example.

[0048] In one possible scenario, when a user uses a camera application in an electronic device to capture a frame of image, the electronic device detects a trigger operation (such as a click operation or a touch operation) of a shooting control by the user. In response to the trigger operation, the electronic device performs a photo-taking operation and captures a frame of image.

[0049] Figure 1 This is an example diagram of an image captured by a camera provided in an embodiment of the present application. Figure 1 (a) shows a graphical user interface (GUI) of a mobile phone, which is a desktop 101 of the mobile phone. When the mobile phone detects that a user clicks on an icon 102 of a camera application on the desktop 101, the camera application can be started and a screen like Figure 1 Another GUI shown in (b) of FIG. 1 may be referred to as a shooting interface 103. The shooting interface 103 may include a viewfinder 104. In a preview state, the viewfinder 104 may display a preview image in real time.

[0050] It is understandable that the size of the viewfinder 104 may be different in the photo mode and the video mode (ie, the video shooting mode). Figure 1 The viewfinder shown in (a) in the figure may be a viewfinder in the photographing mode. In the video recording mode, the viewfinder 104 may cover the entire touch screen.

[0051] For example, see Figure 1In (b), after the mobile phone starts the camera, a preview image may be displayed in the viewfinder 104. The shooting interface may also include a control 105 for indicating the night scene mode, a control 106 for indicating the portrait mode, a control 107 for indicating the photo mode, a control 108 for indicating the video mode, and a shooting control 109. In the night scene mode, when the mobile phone detects that the user clicks the shooting control 109, the mobile phone performs a shooting operation in the night scene mode; in the photo mode, when the mobile phone detects that the user clicks the shooting control 109, the mobile phone performs a photo operation; in the video mode, when the mobile phone detects that the user clicks the shooting control 109, the mobile phone performs a video operation.

[0052] For example, Figure 2 An interactive diagram of a camera capturing a single frame image is provided for related technologies. Figure 2As shown, after the mobile phone starts the camera and enters the photo mode, the camera application creates a preview request according to the preview template. Then, the camera application instantiates the preview request, for example, the camera application uses the following method to create the preview request: previewRequest = cameraService.initPreviewRequest (TEMPLATE_PREVIEW), the camera application points the preview request to the preview stream, for example, the camera application uses the following method to point the preview request to the preview stream: previewRequest.setTarget (previewSurface), the camera application sends the preview request to the preview stream, for example, the camera application calls the setRepeatingRequest (previewRequest, callBack) function to repeatedly send the preview request to the application framework layer. The application framework layer of the mobile phone receives the preview request sent by the camera application, and the application framework layer repeatedly sends the preview request to the hardware abstract layer (HAL). The hardware abstract layer processes the preview request to display the preview image in the viewfinder of the camera application. The camera application creates a photo request according to the photo template, for example, the camera application uses the following method to create a request for taking a photo, captureRequest = cameraService.initCaptureRequest (TEMPLATE_STILL_CAPTURE). After the camera application creates a photo request, the mobile phone detects the user's trigger operation on the shooting control (for example, the operation of clicking the shooting control or the operation of touching the shooting control). In response to the trigger operation, the camera application directs the photo request to the photo stream and sends the photo request to the photo stream. For example, the camera application uses the following method to direct the request to take a photo: captureRequest.appTarget(captureSurface). Then, the camera application sends the photo request to the application framework layer. For example, the camera application calls the capture(captureRequest, callBack) function to send the photo request to the application framework layer. After receiving the photo request, the application framework layer sends the photo request to the hardware abstraction layer, and the hardware abstraction layer processes the photo request. The hardware abstraction layer triggers the camera to capture an image and displays the captured frame of image on the display screen of the mobile phone.

[0053] The preview request is to request the system to output image data in YUV format for the user to preview. The picture that the user sees in the viewfinder after opening the camera is the preview picture. The photo request is to request the system to output image data in JPEG format to save to the local album.

[0054] In another possible scenario, when a user uses a camera application in an electronic device to continuously capture multiple frames of images, the electronic device detects a trigger operation (e.g., a long press operation) of a shooting control by the user. In response to the trigger operation, the electronic device enters a continuous shooting mode and continuously captures multiple frames of images.

[0055] For example, see Figure 1 In (b), after the mobile phone starts the camera, in the photo mode, when the mobile phone detects that the user has long pressed the shooting control 109, the mobile phone performs a continuous photo operation and continuously captures multiple frames of images until the mobile phone detects that the user has ended the long press operation on the shooting control 109.

[0056] For example, Figure 3 An interactive diagram of a camera continuously collecting multiple frames of images is provided for related technologies. Figure 3 As shown, the user long presses the camera's shooting control, and the electronic device responds to the user's long press operation on the shooting control, and the camera application enters the continuous shooting mode. The camera application creates a request based on the shooting template, and the camera application uses the following method to create a request for shooting: captureRequest = cameraService.initCaptureRequest (TEMPLATE_STILL_CAPTURE). Here, the request created by the camera application includes a preview request and a photo request. The camera application points the preview request to the preview stream and the photo request to the photo stream. The camera application calls request.setTarget (preview.Surface) to point the request to the preview stream, and the camera application calls request.setTarget (capture.Surface) to point the request to the photo stream. Then, the camera application sends the preview request and the photo request to the application framework layer of the electronic device at the same time. For example, the camera application calls setRepeatingRequest (request, callBack) to send the preview request and the photo request. After the application framework layer of the electronic device receives the preview request and the photo request, the application framework layer sends the preview request and the photo request to the hardware abstraction layer, and the hardware abstraction layer processes the preview request and the photo request. Typically, in the process of an electronic device continuously capturing multiple frames of images, the time taken by the hardware abstraction layer to process a photo request corresponding to each frame of image is longer than the time taken to process a preview request corresponding to each frame of image, resulting in the preview image output speed when the camera application captures a frame of image being dependent on the output speed of the captured image (also called the photo image). In the process of the system processing the photo request, the preview image is continuously displayed in the viewfinder of the camera application, resulting in preview freezes during continuous shooting.

[0057] For example, Figure 4, assuming that it takes 20ms for the hardware abstraction layer to process a preview request and 30ms for a photo request. After the hardware abstraction layer receives the preview request and the photo request, the hardware abstraction layer processes the preview request and the photo request in parallel, that is, the hardware abstraction layer processes the preview request and the photo request at the same time. After the hardware abstraction layer receives the photo request, it acquires the photo image based on the photo request, and then uploads the photo image to the camera application for saving. However, after the hardware abstraction layer receives the preview request, it acquires the preview image based on the preview request, and then uploads the preview image to the camera application for display and preview. Since it takes a long time for the hardware abstraction layer to process the photo request corresponding to a frame of image, the hardware abstraction layer finishes processing the preview request corresponding to the frame of image. During the process of displaying the preview image on the preview interface, the hardware abstraction layer is still processing the photo request, resulting in an unsmooth presentation of the preview image. It can be seen that in the process of the camera application continuously capturing multiple frames of images, the speed of outputting the photographed images is slow, and multiple preview images are displayed in the preview interface. In the process of displaying multiple preview images, the hardware abstraction layer is still processing the corresponding photo request, resulting in the presentation of multiple preview images being not smooth, and there is a freeze, and the preview effect is not good.

[0058] In order to solve the above problems, the embodiment of the present application provides a continuous shooting method, which is applied to an electronic device. When the camera application is in continuous shooting mode, the camera application creates a first request, and the camera application points the first request to the preview stream. After the camera application sends a preview request to the application framework layer, the camera application points the first request to the photo stream, and the camera application sends a photo request to the application framework layer. As a result, the camera application sends the preview request and the photo request separately to the hardware abstraction layer, and the hardware abstraction layer processes the preview request and the photo request in parallel, so that the preview output speed is no longer dependent on the photo output speed, so that the camera application previews more smoothly during continuous shooting, improving the user's photo experience.

[0059] Exemplarily, the continuous shooting method provided in the embodiments of the present application can be applied to mobile phones, tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, netbooks, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, smart cars, smart speakers and other electronic devices with shooting functions, and the embodiments of the present application do not impose any restrictions on this.

[0060] like Figure 5 As shown, Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0061] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0062] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0063] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0064] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0065] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0067] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0068] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0069] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0070] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0071] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0072] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0073] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0074] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0075] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0076] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0077] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0078] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0079] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0080] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0081] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0082] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0083] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0084] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0085] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0086] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0087] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0088] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0089] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0090] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0091] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0092] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0093] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0094] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0095] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0096] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0097] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0098] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0099] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0100] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .

[0101] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0102] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0103] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0104] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser.

[0105] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 may adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0106] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0107] The temperature sensor 180J is used to detect the temperature.

[0108] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The buttons 190 can be mechanical buttons. It can also be a touch button. The electronic device 100 can receive button input and generate key signal input related to the user settings and function control of the electronic device 100.

[0109] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0110] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0111] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0112] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes a system with a layered architecture as an example to exemplify the software structure of the electronic device.

[0113] Figure 6 A software structure diagram of an electronic device provided in an embodiment of the present application.

[0114] It is understandable that the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system may include an application layer (referred to as the application layer), an application framework layer (referred to as the framework layer), a hardware abstraction layer, and a kernel layer.

[0115] The above-mentioned application layer may include a series of application packages.

[0116] like Figure 6 As shown, the application package may include system applications. System applications refer to applications that are set in the electronic device before leaving the factory. For example, the system applications may include programs such as camera, gallery, calendar, music, short message, and weather.

[0117] The application package may also include third-party applications, which are applications that users install after downloading the installation package from an application store (or application market), such as map applications, food delivery applications, reading applications (such as e-books), social applications, and travel applications.

[0118] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions.

[0119] like Figure 6 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0120] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0121] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0122] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0123] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0124] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0125] The notification manager allows applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a reminder sound is emitted, the phone vibrates, the indicator light flashes, etc.

[0126] The hardware abstraction layer is a layer of encapsulation for the underlying hardware driver, providing a general interface for calling the driver to the application framework layer. In the embodiment of the present application, the hardware abstraction layer may include a camera hardware abstraction module and an image processing module.

[0127] The camera hardware abstraction module is used to control the target camera to perform camera initialization operations. For example, the camera hardware abstraction module can perform initialization operations such as powering on and parameter configuration on the target camera. The target camera is a camera used to capture images. The target camera can be a front camera or a rear camera, which is not limited here.

[0128] The image processing module is used to process the image captured by the target camera. For example, the image processing module can adjust the size of the preview image, and can also perform beauty processing, night scene processing, and denoising processing on the photographed image.

[0129] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver and sensor driver.

[0130] In some embodiments, the sensor driver may be used to obtain information collected by at least one sensor in real time or periodically, and report the information collected by the sensor.

[0131] The camera driver can be used to trigger the target camera that has completed the initialization operation to send and display images.

[0132] In some embodiments, the camera driver may include a front camera driver and a rear camera driver, wherein the front camera driver may be used to drive the front camera, and the rear camera driver may be used to drive the rear camera.

[0133] For ease of understanding, the following embodiments of the present application will take a mobile phone with the above-mentioned hardware structure and software architecture as an example, and combine the accompanying drawings to exemplarily illustrate the continuous shooting method provided in the embodiments of the present application.

[0134] Figure 7 This is an interface diagram of a continuous shooting method provided in an embodiment of the present application. Figure 7 (a) shows a GUI of a mobile phone. When the mobile phone detects that a user clicks on the icon 702 of the camera application on the desktop 701, the camera application is started and the display is as follows: Figure 7 Another GUI shown in (b) is the shooting interface 703. In the preview state, the viewfinder 704 displays the preview image in real time. In the shooting mode, when the mobile phone detects that the user is Figure 7 After the long press operation of the shooting control 705 shown in (b) of FIG. 1 is performed, in response to the long press operation, the camera application enters the continuous shooting mode. In the continuous shooting mode, the camera application continuously captures multiple frames of images until the mobile phone detects that the user stops the long press operation of the shooting control 705, and the camera application exits the continuous shooting mode and displays the following Figure 7 The shooting interface 707 shown in (c) .

[0135] When the camera application continuously acquires images, the display screen can display the number of frames of the image currently acquired by the camera application in real time, for example, Figure 7 The display area 706 in (b) shows that the camera has currently captured 8 frames of images.

[0136] As a possible implementation, the number of image frames collected by the camera application in continuous shooting mode is positively correlated with the length of time the phone detects that the user has long pressed the shooting control. The longer the phone detects that the user has long pressed the shooting control, the more frames of images collected by the camera application. As a possible implementation, when the phone detects that the user has long pressed the shooting control to continuously collect multiple frames of images, when the phone determines that the number of frames of the collected images has reached a preset frame (for example, 100 frames, 200 frames, etc.), the camera application will no longer collect images even if the phone detects that the user is still long pressing the shooting control.

[0137] The camera used by the camera application to collect images can be the front camera of the mobile phone or the rear camera of the mobile phone, which is not limited here. When the mobile phone is a folding screen mobile phone, the camera used by the camera application to collect images can be the internal screen front camera of the folding screen mobile phone, the external screen front camera of the folding screen mobile phone, or the rear camera of the folding screen mobile phone, which is not limited here.

[0138] The above mobile phone detects the user's long press operation on the shooting control, and the camera application enters the continuous shooting mode, which is only a possible implementation method of the embodiment of the present application. There are many ways for the camera application to enter the continuous shooting mode.

[0139] In one embodiment, the shooting interface includes a control for indicating the continuous shooting mode. When the mobile phone detects that the user clicks on the control, the camera application enters the continuous shooting mode.

[0140] Exemplarily, the control for indicating the continuous shooting mode may be: Figure 8 The control 801 shown in (a) in FIG. 1 , or Figure 8 The control 802 shown in (b) in FIG. 1 , or Figure 8 Control 803 shown in (c) .

[0141] When the phone detects that the user clicks the shooting control, the camera application continuously captures multiple frames of images. Therefore, when the camera application is in continuous shooting mode, the phone detects the user's triggering operation on the shooting control and continuously captures multiple frames of images, avoiding the situation where the user presses the shooting control to continuously capture multiple frames of images, and realizing that the user clicks or touches the shooting control once to continuously capture multiple frames of images.

[0142] The mobile phone can preset the number of images collected in the continuous shooting mode in response to the user's setting operation, for example, preset the continuous shooting mode to collect 10 frames of images. When the camera application is in the continuous shooting mode, the mobile phone detects the user's triggering operation on the shooting control, and the camera application continuously collects 10 frames of images.

[0143] In another embodiment, see Fig. 9 In (a), the shooting interface includes more controls 901. After the mobile phone detects that the user clicks on the more controls 901, see Fig. 9 In (b), the mobile phone displays an interface 902. When the mobile phone detects that the user clicks on the shooting control 903, the camera application enters a continuous shooting mode. Fig. 9 In (c), when the mobile phone detects that the user clicks the shooting control 903, the camera application continuously captures multiple frames of images. Therefore, when the camera application is in continuous shooting mode, the mobile phone detects the user's triggering operation on the shooting control and continuously captures multiple frames of images, avoiding the situation where the user presses the shooting control for a long time to continuously capture multiple frames of images, and realizing that the user clicks or touches the shooting control once to continuously capture multiple frames of images.

[0144] In another embodiment, after the mobile phone displays the shooting interface, see Fig. 9 In (a), if the mobile phone detects the user's voice instruction to enter the continuous shooting mode, the camera application enters the continuous shooting mode. Fig. 9 In (c), when the phone detects that the user clicks the shooting control, the camera application continuously captures multiple frames of images.

[0145] In another embodiment, after the mobile phone displays the shooting interface, if the mobile phone detects the user's voice instruction to use the continuous shooting function, the camera application continuously captures multiple frames of images. For example, if the mobile phone detects the user's voice instruction "take 10 photos in a row", the camera application continuously captures 10 frames of images. In an embodiment of the present application, after the camera application enters the continuous shooting mode for the first time or each time, the mobile phone can prompt the user by displaying on the touch screen or by sound to remind the user that the camera application is currently shooting in the continuous shooting mode. For example, the mobile phone can prompt the user that "the continuous shooting mode has been entered" by displaying a text message, see Fig. 9 As shown in (c) in .

[0146] When the mobile phone detects the user's operation for instructing to exit the camera application, the camera application is closed in response to the user's operation. After the camera application is closed, the display screen of the mobile phone may display the following Figure 7The desktop shown in (a) in the figure. The operation for the user to instruct to exit the camera application may be an operation of the user swiping up the display screen, or an operation of the user instructing to exit the camera application through voice, etc. The specific operation method for the user to instruct to exit the camera application is not limited in the embodiments of the present application.

[0147] Fig.10 This is an interactive diagram of a continuous shooting method provided in an embodiment of the present application. Fig.10 The implementation process of the above continuous shooting method is introduced in detail.

[0148] In an embodiment of the present application, when the mobile phone detects that the user instructs to start the camera application, in response to the user's operation, the system calls the CameraManager.openCamera method to start the camera application. Among them, CameraManager is used to open and close the system camera. At this time, the CameraManager.openCamera method passes in three parameters, namely, Srting Cameraid (the identification number of the camera to be started), CameraDevice.stateCallback (an instance of CameraDevice.stateCallback), and Handler (a handle representing the Handler that executes the callback. If the program wants to execute the callback directly in the current thread, the Handler parameter can be set to null).

[0149] When the system detects that the camera application has been successfully started in the CameraDevice.statCallback callback, the camera application initializes the camera device (CameraDevice), creates a capture image request through CameraDevice, and attaches the corresponding Surface to the request to complete the creation of CaptureRequest. The camera application creates a camera capture session (CameraCaptureSession) through CameraDevice, and starts processing requests after the session configuration is completed. The camera capture session comes with a set of output surfaces (output Surfaces) used with CameraDevice, and the session is created using createCaptureSession(SessionConfiguration). Each output surface is pre-configured with a suitable size and format to match the size and format of the camera output, and the output surface can be of multiple types. For example, output surfaces can include SurfaceView, SurfaceTexture, MediaCodec, MediaRecorder, Allocation, and ImageReader.

[0150] It should be noted that when the camera application previews or captures images, both the preview data and the photo data are requested using the CameraCaptureSession session.

[0151] like Fig.10 As shown, the camera application creates a preview request based on the preview template, and then the camera application instantiates the preview request. For example, the camera application uses the following method to create the preview request: previewRequest = cameraService.initPreviewRequest (TEMPLATE_PREVIEW). The camera application points the preview request to the preview stream. For example, the camera application calls the request.addTarget (previewSurface) method to point the preview request to the preview stream. The camera application sends the preview request through the CameraCaptureSession session.

[0152] When the camera application is in preview mode, the camera application sends a preview request to the application framework layer. For example, the camera application calls the setRepeatingRequest(request, callBack) method to repeatedly send a preview request to the application framework layer. After receiving the preview request, the application framework layer sends the preview request to the hardware abstraction layer. The hardware abstraction layer processes the preview request to display the preview image in the preview SurfaceView.

[0153] In an embodiment of the present application, after the hardware abstraction layer receives the preview request, the camera hardware abstraction module of the hardware abstraction layer can control the target camera to perform camera initialization operations. Among them, the camera initialization operation includes power-on, configuration register sequence and other operations. After the camera driver control corresponding to the target camera has completed the initialization of the target camera to collect the preview image, the target camera uploads the collected preview image to the hardware abstraction layer. The hardware abstraction layer uploads the preview image to the camera application through the application framework layer, and the camera application displays the preview image (such as the image in YUV format obtained after processing) in the preview interface. Here, the camera application can call ImageReader to display the preview image on the preview SurfaceView.

[0154] In one embodiment, after the hardware abstraction layer receives the preview image, the image processing module of the hardware abstraction layer can process the preview image to obtain a processed preview image. For example, the image processing module can adjust the size of the preview image to avoid deformation of the preview image when it is displayed on the display screen of the mobile phone. Then, the hardware abstraction layer uploads the processed preview image to the camera application through the application framework layer, and the camera application displays the processed preview image in the preview interface. Here, the camera application can call ImageReader to display the processed preview image in the preview SurfaceView.

[0155] It should be explained that when the system creates an ImageReader, it first obtains the list of photo sizes supported by the phone's display, and selects the appropriate size from the list, and obtains a maximum of several frames of data each time the ImageReader is called. When the camera application monitors that preview image frame data is available in the ImageReader event, the camera application will call back the onImageAvailable method, and the parameter of the onImageAvailable method is the preview image frame data, so that the preview image is displayed in the preview SurfaceView.

[0156] like Fig.10 As shown, when the mobile phone detects that the user instructs the camera to enter the continuous shooting mode, the camera application enters the continuous shooting mode in response to the user's operation. In this case, the camera application calls the stopRepeating method to stop capturing images, that is, the preview image is stopped from being displayed on the display screen of the mobile phone, the camera application closes the preview stream in the preview mode, and restarts the preview stream. The camera application creates a first request based on the photo template. For example, the camera application uses the following method to create a request for taking a photo: request = cameraService.initCaptureRequest(TEMPLATE_STILL_CAPTURE). The camera application points the first request to the preview stream. For example, the camera application calls the request.addTarget(previewSurface) method to point the first request to the preview stream. The camera application sends a preview request through the CameraCaptureSession session.

[0157] Then, the camera application performs an operation to remove the preview stream pointed to by the first request, for example, the camera application calls the request.removeTarget(previewSurface) function to perform an operation to remove the preview stream pointed to by the first request. Then, the camera application points the first request to the photo stream, and the camera application sends a photo request to the application framework layer. For example, the camera application calls the request.addTarget(captureSurface) function to point the first request to the photo stream. After the camera application sends the photo request to the application framework layer, the application framework layer sends the photo request to the hardware abstraction layer. For example, the camera application uses the captureBurst(requests.callBack) function to send four photo requests, and the application framework layer sends four photo requests to the hardware abstraction layer. After the hardware abstraction layer receives the photo request, the camera hardware abstraction module of the hardware abstraction layer determines that the target camera for collecting images is initialized, and the camera driver corresponding to the target camera controls the initialized target camera to collect the photo image. The target camera uploads the collected photo image to the hardware abstraction layer. Then, the hardware abstraction layer uploads the photo image to the camera application through the CameraCaptureSession session, and the camera application saves the photo image locally on the mobile phone. For example, the camera application saves the photographed image (such as the processed JPEG format image) to the memory card of the mobile phone.

[0158] In one possible case, after the hardware abstraction layer receives the photographic image captured by the target camera, the image processing module of the hardware abstraction layer can process the photographic image to obtain a processed photographic image. For example, the image processing module can perform processing operations such as beautification, denoising, and compression on the photographic image to obtain a photographic image with higher image quality. Then, the hardware abstraction layer uploads the processed photographic image to the camera application through the CameraCaptureSession session, and the camera application saves the processed photographic image locally on the mobile phone.

[0159] In an embodiment of the present application, when the camera application creates a CameraCaptureSession, an ImageReader Surface user is added to the output Surface list to obtain the photo stream data. When the camera application creates an ImageReader for taking photos, the photo size is set, and then the photo size is used when the ImageReader is initialized. When the camera application monitors that there is photo stream data available in the ImageReader event, the onImageAvailable method will be called back, and the parameter of the onImageAvailable method is the photo stream data.

[0160] In an embodiment of the present application, after the camera application receives a frame of photographic image returned by the onImageAvailable method, the camera application calls the capture(request, callback) method again to issue a photographic request, that is, the camera application cyclically issues photographic requests until the number of frames of photographic images received by the camera application reaches a preset frame, or until the mobile phone detects an operation by the user to instruct the camera to exit the continuous shooting mode. For example, assuming that the preset frame is 100 frames, when the camera application receives 100 frames of photographic images, the camera application stops issuing photographic requests. Alternatively, when the mobile phone detects that the user stops pressing the shooting control, the camera application stops issuing photographic requests.

[0161] When the camera application stops sending the photo request, the camera application returns to the preview mode, restarts the preview stream, and sends the preview request to the application framework layer again. The implementation process of the camera application in the preview mode can refer to the specific implementation process in the above embodiment, which will not be repeated here.

[0162] When the mobile phone detects the user's instruction to exit the camera application, in response to the user's operation, the camera application exits the preview mode and closes the camera application. Exemplarily, the camera application calls the captureSession.close method to exit the preview mode. The system calls the CameraDevice.close method to close the camera application.

[0163] In summary, in an embodiment of the present application, when the camera application is in continuous shooting mode, the camera application creates a first request, the camera application points the first request to the preview stream, and after the camera application sends the preview request to the application framework layer, the camera application points the first request to the photo stream, and the camera application sends the photo request to the application framework layer. After the application framework layer receives the preview request, the application framework layer sends the preview request to the hardware abstraction layer, and after the hardware abstraction layer processes the preview request, the preview image is uploaded to the camera application through the application framework layer for preview display. After the camera application sends the preview request to the application framework layer, the camera application sends the photo request to the application framework layer, and after the application framework layer receives the photo request, the photo request is sent to the hardware abstraction layer, and after the hardware abstraction layer processes the photo request, the photo image is uploaded to the camera application through the application framework layer for storage. The camera application repeatedly sends the photo request to the application framework layer until the number of frames of the photo image received by the camera application reaches the preset frame, or until the mobile phone detects the user's operation for instructing the camera to exit the continuous shooting mode.

[0164] It can be seen that the camera application sends the preview request and the photo request separately to the hardware abstraction layer, and the hardware abstraction layer processes the preview request and the photo request in parallel, so that the preview output speed is no longer dependent on the photo output speed, thereby making the camera application preview smoother during continuous shooting, improving the user's photo-taking experience.

[0165] It should be noted that when the camera application is in continuous shooting mode, the camera application sends preview requests and photo requests to the application framework layer respectively to continuously capture multiple frames of photo images. This solves the problem in the related technology that when the camera application sends preview requests and photo requests at the same time, the time it takes for the hardware abstraction layer to process the preview request is less than the time it takes to process the photo request, resulting in the preview image being displayed for a long time on the preview interface and the preview being stuck.

[0166] for example, Fig.11 The hardware abstraction layer provided in the embodiment of the present application processes the timing diagram of the preview request and the photo request. Fig.11 It can be seen that, assuming that it takes 20ms for the hardware abstraction layer to process a preview request and 30ms for a photo request. After the hardware abstraction layer receives the preview request sent by the application framework, the hardware abstraction layer obtains the preview image (such as an image in YUV format) based on the preview request, and then uploads the preview image to the preview interface of the camera application for display and preview. After the hardware abstraction layer receives the photo request, it obtains the photo image (such as an image in JPEG format) based on the photo request, and then uploads the photo image to the camera application for saving. Obviously, the output speed of the preview image displayed on the preview interface of the camera application is no longer dependent on the output speed of the photo image. After the hardware abstraction layer processes a preview request, it can upload the preview image collected based on the preview request to the preview interface for preview, thereby avoiding the phenomenon of preview freeze during continuous shooting.

[0167] For example, Fig.12 This is a time sequence diagram of image frame generation provided in the embodiment of the present application. Fig.12 As shown, after receiving the preview request sent by the camera application, the application framework layer stores the preview request in the preview request queue (such as Fig.12 After receiving the photo request sent by the camera application, the application framework layer stores the photo request in the photo request queue (such as Fig.12 The application framework layer sends the preview request in queue 1 and the photo-taking request in queue 2 to the hardware abstraction layer in parallel, and the hardware abstraction layer processes the preview request and the photo-taking request.

[0168] like Fig.12As shown, when the camera application is in continuous shooting mode, at time t1, when a photo request 10 enters the queue 2, the application framework layer sends the photo request 10 to the hardware abstraction layer, and the hardware abstraction layer generates a photo image frame 10 based on the photo request 10. At time t1+30ms, when a photo request 11 enters the queue 2, the application framework layer sends the photo request 11 to the hardware abstraction layer, and the hardware abstraction layer generates a photo image frame 11 based on the photo request 11.

[0169] When the hardware abstraction layer processes a photo request, if there is a preview request in queue 1, the application framework layer will first send the preview request to the hardware abstraction layer, and the hardware abstraction layer will process the preview request and generate a preview image frame. Fig.12 At time t1, when a photo request 10 is queued in queue 2 and a preview request 10 is queued in queue 1, the application framework layer sends the preview request 10 to the hardware abstraction layer, and the hardware abstraction layer generates a preview image frame 10 based on the preview request 10. After that, the application framework layer sends the photo request 10 to the hardware abstraction layer, and the hardware abstraction layer generates a preview image frame 10 based on the preview request 10 at time t1+20ms, and the hardware abstraction layer generates a photo image frame 10 based on the photo request 10 at time t1+30ms. At time t1+20ms, when a preview request 11 is queued in queue 1, the application framework layer sends the preview request 11 to the hardware abstraction layer, and the hardware abstraction layer generates a preview image frame 11 based on the preview request 11 at time t1+40ms. At time t1+30ms, when a photo request 11 is queued in queue 2, the application framework layer sends the photo request 11 to the hardware abstraction layer, and the hardware abstraction layer generates a photo image frame 11 based on the photo request 11 at time t1+60ms.

[0170] In combination with the above embodiments and related drawings, the present application embodiment provides a continuous shooting method. Figure 5 and Figure 6 The electronic device (such as a mobile phone, tablet computer, etc.) with a display screen and a camera is implemented as shown. Fig.13 As shown, the method may include the following steps:

[0171] Step 1301: The electronic device starts a target application.

[0172] Exemplarily, after the electronic device detects an operation by a user to start a target application, the electronic device starts the target application in response to the user's operation.

[0173] The above-mentioned operation of starting the target application by the user's instruction may be an operation of the user clicking the icon of the target application, or an operation of starting the target application by the user's voice instruction, etc., which is not limited here.

[0174] Step 1302: The electronic device displays a first preview image based on a first request sent by a target application.

[0175] The first request includes a preview request.

[0176] In an embodiment of the present application, after the target application of the electronic device sends a first request to the application framework layer, the application framework layer sends the first request to the hardware abstraction layer. After the hardware abstraction layer collects the first preview image based on the first request, the hardware abstraction layer sends the first request to the target application through the application framework layer. After the target application receives the first preview image, it displays the first preview image on the preview interface.

[0177] Step 1303: The electronic device detects a first operation performed by the user to indicate a continuous shooting mode of a target application.

[0178] For example, assuming that the target application is a camera application, the first operation may be a user long pressing Figure 7 The first operation may also be an operation in which the user instructs the camera application to enter the continuous shooting mode by voice, which is not limited in the embodiments of the present application.

[0179] Step 1304: In response to the first operation, the preview request in the first request is removed by the target application, and at least one photo-taking request is added to the first request to obtain a second request.

[0180] In an embodiment of the present application, in response to the first operation, the target application of the electronic device enters a continuous shooting mode. The target application can call a remove function to remove the preview request in the first request, and then call an add function to add at least one photo request to the first request to obtain a second request.

[0181] Step 1305: The electronic device saves the photographed image based on the second request sent by the target application.

[0182] In an embodiment of the present application, the target application may call an image capture function to send the second request to the application framework layer.

[0183] When the target application is in continuous shooting mode, the target application sends a second request to the application framework layer. After the target application receives the first photographed image sent by the application framework layer, the target application sends a second request to capture the next frame of photographed image to the application framework layer. The target application receives the second photographed image sent by the application framework layer, and the electronic device saves the second photographed image.

[0184] After the target application sends the second request to the application framework layer, the application framework layer sends the received second request to the hardware abstraction layer, and the hardware abstraction layer obtains the photographed image through the kernel layer. The hardware abstraction layer sends the photographed image to the target application through the application framework layer, and the target application repeatedly sends the second request to the application framework layer until the number of frames of the photographed image received by the target application reaches a preset frame.

[0185] In an embodiment of the present application, after the electronic device detects a second operation performed by the user to instruct the target application to exit the continuous shooting mode, in response to the second operation, the target application removes the photo request in the second request and adds at least one preview request to the second request to obtain a third request; the electronic device displays the second preview image based on the third request issued by the target application.

[0186] In an embodiment of the present application, the electronic device sends a preview request to the application framework layer based on the target application. After receiving the preview request, the application framework layer stores the preview request in the first queue.

[0187] In the embodiment of the present application, the electronic device sends a photo request to the application framework layer based on the target application, and after receiving the photo request, the application framework layer stores the photo request in the second queue, wherein the second queue is different from the first queue.

[0188] In an embodiment of the present application, when the target application monitors that there is photo frame data in the image reading event, the camera application calls back the image acquisition function, and the parameters of the image acquisition function carry the photo frame data; the electronic device saves the photo image according to the photo frame data.

[0189] When the electronic device displays the photographed image, the electronic device displays prompt information, where the prompt information is used to prompt the number of frames of the photographed image currently collected.

[0190] For example, the electronic device displays Figure 7 The indication information shown in (b) in the figure is used to remind the user that 8 frames of photographic images have been collected.

[0191] like Fig.14As shown, an embodiment of the present application discloses an electronic device, which may be the mobile phone mentioned above. The electronic device may specifically include: a touch screen 1401, wherein the touch screen 1401 includes a touch sensor 1406 and a display screen 1407; one or more processors 1402; a memory 1403; one or more application programs (not shown); and one or more computer programs 1404, and the above-mentioned components may be connected via one or more communication buses 1405. Among them, the above-mentioned one or more computer programs 1404 are stored in the above-mentioned memory 1403 and are configured to be executed by the one or more processors 1402, and the one or more computer programs 1404 include instructions, which may be used to execute the relevant steps in the above-mentioned embodiments.

[0192] It is understandable that, in order to realize the above functions, the above-mentioned electronic devices, etc. include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0193] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0194] In the case of dividing each functional module according to each function, a possible composition diagram of the electronic device involved in the above embodiment, the electronic device may include: a display unit, a transmission unit and a processing unit, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0195] The embodiment of the present application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the above-mentioned related method steps to implement the continuous shooting method in the above-mentioned embodiment.

[0196] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the continuous shooting method in the above-mentioned embodiment.

[0197] An embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the continuous shooting method in the above-mentioned embodiment.

[0198] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the device executes the continuous shooting method executed by the electronic device in the above-mentioned method embodiments.

[0199] Among them, the electronic device, computer-readable storage medium, computer program product or device provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0200] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0201] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0203] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A continuous shooting method, applied to an electronic device, characterized in that: The method comprises: Start the target application; The electronic device displays a first preview image based on a first request sent by the target application; the first request includes a preview request; The electronic device detects a first operation of a user for indicating a continuous shooting mode of the target application; In response to the first operation, the target application calls a removal function to remove the operation of the first request pointing to the preview stream; The target application calls an add function to direct the first request to the photo stream, thereby obtaining a second request; The electronic device saves the photographed image based on the second request sent by the target application.

2. The method according to claim 1, characterized in that: The method further comprises: The electronic device detects a second operation by a user for instructing the target application to exit the continuous shooting mode; In response to the second operation, the target application removes the photo request from the second request and adds at least one preview request to the second request to obtain a third request; The electronic device displays a second preview image based on the third request sent by the target application.

3. The method according to claim 1 or 2, characterized in that: The electronic device saves the photographed image based on the second request sent by the target application, including: The electronic device sends a photographing request to the application framework layer based on the target application, and the target application receives a first photographing image sent by the application framework layer; The electronic device stores the first photographed image; The electronic device sends a photographing request for collecting a next frame of photographing image to the application framework layer based on the target application; The target application receives a second photographed image sent by the application framework layer; The electronic device saves the second photographed image.

4. The method according to any one of claims 1 to 3, characterized in that: The electronic device displays a first preview image based on a first request sent by the target application, including: The electronic device sends a preview request to the application framework layer based on the target application, and the target application receives the first preview image sent by the application framework layer; The electronic device displays the first preview image.

5. The method according to claim 4, characterized in that After the electronic device sends the preview request to the application framework layer based on the target application, the method further includes: The application framework layer stores the preview request in a first queue.

6. The method according to claim 3, characterized in that After the electronic device sends the photo request to the application framework layer based on the target application, the method further includes: The application framework layer stores the photo-taking request in a second queue.

7. The method according to any one of claims 1 to 6, characterized in that: The target application is an application with a shooting function.

8. An electronic device, characterized in that: include: Display screen; one or more processors; Memory; Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the continuous shooting method as described in any one of claims 1-7.

9. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When the instruction is executed on an electronic device, the electronic device executes the continuous shooting method as claimed in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Task scheduling method, task scheduling device and storage medium

    CN112217990A

  • Camera photographing parameter setting method for smart terminal, setting device, and smart terminal

    WO2019056242A1