A photographing method, an electronic device, a storage medium, and a program product

Abstract

The application provides a photographing method, an electronic device, a storage medium and a program product, which are applied to the terminal technical field and realize obtaining an image with higher image quality on the basis of improving the efficiency of the photographed image. The photographing method comprises the following steps: starting a camera application, and displaying a photographing preview interface; in response to a photographing operation, acquiring image frame information corresponding to an image collected by a camera, storing the image frame information, and obtaining a first target image through photographing; after displaying a first gallery interface comprising the first target image in response to a first operation, displaying a second gallery interface comprising a second target image in response to a second operation. The second target image is obtained based on the stored image frame information, and the image quality of the second target image is higher than that of the first target image.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a photographing method, electronic device, storage medium, and program product. Background Technology

[0002] With the development of electronic devices, more and more electronic devices have shooting functions, allowing users to take pictures anytime and anywhere. When users take pictures using the camera application (APP) on their electronic devices, such as during burst shooting or quick shooting, the shooting process may take a long time, requiring a considerable amount of time to obtain the captured image, thus affecting the user's shooting experience. Summary of the Invention

[0003] This application provides a photographing method, electronic device, storage medium, and program product, which not only improves the efficiency of image capture but also enables the acquisition of images with better quality.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] Firstly, a method for taking a photograph is provided, the method comprising:

[0006] Launch the camera application and display the shooting preview interface; in response to the shooting operation, obtain the image frame information corresponding to the image captured by the camera, store the image frame information, and capture the first target image; in response to the first operation, display the first gallery interface including the first target image, and in response to the second operation, display the second gallery interface including the second target image.

[0007] The second target image is obtained based on stored image frame information, and the image quality of the second target image is higher than that of the first target image.

[0008] As can be seen, the electronic device can quickly capture the first target image. Subsequently, the electronic device can generate a second target image with better image quality based on the pre-stored image frame information, further improving the image quality.

[0009] In one possible implementation of the first aspect, the second target image is generated based on stored image frame information when the electronic device meets preset conditions, including the camera application exiting and the electronic device being in an idle state.

[0010] This can be understood as follows: when the camera application is closed and the electronic device is in an idle state, the electronic device can generate a second target image with higher image quality based on the pre-stored image frame information. This achieves the generation of higher quality images without affecting normal shooting and the operation of the electronic device, thus improving the image quality.

[0011] In another possible implementation of the first aspect, the method includes the following steps before storing the image frame information:

[0012] The image frame information is classified to obtain a first type of data and a second type of data. The first type of data includes data that is continuous in memory, and the second type of data includes multidimensional data and / or data that is scattered in memory. The second type of data is flattened to obtain flattened data.

[0013] The flattening process is used to transfer the second type of data to a contiguous block of memory.

[0014] This can be understood as follows: when image frame information includes multi-dimensional data, scattered data in memory, and continuous data, electronic devices can flatten the data into a continuous memory block, which not only avoids the need to request memory multiple times to store discrete data, but also allows for more efficient data processing of the flattened data.

[0015] In another possible implementation of the first aspect, the aforementioned storage of image frame information may include:

[0016] The write request is submitted to the circular queue. The write request is used to request that the first type of data and the flattened data be stored in the preset storage location. The circular queue is a queue for shared memory built between the kernel running space of the electronic device and the user program running space. After the kernel thread of the electronic device obtains the write request from the circular queue, it stores the first type of data and the flattened data in the preset storage location according to the write request.

[0017] Therefore, by storing data in shared memory and transferring it directly between user space and kernel space, there is no need to copy data between user space and kernel space, which reduces the use of the central processing unit (CPU) and the consumption of memory bandwidth.

[0018] In another possible implementation of the first aspect, the circular queue includes a commit queue and a complete queue; submitting a write request to the circular queue includes:

[0019] When processing queue items of a circular queue using the asynchronous read / write io_ur ing method, write requests are submitted to the submission queue.

[0020] After storing the first type of data and the flattened data to the preset storage location according to the write request, the method also includes:

[0021] The write completion status information is stored in the completion queue. The write completion status information is used to indicate that the first type of data and the flattened data have been stored in the preset storage location.

[0022] Therefore, by storing data through shared memory using io_ur ing, there is no need to copy data between user space and kernel space. Furthermore, io_ur ing can submit multiple write requests at once, which not only reduces CPU usage and memory bandwidth consumption, but also reduces the number of context switches between applications and the kernel, thus helping to reduce system power consumption.

[0023] In another possible implementation of the first aspect, the method also includes:

[0024] The performance parameters of io_ur ing are determined based on the status information of the electronic device and the size of the data to be stored.

[0025] The status information of the electronic device includes at least one of the following: system temperature, number of CPU cores, load level, remaining memory or size of data to be stored, where the size of the data to be stored is the size of the first type of data and the size of the flattened data.

[0026] It can be understood that the performance parameters of io_uring affect the efficiency of electronic devices in storing data based on io_uring. Therefore, electronic devices dynamically set the performance parameters of io_uring according to the status information of the electronic device and the size of the data to be stored, which helps to improve the efficiency of data storage or retrieval.

[0027] In another possible implementation of the first aspect, the performance parameters of io_ur ing are determined based on the state information of the electronic device and the size of the data to be stored, including:

[0028] Input at least one of the following into the trained parameter estimation model: system temperature, number of CPU cores, load level, remaining memory, or size of data to be stored; determine the performance parameters of io_ur ing based on the output of the parameter estimation model.

[0029] This can be understood as the parameter estimation model being pre-trained, allowing electronic devices to determine the corresponding io_ur ing performance parameters based on the model's output.

[0030] In another possible implementation of the first aspect, the second type of data includes multidimensional data and / or memory-dispersed data. The second type of data is flattened to obtain flattened data, including:

[0031] Determine the total size of the data in all structures in the multidimensional data and / or memory-dispersed data; based on the total size of the data in all structures, obtain the first memory, the size of which is greater than the total size of the data in all structures; for each structure in the multidimensional data and / or memory-dispersed data, store the structure's data size, storage address, and data in the structure into the first memory in sequence.

[0032] This can be understood as follows: based on the total size of the multidimensional data and / or the data scattered in memory, after allocating a contiguous first memory block, the multidimensional data and / or the data scattered in memory are stored in the contiguous first memory block so that the data can be processed efficiently in the future.

[0033] In another possible implementation of the first aspect, the second type of data also includes contiguous data in memory. Flattening the second type of data to obtain flattened data further includes:

[0034] Add identification information to the contiguous data in memory. The identification information indicates the original start and end positions of the contiguous data in memory. Then store the contiguous data with the added identification information in a second memory. The size of the second memory is larger than the size of the contiguous data in memory.

[0035] Here, when flattening contiguous data in memory, identification information is added so that the data can be restored later based on the original start and end positions indicated in the identification information.

[0036] In another possible implementation of the first aspect, before displaying the gallery interface in response to the second operation, the method further includes:

[0037] When the electronic device meets the preset conditions, it reads the first type of data and the flattened data from the preset storage location; it performs data recovery on the flattened data to obtain the second type of data; and it generates the second target image based on the first type of data and the second type of data.

[0038] This can be understood as follows: without affecting the normal shooting and operation of the electronic device, by reading the pre-stored data and restoring the flattened data, the electronic device can generate a higher quality image based on the restored image frame information.

[0039] In another possible implementation of the first aspect, data recovery is performed on the flattened data to obtain a second type of data, including:

[0040] Based on the data size and storage address of each structure in the flattened data, the structures are restored to their storage addresses, resulting in multi-dimensional data or memory-dispersed data in the second type of data. This allows for the subsequent generation of higher-quality images from the restored data.

[0041] In another possible implementation of the first aspect, data recovery is performed on the flattened data to obtain a second type of data, including:

[0042] Based on the original start and end positions indicated by the identifiers of each structure in the flattened data, the structures are stored at the corresponding memory addresses, resulting in contiguous memory data in the second type of data. This allows for the subsequent generation of higher-quality images from the recovered data.

[0043] In another possible implementation of the first aspect, the second type of data includes a first subtype of data and a second subtype of data. The first subtype of data includes hardware parameters for image processing, and the second subtype of data includes software parameters for image processing. The second type of data is flattened to obtain flattened data, including:

[0044] The hardware and software parameters were flattened separately to obtain the flattened data.

[0045] Data recovery is performed on the flattened data to obtain the second type of data, including:

[0046] Data recovery was performed on the hardware and software parameters of the flattened data to obtain the second type of data.

[0047] This can be understood as follows: when the second type of data includes hardware parameters and software parameters, the electronic device can sequentially flatten the hardware parameters and software parameters. Similarly, when the electronic device restores the flattened data, it sequentially restores the hardware parameters and software parameters. Of course, the order in which the electronic device flattens and restores the hardware parameters and software parameters is only an example, and this embodiment is not limited thereto.

[0048] In another possible implementation of the first aspect, the first type of data includes image data, and the second type of data includes an image processing algorithm, hardware parameters and software parameters for image processing. Generating a second target image based on the first type of data and the second type of data includes:

[0049] The electronic device is configured according to the hardware parameters in the second type of data; the image data in the first type of data is preprocessed to obtain a preprocessed image, the preprocessing operation including at least one of demosaic, white balance or noise reduction; according to the software parameters in the second type of data, the image processing algorithm in the second type of data is used to process the preprocessed image to obtain a second target image.

[0050] This can be understood as the second target image generated by the electronic device based on the stored image frame information, which is a higher quality image obtained through image processing algorithms.

[0051] In another possible implementation of the first aspect, before displaying the second gallery interface in response to the second operation, the method further includes:

[0052] Obtain at least one of the following: the remaining space of the current read-only memory (ROM), the input / output (I / O) throughput, or the battery level; if at least one of the following—the remaining space of the current ROM, the I / O throughput, or the battery level—does not reach the corresponding threshold, then determine that the electronic device is in an idle state.

[0053] This can be understood as acquiring pre-stored image frame information when the electronic device is determined to be in an idle state, in order to generate a higher quality image, so as to ensure that the normal operation of the electronic device is not affected.

[0054] In a second aspect, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the photographing method described in any one of the first aspects above.

[0055] Thirdly, this application provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the photographing method described in any one of the first aspects.

[0056] Fourthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the photographing method described in any one of the first aspects.

[0057] It is understood that the electronic device described in the second aspect, the computer-readable storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to perform the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0058] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0059] Figure 2 A software structure diagram of an electronic device provided in an embodiment of this application;

[0060] Figure 3 This application provides a schematic diagram of a data processing flow as an embodiment of the present application.

[0061] Figure 4 An interface diagram of a photographing method provided in an embodiment of this application;

[0062] Figure 5 A flowchart illustrating a photographing method provided in an embodiment of this application;

[0063] Figure 6 Example diagram of data classification provided for embodiments of this application;

[0064] Figure 7 This is an example diagram of the storage structure after data flattening is provided in an embodiment of this application;

[0065] Figure 8 Structural example diagrams of the outer layer structure data and inner layer structure data provided in the embodiments of this application;

[0066] Figure 9 A schematic diagram illustrating the principle of the io_ur ing mechanism provided for embodiments of this application;

[0067] Figure 10 A schematic diagram of data storage based on io_ur ing provided for embodiments of this application;

[0068] Figure 11 Example diagrams for determining the input parameters of io_ur ing provided in embodiments of this application;

[0069] Figure 12 Example diagrams for restoring flattened data provided in embodiments of this application;

[0070] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0072] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0073] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0074] To improve the shooting speed and quality of electronic devices, the shooting process can be divided into two stages. The first stage focuses on improving the shooting speed (e.g., fast shooting or burst shooting), and the second stage focuses on improving the image quality. To achieve this, the electronic device can store the shooting-related data in the first stage and then read the pre-stored data in the second stage (e.g., when the camera application is closed and the electronic device is idle) to generate a higher-quality image based on the read data.

[0075] Therefore, this application provides a method for taking pictures, which is applied to an electronic device. After starting the camera application, a shooting preview interface is displayed; in response to the shooting operation, image frame information corresponding to the image captured by the camera is obtained and stored, and a first target image is captured; in response to the first operation, a first image library interface is displayed, wherein the first image library interface includes the first target image. Then, in response to the second operation, a second image library interface is displayed, wherein the second image library interface includes a second target image, which is obtained based on the stored image frame information, and the image quality of the second target image is higher than that of the first target image. Thus, it can be seen that in the first stage (i.e., the shooting stage), the electronic device can quickly capture the first target image, and in the second stage (i.e., when the electronic device meets the preset conditions), the electronic device generates a second target image with better image quality based on the pre-stored image frame information, further improving the image quality.

[0076] For example, the photo-taking method provided in this application embodiment can be applied to electronic devices with photo-taking functions such as mobile phones, tablets, personal computers (PCs), personal digital assistants (PDAs), smartwatches, netbooks, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, in-vehicle devices, and smart cars. This application embodiment does not impose any limitations on this.

[0077] like Figure 1 As shown, Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0078] Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a user identification module (SIM) card interface 195, etc.

[0079] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0080] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0081] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0082] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0083] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0084] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0085] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0086] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance).

[0087] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0088] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0089] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0090] In some embodiments, in response to an operation to launch a camera application, the electronic device displays a shooting preview interface on the display screen 194. In response to a shooting operation, after capturing a first target image, the electronic device displays a first gallery interface including the first target image on the display screen 194 in response to a first operation.

[0091] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0092] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0093] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. A digital signal processor is used to process digital signals, including digital image signals and other digital signals. For example, when electronic device 100 selects a frequency, the digital signal processor performs a Fourier transform on the frequency energy.

[0094] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0095] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0096] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0097] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0098] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0099] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0100] Figure 2 This is a software structure diagram of an electronic device provided in an embodiment of this application.

[0101] Understandably, a layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer (referred to as the application layer), an application framework layer (referred to as the framework layer), system libraries, and a kernel layer.

[0102] The application layer described above may include a series of application packages.

[0103] like Figure 2 As shown, the application package may include system applications. System applications refer to applications installed on the electronic device before it leaves the factory. For example, system applications may include programs such as camera, gallery, calendar, music, SMS, memo, and weather.

[0104] Application packages can also include third-party applications, which are applications that users download and install from app stores (or app markets). Examples include map applications, food delivery applications, reading applications (such as e-books), social networking applications, and travel applications.

[0105] The application framework layer described above provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0106] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0107] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0108] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0109] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0110] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0111] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0112] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the phone, and flashing indicator lights.

[0113] The Android Runtime includes the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0114] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0115] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0116] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0117] The kernel layer is the layer between hardware and software. The kernel layer includes at least the display driver, camera driver, audio driver, and sensor driver.

[0118] All the technical solutions involved in the following embodiments can be implemented in the electronic device 100 having the above-described hardware structure and software architecture. The following description uses a mobile phone as an example to illustrate this solution.

[0119] In this embodiment, after the mobile phone receives the user's operation to launch the camera application, it launches the camera application and displays a shooting preview interface in response to the user's operation. After the camera application is launched, upon receiving the user's operation to take a picture, the mobile phone, in response to the operation, controls the camera application to capture an image while acquiring image frame information corresponding to the image captured by the camera. The shooting operation can be a quick capture or a burst capture operation.

[0120] like Figure 3 As shown, after acquiring image frame information, the mobile phone performs data classification processing on the image frame information, dividing it into different types of data. Then, the phone performs data flattening processing on the classified data to integrate multi-dimensional or scattered data, resulting in several integrated storage files for convenient data storage and maintenance. Subsequently, the phone stores these integrated storage files in a preset storage location (e.g., on a hard drive). While the phone is storing the image frame information in the preset storage location, it can capture a first target image. This first target image is the image captured by the camera application in response to the shooting operation, after undergoing lightweight image processing. For example, the phone may only perform brightness enhancement and simple image algorithm optimization on the original image to obtain the first target image. In response to the user's launch of the gallery application (i.e., the first operation), the phone displays the first gallery interface. This first gallery interface includes the first target image.

[0121] When the phone determines that the camera application has exited and the electronic device is idle, it can perform a data reading process. This involves reading several stored files from a preset storage location and restoring the data, which had undergone image flattening during storage, to its original data structure, thus obtaining image frame information. Further, the phone generates a second target image based on the image frame information. This second target image is a depth-processed image with higher quality than the first target image. Subsequently, in response to the user's action of relaunching the gallery application (i.e., the second operation), the phone displays a second gallery interface. This second gallery interface includes the second target image.

[0122] In this embodiment, the mobile phone receives the second operation after receiving the first operation. That is, the mobile phone receives the first operation at a first moment and the second operation at a second moment, and the first moment is before the second moment.

[0123] For example, such as Figure 4 As shown in (a), when the mobile phone receives a trigger operation from the user on the camera application icon 420 on the desktop 410, it responds to the user's trigger operation by launching the camera application and displaying the following: Figure 4 The shooting interface 430 is shown in (b). In shooting mode, when the mobile phone receives a trigger operation from the user on the shooting control 440, the mobile phone executes the shooting process in response to the user's trigger operation.

[0124] During the process of taking a photo on a mobile phone, after the phone obtains the image frame information corresponding to the image captured by the camera, it processes the image frame information and stores it in the disk storage space. At this time, the camera application captures the first target image, such as... Figure 4 Image 460 is displayed in the gallery application interface 450 shown in (c). The mobile phone receives the user's input... Figure 4 The trigger operation of the gallery application 470 on the desktop 410 shown in (d) responds to the user's trigger operation, launches the gallery application, and displays as shown in (d). Figure 4 Image 480 is shown in (e). Image 490 is displayed in the gallery interface. Image 490 is a deep image processed image generated by the phone after retrieving image frame information from the disk storage space when system resources are idle. For example, the phone may use a deep learning model to process the original image.

[0125] The following is combined with Figure 5 The photographing method provided in the embodiments of this application will be described in detail.

[0126] Figure 5 This is a flowchart illustrating a photographing method provided in an embodiment of this application, as shown below. Figure 5 As shown, the method includes the following steps:

[0127] S501, camera application launched.

[0128] In one implementation, after the phone receives a user's command to launch the camera app, the camera app launches in response to that command. For example, the phone launches the camera app in response to a user tapping the app icon. Another example is when the phone receives a user's voice input saying "Open camera app," the camera app launches in response to that voice input.

[0129] S502, in response to the photo-taking operation, executes the photo-taking process.

[0130] In one implementation, after the phone receives a user's command to initiate the photo-taking process, the camera application controls the camera to capture an image in response to this command, thus executing the photo-taking process. For example, after the phone receives a user's tap on the camera control in the camera application's photo-taking interface, the camera application starts taking a picture in response to the user's tap. Another example is when the phone receives a user's voice command to "start taking a picture," the phone controls the camera application to execute the photo-taking process.

[0131] S503 acquires the image frame information corresponding to the image captured by the camera.

[0132] Image frame information refers to all the data corresponding to the image captured by the camera during this photo-taking process. Image frame information may include image data, image processing algorithms, hardware parameters during image processing, and software parameters, etc.

[0133] The hardware parameters for image processing mentioned above may include camera hardware parameters and device hardware parameters. Camera hardware parameters include, but are not limited to, one or more of the following: sensor type, sensor size, number of pixels, aperture size, lens type (e.g., prime lens, wide-angle lens, telephoto lens, etc.), shutter speed, and ISO range. Device hardware parameters include, but are not limited to, one or more of the following: number of processor cores and / or number of threads, number of graphics processor cores and / or clock speed, or remaining memory.

[0134] Image data includes, but is not limited to, one or more of the following: image information (pixel information of the image, for example, each pixel in an RGB image has three channels, representing red, green, and blue respectively), image format (e.g., grayscale, RGB, RGBA, etc.), and image stride (the distance between two consecutive rows of pixels in memory). Software parameters include, but are not limited to, one or more of the following: parameters used in white balance adjustment, parameters used in gamma correction, parameters used in color correction, parameters used in sharpening, etc. Photography parameters may include exposure mode (e.g., automatic or manual exposure), focus mode (e.g., autofocus, manual focus, continuous autofocus, etc.), metering mode, resolution (width and height of the image, such as 3840*2160 pixels), compression ratio, and whether a flash is used, etc. Environmental parameters may include lighting conditions (e.g., natural light, artificial light, front lighting, backlighting, etc.), shooting time, weather information, and geographical location information (e.g., latitude, longitude, altitude, etc. of the shooting location).

[0135] Image processing algorithms include, but are not limited to, one or more of the following: image enhancement algorithms (e.g., histogram equalization, Laplacian sharpening, median filtering, etc.), image segmentation algorithms (e.g., thresholding, region growing, etc.), image white balance processing algorithms, feature extraction algorithms (e.g., scale-invariant feature transform, histogram of oriented gradients, etc.), image recognition and classification algorithms (e.g., convolutional neural networks, support vector machines, etc.), image reconstruction algorithms (e.g., Fourier transform, wavelet transform, etc.), image compression processing algorithms, computer vision algorithms (e.g., target tracking, 3D reconstruction, etc.), and so on. The algorithm parameters used in image processing refer to the parameters corresponding to the algorithm used by the mobile phone to process the image; these will not be described in detail here.

[0136] In some embodiments, image frame information includes all the data that a camera application can use to capture a frame of image. The data listed above is only an example, and all the data will not be listed one by one in this application embodiment.

[0137] S504 classifies the image frame information to obtain different types of data.

[0138] Since the process of taking pictures with a mobile phone is very complex, involving a large amount of complex data, in order to facilitate data maintenance and storage, the acquired image frame information can be classified and processed to obtain different types of data, and these different types of data can be stored in different files so that subsequent operations such as adding, deleting or modifying different types of data can be performed.

[0139] In some embodiments, the mobile phone can classify image frame information into three types of data based on the data type: Type 1 data, Type 2 data, and Type 3 data. For example, Type 1 data can be stored in BUFFER_FILE, Type 2 data can be stored in META_FILE, and Type 3 data can be stored in DB_FILE. This classification of image frame information into different types facilitates subsequent processing of these different data types.

[0140] Type 1 data may include data related to images captured by the camera. For example, Type 1 data includes, but is not limited to, one or more of the following: image data, image width and height, image format, or image stride.

[0141] Type 2 data can include hardware parameters used in image processing. For example, Type 2 data can be frame metadata (also known as in-frame metadata) and frame parameters (frame structure data). Frame metadata includes additional information associated with the image frame, such as timestamps, frame type, image sequence number, camera location, geographic location, etc. Frame parameters include specific parameters associated with the image frame that define its attributes and characteristics. For example, frame parameters can include resolution, pixel format, frame rate, codec, etc.

[0142] Type 3 data can include image processing algorithms and software parameters used during image processing. For example, Type 3 data can include ppinfo, db, and runtimeinfo. ppinfo includes information related to image processing and page layout. db includes a database storing configuration information or other data. Runtimeinfo includes information collected during camera application runtime, which may include performance metrics, logs, and other information.

[0143] For example, such as Figure 6 As shown, after acquiring image frame information, the mobile phone can classify and process the image frame information to obtain different types of data. For example, the image frame information can be divided into data related to the image captured by the camera, data corresponding to the image processing algorithm, hardware parameters during image processing, and software parameters. Figure 6 Files 1 through 6 can all be lossy compressed (joint photographic experts group, JPEG) format files, thus reducing the file size.

[0144] In some embodiments, the mobile phone can further divide the image frame information into a first type of data and a second type of data based on whether the data in the image frame information is continuous in memory. The first type of data includes data that is continuous in memory, while the second type of data includes multidimensional data and / or data that is scattered in memory. For example, the aforementioned type 1 data is the first type of data, and the aforementioned type 2 and type 3 data are the second type of data. This classification of data facilitates subsequent processing of different types of data.

[0145] S505, flatten the second type of data to obtain the flattened data.

[0146] Flattening refers to converting discontinuous, scattered data in memory into a contiguous block of memory, or converting multidimensional data (such as arrays of structures or nested structures) into a single-level structure and storing it in a contiguous block of memory for more efficient data processing. For example, if the second type of data includes data scattered in memory, the mobile phone can flatten this data to obtain the flattened data.

[0147] Since the data stored in META_FILE and DB_FILE includes multidimensional data, and these data are individual and scattered in memory, to avoid repeated memory allocation when storing the scattered data to disk and to facilitate data file maintenance, the multidimensional data and / or scattered data in the META_FILE and DB_FILE files can be flattened to obtain flattened data. For example, if META_FILE contains multidimensional data, the mobile phone can convert the multidimensional data into one-dimensional data through data flattening.

[0148] It should be explained that since the data stored in BUFFER_FILE is continuous data related to the image and is stored in a contiguous block of memory, there is no need to flatten the data stored in BUFFER_FILE.

[0149] In the process of data flattening on multidimensional or memory-dispersed data, the size of the data in each structure within the multidimensional or dispersed data is first determined. For example, for basic data types (such as int and float), their size is usually determined by the compiler; for instance, int data is typically 4 bytes, and float data is typically 4 bytes. Then, the size of the data in a single structure is multiplied by the number of structures to obtain the total size of the data in all structures. Next, based on the total size of the data in all structures, a sufficiently large block of memory is allocated to store the data of all structures. For example, the phone obtains a first memory block with a storage space larger than the total size of the data in all structures. For each structure in the multidimensional or dispersed data, the phone sequentially stores the structure's data size, storage address, and the data within the structure into the first memory block, thus achieving the data flattening process.

[0150] In some embodiments, the second type of data also includes contiguous data in memory. During the flattening process of the second type of data, the contiguous data in memory is stored in a contiguous memory block along with multidimensional data and / or discrete data in memory. During the flattening process of contiguous data in memory on the mobile phone, for example, for variables of directly copyable data types (e.g., int, float, and structures composed of basic data types), the original start and end positions of the contiguous data in memory are marked by adding identification information (e.g., begin and end) before being packaged and stored in newly allocated memory. Then, the contiguous data in memory with the added identification information is copied to the newly allocated memory (e.g., the second memory) until the data copy is complete, thereby realizing the flattening process of the second type of data. Here, during the flattening process of contiguous data in memory, identification information is added so that the data can be subsequently recovered based on the original start and end positions indicated in the identification information.

[0151] In some embodiments, the mobile phone flattens the second type of data, obtains the flattened data, and then stores the flattened data in newly allocated memory, which can release the original memory storing the second type of data, thereby saving the device's storage space.

[0152] For example, Figure 7 This is an example diagram of the storage structure after data flattening provided in an embodiment of this application. For non-contiguous data in the frame metadata stored in META_FILE, the frame metadata needs to be flattened and integrated into a large storage area, and then the flattened data is stored in the corresponding file. For example... Figure 7As shown, the header in the file can include information such as file type and size, while meta refers to frame metadata. The file also includes frame structure data. For contiguous memory data that can be directly copied, identification information (such as begin and end) is added to the contiguous memory data to mark the original start and end positions of the contiguous memory data, so that the data can be recovered later based on the original start and end positions indicated in the identification information.

[0153] In some embodiments, a database comprises an outer structure (OuterDb) and an inner structure (InnerDb), and the data stored in the outer and inner structures has different structures. Therefore, when storing the database, it is necessary to calculate the memory usage of both data structures.

[0154] For example, such as Figure 8 (a) in the diagram is an example of the structure of the outer structure data. The outer structure data stores the data name, the data type determination (i.e., sizeofTb1()), and the array (i.e., Tb l()).

[0155] Figure 8 (b) in the diagram is an example of the structure of the inner layer data. Figure 8 As shown in (b), the inner structure data is a Tab leMgr, which contains multiple Range structures and Tab le structures. It contains multiple m_tab les(std::map) <Range,std::shared_ptr<Tab le> >m_tabs;). Each Range structure stores data identification information. When performing a tab flattening operation, the entire Tab size can be stored first, followed by the index information (i.e., key information) and the corresponding stored data (i.e., item). The stored data contains different data types and their corresponding data types. There is a one-to-one correspondence between the index information and data types stored in the Tab. Based on the index information, the data type and specific data stored in the corresponding stored data can be determined. An example diagram of the inner structure data storage after the flattening operation is still shown below. Figure 7 As shown.

[0156] S506, store the first type of data and the flattened data to the preset storage location.

[0157] After flattening the second type of data, the phone obtains a contiguous block of memory. The phone then stores the flattened data from this contiguous memory block in a preset storage location. For example, the preset storage location could be the corresponding storage space on a disk. In addition, the phone also stores the data from various data types that do not require flattening (i.e., the first type of data) in the preset storage location, using the same method as storing the flattened data.

[0158] As an example, suppose the mobile phone divides image frame information into three different types of data. After the mobile phone flattens the different types of data, it can store the flattened data in three storage files, or store the flattened data in a storage file at a preset storage location.

[0159] As one possible implementation, the phone can submit write requests to a circular queue. The kernel thread retrieves the write request from the circular queue and then stores the first type of data and the flattened data in a preset storage location according to the request. Thus, by storing data through shared memory, and directly transferring data between user space and kernel space, there is no need to copy data between the two spaces, thereby reducing CPU usage and memory bandwidth consumption.

[0160] Among them, the write request is used to request the storage of the first type of data and the flattened data to a preset storage location, and the circular queue is a queue for sharing memory built between the kernel running space (i.e., kernel space) and the user program running space (i.e., user space) of the mobile phone.

[0161] In some embodiments, the circular queue includes a submission queue (SQ) and a completion queue (CQ). The submission queue is a buffer for transferring input / output (I / O) requests between user space and kernel space. I / O requests are submitted by filling the entries in the submission queue. The completion queue is a buffer for the kernel space to report the completion status of I / O requests to user space.

[0162] When the mobile phone processes queued items in a circular queue using the asynchronous read / write method `io_uring`, it submits write requests to the submission queue. After the kernel thread retrieves the write request from the circular queue, it stores the first type of data and the flattened data to a preset storage location according to the write request, and then stores the write completion status information to the completion queue. The write completion status information indicates that the first type of data and the flattened data have been stored in the preset storage location. Therefore, by storing data through shared memory using `io_uring`, there is no need to copy data between user space and kernel space. Furthermore, `io_uring` can submit multiple write requests at once, which not only reduces CPU usage and memory bandwidth consumption but also reduces the number of context switches between the application and the kernel, thus helping to reduce system power consumption.

[0163] In some embodiments, the mobile phone can communicate with the kernel through "shared memory" between user space and kernel space based on the io_ur ing method to store and retrieve data.

[0164] For example, Figure 9 This is a schematic diagram illustrating the principle of the io_ur i ng mechanism provided for embodiments of this application. For example... Figure 9 As shown, when a mobile phone stores and retrieves data using the `io_urling` method, the user process can submit input / output (I / O) operations to be initiated to shared memory. I / O operations involve writing or retrieving data from a preset storage location. Kernel threads can read I / O operations from shared memory and perform the relevant I / O operations. User processes do not need to use system calls to read and write to shared memory, so no context switching occurs, saving data storage or retrieval time. Therefore, the `io_urling` method stores data using shared memory, reducing data copying. Furthermore, the batch I / O requests within the `io_urling` method reduce the number of system calls.

[0165] Figure 10 This is an example diagram of data storage based on io_ur ing provided for embodiments of this application. For example... Figure 10As shown, `io_ur ing` uses two main circular queues: the commit queue and the completion queue. When an I / O operation is complete, the kernel process places the completion status information into the completion queue. Users can poll the completion queue to obtain the status of completed I / O requests. In other words, the completion queue is used to receive I / O completion notifications. A commit queue entry (SQE) is an entry in the commit queue. The SQE stores the task queues, and there is a one-to-one correspondence between the task queues stored in the SQE and the I / O requests committed by the SQE. The SQE contains all the information needed to initiate an I / O operation. For example, the SQE includes the operation type (e.g., read, write, send, or receive), file descriptor, buffer address, buffer size, and other relevant operation parameters.

[0166] When the phone uses the `io_uring` method to store the flattened data to a preset storage location, the phone first creates an `io_uring` and initializes it. Optionally, the phone can create the `io_uring` using polling or time-based modes, and the kernel will create a kernel thread named `io_uring-sq`. For example, the `io_uring_setup` function creates an `io_uring` object, and then the `io_uring_queue_init` function initializes an `io_uring` instance. This initializes the size of the commit queue and the completion queue to manage I / O requests and determine whether to initiate batch processing of commit and completion events. Then, to facilitate the submission of non-contiguous requests in memory via a circular buffer, an array of `io_uring_sqe` structures can be created to store the `SQE`. For example, the `io_uring_prep_write` function can be called to set the `SQE`, specifying the file descriptor, buffer address, buffer size, and the length of data to be written.

[0167] After the phone submits an I / O request (SQE) to the SQ queue using the `io_uring` method, a completion queue entry (CQE) is added to the completion queue to notify the phone that the operation is complete. For example, the `io_uring_submit` function can be used to submit an SQE to the SQ queue, and multiple requests can be submitted at once, thus improving data storage efficiency. The kernel thread continuously reads I / O operations from the SQ and initiates I / O requests. When an I / O request is completed, the kernel thread checks the I / O request in the SQ and begins executing the corresponding I / O operation. When the kernel thread completes an I / O operation, it fills the relevant completion event descriptor into the CQ. Subsequently, the camera application can read the completion time descriptor from the CQ through polling or asynchronous notifications to retrieve the result of the I / O operation. When there are many I / O requests, the phone can repeat the above process until all necessary I / O operations are completed.

[0168] When the mobile phone uses the io_ur ing method to store the flattened data to the preset storage location, it can submit multiple write requests and process multiple completion events at once, thereby reducing the overhead of system calls and context switching and improving system efficiency.

[0169] For example, such as Figure 10 As shown, assuming both the submission queue and the completion queue are divided into 64 parts, and the files to be stored are also divided into 64 blocks, the phone can submit all 64 blocks to the kernel simultaneously. Assuming the kernel has 30 threads, these 30 threads can simultaneously store the 30 blocks to the disk. Therefore, after three rounds of multi-threaded scheduling, the phone can store all 64 blocks of files to the disk.

[0170] The performance parameters of `io_uring` define the behavior of the `io_uring` method within the kernel thread, directly impacting file storage and retrieval performance. Therefore, the phone can dynamically set the performance parameters of `io_uring` based on the phone's status information and the size of the data to be stored, thereby ensuring efficient data storage and retrieval. For example, the main performance parameters of `io_uring` are shown in Table 1 below.

[0171] Table 1

[0172]

[0173] It should be noted that the polling mode in Table 1 above means the program continuously queries whether the operation is complete, while the event mode means receiving a notification upon completion of the operation. For scenarios with high concurrency, the polling mode should be used; otherwise, the event mode should be used.

[0174] In some embodiments, the mobile phone can determine the performance parameters of io_ur ing based on the phone's status information and the size of the data to be stored. The phone's status information includes at least one of system temperature, number of CPU cores, load level, or remaining memory. The size of the data to be stored is the size of the first type of data and the flattened data.

[0175] The system temperature mentioned above refers to the heat generated by the various components inside the phone during operation. CPU core count refers to the number of CPU cores. For example, a CPU can have dual-core, quad-core, hexa-core, octa-core, or twelve cores. Under the same conditions of core frequency and cache size, the more CPU cores, the stronger the overall CPU performance. For example, a 3.8GHz 6-core CPU is more powerful than a 3.8GHz dual-core CPU. Load level refers to the workload and intensity that the phone's hardware withstands when running various applications and functions. Remaining memory refers to the amount of RAM currently available in the phone's storage space.

[0176] One possible implementation is that the mobile phone can acquire different state information and different sizes of data to be stored. Based on these different state information and data sizes, the storage time using the io_uring method under different parameter combinations can be determined. Then, machine learning or statistical analysis methods are used to fit this data, and by adjusting the parameters, the storage time of the io_uring method can be shortened, thus revealing the performance parameters corresponding to the shortest storage time. Afterward, the mobile phone can determine the corresponding performance parameters based on its state information and the size of the data to be stored.

[0177] As another possible implementation, the mobile phone can input its state information and the size of the data to be stored into a trained parameter estimation model. Based on the model's output, the performance parameters of `io_ur ing` can be determined. The parameter estimation model is pre-trained and can determine the `io_ur ing` performance parameters corresponding to the shortest data storage time based on its output. Therefore, the mobile phone can determine the performance parameters corresponding to its state information and the size of the data to be stored based on the output of the parameter estimation model, completing the data storage process in the shortest possible time.

[0178] The following section uses queue depth and basic size, two performance parameters of mobile phone dynamic decision-making, as an example to introduce the process of determining the performance parameters of io_uring based on the mobile phone's state information and the size of the data to be stored.

[0179] For example, firstly, the phone employs a factorial experimental design, systematically varying system temperature, number of CPU cores, load level, remaining memory, size of data to be stored, queue depth, and base size, recording the processing time for each parameter combination. Factorial experimental design refers to combining all levels of all experimental factors to form different experimental conditions, with each condition being independently repeated two or more times. Then, the phone preprocesses all parameter combinations. For instance, the StandardScaler function from Scikit-learn can be used for data standardization to ensure all parameters are of the same magnitude. Next, the phone uses the RandomForestRegressor function from the Scikit-learn library to construct a multi-output random forest regression model. Queue depth and base size are used as target variables, while system temperature, number of CPU cores, load level, remaining memory, and size of data to be stored are used as feature matrices for model training. During model training, the phone evaluates model performance using mean squared error to obtain the parameter estimation model with the minimum mean squared error.

[0180] like Figure 11 As shown, the mobile phone can input system temperature, number of CPU cores, load level, remaining memory, and the size of the data to be stored into a trained parameter estimation model. Based on the model's output, the queue depth and basic size in the ioration performance parameters are determined. At this point, the parameters output by the model are those with the shortest processing time for ioration.

[0181] S507: When the phone meets the preset conditions, it reads the first type of data and the flattened data from the preset storage location.

[0182] In this embodiment, when the mobile phone determines that the camera application has exited and the phone is in an idle state, the phone can read the pre-stored first type of data and the flattened data from a preset storage location to obtain the image frame information corresponding to the image captured by the camera. Thus, without affecting normal shooting or phone operation, a higher quality image can be generated based on the image frame information.

[0183] In some embodiments, after the phone determines that the camera application has exited, it determines whether the phone has the capability to perform image transfer. This transfer capability refers to the phone's ability to generate high-quality images based on pre-stored image frame information. For example, the phone can obtain identifier information 2 and determine whether it has the transfer capability by judging whether identifier information 2 is a preset identifier. Assuming the preset identifier is 1, if the phone obtains identifier information 2 as 1, then the phone has the transfer capability; otherwise, the phone does not have the transfer capability.

[0184] After determining that the phone has the capability to perform data transfer, the phone determines whether it is in an idle state. Optionally, the phone can obtain at least one of the following: the remaining space of the read-only memory (ROM), the I / O throughput, or the battery level, to determine whether the phone is in an idle state. If the phone determines that at least one of the remaining space of the ROM, the I / O throughput, or the system battery level has not reached the corresponding threshold, then the phone is determined to be in an idle state. If the phone determines that the remaining space of the ROM, the I / O throughput, and the system battery level have all reached the corresponding thresholds, then the phone is determined not to be in an idle state. The aforementioned idle state of the phone can refer to the idle state of the phone's system resources.

[0185] Optionally, the process by which the phone reads the pre-stored first type of data and the flattened data from the preset storage location is as follows: The phone first initializes `io_uring`, for example, by calling `io_uring_queue_init` to initialize an `io_uring` instance to create a `SQ` and a `CQ`. Then, the phone obtains an `SQE` and populates it using functions such as `io_uring_prep_read` or `io_uring_prep_readv`. These functions set the structure of the `SQE`, specifying the file descriptor to be read, the buffer address, the buffer size, and the read offset. After the `SQE` is populated, it is submitted to the `SQ` using the `io_uring_submit` function. The `io_uring_submit` function can submit multiple read requests at once, thereby improving efficiency.

[0186] The aforementioned read operation is executed asynchronously by a kernel thread. When the kernel thread completes the read operation, it places a CQE (Confirmation Queue) into the completion queue. Functions such as `io_urging_wait_cqe` or `io_urging_peek_cqe` are then used to check the CQE to determine if the read operation is complete. Once the read operation is complete, the phone can retrieve the result from the CQE, including the number of bytes read and error codes. If the read is successful, the flattened data can be retrieved from a preset storage location.

[0187] Since the preset storage location contains a large amount of flattened data that cannot be processed in one go, the above reading steps can be repeated. After all reading operations are completed, the phone cleans up the io_uring instance and releases related resources.

[0188] S508 performs data recovery on the flattened data, obtains the second type of data, and then obtains the image frame information based on the first type of data and the first data.

[0189] After the phone reads the first type of data and the flattened data from the preset storage location, it restores the flattened data to its original data structure for use in generating the target image later.

[0190] In one scenario, when the flattened data is obtained by flattening multidimensional or memory-dispersed data, the phone can restore each structure to its storage address based on the data size and storage address of each structure in the flattened data, thus obtaining multidimensional or memory-dispersed data in the second type of data. This allows for the subsequent generation of higher-quality images based on the restored data.

[0191] In another scenario, when the flattened data is obtained by flattening contiguous data in memory, the phone can store each structure at its original start and end addresses, based on the identifiers of each structure within the flattened data. This results in contiguous data in memory, a second type of data. Consequently, higher-quality images can be generated subsequently from the recovered data.

[0192] In some embodiments, the second type of data includes a first subtype of data and a second subtype of data. The first subtype of data includes hardware parameters during image processing, and the second subtype of data includes software parameters during image processing. When the mobile phone flattens the second type of data to obtain flattened data, the mobile phone can flatten both the hardware parameters and the software parameters separately. When performing data recovery on the flattened data to obtain the second type of data, the mobile phone can recover both the hardware parameters and the software parameters from the flattened data separately to obtain the second type of data.

[0193] It should be explained that the above-described order of data recovery for the first and second subtypes of data is merely an example. When a mobile phone performs data flattening or recovery on different subtypes of data, the order of flattening and recovery is not limited. For example, when flattening data, the phone can flatten software parameters first, then hardware parameters. Similarly, when recovering data, the phone can recover software parameters first, then hardware parameters. Of course, the phone can also flatten and recover both hardware and software parameters simultaneously.

[0194] For example, the phone flattens the data stored in META_FILE and DB_FILE, and then stores the flattened data in a preset storage location. After reading the flattened data from the preset storage location, the phone can restore the data to obtain the original data structure stored in META_FILE and DB_FILE.

[0195] For example, such as Figure 12 As shown, assume the flattened data read by the phone from the preset storage location is obtained by flattening the data stored in META_FILE and DB_FILE. Assume META_FILE stores frame metadata and frame structure data, while DB_FILE stores ppinfo, db, and runtimeinfo. When the phone recovers the flattened data, it first parses the frame structure data, using the identifiers added during data storage (such as begin and end) to restore the packaged data in the order of storage. Next, the phone parses the frame metadata. After obtaining the flattened frame metadata stored in the preset storage location, the phone restores the data to its original frame metadata based on the data size and storage address. Then, the phone sequentially parses ppinfo, the memory allocated in the copy class (malloc), reconstructs std::string and the values ​​of each std container, and then parses dbinfo and runtimeinfo in a similar manner to obtain all the data needed for taking the picture.

[0196] S509, Generate a second target image based on the image frame information.

[0197] After acquiring image frame information, the mobile phone can generate a second target image based on that information. Optionally, the mobile phone can be configured according to the hardware parameters in the second type of data, and then preprocess the image data in the first type of data to obtain a preprocessed image. For example, the ISP performs preprocessing such as de-mosaicing, white balance adjustment, and noise reduction on the original image, and the mobile phone further processes the preprocessed image, such as sharpening, increasing contrast, and color correction. The mobile phone can also use the software parameters in the second type of data and the image processing algorithms in the second type of data to further process the preprocessed image. For example, the NPU performs scene recognition on the image, and the mobile phone can also perform cropping, rotation, and other processing on the image to finally obtain the second target image.

[0198] At this point, the second target image generated by the mobile phone based on the stored image frame information is an image processed by an image processing algorithm. Compared with the first target image obtained when the camera application takes the picture, the image quality of the second target image is better.

[0199] In summary, in this embodiment, after acquiring image frame information from the camera, the mobile phone classifies the complex image frame information to obtain different types of data. Then, the mobile phone flattens the second type of data within the different data types to obtain flattened data, and stores the first type of data and the flattened data in a preset storage location. When the mobile phone determines that the camera application has exited or the phone is idle, it retrieves the pre-stored data, performs data recovery processing on the retrieved data, obtains the image frame information, and generates a higher-quality image based on the image frame information. Therefore, by storing the image frame information, only a lightly processed image is obtained during this capture, enabling the camera application to quickly obtain captured images during continuous shooting or fast shooting, and subsequently generate higher-quality images based on the stored image frame information.

[0200] like Figure 13 As shown in the illustration, this application discloses an electronic device, which can be the aforementioned mobile phone. Specifically, the electronic device may include: a touchscreen 1301, wherein the touchscreen 1301 includes a touch sensor 1306 and a display screen 1307; one or more processors 1302; a memory 1303; one or more application programs (not shown); and one or more computer programs 1304. These devices can be connected via one or more communication buses 1305. The one or more computer programs 1304 are stored in the memory 1303 and configured to be executed by the one or more processors 1302. The one or more computer programs 1304 include instructions that can be used to perform the relevant steps in the above embodiments.

[0201] It is understood that the aforementioned electronic devices, etc., include hardware structures and / or software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this invention.

[0202] This application embodiment can divide the above-mentioned electronic device into functional modules according to the method example described above. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0203] When each functional module is divided according to its corresponding function, the above embodiments illustrate a possible composition of the electronic device, which may include a display unit, a transmission unit, and a processing unit. It should be noted that all relevant content regarding the steps in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0204] This application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the aforementioned method steps to implement the photographing method in the above embodiments.

[0205] Embodiments of this application also provide a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the photographing method in the above embodiments.

[0206] An embodiment of this application also provides a computer program product, which includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned method steps to implement the photographing method in the above embodiments.

[0207] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the apparatus to perform the photographing method performed by the electronic device in the above method embodiments.

[0208] In this embodiment, the electronic device, computer-readable storage medium, computer program product or device are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0209] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0210] In the embodiments of this application, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0211] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0212] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for taking photos, characterized in that, The method includes: Launch the camera app to display the shooting preview interface; In response to the shooting operation, the system acquires and stores the image frame information corresponding to the image captured by the camera, and captures the first target image. In response to the first operation, a first image gallery interface is displayed, the first image gallery interface including the first target image; In response to the second operation, a second gallery interface is displayed, which includes a second target image. The second target image is generated based on the stored image frame information when the electronic device meets preset conditions, and the image quality of the second target image is higher than that of the first target image. The preset conditions include the camera application being closed and the electronic device being in an idle state.

2. The method according to claim 1, characterized in that, Before storing the image frame information, the method includes: The image frame information is classified to obtain a first type of data and a second type of data. The first type of data includes data that is contiguous in memory, and the second type of data includes multidimensional data and / or data that is scattered in memory. The second type of data is flattened to obtain flattened data. The flattening process is used to transfer the second type of data to a contiguous memory block.

3. The method according to claim 2, characterized in that, The storage of the image frame information includes: A write request is submitted to a circular queue. The write request is used to request that the first type of data and the flattened data be stored in a preset storage location. The circular queue is a queue for shared memory built between the kernel running space and the user program running space of the electronic device. After the kernel thread of the electronic device obtains the write request from the circular queue, it stores the first type of data and the flattened data in the preset storage location according to the write request.

4. The method according to claim 3, characterized in that, The circular queue includes a commit queue and a complete queue; submitting a write request to the circular queue includes: When processing queue items of the circular queue based on the asynchronous read / write io_uring method, the write request is submitted to the submission queue; After storing the first type of data and the flattened data to the preset storage location according to the write request, the method further includes: The write completion status information is stored in the completion queue. The write completion status information is used to indicate that the first type of data and the flattened data have been stored in the preset storage location.

5. The method according to claim 4, characterized in that, The method further includes: The performance parameters of io_uring are determined based on the status information of the electronic device and the size of the data to be stored. The status information of the electronic device includes at least one of the following: system temperature, number of CPU cores, load level, remaining memory, or the size of the data to be stored. The size of the data to be stored is the size of the first type of data and the flattened data.

6. The method according to claim 5, characterized in that, The process of determining the performance parameters of io_uring based on the electronic device's status information and the size of the data to be stored includes: Input at least one of the system temperature, the number of CPU cores, the load level, the remaining memory, or the size of the data to be stored into the trained parameter estimation model; The performance parameters of io_uring are determined based on the output of the parameter estimation model.

7. The method according to any one of claims 2-6, characterized in that, The second type of data includes multidimensional data and / or memory-dispersed data. The flattening process of the second type of data to obtain flattened data includes: Determine the total size of the data in all structures within the multidimensional data and / or memory-dispersed data; Based on the total size of the data in all the structures, a first memory is obtained, and the size of the storage space of the first memory is greater than the total size of the data in all the structures; For each structure in the multidimensional data and / or memory-dispersed data, the data size, storage address, and data in the structure are sequentially stored in the first memory.

8. The method according to any one of claims 2-6, characterized in that, The second type of data also includes contiguous data in memory. The flattening process of the second type of data to obtain flattened data further includes: Add identification information to the contiguous data in memory, the identification information indicating the original start position and the original end position of the contiguous data in memory; The contiguous memory data containing the added identification information is stored in a second memory, the size of which is larger than the size of the contiguous memory data.

9. The method according to any one of claims 2-8, characterized in that, Before displaying the gallery interface in response to the second operation, the method further includes: When the electronic device meets the preset conditions, it reads the first type of data and the flattened data stored in the preset storage location; Data recovery is performed on the flattened data to obtain the second type of data; A second target image is generated based on the first type of data and the second type of data.

10. The method according to claim 9, characterized in that, The process of restoring the data from the flattened data to obtain the second type of data includes: Based on the data size and storage address of each structure in the flattened data, the structure is restored to the storage address to obtain multidimensional data or memory-dispersed data in the second type of data.

11. The method according to claim 9, characterized in that, The process of restoring the data from the flattened data to obtain the second type of data includes: Based on the original start and end positions indicated by the identification information of each structure in the flattened data, the structure is stored at the storage address corresponding to the original start and end positions, thus obtaining contiguous memory data in the second type of data.

12. The method according to claim 9, characterized in that, The second type of data includes a first subtype of data and a second subtype of data. The first subtype of data includes hardware parameters for image processing, and the second subtype of data includes software parameters for image processing. The step of flattening the second type of data to obtain flattened data includes: The hardware parameters and the software parameters are flattened respectively to obtain the flattened data; The process of restoring the data from the flattened data to obtain the second type of data includes: The hardware parameters and software parameters in the flattened data are recovered to obtain the second type of data.

13. The method according to any one of claims 9-12, characterized in that, The first type of data includes image data, and the second type of data includes image processing algorithms, hardware parameters, and software parameters during image processing. Generating a second target image based on the first type of data and the second type of data includes: Configure the electronic device based on the hardware parameters in the second type of data; The image data in the first type of data is preprocessed to obtain a preprocessed image. The preprocessing operation includes at least one of demosaicing, white balance, or noise reduction. Based on the software parameters in the second type of data, the preprocessed image is processed using the image processing algorithm in the second type of data to obtain the second target image.

14. The method according to any one of claims 1-13, characterized in that, Before displaying the second gallery interface in response to the second operation, the method further includes: Obtain at least one of the following: remaining space in the current read-only memory (ROM), input / output I / O throughput, or power consumption; If at least one of the following—the remaining space in the current ROM, the I / O throughput, or the battery power—does not reach the corresponding threshold, the electronic device is determined to be in an idle state.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-14.

16. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-14.

17. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-14.

Images