Data processing method and electronic equipment

Abstract

The invention discloses a data processing method and electronic equipment, and the method comprises the steps: in a process of continuously photographing N times, in response to a first photographing operation, creating a foreground path and a first background path, and collecting a first image; n is a positive integer; processing the data of the first image by using a foreground path to obtain a first thumbnail, and displaying the first thumbnail; after the data of the first image is processed by using the first background channel to obtain a second image, replacing the first thumbnail with the second image; after the foreground path is created, the foreground path is kept from being destroyed before the camera is turned off. According to the method and the device, the condition of frequently creating and destroying algorithm paths during continuous photographing can be reduced, the time consumption is reduced, and the performance of the shot2ee is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular, to data processing methods and electronic devices. Background Art

[0002] Users can take photos through the camera application in the electronic device. When the user clicks the photo button in the camera application, the electronic device will send a photo request message to the hardware abstraction layer (HAL); the HAL layer will create an algorithm path for this photo, and will send a photo request message to the hardware device to obtain image data from the hardware device; then the algorithm path is used to process the image data, and the generated picture is returned to the camera application.

[0003] like Figure 1A As shown, in the process of processing using the algorithm path, it can be divided into a foreground path and a background path. The foreground path is the process of generating thumbnails, and the background path is the process of generating pictures using algorithm processing. Specifically, the electronic device generates a photo request message in response to the photo operation, and sends it to the HAL layer; the algorithm path management module of the HAL layer creates a foreground path and a background path according to the photo request message, and then further sends the photo request message to the kernel layer, and the kernel layer returns the image data to the foreground path of the HAL layer, and generates a thumbnail in the foreground path; as shown Figure 1B As shown, the generated thumbnail can be displayed in the lower left corner of the camera interface, and the thumbnail will also be saved. During the processing of the foreground channel, the image data will also be copied to the background channel. At this time, the background channel will use the algorithm to process the image data. The image that is actually processed by the algorithm often takes a long time. The previously saved thumbnail will be updated after the algorithm is processed, and the foreground channel and the background channel will be destroyed.

[0004] In this process, the time from clicking the photo button to the thumbnail being displayed in the lower left corner of the camera interface is called shot2see; the time from the thumbnail being displayed to the thumbnail being updated to the image processed by the background pipeline is called see2review. In the scenario of taking multiple photos in succession, the algorithm pipeline may be different, so the algorithm pipeline will be recreated and destroyed for each photo. As a result, the algorithm pipeline will be frequently created and destroyed during continuous photography, which will consume more time and lead to poor shot2see performance. Summary of the invention

[0005] The present application provides a data processing method and an electronic device, which can reduce the frequent creation and destruction of algorithm paths during continuous photography, reduce time consumption, and improve the performance of shot2see.

[0006] In a first aspect, the present application provides a data processing method, which also includes: in the process of taking photos N times in succession, in response to the first photo operation, creating a foreground path and a first background path and capturing a first image; N is a positive integer; after using the foreground path to process the data of the first image to obtain a first thumbnail, displaying the first thumbnail; after using the first background path to process the data of the first image to obtain a second image, replacing the first thumbnail with the second image; after creating the foreground path, before turning off the camera, keeping the foreground path from being destroyed.

[0007] Based on the method described in the first aspect, during the process of taking N consecutive photos, since there is no difference in the foreground path, the electronic device only needs to create the foreground path when taking the first photo, and then keep the foreground path resident, that is, before turning off the camera, keep the foreground path from being destroyed, thereby optimizing the shot2see time of multiple consecutive photos, reducing the frequent creation and destruction of algorithm paths during continuous photos, and improving the performance of shot2see.

[0008] In a possible implementation, the method further includes: sending the data of the first image to the foreground path and the first background path simultaneously. Based on this method, it is helpful to save time.

[0009] In a possible implementation, after replacing the first thumbnail with the second image and before detecting the second photo-taking operation, the method further includes: destroying the first background path. Based on this approach, it is beneficial to improve the accuracy of image data processing.

[0010] In a possible implementation, the method further includes: in the process of taking photos N times in a row, in response to the second photo operation, creating a second background path and collecting a third image; after using the foreground path to process the data of the third image to obtain a second thumbnail, displaying the second thumbnail; after using the second background path to process the data of the third image to obtain a fourth image, replacing the second thumbnail with the fourth image, and destroying the second background path. Based on this method, the foreground path is kept resident, and only the background path needs to be created for each photo, which is conducive to saving time and improving the accuracy of image data processing.

[0011] In a possible implementation, the method further includes: in the process of taking photos N times in a row, in response to the second photo operation, collecting a third image; when the scene type corresponding to the second photo operation is different from the scene type corresponding to the first photo operation, destroying the first background path and creating a second background path; after using the foreground path to process the data of the third image to obtain the second thumbnail, displaying the second thumbnail; after using the second background path to process the data of the third image to obtain the fourth image, replacing the second thumbnail with the fourth image. Based on this method, the foreground path is kept resident, and the first background path created during the first photo is destroyed with a delay. Whether to destroy the first background path is determined based on whether the scene type corresponding to the second photo operation is the same as the scene type corresponding to the first photo operation. If they are different, the first background path is destroyed and the second background path is recreated; if they are the same, the first background path can be retained for continued use, thereby further saving time.

[0012] In a possible implementation, the method further includes: when the scene type corresponding to the second photographing operation is the same as the scene type corresponding to the first photographing operation, retaining the first background path; after using the foreground path to process the data of the third image to obtain the second thumbnail, displaying the second thumbnail; after using the first background path to process the data of the third image to obtain the fifth image, replacing the second thumbnail with the fifth image. Based on this method, time consumption can be further saved.

[0013] In a possible implementation, the method further includes: in the process of taking photos N times in a row, when the first background path is retained M times, destroying the first background path; M is less than or equal to N, and M is a positive integer. Based on this method, the accuracy of image data processing can be guaranteed.

[0014] In a second aspect, the present application provides a data processing device, which may be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device; wherein the data processing device may also be a chip system, and the data processing device may execute the method executed by the electronic device in the first aspect. The functions of the data processing device may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. The unit may be software and / or hardware. The operations and beneficial effects performed by the data processing device may refer to the methods and beneficial effects described in the first aspect above, and the repeated parts will not be repeated.

[0015] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the data processing method in any possible implementation of the first aspect.

[0016] In a fourth aspect, the present application provides a data processing device, which includes functions or units for executing any method described in the first aspect.

[0017] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the data processing method in any possible implementation of the first aspect.

[0018] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the data processing method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A is a schematic diagram of an image data processing method based on a foreground path and a background path provided in an embodiment of the present application;

[0020] Figure 1B This is a schematic diagram of taking photos and displaying thumbnails provided by an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 It is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0023] Figure 4 It is a flowchart of a data processing method based on software structure provided in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of another image data processing method based on a foreground path and a first background path provided in an embodiment of the present application;

[0025] Figure 6 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0026] Figure 7It is a flowchart of another data processing method provided in an embodiment of the present application;

[0027] Figure 8 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0028] Fig. 9 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0031] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of a terminal device.

[0032] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0033] 1. Thumbnail

[0034] A thumbnail is a small snapshot, a low-quality preview of the original image after compression. It is an image generated by a specific algorithm that can represent digital images, videos, articles, etc.; it usually contains a hyperlink to the full-size image. Figure 1B As shown in the figure, a thumbnail can be displayed in the lower left corner of the camera interface and saved at the same time; users can click on the thumbnail to jump to the album to view the full-size image. Since the thumbnail is only a low-quality preview, a clear image cannot be seen in the album at this time. The thumbnail needs to be updated later to display a clearer high-quality image.

[0035] 2. shot2see

[0036] Users can take photos through the camera application in the electronic device. When the user clicks the photo button in the camera application, the electronic device will send a photo request message to the hardware abstraction layer (HAL); the HAL layer will create an algorithm path for this photo, and will send a photo request message to the hardware device to obtain image data from the hardware device; then the algorithm path is used to process the image data, and the generated picture is returned to the camera application.

[0037] like Figure 1A As shown, in the process of processing using the algorithm path, it can be divided into a foreground path and a background path. The foreground path is the process of generating thumbnails, and the background path is the process of generating pictures using algorithm processing. Specifically, the electronic device generates a photo request message in response to the photo operation, and sends it to the HAL layer; the algorithm path management module of the HAL layer creates a foreground path and a background path according to the photo request message, and then further sends the photo request message to the kernel layer, and the kernel layer returns the image data to the foreground path of the HAL layer, and generates a thumbnail in the foreground path; as shown Figure 1B As shown, the generated thumbnail can be displayed in the lower left corner of the camera interface, and the thumbnail will also be saved. During the foreground channel processing, the image data will also be copied to the background channel. At this time, the background channel will use the algorithm to process the image data. The image that is actually processed by the algorithm often takes a long time. The previously saved thumbnail will be updated after the algorithm processing is completed. Only then can a clearer high-quality image be viewed, and the foreground channel and the background channel will be destroyed.

[0038] In this process, the time from clicking the photo button to the thumbnail being displayed in the lower left corner of the camera interface is called shot2see; the time from the thumbnail being displayed to the thumbnail being updated to the image processed by the background pipeline is called see2review. In the scenario of taking multiple photos in succession, the algorithm pipeline may be different, so the algorithm pipeline will be recreated and destroyed for each photo. As a result, the algorithm pipeline will be frequently created and destroyed during continuous photography, which will consume more time and lead to poor shot2see performance.

[0039] Therefore, in order to reduce the frequent creation and destruction of algorithm paths during continuous photography, reduce time consumption, and improve the performance of shot2see, the present application provides a data processing method and an electronic device. In a specific implementation, the above-mentioned data processing method can be executed by an electronic device 100. Among them, the electronic device 100 can be a mobile phone, a tablet computer, a laptop computer, or a wearable electronic device with wireless communication function (such as a smart watch), etc., but is not limited to this. The electronic device 100 is configured with a display screen and can be installed with a preset application (application, APP), such as a camera APP. Users can use the camera APP to perform the photo function.

[0040] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0041] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0042] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0043] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0044] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0045] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The processor 110 calls the instructions or data stored in the memory, so that the electronic device 100 executes the data processing method executed by the electronic device in the following method embodiment.

[0046] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0047] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0048] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0049] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0050] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0051] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0052] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0053] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0054] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0055] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0056] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0057] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0058] NPU is a neural-network (NN) computing processor. It can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0059] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0060] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as audio data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0061] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0062] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0063] Speaker 170A, also known as "speaker", is used to convert audio electrical signals into sound signals. Receiver 170B, also known as "earpiece", is used to convert audio electrical signals into sound signals. Microphone 170C, also known as "microphone" and "microphone", is used to convert sound signals into electrical signals. Headphone interface 170D is used to connect wired headphones. Pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, pressure sensor 180A can be set on display screen 194. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Air pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes Hall sensor. Acceleration sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Proximity light sensor 180G can include, for example, light emitting diode (LED) and light detector. Ambient light sensor 180L is used to sense ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0064] In addition, an operating system is running on the above components. For example, operating systems such as iOS and Android. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0065] Figure 3 It is a software structure block diagram of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiment of the present application, the operating system (taking the Android system, the Android system running on the AP as an example) can be divided into six layers, from top to bottom, respectively, the application layer (application, APP), the application framework layer (framework, FWK), the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware abstraction layer, HAL), the kernel layer and the hardware layer.

[0066] The application layer may include a series of application packages. Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. The application layer may also include system UI (system UI), which is used to display the interface of the electronic device, such as the camera interface, etc. In the embodiment of the present application, the camera refers to the camera application.

[0067] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a camera service, etc., and the embodiments of the present application do not impose any restrictions on this.

[0068] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0069] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0070] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0071] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0072] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0073] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0074] The camera service is the core process module of the camera framework. It mainly provides API interface functions to the application layer and calls the camera hardware abstraction layer downward through HIDL (hardware interface definition language).

[0075] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0076] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0077] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0078] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0079] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0080] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats.

[0081] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0082] A 2D graphics engine is a drawing engine for 2D drawings.

[0083] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. The hardware abstraction layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware. Figure 3 As shown, the hardware abstraction layer may include Wi-Fi HAL, audio HAL, camera HAL, etc. Among them, the camera HAL is the core software framework of the Camera, and the camera HAL may include an algorithm path management module, a path module, etc.

[0084] The algorithm path management module is used to create foreground paths and background paths; it can also interact with the kernel layer, for example, it can send a photo request message to the kernel layer, and it can also receive image data returned by the kernel layer. The path module includes foreground paths and background paths. The foreground path is the process of generating thumbnails, and the background path is the process of generating images using algorithm processing.

[0085] The kernel layer is the layer between hardware and software. It is the core of an operating system and the first layer of software expansion based on hardware. It provides the most basic functions of the operating system and is the basis for the operation of the operating system. It is responsible for managing the system's processes, memory, device drivers, files, and network systems, and determines the performance and stability of the system. The kernel layer can include display drivers, audio drivers, camera drivers, sensor drivers, image signal processing drivers, etc. Among them, the camera driver is the driver layer of the Camera device, which is mainly responsible for the interaction with the hardware.

[0086] The hardware layer includes display, camera, sensor, image signal processor (ISP), etc.

[0087] Based on the above software structure, the present application embodiment provides a flowchart of a data processing method based on a software structure. Figure 4 As shown, the data processing method includes the following steps S401 to S408. Optionally, the data processing method also includes step S409. Among them:

[0088] S401. In a process of taking photos N times in succession, a camera application generates a photo request message in response to a first photo operation.

[0089] In the embodiment of the present application, continuous shooting here refers to continuously clicking the shooting button in the camera application to take pictures, and the time interval between two adjacent pictures is less than or equal to the first preset threshold. It can be understood that in the process of taking pictures N times in a row, the time interval between two adjacent pictures is in a very small range (i.e., the first preset threshold). Wherein N is a positive integer. For example, assuming that N is 5, the first preset threshold is 2 seconds, and the time interval between two adjacent pictures in the process of taking pictures 5 times in a row is less than 2 seconds.

[0090] In the process of taking photos N times in succession, for the first photo, the user can click the photo button in the camera application. At this time, the camera application at the APP layer generates a photo request message in response to the user's first photo operation.

[0091] S402: The camera application sends the photo request message to the algorithm path management module. Correspondingly, the algorithm path management module receives the photo request message from the camera application.

[0092] In the embodiment of the present application, the camera application further sends a photo request message to the algorithm path management module of the HAL layer through the APP layer and the FWK layer. It should be noted that the sending and receiving between modules is actually implemented through functions.

[0093] S403, the algorithm path management module creates a foreground path and a first background path according to the photo request message, and sends the photo request message to the camera driver. Correspondingly, the camera driver receives the photo request message from the algorithm path management module.

[0094] In an embodiment of the present application, the algorithm path management module creates a foreground path and a first background path for the first photo operation according to the photo request message. Among them, the foreground path here is a process for generating thumbnails. For example, the foreground path may include a thumbnail module (for generating thumbnails) and a callback module (for returning thumbnails to the APP layer). The first background path is a process for generating pictures using algorithm processing. For example, the first background path may include a first algorithm module, a second algorithm module, a third algorithm module and a callback module (for returning the image processed by the background path to the APP layer). At the same time, the algorithm path management module will also send the photo request message to the camera driver of the kernel layer. It should be noted that for different scene types when taking pictures, different background paths need to be created to process image data. For example, for the photo scene of a person, a beauty algorithm will need to be applied in the background path; for the photo scene of a scene, a scene color adjustment algorithm, a contrast enhancement algorithm, etc. will need to be applied in the background path, which is not limited here.

[0095] S404: The camera driver captures a first image according to the photo request message.

[0096] In an embodiment of the present application, after the camera driver receives the photographing request message, it can use the camera to capture the first image for this photographing operation according to the photographing request message.

[0097] Optionally, the data of the first image is data obtained after the camera driver processes the first image using an image signal processor. That is, after the camera driver acquires the original image, it also processes the image signal processor through the image signal processor driver of the kernel layer, thereby obtaining the processed data of the first image.

[0098] S405: The camera driver sends the data of the first image to the foreground path and the first background path at the same time. Accordingly, the foreground path and the first background path receive the data of the first image from the camera driver at the same time.

[0099] In an embodiment of the present application, the camera driver further sends the data of the first image to the foreground path and the first background path for processing at the same time. Of course, the camera driver can also send the data of the first image to the foreground path and the first background path for processing successively, but it is necessary to ensure that the time interval between sending to the foreground path and the first background path is less than or equal to the second preset threshold. In other words, the data of the first image acquired by the first background path is not copied by the foreground path, but is directly sent to the first background path by the camera driver; at the same time, in order to reduce time consumption, it is necessary to control the time interval between the camera driver sending the data of the first image to the foreground path and the first background path to be within a very small range (i.e., the second preset threshold).

[0100] For example, the camera driver can send the data of the first image to the foreground channel and the first background channel for processing at the same time; or the data of the first image can be sent to the foreground channel first, and then the data of the first image can be sent to the first background channel, wherein the time interval between sending the data of the first image to the foreground channel and the first background channel is less than or equal to 0.5 seconds; or the data of the first image can be sent to the first background channel first, and then the data of the first image can be sent to the foreground channel, wherein the time interval between sending the data of the first image to the first background channel and the foreground channel is less than or equal to 0.5 seconds, which is not limited here.

[0101] S406: After the foreground path is used to process the data of the first image to obtain a first thumbnail, the foreground path sends the first thumbnail to the camera application. Accordingly, the camera application receives the first thumbnail from the foreground path and displays the first thumbnail.

[0102] In the embodiment of the present application, after the foreground channel receives the data of the first image, the thumbnail module processes the data of the first image to obtain a first thumbnail; finally, the first thumbnail is returned to the camera application through the callback module, and the first thumbnail can be displayed in the camera application. Figure 1B As shown, the first thumbnail can be displayed in the lower left corner of the camera interface and saved; the user can click the thumbnail to jump to the album to view the full-size image. Since the thumbnail is only a low-quality preview, a clear image cannot be viewed in the album at this time, and the thumbnail needs to be updated later to display a clearer high-quality image.

[0103] S407: After the first background path is used to process the data of the first image to obtain the second image, the first background path sends the second image to the camera application. Accordingly, the camera application receives the second image from the first background path and replaces the first thumbnail with the second image.

[0104] In an embodiment of the present application, after the first background channel receives the data of the first image, the first algorithm module, the second algorithm module, and the third algorithm module are used to process the data of the first image in sequence to obtain the second image; finally, the second image is returned to the camera application through the callback module, so that the previously saved first thumbnail can be replaced with the second image; the user can click on the thumbnail and jump to the album to view the full-size picture. At this time, the image displayed is the image after the thumbnail is updated, that is, the second image, thereby showing a clearer high-quality image.

[0105] S408. After the algorithm channel management module creates the foreground channel, before the camera is turned off, the foreground channel is kept from being destroyed.

[0106] In the embodiment of the present application, in the scenario of taking multiple consecutive photos, although the algorithm pathways may be different, the processing pathway for generating thumbnails (i.e., the foreground pathway) is the same. Therefore, during the process of taking multiple consecutive photos, the foreground pathway can be kept resident. That is, after being created during the first photo shooting process and before the camera is turned off, the foreground pathway is kept from being destroyed, thereby optimizing the shot2see time of multiple consecutive photos. The foreground pathway is destroyed after the camera is turned off, thereby reducing the time consumed in frequently creating and destroying the algorithm pathways.

[0107] S409: The algorithm path management module destroys the first background path.

[0108] In the embodiment of the present application, since there is no difference in the foreground path in the scenario of taking photos multiple times in a row, the foreground path can be kept resident; however, the difference in the background path is large. For example, the scene type when taking the photo for the first time is a person, and the scene type when taking the photo for the second time is a scene. At this time, the algorithm of the background path used is different. If the first background path is kept resident, the accuracy of processing image data will be reduced. Therefore, after the camera application replaces the first thumbnail with the second image, the algorithm path management module needs to destroy the first background path, and in the subsequent photo process, it needs to re-create the background path for the current photo request message, so as to ensure the accuracy of the background path in processing image data in the subsequent photo process.

[0109] In a possible implementation, the method further includes: during the process of taking photos N times in a row, the camera application responds to the operation of closing the camera, and the algorithm path management module destroys the foreground path. That is, after the foreground path is created, the foreground path is kept resident until the camera application is closed, and the algorithm path management module destroys the foreground path.

[0110] In general, if Figure 5As shown, the camera application generates a photo request message in response to the first photo operation, and sends it to the algorithm path management module of the HAL layer. The algorithm path management module creates a foreground path and a first background path according to the photo request message, and sends the photo request message to the camera driver of the kernel layer. The camera driver collects the first image according to the photo request message, and the data of the first image here can be the data obtained after the camera driver processes the first image using the image signal processor. Then, the camera driver sends the data of the first image to the foreground path (including the thumbnail module and the callback module) and the first background path (including the first algorithm module, the second algorithm module, the third algorithm module and the callback module) at the same time. After the foreground path is used to process the data of the first image to obtain the first thumbnail, the foreground path sends the first thumbnail to the camera application; after the first background path is used to process the data of the first image to obtain the second image, the first background path sends the second image to the camera application, and the camera application receives the second image from the first background path and replaces the first thumbnail with the second image. After the algorithm path management module creates the foreground path, before the camera is turned off, the foreground path is kept from being destroyed. In addition, the algorithm path management module can also destroy the first background path. Based on this method, by keeping the foreground path resident during continuous shooting, the frequent creation and destruction of algorithm paths can be reduced, the shot2see time of multiple continuous shootings can be optimized, and the shot2see performance can be improved.

[0111] 1. In the process of taking photos continuously for multiple times, for each photo, keep the foreground path from being destroyed, and destroy the background path after each photo is taken.

[0112] Based on the above, a data processing method provided by an embodiment of the present application is further described in detail below. Figure 6 As shown, the data processing method includes the following steps S601 to S604. Figure 6 The method execution subject shown can be the electronic device mentioned above. Or, Figure 6 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 6 The method is described by taking an electronic device as an example of an execution subject.

[0113] S601: In a process of taking photos N times continuously, the electronic device creates a foreground path and a first background path and captures a first image in response to a first photo taking operation.

[0114] In the embodiment of the present application, continuous shooting means continuously clicking the shooting button in the camera application to shoot, and the time interval between two adjacent shootings is less than or equal to the first preset threshold. It can be understood that in the process of taking N consecutive shots, the time interval between two adjacent shootings is in a very small range. Wherein N is a positive integer. For example, assuming that N is 5, the first preset threshold is 2 seconds, and the time interval between two adjacent shootings in the process of taking 5 consecutive shots is less than 2 seconds.

[0115] In the process of taking photos N times in a row, for the first photo, the user can click the photo button in the camera application, and the electronic device responds to the first photo operation, creates a foreground path and a first background path and captures the first image. Among them, the foreground path here is the process of generating thumbnails; the first background path is the process of generating pictures using algorithm processing. It should be noted that for different scene types when taking photos, different background paths need to be created to process images. For example, for the photo scene of a person, a beauty algorithm will need to be applied in the background path; for the photo scene of a scene, a scene color adjustment algorithm, a contrast enhancement algorithm, etc. will need to be applied in the background path, which is not limited here.

[0116] In the software structure of the electronic device, the application layer includes the camera application; the application framework layer includes the system service; the hardware abstraction layer includes the algorithm path management module and the path module, wherein the path module includes the foreground path and the background path; the kernel layer includes the camera driver. In the process of taking photos N times in a row, when the electronic device responds to the first photo operation, creates the foreground path and the first background path and collects the first image, the specific implementation method can refer to the above steps S401 to S404, that is:

[0117] In the process of taking photos N times in succession, the camera application generates a photo request message in response to the first photo operation; the camera application sends the photo request message to the algorithm path management module, and accordingly, the algorithm path management module receives the photo request message from the camera application; the algorithm path management module creates a foreground path and a first background path according to the photo request message, and sends the photo request message to the camera driver, and accordingly, the camera driver receives the photo request message from the algorithm path management module; the camera driver captures a first image according to the photo request message.

[0118] In a possible implementation, the method further includes: the electronic device sends the data of the first image to the foreground path and the first background path at the same time. That is, the data of the first image acquired by the first background path is not copied by the foreground path, but is sent directly by the electronic device to the first background path. The electronic device sends the data of the first image to the foreground path and the first background path at the same time, which can reduce time consumption. Of course, the electronic device can also send the data of the first image to the foreground path and the first background path successively for processing, but it is necessary to ensure that the time interval between sending the data to the foreground path and the first background path is less than or equal to the second preset threshold. It can be understood that in order to reduce time consumption, it is necessary to control the time interval between sending the data of the first image to the foreground path and the first background path to be within a very small range (i.e., the second preset threshold).

[0119] For example, the electronic device can send the data of the first image to the foreground path and the first background path for processing at the same time; or it can send the data of the first image to the foreground path first, and then send the data of the first image to the first background path, wherein the time interval between sending the data of the first image to the foreground path and the first background path is less than or equal to 0.5 seconds; or it can send the data of the first image to the first background path first, and then send the data of the first image to the foreground path, wherein the time interval between sending the data of the first image to the first background path and the foreground path is less than or equal to 0.5 seconds; no limitation is made here.

[0120] The specific implementation process can refer to the above step S405, that is, the camera driver sends the data of the first image to the foreground path and the first background path at the same time. Correspondingly, the foreground path and the first background path simultaneously receive the data of the first image from the camera driver.

[0121] S602: After the electronic device processes the data of the first image using the foreground path to obtain a first thumbnail, the first thumbnail is displayed.

[0122] In the embodiment of the present application, the foreground path may include a thumbnail module (for generating thumbnails) and a callback module (for returning thumbnails to the camera application). After the foreground path receives the data of the first image, the thumbnail module processes the data of the first image to obtain a first thumbnail; finally, the callback module returns the first thumbnail to the camera application, and the first thumbnail can be displayed in the camera application. Figure 1B As shown, the first thumbnail can be displayed in the lower left corner of the camera interface and saved; the user can click the thumbnail to jump to the album to view the full-size image. Since the thumbnail is only a low-quality preview, a clear image cannot be viewed in the album at this time, and the thumbnail needs to be updated later to display a clearer high-quality image.

[0123] The specific implementation process can refer to the above step S406, that is, after the foreground path is used to process the data of the first image to obtain the first thumbnail, the foreground path sends the first thumbnail to the camera application. Correspondingly, the camera application receives the first thumbnail from the foreground path and displays the first thumbnail.

[0124] S603: After the electronic device processes the data of the first image by using the first background path to obtain the second image, the first thumbnail is replaced by the second image.

[0125] In an embodiment of the present application, the first background path may include a first algorithm module, a second algorithm module, a third algorithm module and a callback module (for returning the image processed by the background path to the camera application). After the first background path receives the data of the first image, the first algorithm module, the second algorithm module and the third algorithm module are used to process the data of the first image in sequence to obtain the second image; finally, the second image is returned to the camera application through the callback module, and the first thumbnail previously saved can be replaced with the second image; the user can click on the thumbnail to jump to the album to view the full-size picture, and the image displayed at this time is the image after the thumbnail is updated, that is, the second image, thereby showing a clearer high-quality image.

[0126] The specific implementation process can refer to the above step S407, that is, after the first background path is used to process the data of the first image to obtain the second image, the first background path sends the second image to the camera application. Correspondingly, the camera application receives the second image from the first background path and replaces the first thumbnail with the second image.

[0127] In a possible implementation, the first thumbnail is saved in a thumbnail list, and before the electronic device replaces the first thumbnail with the second image, the method further includes: the electronic device searches the thumbnail list for a first thumbnail that matches the second image, and the first thumbnail has the greatest similarity with the second image. That is, after the electronic device processes the data of the first image using the foreground path to obtain the first thumbnail, the first thumbnail can be saved in the thumbnail list. After the electronic device processes the data of the first image using the first background path to obtain the second image, the similarity between each thumbnail in the thumbnail list and the second image is calculated, and the thumbnail with the greatest similarity (i.e., the first thumbnail) is replaced with the second image.

[0128] S604: After the electronic device creates the foreground path, before the camera is turned off, the foreground path is kept from being destroyed.

[0129] In the embodiment of the present application, in the scenario of taking multiple consecutive photos, although the algorithm pathways may be different, the processing pathway for generating thumbnails (i.e., the foreground pathway) is the same. Therefore, during the process of taking multiple consecutive photos, the foreground pathway can be kept resident. That is, after being created during the first photo shooting process and before the camera is turned off, the foreground pathway is kept from being destroyed, thereby optimizing the shot2see time of multiple consecutive photos. The foreground pathway is destroyed after the camera is turned off, thereby reducing the time consumed in frequently creating and destroying the algorithm pathways.

[0130] The specific implementation process may refer to the above step S408, that is, after the algorithm path management module creates the foreground path and before the camera is turned off, the foreground path is kept from being destroyed.

[0131] In a possible implementation, after the electronic device replaces the first thumbnail with the second image and before detecting the second photo operation, the method further includes: the electronic device destroys the first background path. It can be understood that, since there is no difference in the foreground path in the scenario of multiple consecutive photos, the foreground path can be kept resident; however, the differences in the background paths are large, and if the first background path is also kept resident, the accuracy of processing image data will be reduced. Therefore, after the camera application replaces the first thumbnail with the second image, the electronic device needs to destroy the first background path, and re-create the background path for the current photo request in the subsequent photo process, so as to ensure the accuracy of the background path processing image data in the subsequent photo process. The specific implementation process can refer to the above step S409, that is, the algorithm path management module destroys the first background path.

[0132] In a possible implementation, the method further includes: in the process of taking photos N times continuously, in response to the operation of closing the camera, destroying the foreground path. That is, after the foreground path is created, the foreground path is kept resident until the camera application is closed, and the electronic device destroys the foreground path.

[0133] In one possible implementation, the method also includes: in the process of taking photos N times in succession, the electronic device creates a second background path and captures a third image in response to the second photo operation; after the electronic device uses the foreground path to process the data of the third image to obtain a second thumbnail, the second thumbnail is displayed; after the electronic device uses the second background path to process the data of the third image to obtain a fourth image, the second thumbnail is replaced with the fourth image, and the second background path is destroyed.

[0134] It can be understood that, in the process of taking photos N times in a row, the user clicks the photo button in the camera application for the first time, and the electronic device responds to the first photo operation, creates a foreground path and a first background path and collects the first image; after the electronic device uses the foreground path to process the data of the first image to obtain the first thumbnail, the first thumbnail is displayed; after the electronic device uses the first background path to process the data of the first image to obtain the second image, the first thumbnail is replaced with the second image; the electronic device keeps the foreground path resident and destroys the first background path. The user continues to click the photo button in the camera application for the second time, and the electronic device responds to the second photo operation, at this time, it needs to create a second background path and collect the third image; similarly (refer to the above steps S602 and S603), after the electronic device uses the foreground path to process the data of the third image to obtain the second thumbnail, the second thumbnail is displayed; after the electronic device uses the second background path to process the data of the third image to obtain the fourth image, the second thumbnail is replaced with the fourth image, and the second background path is destroyed. In this way, during the process of taking N consecutive photos, the foreground path is always kept before the camera is turned off, and only the background path is created for each photo. The foreground path is not destroyed until the camera is turned off, reducing the time spent on frequent creation and destruction of algorithm paths.

[0135] It can be seen that based on the method described in the present application, in the process of taking N consecutive photos, since there is no difference in the foreground path, the electronic device only needs to create the foreground path when taking the first photo, and then keep the foreground path resident, that is, before turning off the camera, keep the foreground path from being destroyed, thereby optimizing the shot2see time of multiple consecutive photos, reducing the frequent creation and destruction of algorithm paths during continuous photos, and improving the performance of shot2see.

[0136] 2. During the process of taking photos continuously for multiple times, for each photo, the foreground path is kept from being destroyed, and the background path is destroyed with a delay after each photo is taken.

[0137] Based on the above, another data processing method provided by the embodiment of the present application is further described in detail below. Figure 7 As shown, the data processing method includes the following steps S701 to S708. Figure 7 The method execution subject shown can be the electronic device mentioned above. Or, Figure 7 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 7 The method is described by taking an electronic device as an example of an execution subject.

[0138] S701: In a process of taking photos N times in succession, the electronic device creates a foreground path and a first background path and captures a first image in response to a first photo taking operation.

[0139] Wherein, N is a positive integer.

[0140] S702: After the electronic device processes the data of the first image using the foreground path to obtain a first thumbnail, the first thumbnail is displayed.

[0141] S703: After the electronic device processes the data of the first image by using the first background path to obtain the second image, the first thumbnail is replaced by the second image.

[0142] S704: After the electronic device creates the foreground path, before the camera is turned off, the foreground path is kept from being destroyed.

[0143] The specific implementation of steps S701 to S704 may refer to the specific implementation of steps S601 to S604 above, which will not be described in detail here.

[0144] S705. During the process of taking photos N times in succession, the electronic device collects a third image in response to the second photo taking operation.

[0145] S706: When the scene type corresponding to the second photographing operation is different from the scene type corresponding to the first photographing operation, the electronic device destroys the first background path and creates a second background path.

[0146] S707: After the electronic device processes the data of the third image using the foreground path to obtain the second thumbnail, the second thumbnail is displayed.

[0147] S708: After the electronic device processes the data of the third image by using the second background path to obtain the fourth image, the second thumbnail is replaced by the fourth image.

[0148] In an embodiment of the present application, during the process of taking photos N times continuously, the user clicks the photo button in the camera application for the first time, and the electronic device responds to the first photo operation, creates a foreground path and a first background path and collects the first image; after the electronic device uses the foreground path to process the data of the first image to obtain the first thumbnail, the first thumbnail is displayed; after the electronic device uses the first background path to process the data of the first image to obtain the second image, the first thumbnail is replaced with the second image; the electronic device keeps the foreground path resident. At this time, the first background path is not destroyed temporarily, and the user continues to click the photo button in the camera application for the second time, and the electronic device responds to the second photo operation and collects the third image; if the scene type corresponding to the second photo operation is different from the scene type corresponding to the first photo operation, then it is considered that the scene type has changed (for example, from a character scene to a scene scene), and the electronic device needs to destroy the first background path at this time and create a second background path. Similarly (refer to the above steps S602 and S603), after the electronic device uses the foreground path to process the data of the third image to obtain the second thumbnail, the second thumbnail is displayed; after the electronic device uses the second background path to process the data of the third image to obtain the fourth image, the second thumbnail is replaced with the fourth image.

[0149] It can be understood that in the process of taking photos N times in succession, not only can the foreground path be kept resident, but the background path can also be destroyed with delay, that is, after the current photo is taken, the first background path is not destroyed temporarily. When taking the next photo, if the scene type of the next photo is different from the scene type of the current photo, it is considered that the scene type has changed, and the background path used is different. Therefore, it is necessary to destroy the first background path created for the first time and recreate the second background path to ensure the accuracy of image processing.

[0150] In a possible implementation, the method also includes: when the scene type corresponding to the second photographing operation is the same as the scene type corresponding to the first photographing operation, the electronic device retains the first background path; after the electronic device uses the foreground path to process the data of the third image to obtain the second thumbnail, the second thumbnail is displayed; after the electronic device uses the first background path to process the data of the third image to obtain the fifth image, the second thumbnail is replaced with the fifth image.

[0151] It can be understood that in the process of taking photos N times in a row, not only can the foreground path be kept resident, but the background path can also be destroyed with a delay, that is, after the photo is taken this time, the first background path will not be destroyed temporarily. When taking photos next time, if the scene type of the next photo is the same as the scene type of the current photo, it is considered that the scene type has not changed. At this time, the first background path can be retained, and the first background path created before can continue to be used, without the need to recreate a new background path. Similarly (refer to the above steps S602 and S603), after the electronic device uses the foreground path to process the data of the third image to obtain the second thumbnail, the second thumbnail is displayed; after the electronic device uses the first background path to process the data of the third image to obtain the fifth image, the second thumbnail is replaced with the fifth image. Based on this method, time can be further saved, and the frequent creation and destruction of algorithm paths can be reduced during continuous photography, thereby improving the performance of shot2see.

[0152] In a possible implementation, the method further includes: in the process of taking photos N times in a row, when the first background path is retained M times, the electronic device destroys the first background path; M is less than or equal to N, and M is a positive integer. It can be understood that in the process of taking photos N times in a row, when the number of times the first background path is retained reaches a preset value (i.e., M), it is considered that the first background path has been created and used for a long time. In order to ensure the accuracy of image data processing, the first background path needs to be destroyed at this time so that the background path is recreated when taking photos next time.

[0153] It can be seen that based on the method described in the present application, in the process of taking N consecutive photos, since there is no difference in the foreground path, the electronic device only needs to create the foreground path when taking the first photo, and then keep the foreground path resident, that is, before turning off the camera, keep the foreground path from being destroyed, thereby optimizing the shot2see time of multiple consecutive photos, reducing the frequent creation and destruction of algorithm paths during continuous photos, and improving the performance of shot2see; at the same time, the electronic device can also delay the destruction of the first background path created during the first photo, and determine whether to destroy the first background path based on whether the scene type corresponding to the second photo operation is the same as the scene type corresponding to the first photo operation. If they are different, the first background path is destroyed and the second background path is recreated. If they are the same, the first background path can be retained for continued use, thereby further saving time.

[0154] See also Figure 8 , Figure 8 A structural schematic diagram of a data processing device 800 according to an embodiment of the present application is shown. Figure 8 The data processing device shown may be an electronic device, or a device in an electronic device, or a device that can be used in conjunction with an electronic device. Figure 8The data processing device shown may include a processing unit 801 and a display unit 802. Among them:

[0155] The processing unit 801 is used to create a foreground path and a first background path and acquire a first image in response to a first photographing operation during a process of taking photographs N times in succession; N is a positive integer;

[0156] A display unit 802, configured to display the first thumbnail after obtaining the first thumbnail by processing the data of the first image using the foreground path;

[0157] A processing unit 801 is used to replace the first thumbnail with the second image after the first background path is used to process the data of the first image to obtain the second image;

[0158] The processing unit 801 is used to keep the foreground path from being destroyed after the foreground path is created and before the camera is turned off.

[0159] In a possible implementation, the method further includes: sending the data of the first image to the foreground path and the first background path simultaneously.

[0160] In a possible implementation, after the first thumbnail is replaced by the second image and before the second photographing operation is detected, the method further includes: destroying the first background path.

[0161] In a possible implementation, the method also includes: in the process of taking photos N times in succession, in response to the second photo operation, creating a second background path and capturing a third image; after using the foreground path to process the data of the third image to obtain a second thumbnail, displaying the second thumbnail; after using the second background path to process the data of the third image to obtain a fourth image, replacing the second thumbnail with the fourth image, and destroying the second background path.

[0162] In a possible implementation, the method also includes: in the process of taking photos N times in succession, in response to a second photo operation, collecting a third image; when the scene type corresponding to the second photo operation is different from the scene type corresponding to the first photo operation, destroying the first background path and creating a second background path; after using the foreground path to process the data of the third image to obtain a second thumbnail, displaying the second thumbnail; after using the second background path to process the data of the third image to obtain a fourth image, replacing the second thumbnail with the fourth image.

[0163] In a possible implementation, the method also includes: retaining the first background path when the scene type corresponding to the second photographing operation is the same as the scene type corresponding to the first photographing operation; displaying the second thumbnail after using the foreground path to process the data of the third image to obtain the second thumbnail; and replacing the second thumbnail with the fifth image after using the first background path to process the data of the third image to obtain the fifth image.

[0164] In a possible implementation, the method further includes: in the process of taking photos N times in succession, when the first background path is retained M times, destroying the first background path; M is less than or equal to N, and M is a positive integer.

[0165] For the case where the data processing device may be a chip or a chip system, see Fig. 9 Schematic diagram of the chip structure shown. Fig. 9 The chip 900 shown includes a processor 901 and an interface 902. Optionally, it may also include a memory 903. The number of the processor 901 may be one or more, and the number of the interface 902 may be multiple.

[0166] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0167] The interface 902 is used to receive or output signals;

[0168] The processor 901 is used to execute data processing operations of the electronic device.

[0169] It is understandable that some optional features in the embodiments of the present application may be implemented independently in certain scenarios without relying on other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects, or may be combined with other features according to needs in certain scenarios. Accordingly, the data processing device provided in the embodiments of the present application may also implement these features or functions accordingly, which will not be described in detail here.

[0170] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0171] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0172] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0173] The present application also provides a computer program product. When the computer program product is executed on a computer, the computer can implement the functions of any of the above method embodiments.

[0174] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0175] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: In the process of taking photos N times continuously, in response to the first photo-taking operation, a foreground path and a first background path are created and a first image is acquired; N is a positive integer; After the data of the first image is processed by the foreground path to obtain a first thumbnail, the first thumbnail is displayed; After the first background path is used to process the data of the first image to obtain a second image, the first thumbnail is replaced with the second image; After the foreground path is created, the foreground path is kept from being destroyed before the camera is turned off.

2. The method according to claim 1, characterized in that The method further comprises: The data of the first image is sent to the foreground path and the first background path simultaneously.

3. The method according to claim 1 or 2, characterized in that: After replacing the first thumbnail with the second image and before detecting a second photographing operation, the method further includes: Destroy the first background path.

4. The method according to claim 3, characterized in that The method further comprises: In the process of taking photos N times continuously, in response to the second photo taking operation, creating a second background channel and acquiring a third image; After the data of the third image is processed by the foreground path to obtain a second thumbnail, the second thumbnail is displayed; After the second background path is used to process the data of the third image to obtain a fourth image, the second thumbnail is replaced by the fourth image, and the second background path is destroyed.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: In the process of taking photos N times continuously, in response to a second photo-taking operation, collecting a third image; When the scene type corresponding to the second photographing operation is different from the scene type corresponding to the first photographing operation, destroying the first background path and creating a second background path; After the data of the third image is processed by the foreground path to obtain a second thumbnail, the second thumbnail is displayed; After the data of the third image is processed by the second background path to obtain a fourth image, the second thumbnail is replaced by the fourth image.

6. The method according to claim 5, characterized in that The method further comprises: When the scene type corresponding to the second photographing operation is the same as the scene type corresponding to the first photographing operation, retaining the first background path; After the data of the third image is processed by the foreground path to obtain a second thumbnail, the second thumbnail is displayed; After the data of the third image is processed by the first background path to obtain a fifth image, the second thumbnail is replaced by the fifth image.

7. The method according to claim 6, characterized in that The method further comprises: In the process of taking photos N times continuously, when the first background path is retained M times, the first background path is destroyed; the M is less than or equal to the N, and the M is a positive integer.

8. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-7.

9. A chip, characterized in that: It comprises a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images