Camera startup method and electronic devices

Abstract

This application discloses a camera startup method and an electronic device, relating to the field of terminals. The method includes: the electronic device receiving a camera startup operation. In response to the camera startup operation, the electronic device can start the camera to acquire raw image data in real time. The electronic device can detect environmental parameter information and / or read function setting information from historical records. Then, the electronic device can determine a corresponding dynamic image processing algorithm based on the aforementioned environmental parameter information and / or function setting information. The electronic device can load the aforementioned dynamic image processing algorithm. The electronic device can process the raw image data acquired in real time by the camera according to the aforementioned dynamic image processing algorithm to generate a first frame preview image. Next, the electronic device can display the first frame preview image on the shooting interface.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a camera startup method and an electronic device. Background Technology

[0002] With the development of terminal technology, electronic devices are becoming increasingly versatile. For example, users can use cameras on their devices to take photos or videos to record their daily lives. The time it takes for an electronic device to receive a user's request to activate the camera and, in response, display the first preview image on the shooting interface, can be called the camera startup time. Currently, camera startup times are relatively long, causing electronic devices to be unable to respond to user needs promptly, resulting in significant inconvenience for users. Summary of the Invention

[0003] This application provides a camera startup method and electronic device, which optimizes the loading time of image processing algorithms during camera startup, reduces camera startup time, and improves camera startup efficiency.

[0004] In a first aspect, this application provides a camera startup method, comprising: an electronic device receiving a camera startup operation. The electronic device includes a first dynamic image processing algorithm and a second dynamic image processing algorithm. In response to the camera startup operation, the electronic device starts a camera to acquire raw image data in real time. When it is determined that a first function is enabled, the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function. The electronic device processes the raw image data based on the first dynamic image processing algorithm to generate a first frame preview image. The electronic device displays the first frame preview image. The electronic device then loads the second dynamic image processing algorithm. The electronic device processes the raw image data based on the first and second dynamic image processing algorithms to generate a second frame preview image. The electronic device displays the second frame preview image.

[0005] In one possible implementation, before the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function when it is determined that the first function is enabled, the method further includes: the electronic device loading a first basic image processing algorithm and a second basic image processing algorithm. The first and second basic image processing algorithms are used to process the basic quality of the image.

[0006] In one possible implementation, the electronic device processes the original image data based on the first dynamic image processing algorithm to generate the first frame preview image, which includes: the electronic device processes the original image data based on the first dynamic image processing algorithm, the first basic image processing algorithm, and the second basic image processing algorithm to generate the first frame preview image.

[0007] In one possible implementation, the electronic device processes the original image data based on the first dynamic image processing algorithm and the second dynamic image processing algorithm to generate a second frame preview image, which includes: the electronic device processes the original image data based on the first dynamic image processing algorithm, the second dynamic image processing algorithm, the first basic image processing algorithm, and the second basic image processing algorithm to generate a second frame preview image.

[0008] In one possible implementation, when it is determined that the first function has been enabled, the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function by: connecting the input interface of the first dynamic image processing algorithm to the output interface of the first basic image processing algorithm, and connecting the output interface of the first dynamic image processing algorithm to the input interface of the second basic image processing algorithm.

[0009] In one possible implementation, before the electronic device processes the original image data based on the first dynamic image processing algorithm to generate the first frame preview image, the method further includes: the electronic device detecting environmental parameter information. The electronic device determines and loads a third dynamic image processing algorithm corresponding to the environmental parameter information. The electronic device processing the original image data based on the first dynamic image processing algorithm to generate the first frame preview image includes: the electronic device processing the original image data based on the first dynamic image processing algorithm and the third dynamic image processing algorithm to generate the first frame preview image.

[0010] In one possible implementation, the first function includes one or more of the following: background blur function, white balance function, zoom function, and color temperature adjustment function.

[0011] In one possible implementation, the environmental parameter information includes one or more of the following: ambient light intensity and light source frequency.

[0012] In a second aspect, this application provides an electronic device including: one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform a method as described in any of the possible implementations of the first aspect above.

[0013] Thirdly, this application provides a chip system including a processing circuit and an interface circuit. The interface circuit is used to receive code instructions and transmit them to the processing circuit. The processing circuit is used to execute the code instructions so that the chip system performs the method in any of the possible implementations of the first aspect above.

[0014] Fourthly, this application provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any of the possible implementations of the first aspect above. Attached Figure Description

[0015] Figure 1A-Figure 1B A set of user interface diagrams provided for embodiments of this application;

[0016] Figure 2 This is a schematic flowchart illustrating a camera startup method provided in an embodiment of this application.

[0017] Figure 3A A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0018] Figure 3B A schematic diagram of the module interaction process of a camera startup method provided in an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to including one or more of the listed prominent features, any or all possible combinations. In the embodiments of this application, the terms “first” and “second” are used for descriptive purposes only and should not be construed as implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, “a plurality” means two or more.

[0021] Figure 1A-Figure 1B This is a set of user interfaces for starting up a camera on an electronic device provided in the embodiments of this application.

[0022] In this embodiment, the electronic device may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) device, etc. This embodiment does not impose any special limitations on the specific type of the electronic device. In the description of the following embodiments, the electronic device may be referred to as electronic device 100.

[0023] like Figure 1A As shown, the electronic device 100 can display a desktop 1000. The desktop 1000 can display one or more application icons. These one or more application icons may include weather application icons, stock application icons, calculator application icons, settings application icons, email application icons, video application icons, calendar application icons, and gallery application icons, etc. Optionally, the desktop 1000 may also display a status bar, a page indicator, and a tray icon area. The status bar may include one or more signal strength indicators for mobile communication signals (also known as cellular signals), signal strength indicators for wireless fidelity (Wi-Fi) signals, battery status indicators, time indicators, etc. The page indicator can be used to indicate the positional relationship between the currently displayed page and other pages. The tray icon area includes multiple tray icons (e.g., dialer application icon, messaging application icon, contacts application icon, and camera application icon 1001, etc.), which remain displayed when switching pages. The aforementioned page may also include multiple application icons and a page indicator. The page indicator may not be part of the page and may exist independently. The aforementioned tray icons are also optional, and this embodiment does not limit this.

[0024] The electronic device 100 can receive a touch operation (also known as a camera launch operation, such as a click) applied to the camera application icon 1001. In response to the aforementioned camera launch operation, the electronic device 100 can activate the camera to capture images in real time and display the shooting interface.

[0025] In some examples, the camera activation operation is not limited to the touch operation on the camera application icon 1001 described above; it can also be a voice command or gesture operation. That is, the electronic device 100 can also receive and respond to the user's voice command or gesture operation to activate the camera to capture images in real time and display the shooting interface. In other embodiments, the electronic device 100 can also receive and respond to the user's touch operation on the camera activation control of a third application to activate the camera to capture images in real time and display the shooting interface.

[0026] like Figure 1B As shown, the electronic device 100 can display a shooting interface 1100. This shooting interface 1100 may include a shooting control 1101, one or more shooting mode controls (e.g., night scene mode control 1102A, video recording mode control 1102B, still image mode control 1102C, professional mode control 1102D, and more mode controls 1102E), a preview window 1103, etc. Currently, the still image mode control 1102C is selected, meaning the current shooting mode is still image mode. The preview window 1103 can be used to display a preview image.

[0027] In this embodiment of the application, the electronic device 100 receives a user's operation to start the camera (also referred to as a camera start operation), and in response to the camera start operation, displays the first frame preview image in the shooting interface. This time period is called the camera start time. For example, the camera start time can refer to: from Figure 1A The electronic device 100 shown receives a touch operation on the camera application icon 1001, and then... Figure 1B The electronic device shown is displaying the first frame preview image on the shooting interface 1100 during this time.

[0028] In some application scenarios, due to the long startup time of the camera on the electronic device 100, the electronic device 100 cannot display the first frame preview image on the shooting interface in a timely manner, which makes the electronic device 100 unable to respond to the user's needs quickly, thus bringing an extremely inconvenient experience to the user.

[0029] Therefore, this application provides a camera startup method that can be applied to an electronic device 100. The method includes: the electronic device 100 receiving a camera startup operation. In response to the camera startup operation, the electronic device 100 can start the camera to acquire raw image data in real time. The electronic device 100 can detect environmental parameter information and / or read function setting information from historical records. Then, the electronic device 100 can determine a corresponding dynamic image processing algorithm based on the aforementioned environmental parameter information and / or function setting information. The electronic device 100 can load the aforementioned dynamic image processing algorithm. The electronic device 100 can process the raw image data acquired in real time by the camera according to the aforementioned dynamic image processing algorithm to generate a first frame preview image. Next, the electronic device 100 can display the first frame preview image on the shooting interface.

[0030] The function settings information can be used to indicate that a specific function is enabled. Environmental parameter information may include one or more of the following: ambient light intensity, light source frequency, etc. Specified functions may include one or more of the following: background blur function, white balance function, zoom function, color temperature adjustment function, etc. Dynamic image processing algorithms may include one or more of the following: background blur algorithm, image brightness compensation algorithm, white balance algorithm, zoom algorithm, color temperature adjustment algorithm, etc.

[0031] In this way, during camera startup, the electronic device 100 can load the image processing algorithms required for the current scene and generate the first frame preview image based on the loaded algorithms. Image processing algorithms not needed for the current scene do not need to be loaded during camera startup. Therefore, the electronic device 100 can optimize the loading time of image processing algorithms during camera startup, reducing camera startup time and improving camera startup efficiency.

[0032] Figure 2 This is a specific flow of a camera startup method provided in an embodiment of this application.

[0033] like Figure 2 As shown, the specific process of the camera startup method provided in this application embodiment may include:

[0034] S201: Electronic device 100 receives camera start operation.

[0035] In this embodiment of the application, the camera startup operation can be as described above. Figure 1AThe touch operation shown on the camera application icon (e.g., clicking) can also be a user gesture operation (e.g., tapping the display screen of electronic device 100, long-pressing a designated area on the display screen of electronic device 100, etc.), or pressing a physical button on electronic device 100 (e.g., pressing the volume button on the side of electronic device 100, etc.). That is to say, the specific type of operation for camera activation is not limited in this embodiment.

[0036] S202: In response to the camera startup operation, the electronic device 100 loads one or more basic image processing algorithms and starts the camera to acquire raw image data in real time.

[0037] In this embodiment, the basic image processing algorithm can be interpreted as the image processing algorithm that the electronic device 100 automatically loads by default when responding to a camera startup operation. That is, the electronic device 100 does not need to perform subsequent steps of detecting environmental parameter information / reading function setting information to load the basic image processing algorithm. The basic image processing algorithm can be used to process the basic quality of the image. The basic image processing algorithm can be set to an identifier 1. When the electronic device 100 responds to a camera startup operation, it can determine the basic image processing algorithm based on identifier 1 and load the basic image processing algorithm.

[0038] In this embodiment of the application, when the electronic device 100 responds to the camera start operation and starts the camera to collect raw image data in real time, the electronic device 100 can collect each frame of raw image data through the camera at a specified frame rate. The specified frame rate can be 25 frames per second, 30 frames per second, or 50 frames per second, etc., and this application does not impose any limitations.

[0039] S203: Electronic device 100 can detect environmental parameter information corresponding to the current scene.

[0040] In this embodiment, the environmental parameter information may include one or more of the following: ambient light intensity, light source frequency, etc. The electronic device 100 can detect the environmental parameter information through one or more designated sensors. For example, the electronic device 100 can detect the ambient light intensity corresponding to the current scene through an ambient light sensor.

[0041] S204: Electronic device 100 can read function setting information from history records.

[0042] The function setting information indicates that a specific function is enabled. When the electronic device 100 reads a function setting information entry from the history, it can determine that the specified function corresponding to that function setting information is enabled. For example, when the electronic device 100 reads function setting information 1 from the history, it can determine that the specified function 1 corresponding to function setting information 1 is enabled. The specified function may include one or more of the following: background blur function, white balance function, zoom function, color temperature adjustment function, etc.

[0043] In this embodiment of the application, the description of the function setting information can be as follows: For example, during the Nth time the camera application is launched and run, the electronic device 100 can receive and respond to a user's touch operation (e.g., click) on a specified function control, and enable the specified function. When the electronic device 100 closes the camera application, the electronic device 100 can record the enabled specified function and store it as function setting information. When the electronic device 100 launches the camera application for the (N+1)th time, the electronic device 100 can read the function setting information in the history record and determine that a certain specified function was enabled before the (N+1)th time the camera application was launched. Here, "Nth time" and "N+1" are only used to describe the chronological order. "N+1" can also be replaced with "N+2" and "N+3", which is not limited in this application.

[0044] In the embodiments of this application, the electronic device 100 may implement only step S203, only step S204, or both steps S203 and S204. This application does not impose any restrictions on this.

[0045] S205: Electronic device 100 determines and loads one or more corresponding dynamic image processing algorithms based on environmental parameter information / function setting information.

[0046] In this step, one or more dynamic image processing algorithms are determined and loaded based on environmental parameter information / functional setting information, which can be referred to as dynamic image processing algorithm 1.

[0047] The dynamic image processing algorithm is different from the basic image processing algorithm. The dynamic image processing algorithm is not the default image processing algorithm automatically loaded by the electronic device 100 in response to camera startup. The dynamic image processing algorithm requires the electronic device 100 to trigger loading based on environmental parameter information / function setting information, or it is loaded after the electronic device 100 displays the first frame preview image. The electronic device 100 can set an identifier 2 for the dynamic image processing algorithm, which indicates that the algorithm with this identifier is a dynamic image processing algorithm.

[0048] In this embodiment, the electronic device 100 may pre-store a mapping table 1, which may include one or more environmental parameter information / function setting information, one or more dynamic image processing algorithms, and mapping relationships between one or more environmental parameter information / function setting information and one or more dynamic image processing algorithms. When the electronic device 100 detects environmental parameter information / reads function setting information, the electronic device 100 determines one or more dynamic image processing algorithms corresponding to the aforementioned environmental parameter information / function setting information according to the mapping table 1.

[0049] For example, mapping table 1 can be as shown in Table 1 below:

[0050] Table 1

[0051]

[0052]

[0053] As shown in Table 1, when electronic device 100 detects ambient light intensity A1, it can determine the corresponding image processing algorithm Q1 according to Table 1; when electronic device 100 detects light source frequency B1, it can determine the corresponding image processing algorithm Q2 according to Table 1; when electronic device 100 reads function setting information 1 from the historical record, it can determine the corresponding image processing algorithm Q3 according to Table 1. Image processing algorithms Q1, Q2, and Q3 can be collectively referred to as dynamic image processing algorithms.

[0054] It is understandable that the image processing algorithm corresponding to the function setting information also corresponds to the specified function. For example, when function setting information 1 corresponds to specified function 1, mapping table 1 records the image processing algorithm Q3 corresponding to function setting information 1, that is, the specified function 1 corresponds to image processing algorithm Q3.

[0055] Table 1 is merely an illustrative explanation of this application and does not constitute any limitation on this application.

[0056] S206: Electronic device 100 processes the raw image data captured by the camera based on one or more basic image processing algorithms and one or more loaded dynamic image processing algorithms to generate a first frame preview image.

[0057] Specifically, the electronic device 100 can connect the input interface of the aforementioned determined dynamic image processing algorithm 1 to the output interface of the basic image processing algorithm, and connect the output interface of the dynamic image processing algorithm 1 to the input interface of the basic image processing algorithm. The basic image processing algorithm connected to the input interface of the dynamic image processing algorithm 1 is different from the basic image processing algorithm connected to its output interface. Then, the electronic device 100 can process the raw image data captured by the camera based on one or more basic image processing algorithms and one or more loaded dynamic image processing algorithms 1 to generate a first frame preview image. Specific implementation methods can be found in subsequent embodiments and will not be elaborated here.

[0058] S207: Electronic device 100 displays the first frame preview image.

[0059] The electronic device 100 can display a first frame preview image on the shooting interface. This shooting interface can be, for example, as described above. Figure 1B The shooting interface 1100 shown can also be other user interfaces, and this application does not limit this.

[0060] In some embodiments, after generating the first frame preview image, the electronic device 100 may load a dynamic image processing algorithm 2. This dynamic image processing algorithm 2 may refer to an algorithm that processes subsequent frame preview images based on the data from the first frame preview image (e.g., image stabilization algorithm), or it may refer to other types of image processing algorithms; this application does not limit this. The electronic device 100 may process the raw image data captured by the camera based on dynamic image processing algorithm 1 and / or dynamic image processing algorithm 2 and a basic image processing algorithm to generate a second frame preview image. Then, the electronic device 100 may display the second frame preview image.

[0061] In some embodiments, after generating the second frame preview image, the electronic device 100 may load a dynamic image processing algorithm 3. This dynamic image processing algorithm 3 may refer to other dynamic image processing algorithms that have not yet been loaded, besides the already loaded dynamic image processing algorithms 1 and 2. The electronic device 100 may process the raw image data captured by the camera based on dynamic image processing algorithm 1 and / or dynamic image processing algorithm 2 and / or dynamic image processing algorithm 3 and a basic image processing algorithm to generate a third frame preview image. The electronic device 100 may display the third frame preview image. The processing methods for the fourth frame preview image, the fifth frame preview image, and subsequent frame preview images can be referenced to the third frame preview image and will not be repeated here. This improves the efficiency of the electronic device 100 in processing subsequent frame preview images.

[0062] In some embodiments, the electronic device 100 can determine, based on the parameter information of the dynamic image processing algorithms (e.g., the number of dynamic image processing algorithms, the computation time of one or more dynamic image processing algorithms, etc.), that the electronic device 100 needs to load all dynamic image processing algorithms in M ​​frames. Then, the electronic device 100 can load dynamic image processing algorithm 3 after generating the (M-1)th frame preview image; that is, after generating the (M-1)th frame preview image, the electronic device 100 has completed loading all dynamic image processing algorithms. The electronic device 100 can process the raw image data captured by the camera based on dynamic image processing algorithm 1 and / or dynamic image processing algorithm 2 and / or dynamic image processing algorithm 3 and basic image processing algorithms to generate the Mth frame preview image. The electronic device 100 can then display the Mth frame preview image.

[0063] Figure 3A This is a software architecture for an electronic device provided in an embodiment of this application.

[0064] like Figure 3A As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer, and the kernel layer.

[0065] The application layer can include a series of application packages. For example... Figure 3A As shown, the application layer can include applications such as calendar, memo, weather, gallery, map, and browser.

[0066] In this embodiment, the application layer may include an application with shooting functionality, such as a camera application. In some embodiments, when other applications need to use the shooting functionality, they can also call the camera application to implement the shooting functionality.

[0067] like Figure 3A As shown, the application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. The application framework layer includes predefined functions. It can include window managers, content providers, phone managers, resource managers, notification managers, etc.

[0068] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0069] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0070] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0071] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0072] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0073] In this embodiment, the application framework layer may further include a camera service (CameraService). The camera application can start the camera service by calling a preset API. During operation, the camera service can interact with the camera application in the application layer and the camera hardware abstraction layer (Camera HAL) in the hardware abstraction layer.

[0074] like Figure 3A As shown, the Android Runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0075] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0076] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0077] The system library can include multiple functional modules. For example: a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0078] like Figure 3A As shown, the hardware abstraction layer can include a camera hardware abstraction layer (Camera HAL), which may include a device control system, a detection module, a decision-making module, a basic image processing algorithm module, and a dynamic management module. The Camera HAL can interact with the hardware devices (such as cameras) that implement the shooting function in the electronic device 100. The Camera HAL can hide the implementation details of the relevant hardware devices while providing the Android system with interfaces for calling the relevant hardware devices. Specifically:

[0079] The device control system can be used to drive hardware devices such as cameras to acquire raw image data by calling the camera driver in the kernel layer. For example, the camera can pass each frame of raw image data acquired to the Camera HAL through the camera driver at a certain frame rate.

[0080] The detection module can be used to detect environmental parameter information and / or read function setting information from historical records. One or more designated sensors for detecting environmental parameter information can be integrated into the detection module. In some embodiments, when the detection module detects environmental parameter information, it can send the environmental parameter information to the decision module; when the detection module reads function setting information from historical records, it can send that function setting information to the decision module.

[0081] The decision module can receive environmental parameter information / function setting information sent by the detection module, and then determine the dynamic image processing algorithm required during the current camera startup process based on the aforementioned environmental parameter information / function setting information.

[0082] The DynamicPipelineManager module includes one or more dynamic image processing algorithms. That is, one or more dynamic image processing algorithms can be statically configured in the DynamicPipelineManager module. Each dynamic image processing algorithm can have its input and output interfaces set for connection to the input / output interfaces of other image processing algorithms. Specifically, the input interface of each dynamic image processing algorithm can be connected to the output interface of other image processing algorithms, and vice versa.

[0083] In this embodiment, when the decision module determines the dynamic image processing algorithm required during the current camera startup process, the decision module can drive the dynamic management module to load the dynamic image processing algorithm. The input interface of the dynamic image processing algorithm can be connected to the output interface of a loaded image processing algorithm, and its output interface can be connected to the input interface of a loaded image processing algorithm. Specific implementation details can be found in the descriptions of subsequent embodiments and will not be elaborated upon here.

[0084] The basic image processing algorithm module includes one or more basic image processing algorithms. These algorithms are loaded by default during camera startup; that is, the loading of these algorithms does not require the decision module to trigger the dynamic management module based on environmental parameter information / functional setting information.

[0085] One or more basic image processing algorithms in the basic image processing algorithm module correspond to an image processing pipeline. Figure 3A (Not shown in the image), each dynamic image processing algorithm in the decision module corresponds to an image processing pipeline. For example, image processing algorithm Q1 corresponds to pipeline Q1, image processing algorithm Q2 corresponds to pipeline Q2, and so on. The image processing pipeline can be used to manage the disconnection and connection of the corresponding image processing algorithm's input / output interface.

[0086] like Figure 3A As shown, the kernel layer is located below the hardware abstraction layer and serves as the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0087] Figure 3B This is a module interaction flow of a camera startup method provided in an embodiment of this application.

[0088] like Figure 3BAs shown, when an electronic device detects a camera startup operation, the camera application can generate a corresponding camera startup command. The camera application can then send this command to the device control system in the Camera HAL via the camera service. Upon receiving the camera startup command, the device control system can control the camera to acquire raw image data through the camera driver in the kernel layer.

[0089] The basic image processing algorithm module may include basic image processing algorithms. Figure 3B Examples of basic image processing algorithms include Algo1, Algo3, and Algo5; that is, Algo1, Algo3, and Algo5 are image processing algorithms loaded by default when the camera application starts. The electronic device 100 can connect Algo1, Algo3, and Algo5 through their respective interfaces according to a preset image processing algorithm execution order. For example, Figure 3B In the example, Algo1 is executed first, Algo3 is executed second, and Algo5 is executed third. Algo1, Algo3, and Algo5 can be located in the same pipeline.

[0090] The detection module can detect environmental parameter information in the current scene and / or read function setting information from historical records. Then, the detection module can send the environmental parameter information and / or function setting information to the decision module.

[0091] The decision-making module can determine the corresponding dynamic image processing algorithm based on environmental parameter information and / or function setting information. The specific implementation process can be found in the foregoing embodiments.

[0092] Once the decision module determines the corresponding dynamic image processing algorithm ( Figure 3B When the example is Algo2), the decision module can drive the dynamic management module to load the dynamic image processing algorithm. The dynamic management module can connect the dynamic image processing algorithm and the aforementioned basic image processing algorithm through their respective interfaces, according to the preset execution order of image processing algorithms in the electronic device 100. For example, as... Figure 3B As shown, if the electronic device 100 presets the execution order of Algo2 to be between Algo1 and Algo3, the dynamic management module can disconnect the interface connection between Algo1 and Algo3, then connect the input interface (in) of Algo2 to the output interface (out) of Algo1, and connect the output interface (out) of Algo2 to the input interface (in) of Algo3.

[0093] The dynamic management module may include a tag queue (configTopQueue) and a dynamic interface management module (DynamicPluginManager). When the electronic device 100 powers on, it can load the tag queue (configTopQueue). The tag queue can then store dynamic image processing algorithms (e.g., statically configured in the dynamic interface management module (DynamicPluginManager)). Figure 3B The Algo2 and Algo4 shown are cached in the label queue (configTopQueue). When the decision module determines one or more dynamic image processing algorithms (…), Figure 3B When Algo2 (as an example in the text) needs to be loaded, the decision module can drive the dynamic management module to load the dynamic image processing algorithm through the label queue (configTopQueue). Figure 3B The example in the example is Algo2).

[0094] The camera can transmit the captured raw image data to the camera driver, and then the camera driver can transmit the raw image data to the basic image processing module. Figure 3B The example shown is the input interface transmitted to Algo1. Basic image processing algorithms in the basic image processing module ( Figure 3B Examples in the example are Algo1, Algo3, and Algo5) and the loaded dynamic image processing algorithms ( Figure 3B The example in the text uses Algo2 to process the raw image data and generate a first-frame preview image. This first-frame preview image can then be transmitted to the camera application via the camera service.

[0095] Figure 3B The module interaction process described herein is merely an example to explain this application and does not constitute any limitation on this application.

[0096] In some embodiments, the electronic device 100 receives a camera activation operation. The electronic device 100 includes a first dynamic image processing algorithm and a second dynamic image processing algorithm. In response to the camera activation operation, the electronic device 100 activates the camera to acquire raw image data in real time. When it is determined that a first function is enabled, the electronic device 100 determines and loads the first dynamic image processing algorithm corresponding to the first function. The electronic device 100 processes the raw image data based on the first dynamic image processing algorithm to generate a first frame preview image. The electronic device 100 displays the first frame preview image. The electronic device 100 then loads the second dynamic image processing algorithm. The electronic device 100 processes the raw image data based on the first and second dynamic image processing algorithms to generate a second frame preview image. The electronic device 100 displays the second frame preview image.

[0097] In one possible implementation, before the electronic device 100 determines and loads the first dynamic image processing algorithm corresponding to the first function when it is determined that the first function has been enabled, the method further includes: the electronic device 100 loading a first basic image processing algorithm and a second basic image processing algorithm. The first and second basic image processing algorithms are used to process the basic quality of the image.

[0098] In one possible implementation, the electronic device 100 processes the original image data based on the first dynamic image processing algorithm to generate a first frame preview image, which includes: the electronic device 100 processes the original image data based on the first dynamic image processing algorithm, the first basic image processing algorithm, and the second basic image processing algorithm to generate a first frame preview image.

[0099] In one possible implementation, the electronic device 100 processes the original image data based on the first dynamic image processing algorithm and the second dynamic image processing algorithm to generate a second frame preview image, which includes: the electronic device 100 processes the original image data based on the first dynamic image processing algorithm, the second dynamic image processing algorithm, the first basic image processing algorithm, and the second basic image processing algorithm to generate a second frame preview image.

[0100] In one possible implementation, when it is determined that the first function has been enabled, the electronic device 100 determines and loads the first dynamic image processing algorithm corresponding to the first function by: connecting the input interface of the first dynamic image processing algorithm to the output interface of the first basic image processing algorithm, and connecting the output interface of the first dynamic image processing algorithm to the input interface of the second basic image processing algorithm.

[0101] In one possible implementation, before the electronic device 100 processes the original image data based on the first dynamic image processing algorithm to generate the first frame preview image, the method further includes: the electronic device 100 detecting environmental parameter information. The electronic device 100 determines and loads a third dynamic image processing algorithm corresponding to the environmental parameter information. The electronic device 100 processing the original image data based on the first dynamic image processing algorithm to generate the first frame preview image includes: the electronic device 100 processing the original image data based on the first dynamic image processing algorithm and the third dynamic image processing algorithm to generate the first frame preview image.

[0102] Figure 4 This is a hardware structure of an electronic device provided in an embodiment of this application.

[0103] like Figure 4As shown, the electronic device 100 may include a processor 401, a memory 402, a wireless communication module 403 (optional), a display screen 404, a camera 405, an audio module 406 (optional), and a microphone 407 (optional).

[0104] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] Processor 401 may include one or more processor units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0106] The processor 401 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 401 is a cache memory. This memory can store instructions or data that the processor 401 has just used or that are used repeatedly. If the processor 401 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 401, and thus improves the efficiency of the system.

[0107] In some embodiments, the processor 401 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, etc.

[0108] The memory 402 is coupled to the processor 401 and is used to store various software programs and / or multiple sets of instructions. In specific implementations, the memory 402 may include volatile memory, such as random access memory (RAM); it may also include non-volatile memory, such as ROM, flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 402 may also include combinations of the above types of memory. The memory 402 may also store program code so that the processor 401 can call the program code stored in the memory 402 to implement the implementation method of the present application embodiment in the electronic device 100. The memory 402 may store an operating system, such as uCOS, VxWorks, RTLinux, or other embedded operating systems.

[0109] The wireless communication module 403 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 403 can be one or more devices integrating at least one communication processing module. The wireless communication module 403 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 401. The wireless communication module 403 can also receive signals to be transmitted from the processor 401, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via the antenna. In some embodiments, the electronic device 100 can also communicate via the Bluetooth module in the wireless communication module 403 (… Figure 4 (not shown), WLAN module ( Figure 4 (Not shown) The device transmits signals to detect or scan devices near electronic device 100 and establishes wireless communication connections with those devices to transmit data. The Bluetooth module can provide solutions for one or more Bluetooth communication methods, including basic rate / enhanced data rate (BR / EDR) or Bluetooth Low Energy (BLE), and the WLAN module can provide solutions for one or more WLAN communication methods, including Wi-Fi direct, Wi-Fi LAN, or Wi-Fi softAP.

[0110] The display screen 404 can be used to display images, videos, etc. The display screen 404 may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 404, where N is a positive integer greater than 1.

[0111] Camera 405 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 405, where N is a positive integer greater than 1.

[0112] In this embodiment, camera 405 may include a TOF camera and an RGB camera. The TOF camera is used to capture a first image of the first scene, and the RGB camera is used to capture a second image of the first scene. For specific implementation details, please refer to... Figure 2 The description of the illustrated embodiments will not be repeated here.

[0113] The audio module 406 can be used to convert digital audio information into analog audio signal output, and can also be used to convert analog audio input into digital audio signal. The audio module 406 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 406 can also be disposed in the processor 401, or some functional modules of the audio module 406 can be disposed in the processor 401.

[0114] Microphone 407, also known as a "microphone" or "voice transducer," is used to collect sound signals from the environment surrounding the electronic device. This sound signal is then converted into an electrical signal, which undergoes a series of processing steps, such as analog-to-digital conversion, to obtain a digital audio signal that can be processed by the processor 401 of the electronic device. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to the microphone 407, inputting the sound signal into the microphone 407. The electronic device 100 may have at least one microphone 407. In some embodiments, the electronic device 100 may have two microphones 407, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, the electronic device 100 may have three, four, or more microphones 407, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0115] Electronic device 100 may also include a sensor module ( Figure 4 (Not shown in the image). The sensor module may include multiple sensor elements, such as a touch sensor (…). Figure 4 (Not shown in the image). A touch sensor can also be called a "touch device". A touch sensor can be placed on the display screen 404, and the touch sensor and the display screen 404 together form a touch screen, also called a "touchscreen". The touch sensor can be used to detect touch operations applied to or near it.

[0116] It should be noted that, Figure 4 The electronic device 100 shown is merely an illustrative explanation of the hardware structure of the electronic device provided in this application and does not constitute a specific limitation on this application.

[0117] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0118] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A camera startup method, characterized in that, include: The electronic device receives a camera start operation; wherein, the electronic device includes a first dynamic image processing algorithm and a second dynamic image processing algorithm; In response to the camera activation operation, the electronic device activates the camera to acquire raw image data in real time; When it is determined that the first function has been enabled, the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function; The electronic device processes the original image data based on the first dynamic image processing algorithm to generate a first frame preview image; The electronic device displays the first frame preview image; The electronic device loads a second dynamic image processing algorithm; The electronic device processes the original image data based on the first dynamic image processing algorithm, the second dynamic image processing algorithm, and the first frame preview image to generate the second frame preview image; wherein, the second dynamic image processing algorithm runs based on the data of the first frame preview image; The electronic device displays the second frame preview image.

2. The method according to claim 1, characterized in that, Before the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function when it is determined that the first function has been enabled, the method further includes: The electronic device loads a first basic image processing algorithm and a second basic image processing algorithm; wherein the first basic image processing algorithm and the second basic image processing algorithm are used to process the basic quality of the image.

3. The method according to claim 2, characterized in that, The electronic device processes the original image data based on the first dynamic image processing algorithm to generate a first frame preview image, including: The electronic device processes the original image data based on the first dynamic image processing algorithm, the first basic image processing algorithm, and the second basic image processing algorithm to generate a first frame preview image.

4. The method according to claim 2, characterized in that, The electronic device processes the original image data and generates the second frame preview image based on the first dynamic image processing algorithm, the second dynamic image processing algorithm, and the first frame preview image, including: The electronic device processes the original image data and generates a second frame preview image based on the first dynamic image processing algorithm, the second dynamic image processing algorithm, the first basic image processing algorithm, the second basic image processing algorithm, and the first frame preview image.

5. The method according to any one of claims 2-4, characterized in that, When it is determined that the first function has been enabled, the electronic device determines and loads the first dynamic image processing algorithm corresponding to the first function, including: The electronic device connects the input interface of the first dynamic image processing algorithm to the output interface of the first basic image processing algorithm, and connects the output interface of the first dynamic image processing algorithm to the input interface of the second basic image processing algorithm.

6. The method according to claim 1, characterized in that, Before the electronic device processes the original image data based on the first dynamic image processing algorithm to generate the first frame preview image, the method further includes: The electronic device detected environmental parameter information; The electronic device determines and loads the third dynamic image processing algorithm corresponding to the environmental parameter information; The electronic device processes the original image data based on the first dynamic image processing algorithm to generate a first frame preview image, including: The electronic device processes the original image data based on the first dynamic image processing algorithm and the third dynamic image processing algorithm to generate a first frame preview image.

7. The method according to claim 1, characterized in that, The first function includes one or more of the following: Background blur function, white balance function, zoom function, color temperature adjustment function.

8. The method according to claim 6, characterized in that, The environmental parameter information includes one or more of the following: Ambient light intensity and light source frequency.

9. An electronic device, characterized in that, include: One or more processors and one or more memories; the one or more memories are coupled to the one or more processors, the one or more memories being used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-8.

10. A chip system, characterized in that, It includes a processing circuit and an interface circuit, the interface circuit being used to receive code instructions and transmit them to the processing circuit, the processing circuit being used to execute the code instructions to cause the chip system to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The device stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

Images