Shooting method and electronic equipment

Abstract

The invention discloses a shooting method and electronic equipment. An operating system of the electronic equipment can combine a plurality of commands for underlying hardware and / or algorithms to realize a shooting function, and the shooting function is provided for each application of an application layer to call. Therefore, the operating system can shield complex hardware operation and algorithm of the bottom layer of the electronic equipment, and control of the application to the bottom layer is simplified. And moreover, the operation system provides a unified shooting function for each application, so that the consistency of the effect and the capability when each application uses the shooting function can be kept.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a shooting method and an electronic device. Background Art

[0002] Electronic devices such as mobile phones, tablet computers, and laptop computers are equipped with cameras and can provide a shooting function. Providing a more user-friendly and more consistent shooting method for users is the future research direction. Summary of the Invention

[0003] This application provides a shooting method and an electronic device, which can maintain the consistency of the effects and capabilities when each application uses the shooting function.

[0004] In a first aspect, a shooting method is provided. The method is applied to a first electronic device on which a first operating system runs. The method may include: running a first application on the first operating system. The first application is installed on the first electronic device and is a third-party application. The first application provides N shooting functions. The N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for calling hardware and / or algorithms related to shooting. The first application includes program code for respectively calling N shooting function interfaces. One shooting function interface is used to start one shooting function. The N shooting function interfaces are provided by the first operating system; detecting a user operation acting on a first shooting function option in the user interface of the first application. The first shooting function option corresponds to a first shooting function, and the first shooting function belongs to the N shooting functions; starting the first shooting function.

[0005] Through the method of the first aspect, the operating system of the electronic device can combine multiple commands for underlying hardware and / or algorithms to implement shooting function interfaces and provide the shooting function interfaces for each application in the application layer to call. In this way, the operating system can shield the complex hardware operations and algorithms at the bottom layer of the electronic device and simplify the control of the application to the bottom layer. Moreover, the operating system provides a unified shooting function for each application, which can maintain the consistency of the effects and capabilities when each application uses these shooting functions.

[0006] In combination with the first aspect, in some embodiments, the first application further includes program code for calling a first interface. The first interface encapsulates the N shooting function interfaces. The first interface is used to select and run the first shooting function interface from the N shooting function interfaces according to a first scene description parameter. The first interface is provided by the first operating system, and the first scene description parameter indicates the first shooting function.

[0007] In this way, the operating system of the electronic device can further encapsulate the N shooting function interfaces into a first interface for each application in the application layer to call. In this way, the operating system can shield the complex hardware operations and algorithms at the bottom layer of the electronic device, simplifying the control of the application over the bottom layer. Moreover, the operating system provides a unified shooting function to each application, which can maintain the consistency of the effects and capabilities when each application uses these shooting functions.

[0008] Combined with the first aspect, in some embodiments, the first operating system includes: an application layer, a framework layer, a service layer, and a kernel layer. The application layer includes a first application. The N shooting function interfaces are specifically provided by the framework layer of the first operating system. The N shooting function interfaces encapsulate one or more service layer interfaces, and one or more service layer interfaces are provided by the service layer. The service layer interfaces encapsulate one or more kernel layer interfaces, and one or more kernel layer interfaces are provided by the kernel layer. One or more kernel layer interfaces encapsulate the underlying capabilities, and the underlying capabilities include basic capabilities and differentiated capabilities. The basic capabilities are the capabilities shared by multiple platforms, and the differentiated capabilities are the capabilities differentiated by multiple platforms.

[0009] In some embodiments, starting the first shooting function specifically includes: calling, through the first application, the first shooting function interface corresponding to the first shooting function; running the first shooting function interface in the framework layer; running the service layer interface corresponding to the first shooting function interface in the service layer; and running the kernel layer interface corresponding to the first shooting function interface in the kernel layer.

[0010] In some embodiments, the first operating system further includes a media data interface in the framework layer. After the electronic device runs the first shooting function interface in the framework layer, the first shooting function interface can call this media data interface, run this media data interface in the framework layer, and then this media data interface can continue to call the service layer interface and the kernel layer interface below.

[0011] Combined with the first aspect, in some embodiments, the parameters passed into the shooting function interface include one or more of the following: device name, application name of the first application, parameters passed in by the first application. Among them, the parameters passed in by the first application are in the form of key-value. Among them, the key represents the shooting function corresponding to the shooting function interface, and the value includes the parameter content of the shooting function corresponding to the shooting function interface.

[0012] Second aspect, a shooting method is provided. This method is applied to a first electronic device on which a first operating system runs. The method may include: running a first application on the first operating system. The first application is installed on the first electronic device and is a third-party application. The first application provides N shooting functions, and the N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for invoking hardware and / or algorithms related to shooting. The first application includes program code for invoking a first interface. The first interface encapsulates N shooting function interfaces. One shooting function interface is used to start one shooting function. The first interface is used to select and run a first shooting function interface from the N shooting function interfaces according to first scene description parameters. The first interface and the N shooting function interfaces are provided by the first operating system. The first scene description parameters indicate the first shooting function; detecting a user operation on the first shooting function option in the user interface of the first application. The first shooting function option corresponds to the first shooting function, and the first shooting function belongs to the N shooting functions; starting the first shooting function.

[0013] In combination with the second aspect, in some embodiments, the first application further includes program code for respectively invoking the N shooting function interfaces.

[0014] In combination with the second aspect, in some embodiments, the first operating system includes: an application layer, a framework layer, a service layer, and a kernel layer. The application layer includes the first application. The first interface and the N shooting function interfaces are specifically provided by the framework layer of the first operating system. The N shooting function interfaces encapsulate one or more service layer interfaces, and the one or more service layer interfaces are provided by the service layer. The service layer interfaces encapsulate one or more kernel layer interfaces, and the one or more kernel layer interfaces are provided by the kernel layer. The one or more kernel layer interfaces encapsulate underlying capabilities, and the underlying capabilities include basic capabilities and differentiated capabilities. The basic capabilities are capabilities shared by multiple platforms, and the differentiated capabilities are capabilities that are different among multiple platforms.

[0015] In some embodiments, starting the first shooting function specifically includes: invoking the first interface through the first application and passing the first scene description parameters to the first interface; running the first interface in the framework layer; running the first shooting function interface according to the first scene description parameters in the framework layer; running the service layer interface corresponding to the first shooting function interface in the service layer; running the kernel layer interface corresponding to the first shooting function interface in the kernel layer.

[0016] In combination with the second aspect, in some embodiments, the parameters passed into the first interface include one or more of the following: device name, application name of the first application, parameters passed in by the first application. Among them, the parameters passed in by the first application are in the form of key-value pairs, where the key represents the shooting function, and the value includes the parameter content of the shooting function.

[0017] Combined with the first aspect or the second aspect or any of the above embodiments, in some embodiments, the first shooting function is a photographing function, and the parameters passed in by the first application include one or more of the following: photographing stream, preview stream; or, the first shooting function is a face recognition function, and the parameters passed in by the first application include: indication information of the face recognition function, coordinates of the face, image of the face; or, the first shooting function is a beauty function, and the parameters passed in by the first application include: indication information of the beauty function, beauty level, face information.

[0018] Combined with the first aspect or the second aspect or any of the above embodiments, in some embodiments, the first electronic device can turn on multiple cameras in advance, store the data streams of the multiple cameras in a pipeline, and the pipeline is a buffer. In this way, the first electronic device can obtain the data stream of the first camera from the pipeline, process the data stream of the first camera, and use it to complete the first shooting function. In this way, the coupling relationship between the camera and the application can be decoupled.

[0019] In some embodiments, if the electronic device detects that the first camera is damaged, it can obtain the data stream of the second camera from the pipeline, process the data stream of the second camera, and use it to complete the first shooting function. In some embodiments, before obtaining the data stream of the second camera from the pipeline, the electronic device can first obtain the remaining data stream of the first camera from the pipeline, process the data stream of the first camera, and use it to complete the first shooting function. In some embodiments, the matching degree between the first camera and the first application is higher than that between the second camera and the first application. This can avoid the situation where the first application cannot obtain the data stream, avoid the black screen of the user interface of the first application, and give the user a better use experience.

[0020] In some embodiments, the method may further include: obtaining the data stream of the first camera from the pipeline, processing the data stream of the first camera, and using it to complete the second shooting function for starting the second application; or, obtaining the data stream of the second camera from the pipeline, processing the data stream of the second camera, and using it to complete the second shooting function for starting the second application, and the second application is installed on the first electronic device. Equivalently, both the second application and the first application can obtain the data stream from the pipeline, and the two can obtain the same data stream or different data streams.

[0021] Combined with the first aspect or the second aspect or any of the above embodiments, in some embodiments, the first shooting function is specifically a photographing function. After starting the first shooting function, the method may further include: saving the basic quality map of the first picture to the picture library; updating the basic quality map in the picture library to a full quality map, and the quality of the full quality map is higher than that of the basic quality map. This way of generating images in two steps can improve the image output speed and enhance the user experience.

[0022] In some embodiments, after starting the first shooting function, the method may further include: within a first time period after starting the first shooting function, detecting an operation of opening a first picture in the picture gallery, where the first time period is less than a first threshold; first displaying a basic quality picture of the first picture; and after generating a full-quality picture, displaying the full-quality picture. The first time period is the time period from when the electronic device starts the camera function to when the full-quality picture is generated. That is to say, after the electronic device detects an operation of opening the first picture in the picture gallery, if the full-quality picture has not been generated yet, the basic quality picture is first displayed, and after the full-quality picture is generated, the full-quality picture is displayed.

[0023] Combined with the first aspect or the second aspect or any of the above embodiments, in some embodiments, the first shooting function is specifically a video recording function. After starting the first shooting function, the method may further include: saving a basic quality video of the first video to the picture gallery; and updating the basic quality video in the picture gallery to a full-quality video, where the quality of the full-quality video is higher than that of the basic quality video. This way of generating the video in two times can improve the speed of the output video and enhance the user experience.

[0024] In some embodiments, after starting the first shooting function, the method may further include: within a first time period after starting the first shooting function, detecting an operation of opening a first video in the picture gallery, where the first time period is less than a first threshold; first displaying a basic quality video of the first video; and after generating a full-quality video, displaying the full-quality video.

[0025] Combined with the first aspect or the second aspect or any of the above embodiments, in some embodiments, the first shooting function is one of the following: a camera function, a video recording function, a night scene function, a portrait function, a large aperture function, a time-lapse photography function, an AI scene recognition function, a barcode scanning function, a face recognition function, and a face interaction function.

[0026] In a third aspect, there is provided an electronic device, which includes: a memory, a processor, and a computer program stored on the memory. The computer program includes a first operating system. The first electronic device is installed with a first application, where the first application is a third-party application. The first application provides N shooting functions, and the N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program codes for calling hardware and / or algorithms related to shooting. The first application includes program codes for respectively calling N shooting function interfaces. One shooting function interface is used to start one shooting function, and the N shooting function interfaces are provided by the first operating system; the processor executes the computer program to implement the method provided in the first aspect or any of the embodiments of the first aspect.

[0027] Fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes a first operating system. A first electronic device is installed with a first application, where the first application is a third-party application. The first application provides N shooting functions, and the N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for invoking hardware and / or algorithms related to shooting. The first application includes program code for respectively invoking N shooting function interfaces, where one shooting function interface is used to start one shooting function, and the N shooting function interfaces are provided by the first operating system; when the computer program is executed by a processor, it implements the method provided in the first aspect or any implementation manner of the first aspect.

[0028] Fifth aspect, a computer program product is provided. The computer program product includes a computer program, and the computer program includes a first operating system. A first electronic device is installed with a first application, where the first application is a third-party application. The first application provides N shooting functions, and the N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for invoking hardware and / or algorithms related to shooting. The first application includes program code for respectively invoking N shooting function interfaces, where one shooting function interface is used to start one shooting function, and the N shooting function interfaces are provided by the first operating system; when the computer program is executed by a processor, it implements the method provided in the first aspect or any implementation manner of the first aspect. Description of the Drawings

[0029] Figure 1 Schematic diagram of the camera architecture provided for Android;

[0030] Figure 2A Schematic diagram of the unified shooting function provided by the embodiments of the present application;

[0031] Figure 2B Schematic diagram of different applications sharing the same shooting function provided by the embodiments of the present application;

[0032] Figure 2C Schematic diagram of different applications using the shooting function differently provided by the embodiments of the present application;

[0033] Figure 3A Schematic diagram of the camera framework provided by the embodiments of the present application;

[0034] Figure 3B Schematic diagram of an application invoking the shooting function provided by the embodiments of the present application;

[0035] Figure 4 Structural diagram of the OS of the electronic device provided by the embodiments of the present application;

[0036] Figure 5AStructural diagram of another OS for the electronic device provided by the embodiment of the present application;

[0037] Figure 5B Specific example of the camera application directly calling the photographing function interface provided by the embodiment of the present application;

[0038] Figure 6 Schematic diagram of different applications calling the shooting function provided by the embodiment of the present application;

[0039] Figure 7 Schematic diagram of the shooting effects of different applications provided by the embodiment of the present application;

[0040] Figure 8 Comparison diagram of the shooting processes of different applications provided by the embodiment of the present application;

[0041] Figure 9 Session management model provided by the embodiment of the present application;

[0042] Figure 10 Schematic diagram of managing data flow based on pipelines provided by the embodiment of the present application;

[0043] Figure 11 Shows the processing method when the data flow is disconnected or abnormal;

[0044] Figure 12 Flowchart of segmented shooting provided by the embodiment of the present application;

[0045] Figure 13 Flowchart of the shooting method provided by the embodiment of the present application;

[0046] Figure 14 Hardware structure block diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0047] Figure 1Shows the camera architecture related to shooting provided by Android. This camera architecture includes the following modules: camera interface (application program interface, API), camera framework, and camera hardware device. The camera hardware device refers to the hardware used for shooting and subsequent processing, which may include, for example, an optical sensor, a lens, an image signal processor (ISP), a digital signal processor (DSP), etc. The camera framework is a system-level service responsible for managing the access and control of the camera hardware device. The camera interface is the API provided to the application (APP) in the application layer. The APP can interact with the camera framework through these APIs to control the camera hardware and obtain a data stream, which can also be referred to as an image stream.

[0048] The control flow of the camera architecture provided by Android is relatively simple, and the APIs it provides only support outputting basic data streams (which can also be referred to as traditional data streams). The basic data streams may include, for example: preview stream, photo stream, video stream. The parameters such as frame rate and resolution of the above several basic data streams may be different. Each APP in the application layer calls the APIs of the camera framework and can only obtain the above basic streams, and then further processes the above basic data streams to obtain the data stream that the APP really wants. For example, when the APP turns on the large aperture function, the APP can first obtain the above basic data stream, and then perform processing such as exposure adjustment, noise reduction, color correction, and white balance on the basic stream to obtain the desired data stream with the large aperture effect.

[0049] When the APP calls the Android camera architecture, it can call commands for the underlying hardware and / or algorithms. Based on the basic data stream provided by this camera architecture, it can implement the shooting functions that the APP itself wants. The shooting functions refer to the functions that require the use of the camera, such as the large aperture function, the portrait function, etc. Here, the commands can refer to the atomic capabilities provided by the underlying hardware and / or algorithms. The hardware called by the above commands may include hardware related to shooting such as cameras, GPUs, ISPs, DSPs, etc. The algorithms called by the above commands may include various algorithms related to image processing, such as image denoising algorithms, image transformation algorithms, image compression algorithms, image blur processing algorithms, etc. These hardware can be called southbound hardware, and these algorithms can also be called southbound algorithms, which can be provided by other manufacturers other than the electronic device manufacturer, and of course can also be provided by the electronic device manufacturer.

[0050] The method of implementing the shooting function based on the Android camera architecture has the following disadvantages: Each APP in the electronic device needs to face the underlying hardware and / or algorithms and write code independently to control the underlying hardware and / or algorithms to achieve the shooting function it wants, which requires a complex development process. Even if two APPs want to provide the same shooting function, they need to write code separately to implement it, and the development processes of each APP are repetitive. Due to different coding methods of each APP, for the same shooting function, different APPs may present different effects, which is not conducive to the unity of the effects and capabilities of each APP on the electronic device.

[0051] An embodiment of the present application provides a method for providing a unified shooting function for each APP, and this method is implemented by the operating system (OS) of the electronic device.

[0052] Figure 2A Exemplarily shows a way to provide a unified shooting function. As Figure 2A shown, the OS can combine one or more commands for the underlying hardware and / or algorithms to implement a shooting function. The shooting function can be regarded as a unified abstraction of the hardware and / or software capabilities. Compared with the commands directly facing the underlying hardware and / or algorithms, the shooting function is an upper-layer concept abstracted based on the commands. The shooting function can include, for example, but is not limited to: focus function, zoom function, function of switching front and rear cameras, flash function, beauty function, frame rate function, night scene function, high-dynamic range (HDR) image function, etc.

[0053] As Figure 2A shown, the OS can combine one or more commands for the underlying hardware to implement a shooting function. For example, the following three commands can be combined to implement the focus function: the command to obtain the screen focus coordinates, the command to obtain the preview stream according to the screen focus coordinates, and the command to perform focus on the screen focus on the preview stream. The OS can also combine one or more commands for the underlying algorithms to implement a shooting function. The OS can also combine the commands for the underlying hardware and the commands for the underlying algorithms to implement a shooting function.

[0054] It Figure 2A can be seen that in the method provided by the present application, the commands for the underlying hardware and / or algorithms can be converted into shooting functions, and these shooting functions are uniformly provided to each APP accessing northward. Each APP at the application layer only needs to use the combined shooting functions without caring about the specific commands for calling the underlying hardware or algorithms. In this way, when developing each APP, developers only need to face the explicit shooting functions instead of the implicit command sets, which can make the development of each APP simpler.

[0055] Commands for underlying hardware and / or algorithms are single and independent and cannot be shared among different APPs. However, through the method of providing a unified shooting function in this application, various shooting functions provided by the OS can be shared among different APPs, and different APPs can call the same shooting function, thus expanding the application scope of the underlying hardware and / or algorithms in the electronic device. As Figure 2B shown, the OS combines several commands for scheduling the underlying hardware to form a command group A, and then encapsulates the command group A as a shooting function A, which can be provided for use by Application A and Application B.

[0056] Some APPs may need to differentiate their own capabilities. They can perform some personalized processing on the basis of the unified shooting function provided by the OS, so as to provide some personalized functions. Refer to Figure 2C , Figure 2C which is a schematic diagram of different APPs using the unified shooting function provided by the OS in a differentiated manner. As Figure 2C shown, if an APP needs a differentiated effect presentation, it can perform some personalized processing on the basis of the shooting function provided by the OS, so as to provide the personalized functions required by the APP. In other embodiments, the application can also be oriented to the underlying hardware and / or algorithms, that is, develop the personalized functions required by itself for the atomic capabilities.

[0057] In the way of providing a unified shooting function in this application, for the capabilities of the underlying hardware and / or algorithms, they are unified into a shooting function at the functional level and can be called by multiple APPs. Moreover, the shooting functions in this application can be provided based on user scenarios, such as the commonly used focusing function, beauty function, night scene function, etc., which is convenient for APP developers to understand. The OS provides a unified shooting function, so the overall architecture is normalized and has good scalability. For example, the OS can add new shooting functions according to actual needs.

[0058] With the development of electronic devices and various APPs, each APP can provide some shooting functions with richer capabilities for users, and the shooting functions provided by each APP for users are becoming increasingly similar. These shooting functions usually provide corresponding shooting function options on the user interface of the APP for users to use these shooting functions conveniently and directly. These shooting functions can include, but are not limited to: photo-taking function, video-recording function, night scene function, portrait function, large aperture function, time-lapse photography function, artificial intelligence (AI) scene recognition function, barcode scanning function, face recognition function, face interaction function, etc.

[0059] Based on this, the OS of the electronic device can provide unified shooting functions and offer these shooting functions to each APP accessing from the northbound, enabling each APP at the application layer to use the combined shooting functions as needed without having to concern themselves with the specific shooting functions or the commands for invoking the underlying hardware or algorithms. When providing the above-mentioned unified shooting functions, the OS can set the cameras to be used for these functions as well as the shooting parameters. In this way, when developing each APP, developers can focus on the shooting functions at the upper layer without having to deal with the detailed shooting functions, making the development of each APP simpler.

[0060] From the above introduction, it can be seen that the commands for the underlying hardware and / or algorithms and the shooting functions can be regarded as a relationship of encapsulation layer by layer upwards. The shooting functions are obtained by the OS through the conversion of the commands for the underlying hardware and / or algorithms.

[0061] The OS provided by this application can shield the complex hardware operations and algorithms at the bottom layer of the electronic device and simplify the control of the APP over the bottom layer. Moreover, the OS provides unified shooting functions to each APP at the application layer, which can maintain the consistency of the effects and capabilities when each APP uses these shooting functions.

[0062] Figure 3A An exemplary camera framework provided by this application is shown.

[0063] As Figure 3A shown, the bottom layer of the camera framework provides a unified ecological interface for the camera hardware devices and / or algorithms, which is compatible with the camera hardware devices provided by each manufacturer and can also be compatible with the algorithms provided by different manufacturers. The bottom layer interface is used to schedule the underlying hardware and / or algorithms. Based on the bottom layer interface, the camera framework of this application constructs the commands for the underlying hardware and / or algorithms upwards into various shooting functions. The shooting functions can be divided into basic functions and advanced functions for example. The camera framework can also provide these shooting functions to each application at the application layer in the form of an interface (API) for use. In some embodiments, the basic functions can be provided for system applications and third-party applications to call, and the advanced functions can be limited to being provided for system applications to call. Of course, in other embodiments, the advanced functions can also be provided for third-party applications to use.

[0064] Advanced functions are relatively more complex to implement compared to basic functions, and may provide richer or more targeted functions. Both basic functions and advanced functions can include multiple shooting functions. For example, basic functions may include functions such as taking photos and recording videos, while advanced functions may include functions such as night scene function, portrait function, large aperture function, time-lapse photography function, AI scene recognition function, barcode scanning function, face recognition function, and face interaction function. The photo-taking function provides the function of taking photos, the video-recording function provides the function of recording videos, the night scene function provides special processing functions in night scene scenarios (such as shooting with supplementary light), and the portrait function provides functions such as skin beautification of portraits, foreground highlighting, and background blurring. The distinction between basic functions and advanced functions is not fixed, and can be customized by the electronic device or by developers of device manufacturers.

[0065] From Figure 3A the camera framework shown, it can be seen that this camera framework is compatible with the northward application ecosystem and southward device manufacturers, allowing the hardware provided by each device manufacturer to be connected to this framework, and also allowing each application to be connected to this framework to use the unified shooting functions provided by this framework.

[0066] Figure 3B , Figure 3B Exemplarily shows the process of an APP calling the shooting functions uniformly provided by the OS. As Figure 3B shown, the APP can start the camera and then start a certain shooting function such as the video-recording function. After the OS receives the instruction from the APP to call the video-recording function, it can decompose this video-recording function into some clear shooting operations, such as preview, recording, zooming, focusing, beautifying, etc., and these operations are all uniformly implemented by the OS.

[0067] The following will introduce the implementation of the unified shooting function in combination with the OS of the electronic device provided in the embodiments of the present application.

[0068] An OS can run on the electronic device provided in the embodiments of the present application, and this OS can be one of etc.

[0069] Figure 4 Shows the software structure of the OS of the electronic device provided in the embodiments of the present application. This software structure includes a unified media framework. The unified media framework is used to implement the unified shooting functions mentioned above.

[0070] As Figure 4 shown, the OS can include the following four layers from top to bottom: application layer, framework layer, service layer, and kernel layer. The higher the layer, the more interactions with the user; the lower the layer, the more system capabilities it represents. Figure 4 The OS shown is only an example, and it may also include more or fewer modules, which are not limited here.

[0071] The application layer includes a series of applications, such as camera, recorder, gallery, file manager, calendar, call, map, wireless local area network (WLAN), Bluetooth, music, video, short message and other applications (not shown in the figure). Figure 4 As shown, the application layer may include some apps with shooting functions, such as system camera apps, third-party camera apps (such as Meitu XiuXiu), WeChat, Taobao, etc.

[0072] The framework layer can also be called the program framework layer, which is used to provide APIs and programming frameworks for the application programs in the application layer. The program framework layer may include some predefined functions. The framework layer may also include modules such as window manager, content provider, view system, phone manager, resource manager, etc., which will not be introduced here.

[0073] like Figure 4 As shown, the framework layer may include: a unified shooting function interface (API). These unified shooting function interfaces are the encapsulation form of the unified shooting function provided by this application. The definition of the unified shooting function can refer to the relevant description above. According to the capabilities provided by the shooting function interface, these unified shooting function interfaces can be divided into two categories: basic function interfaces and advanced function interfaces. Compared with the basic function interface, the advanced function interface is more complex to implement, and the functions provided may be richer or more targeted. Both the basic function interface and the advanced function interface may include multiple shooting function interfaces. For example, the basic function interface may include a photo function interface, a video recording function interface, etc., and the advanced function interface may include a night scene function interface, a portrait function interface, a large aperture function interface, a time-lapse photography function interface, an AI scene recognition function interface, a code scanning function interface, a face recognition function interface, a face interaction function interface, etc.

[0074] In some implementations, the basic function interface can be provided to system applications and third-party applications for calling, and the advanced function interface can be limited to being provided to system applications for calling. Of course, in other implementations, the advanced function interface can also be provided to third-party applications for use.

[0075] The APP in the application layer can call the shooting function interface in the framework layer. The APP can interact with the modules of the service layer through the interface to control the underlying hardware and algorithms, obtain the data stream and process the data stream to obtain the result data of the corresponding shooting function interface. When the shooting function interface in the framework layer is called by the APP in the application layer, it can further call other interfaces provided by the framework layer, and further interact with the service layer and the kernel layer through other interfaces to realize the corresponding shooting function. The calling process of the shooting function interface in the framework layer can refer to the introduction of the subsequent embodiments, which will not be expanded here.

[0076] As shown Figure 4 in the figure, the framework layer may further include: an image algorithm interface and other algorithm interfaces. The image algorithm interface is provided to each APP in the application layer. The APP can interact with the image algorithm engine in the service layer through this image algorithm interface, so as to control the operation of the image algorithm. The other algorithm interfaces are provided to each APP in the application layer. The APP can interact with the image algorithm engine in the service layer through this other algorithm interface, so as to control the operation of other algorithms other than the image algorithm. The image algorithm may include, for example, an image denoising algorithm, an image transformation algorithm, an image compression algorithm, an image blurring processing algorithm, etc.

[0077] Through each interface in the framework layer provided by this application, the minimalist use of the camera function can be realized, and the consistency of the ecological effect can be maintained. Specifically, the following functions can be included: shielding complex underlying hardware operations and algorithms, simplifying the programming work of application developers; abstracting for the shooting function and uniformly providing shooting function interfaces for each APP to call; standardizing business interfaces and business processes; designing interfaces explicitly for the business to avoid unauthorized extension of interfaces by applications.

[0078] The unified media framework provided by this application may include various interfaces related to the shooting function provided by the framework layer, such as a unified shooting function interface and algorithm interfaces, etc. The functions of the unified media framework and each of the above-introduced modules have been clearly introduced above, and their names do not constitute a limitation. Each of the above modules may also be called other names. The unified media framework provided by the embodiments of this application layers the interfaces and services, allowing APP developers to only need to focus on the shooting functions provided by the interfaces and not need to focus on the complex implementation of underlying commands.

[0079] As shown Figure 4 in the figure, the service layer may include: a camera management service, basic capabilities, advanced capabilities, an image algorithm engine, other algorithm engines, etc. The camera management service is used to manage the business logic related to shooting. The basic capabilities provide the business logic of the basic function interfaces. The advanced capabilities provide the business logic of the advanced shooting function interfaces. The image algorithm engine is used to provide the operation and management of the image algorithm. It provides basic atomic image algorithms and supports personalized image processing. The other algorithm engines can be used to provide the operation and management of other algorithms other than the image algorithm, such as some deep learning algorithms, etc. The other algorithm engines are used to provide the business logic of other algorithms other than the image algorithm.

[0080] The unified media framework of this application can be implemented as a system capability, or as a software development kit (SDK), a resident service, a binary shared object (SO) file, etc. Among them, the resident service can be regarded as a resident process that is always on after the electronic device is powered on. The SO capability can be regarded as a capability provided by the underlying layer of the electronic device and is started when needed.

[0081] The kernel layer is the layer between hardware and software. As Figure 4 shown, the kernel layer may include the following modules: camera driver, display driver, audio driver, sensor driver, underlying algorithms, etc. The underlying algorithms may include image algorithms and other algorithms other than image algorithms.

[0082] Figure 5A Another form of the electronic device OS is exemplarily shown.

[0083] The OS of the electronic device also includes four layers: application layer, framework layer, service layer, and kernel layer. The application layer may include various application programs, such as system applications, third-party applications, etc.

[0084] The framework layer may include: camera system interface, camera picker. Both the camera system interface and the camera picker are provided by the unified media framework of the OS.

[0085] The camera system interface may include a shooting function interface uniformly provided by the OS. The camera system interface can be provided for each application in the application layer to call, keeping the interfaces called by each application consistent.

[0086] The camera picker can be called by each application in the application layer. The camera picker encapsulates N shooting function interfaces, and one shooting function interface is used to start one shooting function. The N shooting function interfaces encapsulated in the camera picker are different from the shooting function interfaces in the camera system interface, and their functions are more abundant or more targeted. Usually, the N shooting functions encapsulated in the camera picker usually provide corresponding shooting function options on the user interface of the APP for users to conveniently and directly use these shooting functions. The N shooting functions may include, for example, but are not limited to: photo-taking function, video-recording function, night scene function, portrait function, large aperture function, time-lapse photography function, AI scene recognition function, barcode scanning function, face recognition function, face interaction function, etc. And the shooting function interfaces included in the camera system interface provide simpler functions, such as a focus function interface, a zoom function interface, and a function interface for switching front and rear cameras.

[0087] The camera picker can be implemented as an externally exposed interface (such as the first interface) or as a system service. Applications in the application layer can call the camera picker as needed. The camera picker shields the applications from direct access to the underlying hardware and / or algorithms. When each application in the application layer calls the camera picker, a unified shooting function is provided through the camera picker, so that a consistent experience can be ensured when each application uses these shooting functions.

[0088] Taking the first application running in the OS of an electronic device calling the camera picker as an example, the process of the electronic device starting the first shooting function in the first application is introduced below. The first application provides N shooting functions, and there are corresponding shooting function options on the user interface of the first application. The N shooting functions can include, for example, but are not limited to: photo-taking function, video-recording function, night scene function, portrait function, large aperture function, time-lapse photography function, AI scene recognition function, barcode scanning function, face recognition function, face interaction function, etc. The first application includes program code for calling the camera picker (i.e., the first interface). During the running of the first application, the electronic device can detect a user operation acting on the first shooting function option. The first shooting function option corresponds to the first shooting function, and the first shooting function belongs to the N shooting functions corresponding to the N shooting function interfaces encapsulated by the camera picker. After that, the first application can call the camera picker and transmit a first scene description parameter to the camera picker. The first scene description parameter can be used to describe the shooting scene currently started by the first application or can be used to indicate the first shooting function. For example, it can indicate scenes such as photo-taking, video-recording, night scene, portrait, barcode scanning, face recognition, etc. After receiving the first scene description parameter, the camera picker can select and run the first shooting function interface from the N shooting function interfaces according to the first scene description parameter.

[0089] In this way, each application in the application layer only needs to call the camera picker and transmit the scene description parameter to it, so that the camera picker of the OS can start the corresponding shooting function for the application, simplifying the application operation and ensuring the consistency of the effects of starting the shooting function for each application.

[0090] In some embodiments, when the camera picker is called by an application in the application layer, it can require the caller to provide the device name and the application name. In this way, the camera picker can verify the legitimacy of the caller. When the caller is legitimate, it can provide the required shooting function for it. When the caller is not legitimate, it can refuse to provide the required shooting function for it, so as to ensure device security.

[0091] In some embodiments, the N shooting function interfaces encapsulated by the camera picker can also be provided to various applications in the application layer. For example, the first application may include program codes for separately invoking the N shooting function interfaces. During the running of the first application, the electronic device can detect a user operation acting on the first shooting function option, where the first shooting function option corresponds to the first shooting function, and the first shooting function belongs to the N shooting functions corresponding to the N shooting function interfaces encapsulated by the camera picker. After that, the first application can directly invoke the first shooting function interface corresponding to the first shooting function. In this way, the applications in the application layer can directly invoke the shooting functions provided by the OS according to their own needs.

[0092] Whether the application in the application layer first invokes the camera picker and then invokes the shooting function interface through the camera picker, or the application directly invokes the shooting function interface, the shooting function corresponding to the shooting function interface can be realized. The shooting function corresponding to the shooting function interface is implemented by the OS and is usually realized by calling the interfaces layer by layer downwards.

[0093] Figure 5B Taking the latter calling method as an example, a specific example of a camera application directly invoking the take photo function interface is shown. As Figure 5B shown, after receiving the user operation to start the take photo function, the camera application invokes the PhotoMode interface corresponding to the take photo function and passes the following parameters to the PhotoMode interface: the device name (deviceName) of the device where the camera application is located, the application name (appName) of the camera application, the parameter (key) indicating the take photo function, the parameter content (value) required for the take photo function, and the extended parameter (extend). Among them, the device name, application name, and extended parameter are optional. The parameter (key) indicating the take photo function and the parameter content required for the take photo function form a key-value pair. The parameter indicating the take photo function can be a name, index, identifier, number, etc., and the parameter content required for the take photo function includes the picture flow (pictureFlow) and the preview flow (previewFlow). The picture flow is used to store the shooting data, and the preview flow is used for the electronic device to display the preview picture in real time.

[0094] After the PhotoMode interface is invoked, it can continue to call the MediaDataKit interface downwards. The form of the MediaDataKit interface is MediaDataKit(type, key1, value1, key2, value2, extend). It includes the following parameters:

[0095] Type, which represents the type to which the operation performed by the current electronic device belongs. The type may include, but is not limited to, the following: general type (normal), acquisition, encoding, storage, sharing, decoding, playing, other, etc. The type can be determined according to the previous interface that calls the MediaDataKit interface. For example, if the previous interface is the PhotoMode interface, the electronic device can determine that it needs to start the camera mode and perform the action of acquiring data, so it is regarded as the acquisition type.

[0096] The indication parameter (key) of the function point, which is used to indicate the function point to be started. Multiple different function points can be included under each type. For example, for the acquisition type, it can correspond to functions such as taking pictures, recording videos, night scene function, portrait function, recording function, etc. The indication parameter (key) of the function point can be directly determined by the name or key of the previous interface that calls the MediaDataKit interface. For example, if the previous interface is the PhotoMode interface, the indication parameter (key) of the function point is used to indicate the taking picture function.

[0097] The value (value) required for the function point, which is used to indicate the data required to implement the function point. For example, for the taking picture function, a picture flow (pictureFlow) and a preview flow (previewFlow) are required. The value (value) required for the function point is determined by the value of the previous interface that calls the MediaDataKit interface.

[0098] The indication parameter (key) of the function point and the value (value) required for the function point are in a corresponding relationship, which can be one pair or multiple pairs. The value (value) corresponding to the indication parameter (key) of a function point can be one or include multiple. The indication parameter (key) of the function point and the value (value) required for the function point can also be determined independently by the OS.

[0099] The extension field (extend) can be used for the construction of embedded capabilities or differentiated capabilities.

[0100] After the MediaDataKit interface is called, other interfaces provided by the framework layer can be called continuously, such as interfaces that provide more refined acquisition functions, etc. There can be multiple implementation methods for subsequent interface calls in the framework layer, which are not shown in detail here.

[0101] In some embodiments, after the PhotoMode interface is called, it can bypass the MediaDataKit interface and directly call other framework layer interfaces below the MediaDataKit interface to implement the camera function.

[0102] In some embodiments, the PhotoMode interface can be a kit or a selector (picker). When the PhotoMode interface is implemented as a picker, the MediaDataKit interface can also be correspondingly subdivided into multiple interfaces with more detailed functions, so that the PhotoMode interface can select a suitable interface from these multiple interfaces for calling.

[0103] After the framework layer interface is called, it will continue to call the interfaces of the service layer and the kernel layer below to implement the corresponding shooting function.

[0104] In the embodiments of the present application, the shooting function interface provided by the framework layer encapsulates one or more service layer interfaces provided by the service layer.

[0105] The service layer interface provided by the service layer encapsulates one or more kernel layer interfaces provided by the kernel layer.

[0106] As the connection layer between software and hardware, the kernel layer can be compatible with different hardware devices. The models or types of the processing chips used by different electronic devices may be different, that is, there are differences in hardware. In the OS of the electronic device provided in the embodiments of the present application, the kernel layer can provide one or more kernel layer interfaces, and the one or more kernel layer interfaces encapsulate underlying capabilities. The underlying capabilities include two types of capabilities: basic capabilities and differentiated capabilities. Basic capabilities refer to the capabilities shared by most platforms, and differentiated capabilities refer to the differentiated capabilities unique to each platform rather than shared. Here, the platform refers to hardware devices such as chips. The kernel layer can provide driver programs and hardware access interfaces for calling basic capabilities, and can also provide driver programs and hardware access interfaces for calling differentiated capabilities, etc. In this way, no matter which chips the electronic device is loaded with, the OS of the electronic device is ready to call the capabilities of these chips to achieve compatibility. The kernel layer implements a compatible southbound ecological interface design, provides southbound interfaces for docking hardware devices, and simplifies the docking cost.

[0107] The specific implementation of the electronic device starting the first shooting function can include the following two:

[0108] When starting the first shooting function through the camera picker, the electronic device first calls the camera picker through the first application and transmits the first scene description parameters to the camera picker, and then runs the camera picker at the framework layer; the camera picker selects and runs the first shooting function interface from N shooting function interfaces at the framework layer through the first scene description parameters; then runs the kernel layer interface corresponding to the first shooting function interface at the kernel layer.

[0109] When the first application directly starts the first shooting function, the electronic device first calls the first shooting function interface through the first application, then runs the first shooting function interface at the framework layer, and then runs the kernel layer interface corresponding to the first shooting function interface at the kernel layer.

[0110] Figure 6 Exemplarily shows the ways for different APPs in the electronic device to call the shooting function.

[0111] Such as Figure 6 As shown, the unified media framework can provide unified shooting functions, etc. through the basic capabilities and differentiated capabilities provided by the kernel layer. Both Camera Application A and Camera Application B can call these unified shooting functions. In addition to the unified shooting functions, each APP can also provide its own unique shooting functions. For example, Camera Application A can provide a unique shooting function, and Camera Application B can provide another unique shooting function. The above-mentioned unique shooting functions can be provided by corresponding APPs by further processing on the basis of the unified shooting functions, or can be directly developed and provided by corresponding APPs facing the underlying hardware and / or algorithms.

[0112] Camera Application A and Camera Application B can be applications with shooting functions, which can be system applications or third-party applications. System applications include applications developed by electronic device manufacturers, and third-party applications can include applications developed by other developers other than electronic device manufacturers.

[0113] Such as Figure 6 As shown, the unified media framework solves the dependencies of various applications in the application layer on the kernel layer or device hardware. Each application in the application layer can directly call the shooting functions provided by the unified media framework, and each application can also reuse these shooting functions, thereby providing users with a consistent shooting experience.

[0114] Figure 7 Is a schematic diagram of the shooting effects of different applications. Such as Figure 7 As shown, the third-party camera application can display the basic effect and the third-party enhanced effect. The basic effect can be the effect obtained by the third-party camera application calling the shooting function provided by the unified media framework, and the third-party enhanced effect can be the effect obtained by the third-party camera application based on its own independently developed shooting function. Such as Figure 7As shown, the system camera application can also display the basic effect and the system enhancement effect. The basic effect can be the effect obtained by the system camera application calling the shooting function provided by the unified media framework, and the system enhancement effect can be the effect obtained by the system camera application based on its own independently developed shooting function.

[0115] Figure 8 It is a comparison diagram of the shooting processes of different applications. As Figure 8 shown, during the shooting process, both the third-party camera application and the system camera application can use the basic session or the basic pipeline. The third-party camera application can also use its own unique enhanced session or enhanced pipeline, and the system camera application can also use its own unique enhanced session or enhanced pipeline. The above sessions or pipelines are used to call the algorithms related to shooting and provide them for the corresponding applications to use. Equivalently, both the third-party camera application and the system camera application can separately call the algorithms they need to meet their own special requirements. The above sessions or pipelines can be located below the unified media framework, and each application can call the algorithms in this session or pipeline to process the data stream obtained through the unified media framework, so as to obtain the data stream desired by the application. The above sessions or pipelines can be located above the unified media framework, so that the data stream obtained by the unified media framework can be processed by this session or pipeline to obtain the data stream desired by the application.

[0116] Combined with Figure 7 and Figure 8 , whether it is the effect or the shooting process, different applications maintain basic consistency, and each application can also maintain its own differential competitiveness. In this way, each application on the electronic device can achieve similar effects in shooting.

[0117] Based on Figure 1 the Android camera architecture shown, the data stream output by the currently activated camera and the application that activates the camera have a strong binding relationship.

[0118] Switching the camera will affect the performance. For example, if application A activates camera 1 for shooting, the electronic device will activate camera 1, create a session, and configure the parameters of the data stream required by application A in the session. If the user switches camera 1 to camera 2, the electronic device needs to reconfigure the parameters of the data stream required by application A in the session.

[0119] The data stream is strongly bound to the application, and switching applications affects performance. For example, if application A starts the camera 1 for shooting, the electronic device will start the camera 1, create a session, and configure the parameters of the data stream required by application A in this session. If other applications want to use the shooting function, since application A occupies the camera of the electronic device, application A needs to exit the use of the camera first. After the electronic device turns off the camera 1, the electronic device can start application B, start the camera 2, create a session, and configure the parameters of the data stream required by application B in this session.

[0120] To improve the overall shooting performance of the electronic device, the embodiments of the present application provide a method for managing data streams based on sessions or pipelines. The session or pipeline here is not the same concept as the session or pipeline mentioned above. Figure 8 in the session or pipeline mentioned above.

[0121] Figure 9 An exemplary model of this management method is shown. As Figure 9 shown, the camera session management in the electronic device creates a session, which is responsible for configuring the parameters of the data streams output by multiple cameras in the electronic device. The multiple cameras can be all or part of the cameras in the electronic device, and the multiple cameras can be turned on in advance before the application is started.

[0122] Figure 10 An exemplary schematic diagram of managing data streams based on a pipeline is shown. As Figure 10 shown, the electronic device can create a pipeline. The pipeline can be regarded as a buffer that can be used to store data streams. The pipeline can be used to configure the input data stream and the output data stream. The pipeline can be used to receive the data streams output by multiple cameras, and these data streams can be output after being stored in the pipeline for a period of time. The pipeline can also be used to output the data stream to the corresponding application. For example, as Figure 10 shown, the electronic device can output the data stream of camera 1 in the pipeline to the application that calls camera 1, output the data stream of camera 2 in the pipeline to the application that calls camera 2, and output the data stream of camera 3 in the pipeline to the application that calls camera 3.

[0123] The input and output of the pipeline can be a one-to-many or many-to-one relationship. For example, two applications can both call camera 1, and the electronic device can output the data stream of camera 1 in the pipeline to these two applications at the same time. Another example is that an application can first call camera 1, and the electronic device can output the data stream of camera 1 in the pipeline to this application. Then the application can switch to call camera 2, and then the data stream of camera 2 in the pipeline will be output to this application.

[0124] Through a pipeline, the data stream output by the camera and the input stream of the application can be separated, relieving the input stream of the application from relying on the camera. The data stream output by the camera can be stored in the pipeline, and when the application needs it, the electronic device can retrieve it as needed and output it to the application.

[0125] Since the relationship between the camera output stream and the application input stream is decoupled, the shooting performance of each application will be improved. For example, after the application switches the camera, it can quickly obtain the data stream, and the user can hardly feel the lag. Another example is that two applications can simultaneously obtain the data stream of the same camera or different cameras, and the electronic device can display the shooting interfaces provided by the two applications in split-screen mode.

[0126] In some cases, the data stream output by the camera may be disconnected or abnormal. The data stream may be disconnected when the camera loses power or is damaged, and the data stream may be abnormal when the camera's view is blocked, etc. In the above cases, the camera will no longer be able to output a valid and usable data stream. For such a situation, the electronic device provides a pipeline-based management method to ensure that the functions of the application that calls the camera can continue to be used.

[0127] Figure 11 An exemplary processing method when the data stream is disconnected or abnormal is shown. As Figure 11 shown, if Application A calls the data stream of Camera 1 and the data stream output by Camera 1 is disconnected or abnormal, the electronic device will make adjustments according to the actual situation.

[0128] As Figure 11 shown, the data stream of Camera 1 cached before the disconnection is stored in the pipeline. If the data stream of Camera 1 is disconnected or abnormal for a short period, the electronic device can use the data stream of Camera 1 in the pipeline to output to Application A. After the data stream of Camera 1 is restored, for example, after Camera 1 loses power and then resumes power or the view of Camera 1 is no longer blocked, the pipeline can continue to receive the data stream of Camera 1, and the electronic device can output the data stream of Camera 1 in the pipeline to Application A.

[0129] As Figure 11As shown, if the data stream of camera 1 is disconnected or abnormal for a long time, the electronic device can use the data output of other cameras (such as camera 2) other than camera 1 in the pipeline to application A. In this way, although application A can no longer obtain the data stream of camera 1 it wants, it can use the data stream of other cameras as a replacement. Therefore, application A continuously obtains the data stream and displays it on the display screen for the user, avoiding the situation where application A cannot output a picture to the user because it cannot obtain the data stream. The other cameras other than camera 1 can be a camera with the closest parameters to camera 1, or a default-set camera. In this implementation, the matching degree between camera 2 and application A is lower than that between camera 1 and application A. For example, camera 1 may be a camera manually activated by the user in application A, or camera 1 is the default-activated camera of the electronic device in application A. In this way, although the matching degree is reduced, the electronic device can try to output a picture to the user to avoid black screens or other user-unfriendly scenarios.

[0130] Figure 11 The two processing methods shown can be combined and implemented. For example, when the electronic device detects that the data stream of camera 1 is disconnected or abnormal at the initial stage, it first uses the data stream of camera 1 in the pipeline to output to application A. If the data stream of camera 1 is not detected to recover after a period of time, it uses the data stream of other cameras in the pipeline to output to application A. The period of time here refers to the time of using the remaining data stream of camera 1 in the pipeline, and the length of this time depends on the storage space size of the pipeline and the flow rate of the pipeline outputting the data stream of camera 1.

[0131] In the embodiments of the present application, the pipeline can keep all cameras or all applications open, so that the pipeline can receive the data streams from all cameras and can also output the data streams to the applications in need according to the actual requirements of each application. In some other implementations, it can be determined whether to close the pipeline for some cameras or some applications according to the operating conditions of the electronic device. For example, if the current load of the electronic device is large, such as when it is notified that multiple applications call the same data stream, the electronic device can choose to close this pipeline for some of these applications and no longer output the data stream to these applications through the pipeline. This can reduce the load of the electronic device.

[0132] The unified media framework provided by the embodiments of the present application also realizes the phased shooting process, divides the entire shooting process into two stages as needed, and meets the user's needs for viewing images at different stages.

[0133] Figure 12 Exemplarily shows the process of segmented shooting. As Figure 12 shown, this process is executed by the electronic device and may include the following steps:

[0134] Phase 1:

[0135] 1. The electronic device launches an application with a shooting function and activates the shooting function. After the electronic device activates the shooting function, it can activate the camera and display an image in the preview interface based on the data collected by the camera for the user to view.

[0136] 2. Receive a shooting operation input by the user, such as a photo-taking operation, a start-recording operation, etc.

[0137] 3. Invoke the real-time shooting service.

[0138] 4. Shooting is completed. If the electronic device has activated the video-recording function, shooting is completed after receiving an operation to end video recording.

[0139] 5. Generate a thumbnail of the first image based on the data collected by the camera during the above shooting operation, and display the thumbnail in the preview interface, such as in the lower left corner of the preview interface. Here, the first image is an image defined by its content.

[0140] 6. Use the stage-one algorithm provided by the real-time shooting service to generate a basic quality map of the first image based on the data collected by the camera during the above shooting operation, and save the basic quality map to the gallery.

[0141] 7. When receiving an operation by the user on the thumbnail, display the basic quality map.

[0142] Stage Two:

[0143] 8. After a period of time has passed since the basic quality map was generated, or, receive an operation to view the image after a period of time, such as an operation to view the image in the gallery.

[0144] 9. Access the media library and find the basic quality map of the corresponding image.

[0145] 10. Invoke the time-lapse shooting service.

[0146] 11. Use the stage-two algorithm in the time-lapse shooting service to generate a full-quality map of the first image, and update the basic quality map in the gallery to the full-quality map. The full-quality map can be generated based on the basic quality map. The quality of the full-quality map is higher than that of the basic quality map.

[0147] 12. In response to an operation to view the image, display the full-quality map.

[0148] After generating the basic quality map in stage one, the electronic device can save the basic quality map in the corresponding storage space in the gallery. After generating the full-quality map in stage two, the electronic device can use the full-quality map to replace the basic quality map in the gallery storage space.

[0149] From Figure 12From the segmented shooting process shown, during the entire shooting process of the unified media framework provided by this application, the real-time shooting service and the time-lapse shooting service are respectively called for processing. The real-time shooting service and the time-lapse shooting service can be two services provided by the software framework of the electronic device and can be located in the service layer or the framework layer.

[0150] After the user triggers the shooting, the electronic device first calls the real-time shooting service. During this process, the algorithms used to generate the basic quality map can be collectively referred to as the stage-one algorithms. The stage-one algorithms can include, for example, basic image denoising algorithms, image transformation algorithms, image blurring processing algorithms, etc. After generating the basic quality map, the shooting process of stage one ends.

[0151] After generating the basic quality map, the electronic device can call the time-lapse shooting service. During this process, the algorithms used to generate the full-quality map based on the basic quality map can be called the stage-two algorithms. The stage-two algorithms may involve post-processing of the image and can include, for example, beauty algorithms, filter algorithms, anti-shake algorithms, night scene algorithms, high-definition algorithms, highlight algorithms, etc. If the image quality is distinguished by scores, the score of the full-quality map is higher than that of the basic quality map. For example, the score of the basic quality map can be 80 points, and the score of the full-quality map is 100 points.

[0152] The stage-one algorithms and the stage-two algorithms can be set as needed. For different shooting scenarios (such as shooting still-life scenes, shooting dynamic object scenes, etc.), developers can customize the stage-one algorithms and the stage-two algorithms for each shooting scenario. If an example is given according to the number of algorithms, assuming that the algorithms for normally generating the full-quality map based on the data collected by the camera include 10, then the algorithms for obtaining the basic quality map based on the data collected by the camera in stage one can include 8 of them, and the algorithms for obtaining the full-quality map based on the basic quality map in stage two can include the remaining 2 algorithms. The stage-one algorithms and the stage-two algorithms can both include algorithms on the local device of the electronic device and can also include algorithms in the cloud, that is, the electronic device can generate the basic quality map and the full-quality map on the local device and / or the cloud device.

[0153] The electronic device can call the stage-two algorithms to generate the full-quality map after a period of time when generating the basic quality map. Or, the electronic device can also call the stage-two algorithms to generate the full-quality map when the user clicks on the thumbnail in the gallery to view the image. Both of the above methods can save the computing power for generating the full-quality map during the startup of the shooting function of the electronic device and better serve the current shooting function of the electronic device.

[0154] In some other embodiments, the electronic device may also generate a full quality map after generating the basic quality map, that is, calling the stage 2 algorithm. In this way, the electronic device can present the full quality map to the user in a short time. For example, the user can input the operation of clicking the thumbnail of the preview interface within the first time length of inputting the photo operation, and then the electronic device can first display the basic quality map in the user interface provided by the gallery, and after staying in the user interface for a period of time, if the electronic device generates a full quality map, the basic quality map displayed in the user interface can be replaced with the full quality map.

[0155] In the normal shooting process, the stage one algorithm and the stage two algorithm are usually used to generate a full-quality image, which takes a long time. For users, after shooting, they may click on the thumbnail to view the shot. If the electronic device has not completed the processing of the full-quality image at this time, they need to wait for a while until the electronic device generates the full-quality image before they can see the shooting result. Figure 12 The segmented shooting method shown first only needs to use the stage one algorithm to generate a basic quality image, which greatly shortens the image output time and allows users to see the basic quality image first, which is equivalent to seeing the shooting result, which helps users make real-time adjustments during the shooting process, such as adjusting the viewing angle and angle. After a period of time, the electronic device uses the stage two algorithm to generate a full quality image and output a high-quality shot image, so that the user can see a higher-quality full quality image when viewing or using the image again.

[0156] The segmented shooting provided in the embodiment of the present application can provide users with fast image output feedback in some emergency scenarios, such as snapshot, continuous shooting, shooting fast-moving objects and other time-sensitive scenes, allowing users to quickly view the captured images and adjust the shooting strategy. Since the basic quality map can be generated using only the algorithm of stage one, the time for generating the basic quality map is short, which saves the image output time and improves the performance in various shooting scenarios, such as ensuring the continuous shooting speed.

[0157] The segmented shooting provided in the embodiment of the present application does not affect the output of the final full-quality map result, but only extracts an intermediate state (i.e., the basic quality map) in the process of generating the full-quality map, so that users can see the content they want in a timely and efficient manner. The one-time shooting process of directly using the data collected by the camera to generate the full-quality map may fail due to reasons such as device freezes and power failures, resulting in the electronic device being unable to obtain the final full-quality map. However, by adopting the segmented shooting provided by the present application, the electronic device can ensure successful shooting as long as it completes the shooting of stage one, thereby improving the efficiency of successful shooting.

[0158] The segmented shooting provided by the embodiments of the present application can also be applied to the scenario of shooting videos. For example, after starting to shoot a video, the electronic device can receive an operation input by the user to end the video shooting. In response to this operation, the electronic device first uses the stage-one algorithm to generate a base-quality video, and the base-quality video can include multiple frames of base-quality images. After a period of time, the electronic device then uses the stage-two algorithm to generate a full-quality video, and the full-quality video can include multiple frames of full-quality images.

[0159] In some embodiments, the segmented shooting method provided by the present application can also be used as a unified shooting function and provided to each APP in the application layer for calling in the unified media framework. In other embodiments, the segmented shooting method can be combined with all the shooting functions provided by the unified media framework, that is, all the shooting functions provided by the unified media framework can be combined with the segmented shooting method to provide corresponding functions for each APP in the application layer.

[0160] Each of the embodiments described above in the embodiments of the present application can be combined and implemented.

[0161] Figure 13 is a flowchart of the shooting method provided by the embodiments of the present application. As Figure 13 shown, the method may include the following steps:

[0162] S1301, a first electronic device runs a first application on a first operating system and displays a user interface of the first application. The user interface of the first application includes a first shooting function option. The first application can be a third-party application.

[0163] S1302, the first electronic device detects a user operation acting on the first shooting function option.

[0164] S1303, the first electronic device starts the first shooting function corresponding to the first shooting function option.

[0165] In the shooting method provided by the embodiments of the present application, the first electronic device runs the OS mentioned above, and this OS can be called the first operating system. A first application runs on the first operating system. The first application provides N shooting functions, and the N shooting functions correspond to the shooting function options on the user interface of the first application. The N shooting functions can include, for example, but are not limited to: photo-taking function, video-recording function, night-scene function, portrait function, large-aperture function, time-lapse photography function, AI scene recognition function, barcode-scanning function, face recognition function, face interaction function, etc.

[0166] The first operating system can provide N shooting function interfaces, such as a photo shooting function interface (PhotoMode), a video recording function interface (VideoMode), a night scene function interface (NightMode), a portrait function interface (PortraitMode), etc. The N shooting function interfaces include a first shooting function interface corresponding to the first shooting function.

[0167] The shooting function interface can be implemented as (deviceName, appName, key1, value1, key2, value2, extend). The shooting function interface can include one or more of the following parameters: the name of the device that calls the shooting function interface (deviceName), the name of the application that calls the shooting function interface (appName), the parameter indicating the shooting function (key), the parameter content of the shooting function (value), and the extended parameter (extend). The device that calls the shooting function interface can be the local device or a remote device connected to the local device. The application that calls the shooting function interface can be an application on the local device or an application on the remote device. The parameter indicating the shooting function can be a name, an index, a number, etc., and the parameter content of the shooting function can include a data stream name, a camera identifier, a beauty level, etc. Among them, the device name, the application name, and the extended parameter are optional. When different shooting function interfaces are called, the application or the camera picker that calls the shooting function interface can pass different parameters to the shooting function interface. The key and the value form a key-value pair, and a shooting function interface can include multiple key-value pairs. The following shows several key-value pairs: the key indicates face recognition, and the value includes the coordinates of the face, the face image, etc.; the key indicates the beauty function, and the value includes the beauty level, face information (such as coordinates, images); the key indicates taking a photo, and the value includes the identifier of the camera used (such as a front camera, a rear camera, a telephoto camera, a wide-angle camera, etc.). In some embodiments, the key and the value can be determined by an application in the application layer and then transmitted to the corresponding shooting function interface in the framework layer; in other embodiments, if the application in the application layer does not transmit the key and the value, the OS of the electronic device can also fill the default key and the value into the shooting function interface to implement the shooting function corresponding to the shooting function interface.

[0168] There are the following two implementation methods for the first application and the first operating system:

[0169] 1. The first application does not include program code for calling hardware and / or algorithms related to shooting. The first application includes program code for calling N shooting function interfaces respectively. One shooting function interface is used to start one shooting function, and the N shooting function interfaces are provided by the first operating system.

[0170] Correspondingly, N shooting function interfaces are directly provided for each application in the application layer to call.

[0171] In some embodiments, the first operating system includes: an application layer, a framework layer, a service layer, and a kernel layer. The application layer includes a first application. The N shooting function interfaces are specifically provided by the framework layer of the first operating system. The N shooting function interfaces encapsulate one or more service layer interfaces, and the one or more service layer interfaces are provided by the service layer. The service layer interfaces encapsulate one or more kernel layer interfaces. The one or more kernel layer interfaces are provided by the kernel layer, and the one or more kernel layer interfaces encapsulate underlying capabilities. The underlying capabilities include basic capabilities and differentiated capabilities. The basic capabilities are capabilities shared by multiple platforms, and the differentiated capabilities are capabilities that are different among multiple platforms.

[0172] In the first implementation manner, the specific implementation of the first electronic device starting the first shooting function may include: calling, through the first application, the first shooting function interface corresponding to the first shooting function, running the first shooting function interface in the framework layer, running the service layer interface corresponding to the first shooting function interface in the service layer, and running the kernel layer interface corresponding to the service layer interface of the first shooting function interface in the kernel layer. When running the first shooting function interface in the framework layer, other interfaces in the framework layer may also be called. For details, reference may be made to Figure 5B and the relevant introduction, which will not be elaborated here.

[0173] 2. The first application does not include program codes for calling and shooting-related hardware and / or algorithms. The first application includes program codes for calling a first interface. The first interface encapsulates N shooting function interfaces. One shooting function interface is used to start one shooting function. The first interface is used to select and run the corresponding shooting function interface from the N shooting function interfaces according to the scene description parameters. Both the first interface and the N shooting function interfaces are provided by the first operating system. The first interface is the camera picker mentioned in the previous embodiments.

[0174] Correspondingly, the camera picker (i.e., the first interface) is encapsulated above the N shooting function interfaces, and the camera picker is directly provided for each application in the application layer to call.

[0175] In some embodiments, the first operating system includes: an application layer, a framework layer, a service layer, and a kernel layer. The application layer includes a first application. The first interface and N shooting function interfaces are specifically provided by the framework layer of the first operating system. The N shooting function interfaces encapsulate one or more service layer interfaces, and the one or more service layer interfaces are provided by the service layer. The service layer interfaces encapsulate one or more kernel layer interfaces. The one or more kernel layer interfaces are provided by the kernel layer. The one or more kernel layer interfaces encapsulate underlying capabilities, and the underlying capabilities include basic capabilities and differentiated capabilities. The basic capabilities are capabilities shared by multiple platforms, and the differentiated capabilities are capabilities that differentiate multiple platforms.

[0176] In the second implementation manner, the specific implementation of the first electronic device starting the first shooting function may include: calling the first interface through the first application and passing a first scene description parameter to the first interface, where the first scene description parameter indicates the first shooting function; running the first interface in the framework layer; running the first shooting function interface in the framework layer according to the first scene description parameter; running the service layer interface corresponding to the first shooting function interface in the service layer; and running the kernel layer interface corresponding to the service layer interface of the first shooting function interface in the kernel layer. When running the first shooting function interface in the framework layer, other interfaces in the framework layer may also be called. For specific references, please refer to Figure 5B and the relevant introduction, which will not be elaborated here.

[0177] The above two implementation manners can also be combined. That is, the first operating system can either provide the first interface externally for each application in the application layer to call, or provide the N shooting function interfaces encapsulated by the first interface externally for each application in the application layer to call.

[0178] In some embodiments, the specific implementation of S1303 may include: obtaining the data stream of the first camera from the pipeline and processing the data stream of the first camera to complete the first shooting function. Among them, the electronic device can turn on multiple cameras in advance and store the data streams of the multiple cameras in the pipeline, and the pipeline is a buffer. The first camera belongs to the multiple cameras.

[0179] In some embodiments, if the electronic device detects that the first camera is damaged, it can obtain the data stream of the second camera from the pipeline and process the data stream of the second camera to complete the first shooting function. In some embodiments, before obtaining the data stream of the second camera from the pipeline, the electronic device can first obtain the remaining data stream of the first camera from the pipeline and process the data stream of the first camera to complete the first shooting function. In some embodiments, the matching degree between the first camera and the first application is higher than that between the second camera and the first application. For the solution regarding the pipeline in this part, please refer to the previous text.

[0180] In some embodiments, a second application is also running on the first operating system. The electronic device can also obtain the data stream of the first camera from the pipeline and process the data stream of the first camera to complete the second shooting function initiated by the second application. Alternatively, the electronic device can also obtain the data stream of the second camera from the pipeline and process the data stream of the second camera to complete the second shooting function initiated by the second application. In this way, the electronic device can provide the data stream of the same camera to different applications, or can provide the data streams of different cameras to different applications, so that different applications can use the shooting function simultaneously.

[0181] S1304. When the first shooting function is specifically a photo-taking function, first save the base-quality image of the first photo to the gallery, and then update the base-quality image in the gallery to a full-quality image, where the quality of the full-quality image is higher than that of the base-quality image.

[0182] S1305. Within the first duration of starting the first shooting function, when an operation to open the first photo in the gallery is detected, and the first duration is less than the first threshold, first display the base-quality image of the first photo, and then display the full-quality image.

[0183] The first duration is the duration from when the electronic device starts the photo-taking function to when the full-quality image is generated. That is to say, after the electronic device detects an operation to open the first photo in the gallery, if the full-quality image has not been generated yet, first display the base-quality image, and after the full-quality image is generated, then display the full-quality image.

[0184] S1304 - S1305 are optional steps, and this part can refer to the previous text Figure 12 and the corresponding introduction to the segmented shooting process.

[0185] S1304 - S1305 can also be replaced by S1306 - S1307.

[0186] S1306. When the first shooting function is specifically a video-recording function, first save the base-quality video of the first video to the gallery, and then update the base-quality video in the gallery to a full-quality video, where the quality of the full-quality video is higher than that of the base-quality video.

[0187] S1307. Within the first duration of starting the first shooting function, when an operation to open the first video in the gallery is detected, and the first duration is less than the first threshold, first display the base-quality video of the first video, and then display the full-quality video.

[0188] The first duration is the duration from when the electronic device starts the video-recording function to when the full-quality video is generated. That is to say, after the electronic device detects an operation to open the first video in the gallery, if the full-quality video has not been generated yet, first display the base-quality video, and after the full-quality video is generated, then display the full-quality video.

[0189] S1306 - S1307 are optional steps, and this part can refer to the foregoing content. Figure 12 And the corresponding introduction to the segmented shooting process.

[0190] Refer to Figure 14 , Figure 14 This is the hardware structure diagram of the electronic device 100 provided by the embodiment of the present application. The electronic device 100 can be the electronic device and the first electronic device mentioned above. The OS introduced above runs on the electronic device 100.

[0191] Such as Figure 14 shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a microphone 170C, a camera 193, a display screen 194, etc.

[0192] It can be understood that the structure schematically shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0193] The processor 110 may include one or more processing units. For example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors.

[0194] The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0195] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0196] The electronic device 100 implements the display function through a GPU, a display screen 194, an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0197] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD). The display panel can also use an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniled, a microled, a micro-oled, a quantum dot light-emitting diode (QLED), etc. to manufacture. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0198] The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, etc.

[0199] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera photosensitive element, where the optical signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0200] The camera 193 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard formats such as RGB and YUV. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0201] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0202] The video codec is used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0203] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission function between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0204] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0205] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices, flash memory.

[0206] Flash memory can be classified into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle, and can be classified into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the number of potential levels of storage cells, and can be classified into universal flash storage (UFS), embedded multi media Card (eMMC), etc. according to the storage specification.

[0207] Random access memory can be directly read and written by the processor 110, can be used to store the operating system or executable programs (such as machine instructions) of other running programs, and can also be used to store data of users and application programs, etc.

[0208] Non-volatile memory can also store executable programs and store data of users and application programs, etc., and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0209] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to achieve the data storage function. For example, files such as music and videos are saved in the external non-volatile memory.

[0210] The microphone 170C, also known as a "microphone" or "transmitter", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C with their mouth to input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0211] In the embodiments of the present application, the internal memory 121 is used to store the computer program for implementing the shooting method provided in the embodiments of the present application, and the processor 110 is used to execute the computer program to implement the shooting method provided in the embodiments of the present application. For example, the processor 110 can generate a basic quality map and a full quality map successively in the segmented shooting process, and can also be used to manage the input data stream and output data stream of the pipeline, etc.

[0212] It should be understood that each step in the above method embodiments can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The method steps disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware processor, or can be executed and completed by the combination of the hardware and software modules in the processor.

[0213] The present application also provides an electronic device, which may include a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the method executed by the electronic device in any of the above embodiments.

[0214] The present application also provides a chip system, including a processing circuit and an interface circuit. The interface circuit is used to receive computer instructions and transmit them to the processing circuit, and the processing circuit is used to run the computer instructions to implement the method executed by the electronic device in any of the above embodiments.

[0215] The present application also provides a chip system, which includes at least one processor for implementing the method executed by the electronic device in any of the above embodiments. In a possible design, the chip system further includes a memory, and the memory is used to store program instructions and data. The memory is located inside or outside the processor.

[0216] The chip system may be composed of chips, or may include chips and other discrete devices.

[0217] Optionally, there may be one or more processors in the chip system. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor that implements its functions by reading software code stored in a memory.

[0218] Optionally, there may also be one or more memories in the chip system. The memory may be integrated with the processor or may be separately provided from the processor, and the embodiments of the present application do not limit this. Exemplarily, the memory may be a non-transitory processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or may be separately provided on different chips. The embodiments of the present application do not specifically limit the type of the memory and the setting manner of the memory and the processor.

[0219] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processing circuit (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0220] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method executed by the electronic device in any one of the above embodiments is implemented.

[0221] The present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method executed by the electronic device in any one of the above embodiments is implemented.

[0222] The various embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)), etc.

[0224] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage media include: various media such as ROM or random access memory RAM, magnetic disks, or optical discs that can store program codes.

[0225] In the description of the embodiments of this application, unless otherwise specified, " / " means "or". For example, A / B can represent A or B; "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "a plurality of" means two or more than two.

[0226] The terms "first" and "second" are only used for descriptive purposes and cannot be understood as implying or indicating relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" can explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality of" is two or more than two.

[0227] In summary, the above description is only an embodiment of the technical solution of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made in accordance with the disclosure of the present application shall be included within the protection scope of the present application.

Claims

1. A shooting method, characterized in that, The method is applied to a first electronic device on which a first operating system runs. The method includes: Run a first application on the first operating system. The first application is installed on the first electronic device and is a third-party application. The first application provides N shooting functions, and the N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for invoking hardware and / or algorithms related to shooting. The first application includes program code for respectively invoking N shooting function interfaces, and one shooting function interface is used to start one shooting function. The N shooting function interfaces are provided by the first operating system. Detect a user operation on a first shooting function option in the user interface of the first application. The first shooting function option corresponds to a first shooting function, and the first shooting function belongs to the N shooting functions. Start the first shooting function.

2. The method according to claim 1, wherein The first application further includes program code for invoking a first interface. The first interface encapsulates the N shooting function interfaces and is used to select and run a first shooting function interface from the N shooting function interfaces according to a first scene description parameter. The first interface is provided by the first operating system, and the first scene description parameter indicates the first shooting function.

3. The method according to claim 1 or 2, characterized in that, The first operating system includes: an application layer, a framework layer, a service layer, and a kernel layer. The application layer includes the first application. The N shooting function interfaces are specifically provided by the framework layer of the first operating system. The N shooting function interfaces encapsulate one or more service layer interfaces, and the one or more service layer interfaces are provided by the service layer. The service layer interface encapsulates one or more kernel layer interfaces, and the one or more kernel layer interfaces are provided by the kernel layer. The one or more kernel layer interfaces encapsulate underlying capabilities, and the underlying capabilities include basic capabilities and differentiated capabilities. The basic capabilities are capabilities shared by multiple platforms, and the differentiated capabilities are capabilities that differentiate the multiple platforms.

4. The method according to claim 3, characterized in that Starting the first shooting function specifically includes: Invoking, through the first application, the first shooting function interface corresponding to the first shooting function; Running the first shooting function interface in the framework layer; Running the service layer interface corresponding to the first shooting function interface in the service layer; Running the kernel layer interface corresponding to the first shooting function interface in the kernel layer.

5. The method according to any one of claims 1-4, characterized in that, The parameters passed into the shooting function interface include one or more of the following: device name, application name of the first application, parameters passed in by the first application. Among them, the parameters passed in by the first application are in the form of key-value pairs, where the key represents the shooting function corresponding to the shooting function interface, and the value includes the parameter content of the shooting function corresponding to the shooting function interface.

6. The method according to claim 5, wherein The first shooting function is a photographing function, and the parameters passed in by the first application include one or more of the following: photographing stream, preview stream. Alternatively, the first shooting function is a face recognition function, and the parameters passed in by the first application include: indication information of the face recognition function, coordinates of the face, and an image of the face. Alternatively, the first shooting function is a beauty function, and the parameters passed in by the first application include: indication information of the beauty function, beauty level, and face information.

7. The method according to any one of claims 1-6, characterized in that The method further includes: turning on multiple cameras, and storing the data streams of the multiple cameras in a pipeline, where the pipeline is a buffer; Starting the first shooting function specifically includes: obtaining the data stream of the first camera from the pipeline, and processing the data stream of the first camera to complete the first shooting function.

8. The method according to claim 7, wherein The method further includes: Detecting that the first camera is damaged; Obtaining the data stream of the second camera from the pipeline, and processing the data stream of the second camera to complete the first shooting function.

9. The method according to claim 8, wherein Before obtaining the data stream of the second camera from the pipeline, the method further includes: First obtaining the remaining data stream of the first camera from the pipeline, and processing the data stream of the first camera to complete the first shooting function.

10. The method according to claim 8 or 9, characterized in that The matching degree between the first camera and the first application is higher than the matching degree between the second camera and the first application.

11. The method according to any one of claims 7-10, characterized in that, The method further includes: Obtaining the data stream of the first camera from the pipeline, and processing the data stream of the first camera to complete the second shooting function started by the second application; Alternatively, obtaining the data stream of the second camera from the pipeline, and processing the data stream of the second camera to complete the second shooting function started by the second application, where the second application is installed on the first electronic device.

12. The method according to any one of claims 1-11, characterized in that The first shooting function is specifically a photographing function. After starting the first shooting function, the method further includes: Saving the base quality map of the first picture to the picture library; Updating the base quality map in the picture library to a full quality map, where the quality of the full quality map is higher than the quality of the base quality map.

13. The method according to claim 12, wherein After starting the first shooting function, the method further includes: Within a first time period after starting the first shooting function, detecting an operation to open the first picture in the picture library, where the first time period is less than a first threshold; First displaying the base quality map of the first picture; After generating the full quality map, then displaying the full quality map.

14. The method according to any one of claims 1-13, characterized in that, The first shooting function is specifically a video recording function. After starting the first shooting function, the method further includes: Saving the base quality video of the first video to the picture library; Updating the base quality video in the picture library to a full quality video, where the quality of the full quality video is higher than the quality of the base quality video.

15. The method according to claim 14, wherein After starting the first shooting function, the method further includes: Within a first time period after starting the first shooting function, detecting an operation to open the first video in the picture library, where the first time period is less than a first threshold; First displaying the base quality video of the first video; After generating the full quality video, then displaying the full quality video.

16. The method according to any one of claims 1-15, characterized in that, The first shooting function is one of the following: photo shooting function, video recording function, night scene function, portrait function, large aperture function, time-lapse photography function, AI scene recognition function, barcode scanning function, face recognition function, and face interaction function.

17. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored on the memory. The computer program includes a first operating system. The first electronic device is installed with a first application, which is a third-party application. The first application provides N shooting functions. The N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for calling hardware and / or algorithms related to shooting. The first application includes program code for respectively calling N shooting function interfaces. One shooting function interface is used to start one shooting function. The N shooting function interfaces are provided by the first operating system. The processor executes the computer program to implement the method according to any one of claims 1-16.

18. A computer-readable storage medium, characterized in that, Stored thereon is a computer program, which includes a first operating system. The first electronic device is installed with a first application, which is a third-party application. The first application provides N shooting functions. The N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for calling hardware and / or algorithms related to shooting. The first application includes program code for respectively calling N shooting function interfaces. One shooting function interface is used to start one shooting function. The N shooting function interfaces are provided by the first operating system. When the computer program is executed by the processor, it implements the method according to any one of claims 1-16.

19. A computer program product, characterized in that, The computer program product includes a computer program, which includes a first operating system. The first electronic device is installed with a first application, which is a third-party application. The first application provides N shooting functions. The N shooting functions correspond to shooting function options on the user interface of the first application. The first application does not include program code for calling hardware and / or algorithms related to shooting. The first application includes program code for respectively calling N shooting function interfaces. One shooting function interface is used to start one shooting function. The N shooting function interfaces are provided by the first operating system. When the computer program is executed by the processor, it implements the method according to any one of claims 1-16.

Images