Image processing method, electronic device, chip system, storage medium and program product

Abstract

The embodiment of the invention provides an image processing method, electronic equipment, a chip system, a storage medium and a program product, relates to the technical field of terminals, and is beneficial to shortening the overall time from triggering photographing to obtaining a photographed image by a user and improving the user experience. The method comprises the following steps: in response to a trigger operation for a photographing button, loading a first algorithm and issuing a photographing instruction in parallel; acquiring a first image based on the photographing instruction; and under the condition that the loading of the first algorithm is completed, processing the first image by using the first algorithm.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image processing method, electronic device, chip system, storage medium and program product. Background Technology

[0002] Electronic devices can support shooting functions, allowing users to record what they see using various shooting functions in camera apps, such as night scene photography, telephoto photography, high dynamic range (HDR) photography, and time-lapse photography.

[0003] However, in some scenarios, users may find that the time from triggering the photo-taking function to obtaining the image is too long, which affects the user experience. Summary of the Invention

[0004] This application provides an image processing method, electronic device, chip system, storage medium, and program product, which are applied in the field of terminal technology and help to shorten the overall time from when a user triggers a photo to when the photo is obtained, thereby improving the user experience.

[0005] In a first aspect, embodiments of this application propose an image processing method applied to an electronic device. The method includes: in response to a trigger operation on a camera button, loading a first algorithm and issuing a camera command in parallel; acquiring a first image based on the camera command; and processing the first image using the first algorithm after the first algorithm has been loaded.

[0006] In this embodiment, the electronic device can load the first algorithm and issue the photo-taking command in parallel in response to a trigger operation on the camera button. This means the photo-taking command can be issued without waiting for the first algorithm to finish loading, allowing the electronic device to capture the first image while the first algorithm is loading, unlike related technologies where the photo-taking command and image acquisition only occur after the first algorithm has finished loading. The method provided in this embodiment saves time loading the first algorithm in the image processing flow, shortens the overall image processing time, and improves the user experience.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the photo-taking command is also used to instruct the loading of the second algorithm. The method further includes: loading the second algorithm based on the photo-taking command, wherein the loading process of the first algorithm and the loading process of the second algorithm overlap in time.

[0008] It is worth noting that loading the first algorithm takes time, and the loading of the first algorithm and the issuance of the photo-taking command can be carried out in parallel. When the electronic device loads the second algorithm based on the photo-taking command, the loading of the first algorithm may still be in progress, which may result in an overlap in the loading time of the first and second algorithms. Since the issuance of the photo-taking command in this embodiment does not need to wait for the first algorithm to finish loading, and since the photo-taking command can also be used to instruct the loading of the second algorithm, the loading of the first and second algorithms in the electronic device can also be carried out in parallel. This means that the loading of the first algorithm, the loading of the second algorithm, and the acquisition of the first image in the electronic device can also be carried out in parallel, which helps to shorten the overall image processing time and improve the user experience.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, before processing the first image using the first algorithm, the method further includes: processing the first image using the second algorithm after the second algorithm has been loaded; processing the first image using the first algorithm includes: processing the first image processed by the second algorithm using the first algorithm.

[0010] In this way, since the first algorithm runs after the second algorithm, and the method of this application allows the loading of the second algorithm and the acquisition of the first image to not have to wait for the first algorithm to finish loading, that is, during the loading of the first algorithm, it is possible to acquire the first image and process the first image by the second algorithm in parallel. Compared with the related technologies, which load the second algorithm and acquire the first image only after the first algorithm has finished loading, the image processing efficiency is improved, the image processing time is shortened, and the user experience is improved.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, before processing the first image using the first algorithm, the method further includes: determining whether the first algorithm has been fully loaded; and processing the first image using the first algorithm, including: if it is determined that the first algorithm has not been fully loaded, then after waiting for a preset time, determining whether the first algorithm has been fully loaded again, until it is determined that the first algorithm has been fully loaded, and then processing the first image using the first algorithm.

[0012] Optionally, the term "preset time" here may have a similar meaning to "segment 2" in method 500, but this application does not limit it in this respect.

[0013] In this embodiment of the application, before processing the first image using the first algorithm, it is determined whether the first algorithm has been fully loaded. If it is determined that the first algorithm has not been fully loaded, the process waits. After each preset time, it is determined again whether the first algorithm has been fully loaded. Only when it is determined that the first algorithm has been fully loaded is the first algorithm used to process the first image. This helps to ensure the processing effect of the first image.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the first algorithm includes one or more first-class image processing algorithms. During the loading process of the first algorithm, a feedback value is recorded after each first-class image processing algorithm is loaded. The method for determining whether the first algorithm has been loaded includes: determining whether the number of feedback values ​​is equal to the number of one or more first-class image processing algorithms included in the first algorithm; if the number of feedback values ​​is equal to the number of one or more first-class image processing algorithms, it is determined that the first algorithm has been loaded; if the number of feedback values ​​is not equal to the number of one or more first-class image processing algorithms, it is determined that the first algorithm has not been loaded.

[0015] Optionally, before processing the first image using the second algorithm, it can be determined whether the second algorithm has finished loading. The method for determining whether the second algorithm has finished loading, and the subsequent processing after determining whether it has finished loading, can be similar to the method for determining whether the first algorithm has finished loading, i.e., the subsequent processing, and will not be described in detail here.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the electronic device includes a camera application, a first module, and an image sensor; in response to a trigger operation on a shutter button, the loading of a first algorithm and the issuance of a shutter command are performed in parallel, including: the camera application, in response to a trigger operation on a shutter button, determines shutter parameters and an identifier of the first algorithm based on the identifier of the camera application's shooting mode and the scene parameters of the shooting mode at the time the shutter button is triggered; the camera application transmits a first instruction to the first module, the first instruction including the identifier of the first algorithm, to instruct the first module to load the first algorithm; the camera application issues a shutter command containing shutter parameters to the image processor, to instruct the image processor to acquire a first image; wherein, the loading of the first algorithm by the first module and the issuance of the shutter command to the image sensor by the camera application are performed in parallel.

[0017] Optionally, the first module may be a camera access interface, and the first instruction may be understood as instruction 1 in method 500, but this application does not limit it.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the electronic device further includes a second module, the first instruction further includes an identifier of the shooting mode and scene parameters of the shooting mode, and the first instruction is also used to instruct the first module to determine the identifier of the second algorithm. The method further includes: the first module determining the identifier of the second algorithm based on the identifier of the shooting mode and scene parameters of the shooting mode included in the first instruction, and transmitting the identifier of the second algorithm to the camera application; the camera application sending a shooting instruction containing shooting parameters to the image sensor, including: the camera application transmitting a shooting request to the second module, the shooting request also including the identifier of the second algorithm; the second module loading the second algorithm based on the identifier of the second algorithm, and forwarding the shooting instruction containing shooting parameters to the image sensor.

[0019] Optionally, the second module may be the camera hardware abstraction layer described in method 500, but this application does not specifically limit it.

[0020] Secondly, embodiments of this application provide an image processing apparatus, which may be an electronic device, a chip, or a chip system within an electronic device. The image processing apparatus may include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit may be a display screen. The display unit is used to perform display steps to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is an electronic device, the processing unit may be a processor. The image processing apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing apparatus is a chip or a chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0021] Thirdly, embodiments of this application provide an electronic device including one or more processors and a memory, the memory being coupled to one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and one or more processors calling the computer instructions to cause the electronic device to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium, which includes a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0023] Fifthly, embodiments of this application provide a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0024] Sixthly, this application provides a chip or chip system including one or more processors and a communication interface. The communication interface and the one or more processors are interconnected via a circuit. The one or more processors are used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0025] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0026] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0028] Figure 2 An exemplary flowchart of a photo-taking process provided in an embodiment of this application;

[0029] Figure 3 An exemplary flowchart of another photographing process provided in this application embodiment;

[0030] Figure 4 A schematic block diagram of the software structure of an electronic device provided in an embodiment of this application;

[0031] Figure 5 An exemplary flowchart of an image processing method provided in an embodiment of this application;

[0032] Figure 6An exemplary flowchart of an image processing method provided in an embodiment of this application;

[0033] Figure 7 A schematic block diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0034] Figure 8 This is a schematic block diagram of the chip structure provided in an embodiment of this application. Detailed Implementation

[0035] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0036] 1. Algorithm instantiation: In programming, instantiation usually refers to creating a specific instance (or object) of a class (or data type). Algorithm instantiation can be understood as combining the specific implementation of an algorithm (or the template of the algorithm) with specific data or problem environment to form an executable program or code segment.

[0037] In this embodiment of the application, algorithm instantiation can also be understood as loading the algorithm's code into the running memory of the electronic device, so that the algorithm is in an executable state.

[0038] 2. Photographed Image: This can be understood as the image captured by the camera button on an electronic device. For example, when an electronic device receives a user's trigger action on the camera button, the device can capture an image at the moment of taking the photo, based on the image taken by the camera, which the user can view from the camera application interface (e.g., ...). Figure 1 The image seen in the control 105 (corresponding area) or in the gallery.

[0039] 3. RAW image: can be understood as an unprocessed image captured by an image sensor; RAW image can also be understood as the raw image.

[0040] 4. YUV image: can be understood as an image processed using a color encoding method, where Y represents luminance, which is the gray value, U represents chrominance, which is used to describe the color of the image, and V represents chroma, which is used to describe the saturation of the image.

[0041] 5. HDR Mode: High Dynamic Range (HDR) mode, also known as dual-frame HDR mode, synthesizes multiple photos with different exposures, resulting in a wider brightness range and more detail and color in the image. HDR mode photography can capture multiple frames with different exposure parameters, such as obtaining long and short frames in a single shot.

[0042] In this application embodiment, "long frame," "short frame," and "normal frame" are all relative concepts. A normal frame can be understood as an image frame output by the image sensor during the preview stage, used as a reference frame to provide a baseline brightness; long frames and short frames can be frames with different exposure times than normal frames, produced by the image sensor in response to the shooting parameters issued in response to the button press event. Long frames can be image frames with an exposure time longer than normal frames, which can provide information about dark areas, so that details in dark areas can also be presented in the photo; short frames can be image frames with an exposure time shorter than normal frames, which can provide information about bright areas and minimize overexposed areas in the photo.

[0043] In this embodiment, the camera algorithm can output a high-quality image by fusing long frames, short frames, and normal frames.

[0044] 6. Other terms

[0045] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0046] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0047] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0048] 7. Electronic equipment

[0049] The electronic devices in this application embodiment may include handheld devices with camera functions, vehicle-mounted devices, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, wearable devices, electronic devices in 5G networks, or future evolution of public land mobile communication networks. The embodiments of this application do not limit the scope of electronic devices in a network (PLMN).

[0050] By way of example and not limitation, in this embodiment, the electronic device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0051] Furthermore, in this application embodiment, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0052] The electronic device in this application embodiment may also be referred to as: electronic device, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0053] Figure 1 An exemplary application scenario diagram of an embodiment of this application is shown. Figure 1 Image 'a' shows the camera application's photo-taking interface 101, where users can take photos by clicking the photo-taking button 103. Optionally, users can also select the camera application's shooting mode, for example, by clicking the "More" option in interface 101. Figure 1 The interface 102 shown in b is used to select "HDR" to take photos in HDR mode. It should be understood that not all controls on the camera interface are displayed in this embodiment; the camera interface can be designed with more or fewer controls, and this application does not limit this. The user holds the electronic device and points it at the target object (e.g., ...). Figure 1 In the image (object 104 shown in the image), clicking the camera button 103 triggers the phone to take a picture of the target object. Optionally, after clicking the camera button, the user can view the captured photo by clicking the control 105.

[0054] In some scenarios, HDR mode can generate images with a high dynamic range, better preserving image details in both bright and low light conditions. When a user presses the shutter button, the electronic device responds to this event by processing the image captured by the image sensor to obtain a high-quality image for display.

[0055] In some implementations, electronic devices can process images through image processing algorithms. In response to the pressing of the camera button, the electronic device can instantiate the image processing algorithm, load the image processing algorithm into memory, and make the image processing algorithm executable, so as to facilitate subsequent processing of the images acquired by the image sensor.

[0056] The photographing process of electronic devices in related technologies can be as follows: Figure 2 As shown. For example, when the user clicks... Figure 1 After the camera button 103 shown in Figure a is pressed, the electronic device responds to the trigger operation of the camera button by obtaining the identifier of the first algorithm link corresponding to the moment the camera button is pressed. Then, it instantiates the first type of image processing algorithm contained in the first algorithm link corresponding to the identifier of the first algorithm link. If the instantiation of the first type of image processing algorithm contained in the first algorithm link is successful, the electronic device sends the camera request to the underlying hardware, such as the image sensor, and uses the image sensor to acquire an image. Furthermore, the electronic device uses the algorithm included in the first algorithm link to process the image acquired by the image sensor to obtain the captured image.

[0057] As can be seen from the above steps, the electronic device will only instantiate the second type of algorithm and use the image sensor to capture images after the first type of algorithm is successfully instantiated. This means that after the user triggers the shutter button, the electronic device will only execute the subsequent process after the first type of algorithm is instantiated, which brings the user a lot of waiting time and affects the efficiency of the electronic device in outputting images.

[0058] In view of this, embodiments of this application provide an image processing method, an electronic device, a chip system, a storage medium, and a program product. The loading of the first algorithm and the issuance of the photo-taking command are executed in parallel. That is to say, image acquisition does not have to wait for the first algorithm to finish loading, which is beneficial to improving the efficiency of the electronic device in outputting images and reducing the time the user waits for image output.

[0059] It should be understood that the first algorithm in the embodiments of this application may be a general description of a first type of image processing algorithm included in the first algorithm link.

[0060] For example, the photographing process of the electronic device provided in the embodiments of this application can be as follows: Figure 3 As shown, in response to pressing the shutter button, the electronic device obtains the identifier of the first algorithm link corresponding to the moment the shutter button is pressed. It can then execute the instantiation of the first type of image processing algorithm contained in the first algorithm link and the sending of the shutter request in parallel, without waiting for the instantiation of the first type of image processing algorithm in the first algorithm link to complete. After acquiring the image, the electronic device processes the image acquired by the image sensor using the algorithms included in the first algorithm link to obtain the photographed image. The main difference between the shutter capture process of the electronic device provided in this embodiment and that in related technologies is that the electronic device does not need to wait for the instantiation of the first type of image processing algorithm in the first algorithm link before acquiring the image. This reduces the user's waiting time for image output and improves the efficiency of the electronic device's image output.

[0061] Figure 3 The illustrated photographing process of the electronic device is a brief description provided to give the reader a quick understanding of this application. The image processing method provided by the embodiments of this application will be described in detail below. Before proceeding, the software architecture of the electronic device will be explained first.

[0062] Figure 4 An exemplary schematic diagram of the software architecture of an electronic device is shown. The software system of the electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. The layered architecture can use the Android system, Linux operating system, Unix operating system, iOS operating system, Windows operating system, or other operating systems, and this application embodiment is not limited to this. The following uses the layered architecture Android system as an example to illustrate the software architecture of the electronic device provided in this application embodiment.

[0063] In some implementations, a layered architecture divides the software system of an electronic device into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom: applications, application framework, hardware abstraction layer (HAL), kernel, and hardware layer.

[0064] like Figure 4 As shown, the application layer may include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. For example, it may include camera applications, gallery applications, or other system applications or third-party applications. This application does not limit this.

[0065] The application framework layer provides APIs and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer may include at least a first module and a first type of algorithm. The first module can be used to instantiate the first type of algorithm. Optionally, the first module may be, for example, a camera access interface, which can also be used to provide an application programming interface and programming framework for camera applications.

[0066] It is worth noting that the first type of algorithm may include one or more first type of image processing algorithms. These one or more first type of image processing algorithms may form one or more first type of algorithm chains. Each first type of algorithm chain may include at least one first type of image processing algorithm. These at least one first type of image processing algorithms may have a certain execution order. Running the first type of image processing algorithms on the first type of algorithm chain based on this execution order can realize functions such as image enhancement, face recognition, image blurring, portrait beautification, and image fusion.

[0067] In some implementations, the first type of algorithm is called a framework-side algorithm, which can be understood as an image processing algorithm provided by application developers, mobile phone manufacturers, or third-party libraries that runs on the upper-level software architecture.

[0068] Optionally, the camera access interface can be an Android Interface Definition Language (AIDL) interface, which is not limited in this application.

[0069] The hardware abstraction layer can include at least the camera hardware abstraction layer and a second type of algorithm.

[0070] In this embodiment, the camera hardware abstraction layer can provide virtual hardware for the camera device and can also be used to instantiate the second type of algorithm.

[0071] The second type of algorithm may include one or more second type image processing algorithms. These one or more second type image processing algorithms may form one or more second type algorithm chains. Each second type algorithm chain may include at least one second type image processing algorithm. These at least one second type image processing algorithm may also have a certain execution order. Running the second type image processing algorithms on the second type algorithm chain based on this execution order can realize basic image processing functions such as image downsampling, noise reduction, white balance adjustment, demosaicing, and color correction.

[0072] In some implementations, the second type of algorithm can be called a platform-side algorithm, which can be understood as an image processing algorithm that runs on the underlying hardware and is provided by the operating system of the electronic device or the hardware (such as a chip) used by the electronic device.

[0073] Optionally, one or more first-class image processing algorithms included in the first-class algorithm and one or more second-class image processing algorithms included in the second-class algorithm can be completely the same, partially the same (or partially different), or completely different, and this application does not limit this.

[0074] Optionally, the functions that the first algorithm link can achieve and the functions that the second algorithm link can achieve can be completely the same, partially the same (or partially different), or completely different, and this application does not make any specific limitations in this regard.

[0075] Each of the first-type image processing algorithms in the first-type algorithm chain and each of the second-type image processing algorithms in the second-type algorithm chain can be understood as an image processing node. During image processing, the functions corresponding to the first-type or second-type algorithm chain can be implemented according to the execution order of each node. In one possible implementation, the first-type algorithm chain and / or the second-type algorithm chain can also be referred to as a photography path; this application does not limit the specific name used.

[0076] The kernel layer is the layer between hardware and software. In this embodiment, the kernel layer may include a camera device driver and a display driver, etc. The camera device driver is used to drive the camera's sensor to acquire images; the display driver is used to transmit images to the display screen.

[0077] The hardware layer may include sensors, such as image acquisition sensors (which can also be understood as cameras) and displays. Optionally, it may also include hardware such as image signal processors (ISPs).

[0078] In this embodiment, the first module used to instantiate the first type of algorithm can be in the application framework layer, the hardware abstraction layer, or the kernel layer. This application does not specifically limit which layer of the software structure the module used to instantiate the first type of algorithm is located in.

[0079] It should be understood that in some embodiments, layers that perform the same function may be called by other names, or layers that can perform the functions of multiple layers may be considered as one layer, or layers that can perform the functions of multiple layers may be divided into multiple layers. This application does not impose any limitations on this.

[0080] The following is combined Figure 4 The modules in the software architecture of the described electronic device are referred to... Figure 5 The image processing method 500 provided in the embodiments of this application is described from the perspective of the interaction between various modules.

[0081] In one possible scenario, in response to a user's request to launch the camera application, the electronic device's launcher sends a camera application launch request (startActivity) to the system service process (System_server) via Binder IPC. Upon receiving the startActivity request, the activity management service in the system service process checks if a running camera application process already exists. If not, it creates the camera application process. During the camera application process creation, while initializing camera-related hardware (such as the image sensor), the camera application can create a preview window. When the camera application's preview interface is successfully created and rendered on the screen, the user can see... Figure 1 The image 'a' shows the interface of the camera application.

[0082] It should be understood that the preview image in the camera application interface can be understood as an image captured by the image sensor and displayed in real time in the preview screen of the camera application. The preview image can be an image processed by the first type of algorithm and / or the second type of algorithm, or it can be an image not processed by the first type of algorithm and / or not processed by the second type of algorithm. This application does not make any specific limitation in this regard.

[0083] It should also be understood that while the camera app is running but the user has not yet taken a photo, the image sensor can continuously capture images and send them to the display screen as preview images. During this time, the user can choose a shooting mode, such as portrait mode, night mode, HDR mode, etc., or can manually adjust the shooting parameters.

[0084] Furthermore, the user can click the shutter button to take a photo, and the electronic device can respond to the shutter button press event by executing method 500.

[0085] It is worth noting that clicking the shutter button to take a photo is only one of the operations that triggers taking a photo. Users can also trigger taking a photo by using preset photo-taking gestures to take a photo in the air, or by waking up the voice assistant to trigger taking a photo by voice control. This application does not limit the specific operations that can trigger taking a photo. Regardless of how the photo is triggered, it can be regarded as a press event of the shutter button for the camera application in the electronic device.

[0086] Method 500 may specifically include the following steps:

[0087] S501. The camera application responds to the press event of the shutter button, obtains information 1, which includes the identifier of the current shooting mode of the camera application and the first scene parameter of the shooting mode; based on information 1 and preset algorithm 1, it determines the shooting parameters and the identifier of the first algorithm link corresponding to the first scene parameter of the shooting mode.

[0088] It should be understood that the first algorithm link mentioned here refers to one or more first-class algorithm links composed of one or more first-class image processing algorithms included in the first-class algorithm.

[0089] Optionally, the shooting mode can be, for example, a large aperture mode, a night scene mode, a portrait mode, and a photo mode (or it can also be understood as a normal mode, an automatic mode, etc.), and this application does not specifically limit it. Among them, the large aperture mode can use algorithms to achieve background blurring to make the image present a background blurring effect for the target object; the night scene mode can use noise reduction algorithms to make the image clear and bright even when taken at night; the portrait mode can use beauty and skin smoothing algorithms to make the target person in the image appear to have fair skin; the photo mode can also be understood as an automatic mode, in which the camera can automatically adjust parameters such as exposure and white balance to obtain a better quality image.

[0090] It should also be understood that each shooting mode can have corresponding scene parameters. These scene parameters can be selected by the user or automatically set by the camera based on information such as the current shooting mode, ambient lighting, previous preview images, and the user's historical usage habits. Scene parameters can include, for example, environmental depth information, the number of objects in the environment (e.g., the number of faces in portrait mode), ISO, exposure time, exposure compensation, white balance, focus mode, color saturation, or aperture size.

[0091] The first scene parameter can be understood as a set of one or more scene parameters corresponding to the current shooting mode of the camera application when the shutter button is pressed.

[0092] Optionally, the preset algorithm 1 may include some predefined functions, trained models, or predefined mapping relationships, which can be referred to in relevant prior art, and this application does not make specific limitations in this regard. In one possible implementation, the camera application can obtain the shooting parameters and the identifier of the first link by calling the preset algorithm 1, taking the identifier of the shooting mode and the first scene parameters as input parameters.

[0093] Optionally, the shooting parameters may include all or part of the scene parameters exemplified above, or may include parameters not included in the scene parameters exemplified above, and this application does not limit this.

[0094] S502, the camera application sends instruction 1 to the first module. Instruction 1 includes information 1 and the identifier of the first algorithm link. Instruction 1 is used to instruct the first module to determine the identifier of the second algorithm link and to instantiate the image processing algorithm corresponding to the first algorithm link. Correspondingly, the first module receives instruction 1.

[0095] S503. The first module determines the identifier of the second algorithm link based on information 1 and preset algorithm 2.

[0096] It should be understood that the second algorithm link mentioned here refers to one or more second-class algorithm links composed of one or more first-class image processing algorithms included in the second-class algorithm.

[0097] Optionally, the meanings of preset algorithm 1 and preset algorithm 2 can be similar. The first module can obtain the identifier of the second algorithm link by calling preset algorithm 2 and taking information 1 as the input parameter.

[0098] S504, the first module sends the identifier of the second algorithm link to the camera application, and the camera application receives the identifier of the second algorithm link.

[0099] S505, The camera application sends a photo-taking request to the camera hardware abstraction layer. The photo-taking request includes photo-taking parameters and the identifier of the second algorithm link. Correspondingly, the camera hardware abstraction layer receives the photo-taking request and forwards it to the image sensor.

[0100] Alternatively, the camera application can forward the photo-taking request to the camera application abstraction layer through the camera access interface of the application framework layer.

[0101] Optionally, the photo-taking request forwarded by the camera hardware abstraction layer to the image sensor may not include the identifier of the second algorithm link, but this application does not limit this.

[0102] S506. The first module determines the image processing algorithms included in the first algorithm link based on the first algorithm link identifier, instantiates the image processing algorithms included in the first algorithm link, and records the feedback value 1 of each image processing algorithm after instantiation.

[0103] S507. The camera hardware abstraction layer determines the image processing algorithm included in the second algorithm link based on the identifier of the second algorithm link included in the photo capture request, instantiates the image processing algorithm included in the second algorithm link, and records the feedback value 2 of the image processing algorithm after each instantiation is completed.

[0104] In one possible implementation, the first module can execute S506 immediately after executing S503 to S504, and the camera hardware abstraction layer can also execute S507 immediately after receiving the photo-taking request. In this way, the camera hardware abstraction layer can execute S507 without waiting for the first module to complete S506. That is to say, the execution of S506 by the first module and the execution of S507 by the camera hardware abstraction layer in the electronic device can be carried out in parallel. Compared with the implementation method in related technologies, this is beneficial to save the time of the first module executing S506 in the camera photo-taking process, which is beneficial to reduce the overall time required for the photo-taking process, improve the efficiency of photo-taking, and enhance the user experience.

[0105] Furthermore, algorithm instantiation can be understood as loading the algorithm's code into the electronic device's runtime memory. This means that before the first module instantiates the image processing algorithm corresponding to the first algorithm link and / or the camera hardware abstraction layer instantiates the image processing algorithm corresponding to the second algorithm link, runtime memory needs to be allocated for these algorithms. Under heavy load conditions (which can be understood as the real-time processor occupancy rate of the electronic device being high and the processor's processing efficiency being insufficient to meet the current demand), memory is tight, and the processor's remaining available memory is insufficient. However, the memory required by the image processing algorithms related to the camera application is usually large, and the processor's efficiency cannot meet the demand. In this case, the time for the image processing algorithms related to the camera application to obtain memory may increase several times compared to the time for obtaining memory when the processor is lightly loaded or idle. However, in the method provided in this application embodiment, the execution of S506 by the first module in the electronic device and the execution of S507 by the camera hardware abstraction layer can be carried out in parallel, which can reduce the overall time required for the shooting process and optimize the user's performance experience.

[0106] S508, The image sensor acquires an image based on the image capture parameters included in the image capture request, and obtains image 1.

[0107] It should be understood that the process of an image sensor acquiring an image can be understood as the process of exposing the pixel array of the image sensor. After the pixel array of the image sensor is exposed, the image sensor generates a frame of image. The image acquired by the image sensor without being processed by the image signal processor can be called a RAW image.

[0108] Optionally, the camera hardware abstraction layer may execute S507 after forwarding the photo capture request to the image sensor, or it may forward the photo capture request to the image sensor during the execution of S507, or it may forward the photo capture request to the image sensor after the execution of S507. This application does not limit this.

[0109] Regardless of when the camera hardware abstraction layer forwards the photo-taking request to the image sensor, since the instantiation of the image processing algorithm corresponding to the first algorithm link by the first module takes a certain amount of time, the electronic device can simultaneously execute the image sensor's image acquisition process, the camera hardware abstraction layer's instantiation of the image processing algorithm corresponding to the second algorithm link, and even the subsequent processing of the image sensor's image acquired by the image sensor using the image processing algorithm corresponding to the second algorithm link by the camera hardware abstraction layer (for example, it may include the following S510 and / or S511). In this way, compared with the related technology that requires the image processing algorithm corresponding to the first algorithm link to be instantiated before the photo-taking command is issued, image processing time can be saved and image processing efficiency can be improved.

[0110] S509, The image sensor transmits image 1 to the camera hardware abstraction layer, and the corresponding camera hardware abstraction layer receives image 1.

[0111] In one possible implementation, the image sensor transmits the acquired RAW image, which has not been processed by the image signal processor, to the image signal processor. The image signal processor then processes the RAW image into a YUV image and transmits it to the camera hardware abstraction layer via the camera device driver. The camera hardware abstraction layer then executes the above-described S507 based on the YUV image. In another possible implementation, the image sensor directly transmits the acquired RAW image, which has not been processed by the image signal processor, to the camera hardware abstraction layer, which then executes the above-described S507 based on the RAW image. That is, the format of image 1 can be either RAW or YUV format, and this application does not specifically limit it.

[0112] After receiving image 1, the camera hardware abstraction layer can continue to execute S510.

[0113] S510: The camera hardware abstraction layer determines whether all image processing algorithms corresponding to the second algorithm link have received feedback value 2. If yes, it continues to execute S511. If no, it continues to execute S510 after waiting period 1 until the camera hardware abstraction layer outputs "yes" after executing S510, and then continues to execute S511.

[0114] S511, the camera hardware abstraction layer processes image 1 using the execution order of each image processing algorithm corresponding to the second algorithm link to obtain image 2.

[0115] S512, the camera hardware abstraction layer transmits image 2 to the first module, and the corresponding first module receives image 2.

[0116] In some implementations, the camera hardware abstraction layer determines whether all image processing algorithms corresponding to the second algorithm link have received feedback value 2 by judging whether the number of feedback value 2 is the same as the number of nodes in the second algorithm link. If they are the same, it can be understood that all image processing algorithms corresponding to the second algorithm link have received feedback value 2, that is, all image processing algorithms corresponding to the second algorithm link have been instantiated.

[0117] Optionally, time period 1 can be 3 seconds, but this application does not specifically limit it.

[0118] In one possible implementation, if the result of the camera hardware abstraction layer executing S510 is "no", after waiting period 3, if the result of the camera hardware abstraction layer executing S510 is still "no", then the camera hardware abstraction layer can skip S511 and execute S512. At this time, the image 2 transmitted by the camera hardware abstraction layer to the first module in S512 can be the unprocessed image 1.

[0119] Optionally, the preset 3 can be a non-negative integer multiple of time period 1, but this application does not limit this.

[0120] Furthermore, after receiving image 2, the first module can execute the following S513 to S515.

[0121] S513: The first module determines whether all image processing algorithms corresponding to the first algorithm link have received feedback value 1. If yes, it continues to execute S514; otherwise, it continues to execute S513 after waiting period 2, until the camera hardware abstraction layer outputs "yes" in S513, and then continues to execute S514. This ensures that all image processing algorithms corresponding to the first algorithm link are instantiated before processing the image, thus ensuring the integrity of image processing and guaranteeing image quality.

[0122] S514. The first module processes image 2 using the execution order of each image processing algorithm corresponding to the first algorithm link to obtain the captured image.

[0123] S515, the first module transmits the captured image to the camera application, and the corresponding first module receives the captured image.

[0124] Optionally, the duration of time period 2 and time period 1 can be the same or different, and this application does not limit this.

[0125] The method by which the first module determines whether all image processing algorithms corresponding to the first algorithm link have received feedback value 1 is similar to the method by which the camera hardware abstraction layer determines whether all image processing algorithms corresponding to the second algorithm link have received feedback value 2, and will not be elaborated here.

[0126] In one possible implementation, if the output of the first module in S513 is "No", and after waiting period 4, if the output of the first module in S513 is still "No", then the first module can skip S514 and execute S515. In this case, the image transmitted by the first module to the camera application in S512 can be the unprocessed image 2, or the first module can transmit information about an image output error to the camera application. This avoids excessively long waiting times in the image processing flow when problems occur. Timely skipping or error reporting can promptly instruct the user to re-trigger the photo taking or check the electronic device, avoiding excessively long and ineffective waiting times and improving the user experience.

[0127] Optionally, the preset 4 can be a non-negative integer multiple of the time period 2, but this application does not limit this.

[0128] In one possible implementation, the image processing algorithm corresponding to the first algorithm link may include an image fusion algorithm that can fuse multiple frames of images to obtain a high-quality image. In this case, the first module can execute S514 after receiving the target number of images to obtain the captured image.

[0129] In one possible implementation, the first module may use the image processing algorithm in the first algorithm link to process image 2 into a photographed image in JPG or JPEG format, but this application does not limit this.

[0130] In one implementation, after obtaining the captured image, the first module can execute S515 to transmit the captured image to the camera application so that the captured image can be displayed as a thumbnail. Figure 1 As shown in Figure 'a', in the camera application interface, at control 105, users can click on control 105 to view the captured image.

[0131] In another implementation, the first module can also transmit the captured image to a gallery application for storage, and the user can also find the captured image in the gallery.

[0132] In one possible implementation, all steps related to the second algorithm link in the above method 500 can be optional. In method 500, if steps related to the second algorithm link are not included, the camera application can send instruction 1 and the photo request simultaneously or in any order. The above method 500 may not include steps S503-S504, S507, S510-S512, etc. Instruction 1 may not include information 1, the photo request may not include the identifier of the second algorithm link, and S509 may be the image sensor sending image 1 to the first module, but this application does not specifically limit this.

[0133] In another possible implementation, S503 described above can be executed by the camera application. In this case, the camera application can send instruction 1 and the photo request simultaneously or in any order, without waiting for the algorithm instantiation corresponding to the first algorithm link to be completed.

[0134] It is worth noting that the numbering of each step in the various methods described in the embodiments of this application does not constitute a limitation on their execution order. The specific execution order of each step in each method should be based on its internal logic.

[0135] Figure 6 This is a schematic flowchart illustrating an image processing method 600 provided in an embodiment of this application. The method 600 can be applied to an electronic device, the software structure of which can be as follows: Figure 4 As shown, the hardware structure of this electronic device can be as follows: Figure 7 As shown, but this application does not limit it.

[0136] The method 600 may include the following steps:

[0137] S601, in response to the trigger operation of the camera button, load the first algorithm and issue the camera command in parallel.

[0138] S602, Acquire the first image based on the photo capture command.

[0139] S603. After the first algorithm is loaded, process the first image using the first algorithm.

[0140] The first algorithm can be understood as a summary of the first type of image processing algorithms included in the aforementioned first algorithm chain.

[0141] In this embodiment, the electronic device can load the first algorithm and issue the photo-taking command in parallel in response to a trigger operation on the camera button. This means the photo-taking command can be issued without waiting for the first algorithm to finish loading, allowing the electronic device to capture the first image while the first algorithm is loading, unlike related technologies where the photo-taking command is issued only after the first algorithm has finished loading. The method provided in this embodiment saves time in loading the first algorithm during the image processing flow, shortens the overall image processing time, and improves the user experience.

[0142] It should also be understood that the algorithm instantiation mentioned above can be understood as the specific implementation means of algorithm loading. This application does not specifically limit how the first algorithm achieves the loading purpose.

[0143] As an optional embodiment, the photo-taking command is also used to instruct the loading of a second algorithm, and the method further includes: loading the second algorithm based on the photo-taking command, wherein the loading process of the first algorithm and the loading process of the second algorithm overlap in time.

[0144] It should be understood that the second algorithm link described in the second algorithm and method 500 may have a similar meaning and will not be explained again.

[0145] It is worth noting that loading the first algorithm takes time, and the loading of the first algorithm and the issuance of the photo-taking command can be carried out in parallel. When the electronic device loads the second algorithm based on the instruction of the photo-taking command, the loading of the first algorithm may not be completed yet. This may result in the loading processes of the first and second algorithms overlapping in time. Since in this embodiment, the issuance of the photo-taking command does not need to wait for the first algorithm to finish loading, and the photo-taking command is also used to instruct the loading of the second algorithm, the loading of the first and second algorithms in the electronic device can also be carried out in parallel. This means that the loading of the first algorithm, the loading of the second algorithm, and the acquisition of the first image in the electronic device can be carried out in parallel. This is beneficial to shorten the overall image processing time and improve the user experience.

[0146] As an optional embodiment, before processing the first image using the first algorithm, the method further includes: processing the first image using the second algorithm after the second algorithm has been loaded; processing the first image using the first algorithm includes: processing the first image processed by the second algorithm using the first algorithm.

[0147] In this way, since the first algorithm runs after the second algorithm, and the method of this application allows the loading of the second algorithm and the acquisition of the first image to not have to wait for the first algorithm to finish loading, that is, during the loading of the first algorithm, it is possible to acquire the first image and process the first image by the second algorithm in parallel. Compared with the related technologies, which load the second algorithm and acquire the first image only after the first algorithm has finished loading, the image processing efficiency is improved, the image processing time is shortened, and the user experience is improved.

[0148] As an optional embodiment, before processing the first image using the first algorithm, the method further includes: determining whether the first algorithm has been fully loaded; and processing the first image using the first algorithm, including: if it is determined that the first algorithm has not been fully loaded, then after waiting for a preset time, determining whether the first algorithm has been fully loaded again, until it is determined that the first algorithm has been fully loaded, and then processing the first image using the first algorithm.

[0149] Optionally, the term "preset time" here may have a similar meaning to "segment 2" in method 500, but this application does not limit it in this respect.

[0150] In this embodiment of the application, before processing the first image using the first algorithm, it is determined whether the first algorithm has been fully loaded. If it is determined that the first algorithm has not been fully loaded, the process waits. After each preset time, it is determined again whether the first algorithm has been fully loaded. Only when it is determined that the first algorithm has been fully loaded is the first algorithm used to process the first image. This helps to ensure the processing effect of the first image.

[0151] In one possible implementation, after determining that the first algorithm has not finished loading and the waiting time exceeds a threshold time, the electronic device can directly output the image that has not been processed by the first algorithm, or process the first image using the first type of image processing algorithm that has been loaded in the first algorithm, or display information through the user interface (UI) to prompt the user to retake the photo. This helps avoid the user waiting blindly for too long and improves the user experience when the first algorithm times out.

[0152] Optionally, the “threshold time” here may have a similar meaning to “time period 4” in method 500, but this application does not specifically limit it.

[0153] As an optional embodiment, the first algorithm includes one or more first-class image processing algorithms. During the loading process of the first algorithm, a feedback value is recorded after each first-class image processing algorithm is loaded. The method for determining whether the first algorithm has been loaded includes: determining whether the number of feedback values ​​is equal to the number of one or more first-class image processing algorithms included in the first algorithm; if the number of feedback values ​​is equal to the number of one or more first-class image processing algorithms, it is determined that the first algorithm has been loaded; if the number of feedback values ​​is not equal to the number of one or more first-class image processing algorithms, it is determined that the first algorithm has not been loaded.

[0154] Optionally, before processing the first image using the second algorithm, it can be determined whether the second algorithm has finished loading. The method for determining whether the second algorithm has finished loading, and the subsequent processing after determining whether it has finished loading, can be similar to the method for determining whether the first algorithm has finished loading, i.e., the subsequent processing, and will not be described in detail here.

[0155] As an optional embodiment, the electronic device includes a camera application, a first module, and an image sensor; in response to a trigger operation on a shutter button, the loading of a first algorithm and the issuance of a shutter command are performed in parallel, including: the camera application, in response to the trigger operation on the shutter button, determines shutter parameters and an identifier of the first algorithm based on the identifier of the camera application's shooting mode and the scene parameters of the shooting mode at the time the shutter button is triggered; the camera application transmits a first instruction to the first module, the first instruction including the identifier of the first algorithm, to instruct the first module to load the first algorithm; the camera application issues a shutter command containing shutter parameters to the image processor, to instruct the image processor to acquire a first image; wherein, the loading of the first algorithm by the first module and the issuance of the shutter command to the image sensor by the camera application are performed in parallel.

[0156] Optionally, the first instruction can be understood as instruction 1 in method 500, but this application does not limit it in this way.

[0157] As an optional embodiment, the electronic device further includes a second module. The first instruction also includes an identifier of the shooting mode and scene parameters of the shooting mode. The first instruction is also used to instruct the first module to determine the identifier of the second algorithm. The method further includes: the first module determining the identifier of the second algorithm based on the identifier of the shooting mode and scene parameters of the shooting mode included in the first instruction, and transmitting the identifier of the second algorithm to the camera application; the camera application sending a shooting instruction containing shooting parameters to the image sensor, including: the camera application transmitting a shooting request to the second module, the shooting request including the identifier of the second algorithm; the second module loading the second algorithm based on the identifier of the second algorithm, and forwarding the shooting instruction containing shooting parameters to the image sensor.

[0158] Optionally, the second module may be the camera hardware abstraction layer described in method 500 above, but this application does not specifically limit it.

[0159] The image processing method of the present application embodiments has been described above. The apparatus for performing the above method provided in the present application embodiments is described below. Those skilled in the art will understand that the methods and apparatus can be combined with and referenced by each other, and the related apparatus provided in the present application embodiments can perform the steps in the above method.

[0160] It should be understood that, in order to achieve the functions described in the above embodiments, the electronic device may include hardware structures and / or software modules corresponding to perform each function. Those skilled in the art should readily recognize that, based on the method steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] This application embodiment can divide the apparatus for implementing the method into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0162] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown. For example... Figure 7 As shown, the electronic device may include a processor 710, a memory 720, a display screen 730, a sensor module 740, a universal serial bus (USB) interface 750, a charging management module 760, a power management module 761, a battery 762, a camera 770, etc.

[0163] The processor 710 may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0164] The memory 720 can be used to store computer executable program code, which includes instructions, such as the program code corresponding to the method provided in the embodiments of this application.

[0165] Display screen 730 is used to display images, videos, etc. Display screen 730 includes a display panel. In some embodiments, the electronic device may include one or N displays screens 730, where N is a positive integer greater than 1. The electronic device implements display functions through a GPU, display screen 730, and application processor, etc. In some implementations, the display screen can communicate with a touch sensor to collect signals generated by the user's touch operations.

[0166] Camera 770, also known as an image sensor, is used to capture still images or videos. In some embodiments, the electronic device may include one or P cameras, where P is a positive integer greater than 1. In this application embodiment, the camera can be used to acquire images and realize the photo-taking function.

[0167] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0168] The image processing method provided in this application embodiment can be applied to electronic devices with camera functions, and the software structure of the electronic device can be as follows: Figure 4 As shown, the hardware structure can be as follows Figure 7 As shown, the specific form of the electronic device can be referred to the above description, and will not be repeated here.

[0169] Figure 8 An exemplary schematic diagram of a chip structure provided in an embodiment is shown. Chip 800 includes one or more (including two) processors 801, communication lines 802, communication interfaces 803, and memory 804.

[0170] In some implementations, memory 804 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0171] The methods described in the embodiments of this application can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit in the hardware of processor 801 or by instructions in software form. The processor 801 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 801 can implement or execute the various processing-related methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0172] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 804, and processor 801 reads the information in memory 804 and, in conjunction with its hardware, completes the steps of the above method.

[0173] The processor 801, memory 804 and communication interface 803 can communicate with each other through communication line 802.

[0174] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0175] In the embodiments of this application, the chip 800 described above may also be a chip system, such as a system on chip (SOC), and this application does not limit it in this way.

[0176] This application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and stores computer program code, including computer instructions; one or more processors invoke the computer instructions to cause the electronic device to perform the methods described in the above embodiments.

[0177] This application provides a chip or chip system. The chip or chip system is applied to an electronic device and includes one or more processors. These processors invoke computer instructions to cause the electronic device to execute the methods described in the above embodiments. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0178] This application also provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When these computer instructions are executed on an electronic device, they cause the electronic device to perform the methods described in the above embodiments. The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted on the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0179] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data using laser optics. Combinations of the above should also be included within the scope of computer-readable media.

[0180] This application provides a computer program product, which includes computer program code. When the computer program code is run on an electronic device, the electronic device performs the method described in the above embodiments.

[0181] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0182] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized by, The method is applied to an electronic device, and the method comprises: In response to a triggering operation on a photographing button, loading of a first algorithm and issuing of a photographing instruction are performed in parallel; A first image is captured based on the photographing instruction; In a case where the loading of the first algorithm is completed, the first image is processed by using the first algorithm.

2. The method of claim 1, wherein, The photographing instruction is also used to instruct loading of a second algorithm, and the method further comprises: Based on the photographing instruction, the second algorithm is loaded, and the loading process of the first algorithm overlaps in time with the loading process of the second algorithm.

3. The method of claim 2, wherein, Before the processing of the first image by using the first algorithm, the method further comprises: In a case where the loading of the second algorithm is completed, the first image is processed by using the second algorithm; The processing of the first image by using the first algorithm comprises: The first image processed by using the second algorithm is processed by using the first algorithm.

4. The method according to any one of claims 1 to 3, characterized in that, Before the processing of the first image by using the first algorithm, the method further comprises: It is judged whether the loading of the first algorithm is completed; The processing of the first image by using the first algorithm comprises: If it is judged that the loading of the first algorithm is not completed, it is judged again whether the loading of the first algorithm is completed after waiting for a preset time, and in a case where it is judged that the loading of the first algorithm is completed, the first image is processed by using the first algorithm.

5. The method of claim 4, wherein, The first algorithm comprises one or more first-type image processing algorithms, in the process of loading the first algorithm, a feedback value is recorded after the loading of each first-type image processing algorithm is completed, and the method of judging whether the loading of the first algorithm is completed comprises: It is judged whether the number of the feedback values is equal to the number of the one or more first-type image processing algorithms included in the first algorithm, in a case where the number of the feedback values is equal to the number of the one or more first-type image processing algorithms, it is judged that the loading of the first algorithm is completed; In a case where the number of the feedback values is not equal to the number of the one or more first-type image processing algorithms, it is judged that the loading of the first algorithm is not completed.

6. The method according to any one of claims 1 to 5, characterized in that, The electronic device comprises a camera application, a first module and an image sensor; The response to the triggering operation on the photographing button, the loading of the first algorithm and the issuing of the photographing instruction in parallel comprises: The camera application determines photographing parameters and an identifier of the first algorithm based on an identifier of a shooting mode of the camera application at a photographing button triggering time and scene parameters of the shooting mode in response to the triggering operation on the photographing button; The camera application transmits a first instruction to the first module, the first instruction comprising the identifier of the first algorithm, to instruct the first module to load the first algorithm; The camera application issues the photographing instruction containing the photographing parameters to the image processor to instruct the image processor to capture the first image; The loading of the first algorithm by the first module and the issuing of the photographing instruction by the camera application to the image sensor are performed in parallel.

7. The method of claim 6, wherein, The electronic device further includes a second module, the first instruction further includes an identifier of the shooting mode and a scene parameter of the shooting mode, and the first instruction is further used to instruct the first module to determine an identifier of a second algorithm, and the method further includes: The first module determines the identifier of the second algorithm based on the identifier of the shooting mode and the scene parameter of the shooting mode included in the first instruction, and transmits the identifier of the second algorithm to the camera application; The camera application issues the shooting instruction containing the shooting parameter to the image sensor, including: The camera application transmits the shooting request to the second module, and the shooting request further includes the identifier of the second algorithm; The second module loads the second algorithm based on the identifier of the second algorithm, and forwards the shooting instruction containing the shooting parameter to the image sensor.

8. An electronic device, comprising: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code including computer instructions, and the one or more processors invoke the computer instructions to enable the electronic device to perform the method in any one of claims 1 to 7.

9. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors used to invoke computer instructions to enable the electronic device to perform the method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions, and when the computer instructions run on an electronic device, enable the electronic device to perform the method in any one of claims 1 to 7.

11. A computer program product, characterised in that, The computer program product includes computer program code, and when the computer program code runs on an electronic device, enables the electronic device to perform the method in any one of claims 1 to 7.

Images