Shooting methods, electronic equipment

Abstract

This application discloses a shooting method and electronic device. When an electronic device uses any camera to shoot an image, it uses the image generated by the reference camera to perform color and brightness processing on the image of that camera. This ensures that the color and brightness of the image output by each camera are consistent with the output of the reference camera. This solution can provide users with a consistent color and brightness experience. When the user switches the camera of the electronic device, the user will not perceive any jumps in the color and brightness of the preview image, and can provide a good shooting experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to shooting methods and electronic equipment. Background Art

[0002] Due to device size constraints, compact camera modules (CCMs) in electronic devices like mobile phones and tablets are often limited in size. Consequently, multiple camera modules are typically used to capture different scenes, such as a main camera, wide-angle camera, and telephoto camera. Differences between different camera modules, such as the varying spectral response of the color filter array (CFA) in the sensor, result in varying color and brightness in the output images. Summary of the Invention

[0003] The present application provides a shooting method and an electronic device that can provide users with a consistent color and brightness experience.

[0004] In a first aspect, a shooting method is provided, which is applied to an electronic device, the electronic device including a first camera and a second camera, the method may include: receiving a user operation to start the second camera; collecting first original image data through the first camera, performing RAW domain processing and color and brightness processing on the first original image data to obtain a first reference image; starting the second camera, collecting second original image data through the second camera, the second original data and the first original data being collected at the same time or the difference between the collection times does not exceed a threshold, performing RAW domain processing on the second original image data, or performing RAW domain processing and partial color and brightness processing on the second original image data to obtain a first image to be processed; determining a first parameter, the first parameter being obtained based on the first reference image and the first image to be processed; performing further color and brightness processing on the first image to be processed using the first parameter to obtain a first preview image; and displaying the first preview image.

[0005] Implementing the method of the first aspect synchronizes the color and brightness of the image output by the second camera with that of the first camera. This method uses the intermediate image processed by the first camera to predict the parameters required by the second camera. This method eliminates the need for calibration mapping relationships and statistical information, resulting in lower power consumption. Furthermore, this solution enables real-time parameter validation, ensuring that the color and brightness of the images output by the second camera and the first camera remain consistent.

[0006] In combination with the first aspect, in some embodiments, the method may further include: generating first metadata when the first original image data is acquired; wherein the first parameter is also obtained based on the first metadata.

[0007] In combination with the first aspect, in some embodiments, the method may further include: generating second metadata when the second original image data is acquired; wherein the first parameter is also obtained based on the second metadata.

[0008] In the above two embodiments, the metadata, as a priori information, can enable the electronic device to output first parameters that are more suitable for the second camera.

[0009] In conjunction with the first aspect, in some embodiments, the electronic device can input the first reference image and the first image to be processed into a neural network, and output the first parameter through the neural network. The neural network can accelerate the acquisition of the first parameter, thereby improving the efficiency of the present application.

[0010] In combination with the previous embodiment, the neural network is stored in the neural network processor NPU of the electronic device.

[0011] In combination with the first aspect, in some embodiments, the first parameter includes any one or more of the following: a color correction matrix CCM, a color lookup table LUT, a dynamic range correction DRC, a global tone mapping GTM, a gamut mapping, or a local tone mapping LTM relationship.

[0012] In combination with the first aspect, in some implementations, the electronic device may further detect the stability of the image captured by the second camera, and adjust the frequency of image processing using the above method according to the stability.

[0013] In some embodiments, after the method provided in the first aspect, the electronic device may also collect third original image data through a second camera, and the movement distance of the electronic device when collecting the third original image data and collecting the second original image data is less than a threshold; perform RAW domain processing on the third original image data, or perform RAW domain processing and partial color brightness processing on the third original image data to obtain a second image to be processed; use the first parameter to further perform color brightness processing on the second image to be processed to obtain a second preview image; and display the second preview image. In some embodiments, the electronic device may also perform smoothing processing on the first parameter to obtain a new parameter, and use the new parameter to perform color brightness processing on the second image to be processed to obtain a second preview image. Using the first parameter or similar parameters can ensure the continuity and smoothness of the preview effect, and can also reduce the power consumption of the device. This embodiment can ensure that the color brightness of the image output by the second camera is consistent with the color brightness of the image output by the first camera, and can also reduce the power consumption of the electronic device as much as possible.

[0014] In some embodiments, after the method provided in the first aspect, the electronic device can collect fourth original image data through the second camera, and the movement distance of the electronic device is greater than a threshold when collecting the fourth original image data and collecting the second original image data; perform RAW domain processing on the fourth original image data, or perform RAW domain processing and partial color and brightness processing on the fourth original image data to obtain a third image to be processed; collect fifth original image data through the first camera, perform RAW domain processing and color and brightness processing on the fifth original image data to obtain a second reference image, and the fifth original data and the fourth original data are collected at the same time or the difference between the collection times does not exceed a threshold; determine a second parameter, and the second parameter is obtained based on the second reference image and the third image to be processed; use the second parameter to further perform color and brightness processing on the third image to be processed to obtain a third preview image; and display the third preview image.

[0015] In combination with the first aspect, in some embodiments, the user operation of starting the second camera specifically includes any one of the following: a user operation of starting a camera application in the electronic device; or a user operation for switching the third camera to the second camera, and the electronic device also includes a third camera.

[0016] In combination with the first aspect, in some embodiments, the electronic device can also switch cameras. Specifically, the electronic device also includes a fourth camera. After displaying the first preview image, the method can also include: receiving a user operation to switch the second camera to the fourth camera; collecting sixth original image data through the first camera, performing RAW domain processing and color brightness processing on the sixth original image data to obtain a third reference image; starting the fourth camera, collecting seventh original image data through the fourth camera, the seventh original data and the sixth original data being collected at the same time or the difference between the collection times does not exceed a threshold, performing RAW domain processing on the seventh original image data, or performing RAW domain processing and partial color brightness processing on the seventh original image data to obtain a fourth image to be processed; determining a third parameter, the third parameter being obtained based on the third reference image and the fourth image to be processed; performing further color brightness processing on the fourth image to be processed using the third parameter to obtain a fourth preview image; and displaying the fourth preview image.

[0017] As can be seen from the above embodiment, when an electronic device uses different cameras to capture images, the image generated by the first camera is used to perform color and brightness processing on the image of that camera. This ensures that the color and brightness of the images output by each camera are consistent with those output by the first camera. At the moment when the electronic device switches cameras and during the steady state after the switch, the color and brightness of the preview image displayed by the electronic device can remain consistent with those of the first camera, thus resolving the problem of inconsistent preview image color and brightness caused by differences between different cameras.

[0018] In combination with the first aspect, in some embodiments, the electronic device can use different reference cameras in different scenarios. The electronic device also includes a fifth camera and a sixth camera, the fifth camera and the first camera being different cameras. The method may further include: receiving a user operation to activate the sixth camera; acquiring eighth raw image data through the fifth camera, performing RAW domain processing and color brightness processing on the eighth raw image data to obtain a fourth reference image; activating the sixth camera, acquiring ninth raw image data through the sixth camera, the ninth raw image data and the eighth raw data being acquired at the same time or the difference between the acquisition times does not exceed a threshold, performing RAW domain processing on the ninth raw image data, or performing RAW domain processing and partial color brightness processing on the ninth raw image data to obtain a fifth image to be processed; determining a fourth parameter, the fourth parameter being obtained based on the fourth reference image and the fifth image to be processed; performing further color brightness processing on the fifth image to be processed using the fourth parameter to obtain a fifth preview image; and displaying the fifth preview image.

[0019] In a second aspect, an electronic device is provided, comprising: a first camera, a second camera, a memory, and one or more processors; the first camera, the second camera, and the memory are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method provided in the first aspect or any one of the embodiments of the first aspect.

[0020] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method provided in the first aspect or any embodiment of the first aspect.

[0021] In a fourth aspect, a computer-readable storage medium is provided, comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method provided in the first aspect or any one of the embodiments of the first aspect.

[0022] In a fifth aspect, a computer program product comprising instructions is provided. When the computer program product is run on an electronic device, the electronic device executes the method provided in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of the shooting method provided in an embodiment of the present application;

[0024] Figure 2 A hardware structure block diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 3 The software architecture of the electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0026] The structure and working principle of the camera module

[0027] The camera module may primarily include a lens, a sensor, and an infrared filter. The sensor may also be called an image sensor. In some embodiments, the camera module may also include a base bracket, a voice coil motor (VCM), a flexible printed circuit (FPC), a printed circuit board (PCB), a driver integrated circuit (driver IC), and the like.

[0028] The lens includes one or more lenses, which are used to receive light signals and focus them on the photosensitive element.

[0029] The photosensitive element can be a charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) phototransistor. Essentially, a photosensitive element is a semiconductor chip, primarily composed of modules such as a photosensitive area array (Bayer array or pixel array), timing control, analog signal processing, and analog-to-digital conversion. The surface of the photosensitive element contains hundreds of thousands to millions of photodiodes. When exposed to light, the photodiodes generate an electric charge, converting the light into an electrical signal, which is then converted internally into a digital image signal. Each pixel of the photosensitive element can only sense one of the following: red (R), green (G), or blue (B). Therefore, the data stored in each pixel is monochrome. The digital image signal converted from the light source signal captured by the photosensitive element is called RAW Data. RAW Data is an unprocessed and uncompressed format and can also be referred to as raw image data.

[0030] A file that records the original information of the photosensitive element in an electronic device can be called a RAW file. It records some metadata generated by shooting, such as ISO settings, shutter speed, aperture value, white balance, shooting mode (such as night mode, face mode, large aperture mode, movie mode, etc.), face frame information, etc.

[0031] The infrared filter is located between the lens and the photosensitive element and is used to filter out near-infrared light from the light entering the lens. This is because the human eye cannot see infrared light, but the photosensitive element can sense infrared light, so the infrared light in the light needs to be filtered out to make the image closer to what the human eye sees.

[0032] The photosensitive element transmits the converted digital image signal to the image signal processor (ISP), which processes the digital image signal using a series of digital image processing algorithms. The ISP then outputs the processed digital image signal to the digital signal processor (DSP) for further processing.

[0033] An ISP can be configured either internally or externally. External means the ISP is installed separately from the electronic device's application processor (AP), while internal means the ISP is embedded within the AP. Multiple camera modules can share a single ISP. Multiple camera modules can share a single DSP.

[0034] During the imaging process of the camera module, the ISP is responsible for receiving the raw image data output by the photosensitive element and performing a series of digital image algorithm processing on the raw image data. These processing may include any one or more of the following:

[0035] RAW processing refers to compensating for hardware-induced defects in the original image. This includes, but is not limited to, any one or more of the following: black level compensation (BLC), lens shading correction (LSC), bad pixel correction (BPC), RAW denoise, and color interpolation (demosaic).

[0036] RGB domain processing refers to the process of converting a RAW image into an RGB image and then performing some corrections on the RGB image. RGB domain processing may include, but is not limited to, any one or more of the following: automatic white balance (AWB), RGB noise reduction (NR color), X-tasing, color correction, wide dynamic range (WDR), gamma enhancement, and color space matrix conversion, such as RGB to YUV conversion.

[0037] YUV domain processing involves converting RGB images to YUV format, performing color denoising, image sharpening, and color correction on the images in the YUV domain, and then performing multi-frame synthesis to improve image tolerance or reduce noise. YUV domain processing includes, but is not limited to, any one or more of the following: color denoise, edge enhancement, contract brightness, and hue saturation.

[0038] After the above processing, the ISP outputs data in YUV or RGB format to the DSP, which encodes the data and outputs the processed image. The output image can be stored in a cache or other storage location.

[0039] Among the above-mentioned processes, RGB domain processing and YUV domain processing involve processing the color and brightness of the image, and therefore can be referred to as color and brightness processing of the image. In other words, color and brightness processing can include the above-mentioned RGB domain processing and / or YUV domain processing.

[0040] For the sake of simplicity, the camera module will be referred to as the camera in the following text.

[0041] The electronic device provided in this application may include multiple cameras, and the multiple cameras can meet the shooting needs of the electronic device in different scenarios. For example, the electronic device may include a rear main camera, a telephoto camera and a wide-angle camera, and may also include a front camera, and may also include image sensors such as a multispectral sensor, a CMY sensor, an RGGB sensor, an RYYB sensor, and an RGBW sensor. Among them, the main camera can be used to meet general rear shooting needs, the telephoto camera can meet telephoto shooting needs, and the wide-angle camera can meet large depth of field shooting needs. The images taken by the main camera, the telephoto camera, the wide-angle camera and the front camera can all be displayed to the user, and the images taken by the multispectral sensor generally do not need to be displayed to the user.

[0042] A multispectral sensor is also a type of camera, consisting of an optical head and electronics. The optical head includes filters, a camera lens, and a detector. The optical head uses several independent area array CCDs to acquire data in frames. Spectral selection and resolution are ensured by filters. Multispectral sensors can capture images with accurate color, but their resolution is relatively low. Multispectral sensors can be either single-point or multi-point multispectral sensors.

[0043] After an electronic device switches cameras, differences between cameras, such as hardware (e.g., inconsistent spectral responses of infrared filters) or software (e.g., exposure strategies of photosensitive elements), can cause differences in color and brightness in the image displayed on the preview interface, impacting the user experience. These differences can arise between models produced by different manufacturers, models produced by the same manufacturer, the same model produced by the same manufacturer but in different batches, or the same model produced by different manufacturers.

[0044] To solve the above problems, the present application provides a shooting method and an electronic device. When the electronic device uses any camera to shoot an image, the image generated by the reference camera is used to perform color and brightness processing on the image of the camera. This ensures that the color and brightness of the image output by each camera are consistent with the output of the reference camera. This solution can ensure that when the electronic device uses any camera, there will not be a large difference in the color and brightness of the preview image, and can provide the user with a consistent color and brightness experience. In this way, when the user switches the camera of the electronic device, the user will not perceive the jump in the color and brightness of the preview image, and can have a good shooting experience.

[0045] The shooting method provided by this application

[0046] Figure 1 The flowchart of the shooting method provided by the embodiment of the present application is exemplarily shown. The method is applied to an electronic device including multiple cameras, the multiple cameras including at least a first camera and a second camera, the first camera being the reference camera in the present application.

[0047] like Figure 1 As shown, the method may include the following steps:

[0048] S101: The electronic device receives a user operation to activate a second camera.

[0049] The user operation to start the second camera can be implemented as any of the following:

[0050] 1. The user operation is a user operation of starting a camera application or a shooting function of an application in the electronic device.

[0051] Camera applications include, for example, system camera applications in electronic devices and third-party camera applications. Some applications, such as WeChat and Taobao, have integrated camera functions. When these applications activate the camera function, they can access the electronic device's camera.

[0052] In the first case, the second camera can be the default camera for a camera app or a photography function within an application. The default camera can be the main camera on the back of the electronic device or the front camera. For example, some selfie apps can use the front camera as the default camera.

[0053] 2. The user operation is a user operation that switches cameras after launching a camera application or a shooting function of an application on the electronic device. For example, the user operation is a user operation that switches the third camera to the second camera. The electronic device also includes a third camera.

[0054] In the second case, the third camera and the second camera are different cameras. The third camera or the second camera can be any camera in the electronic device that can be used to display images, such as a main camera, a telephoto camera, a wide-angle camera, a front camera, etc.

[0055] The user operation in S101 of the present application can be implemented as a user operation on the display screen of an electronic device, such as clicking on the icon of a camera application on the desktop or clicking on a shooting function control. It can also be implemented as a voice user operation or as a gesture, etc., which is not limited here.

[0056] S102: The electronic device collects first original image data through a first camera, and performs RAW domain processing and color and brightness processing on the first original image data to obtain a first reference image.

[0057] The first camera is a reference camera, which can be set by default in the electronic device, for example, it can be set in advance before the electronic device leaves the factory, or the user can independently select one of multiple cameras as a reference camera. The reference camera can be a camera among the various cameras of the electronic device that captures images with more accurate color and brightness, more realistic images, or more in line with the user's aesthetic needs. The reference camera can be any one of the multiple cameras of the electronic device, such as any one of the main camera, telephoto camera, wide-angle camera, front camera, or multispectral sensor. The first camera and the second camera are different cameras.

[0058] In an embodiment of the present application, the first camera may be turned on by default after the electronic device starts a camera application or a shooting function of an application. If the electronic device receives the first user operation in S101, the electronic device may simultaneously start the first camera and the second camera in response to the user operation. If the electronic device receives the second user operation in S101, the electronic device has already started the first camera when the camera application or the shooting function of an application in the electronic device was started, and may start the second camera in response to the user operation.

[0059] The first original image data collected by the electronic device through the first camera refers to RAWData collected by the first camera.

[0060] The electronic device performs RAW domain processing and color and brightness processing on the first original image, as described above. The first reference image is obtained through color and brightness processing, and therefore contains reference information about the color and brightness of the image when processed by the first camera. The first reference image also contains reference information about the color and brightness of the content currently captured by the first camera.

[0061] In some embodiments, when the electronic device captures the first original image data through the first camera, it may also generate first metadata when the first original image data is captured.

[0062] In step S103, the electronic device responds to the user operation in step S101 by activating a second camera and capturing second original image data through the second camera. The second original image data and the first original data are captured at the same time or the difference between the capture times does not exceed a threshold. The electronic device performs RAW domain processing on the second original image data, or performs RAW domain processing and partial color and brightness processing on the second original image data to obtain a first image to be processed.

[0063] The second original image data collected by the electronic device through the second camera refers to RAWData collected by the second camera.

[0064] The second original image data and the first original image data are collected at the same time or the difference between the collection times does not exceed a threshold. If the second camera and the first camera have the same collection period and collection frequency, the two original image data can be collected at the same time. If the second camera and the first camera have different collection periods and collection frequencies, the first original data can be the original image data collected by the first camera at the time closest to the collection time of the second original data, either before or after the collection time. Based on the above definition, the second original image data and the first original image data are collected at a similar or identical time, and both are collected in the same environment. If the first camera and the second camera are both rear cameras, the content captured by both is the same or similar.

[0065] The process of the electronic device performing RAW domain processing on the second original image can refer to the introduction to RAW domain processing in the previous article. The partial color brightness processing performed by the electronic device on the second original image data may include some operations in the RGB domain processing and YUV domain processing mentioned above, such as automatic white balance, color correction and other operations, of course, it may not include automatic white balance, color correction and other operations. The actual processing content of the partial color brightness processing here is related to the position of the neural network mentioned later in the entire image processing flow. If the partial color brightness processing here performs more operations, then the color brightness processing after the neural network will be less. If the partial color brightness processing here performs fewer operations, then the color brightness processing after the neural network will be more. The color brightness processing before and after the neural network constitutes the complete brightness processing of the image. The specific operations of this partial color brightness processing can be set by the R&D personnel as needed.

[0066] The content of the first to-be-processed image obtained by processing is the same as or similar to the content of the first reference image.

[0067] In some embodiments, when the electronic device captures the second original image data through the second camera, it can also generate second metadata when the second original image data is captured.

[0068] S104: The electronic device determines a first parameter.

[0069] In some embodiments, the electronic device may determine a first parameter based on a first reference image and a first image to be processed. Based on the color and brightness difference between the first reference image and the first image to be processed, the electronic device may output a parameter, i.e., a first parameter, suitable for the first image to be processed that is close to or identical to the color and brightness of the first reference image. The first reference image and the first image to be processed have identical or similar content, and the first parameter determined by the electronic device is based on the identical or similar image content and is therefore suitable for the first image to be processed. Furthermore, the first parameter has a color and brightness close to or identical to that of the first reference image.

[0070] In some embodiments, in addition to the first reference image and the first image to be processed, the electronic device may also determine the first parameter based on the first metadata and / or the second metadata. The metadata, as prior information, allows the electronic device to output first parameters that are more suitable for the second camera and the current shooting scene.

[0071] In some embodiments, the electronic device may utilize a neural network to determine the first parameter. Specifically, the electronic device may utilize the data used to determine the first parameter as input to the neural network, and output the first parameter via the neural network. The data used to determine the first parameter may include a first reference image and a first image to be processed, and may also include first metadata and / or second metadata. The neural network may be stored in a neural-network processing unit (NPU) of the electronic device.

[0072] In some embodiments, the electronic device may pre-process the data used to determine the first parameter so that it meets the input requirements of the neural network. For example, the electronic device may perform operations such as translation and cropping on the first reference image to reduce the difference in field of view (FOV) between the first image to be processed and the first image to be processed. For another example, the electronic device may process the first reference image and the first image to be processed into small-resolution images (i.e., tiny images) of a preset size, or process these data into a preset number of channels, etc.

[0073] The neural network can be pre-trained. Different types of data used to determine the first parameter may correspond to different neural networks. For example, the training data may include a reference image and an image to be processed, as well as standard parameters corresponding to the image to be processed that have similar color and brightness effects as the reference image. For another example, the training data may include a reference image and an image to be processed, as well as metadata for each image, as well as standard parameters corresponding to the image to be processed that have similar color and brightness effects as the reference image.

[0074] The first parameter is a parameter related to color and brightness, and may include, but is not limited to, any one or more of the following: color correction matrix (CCM), color lookup table (LUT), dynamic range calibration (DRC), global tone mapping (GTM), gamut mapping, or local tone mapping (LTM). The parameter categories included in the first parameter can be pre-defined by device developers.

[0075] The number of parameters included in the first parameter is related to the color brightness processing performed by the electronic device in S104. The color brightness processing performed by the electronic device in S103 and the color brightness processing performed in S105 constitute a complete color brightness processing. If no color brightness processing is performed in S103, the first parameter includes all parameters required for the partial color brightness processing (i.e., the color brightness processing required in S105). If partial color brightness processing is performed in S103, the first parameter includes parameters required for the remaining color brightness processing (i.e., the color brightness processing required in S105).

[0076] S105 : The electronic device performs further color and brightness processing on the first image to be processed using the first parameter to obtain a first preview image.

[0077] The electronic device can use the first parameters to perform further color and brightness processing on the first image to be processed through the ISP to obtain a first preview image. This color and brightness processing may include the color and brightness processing not performed on the second original image data in S103 above. If the electronic device only performed RAW domain processing on the second original image data in S103, then complete color and brightness processing, such as the RGB domain processing and YUV domain processing described above, needs to be performed on the second original image data here. If the electronic device only performed RAW domain processing and partial color and brightness processing on the second original image data in S103, then only the remaining unperformed color and brightness processing needs to be performed on the second original image data here.

[0078] In a specific implementation, the first parameter may be passed to the ISP via a hardware abstraction layer (HAL), and then the ISP performs color and brightness processing on the first image to be processed using the first parameter.

[0079] S106: The electronic device displays a first preview image.

[0080] After the electronic device generates the first preview image, it may pass the first preview image to the application layer, which may then process the first preview image. For example, the electronic device may display a preview interface through the camera application or application, and display the first preview image in the preview area of the preview interface.

[0081] The above steps S101-S106 are a processing process for a frame of preview image. Through the above process, the electronic device can display the frame of preview image captured by the second camera on the display screen. In addition, the display method has at least the following technical effects:

[0082] 1. Using the intermediate image processed by the reference camera to predict the color and brightness parameters of each camera, the color and brightness of the output image of each camera can be synchronized with the reference camera, which can improve the synchronization accuracy. The color and brightness parameters are obtained based on the actual image content and shooting style of each camera, which can better ensure the consistency of color and brightness of different frames.

[0083] 2. This solution uses the intermediate image processed by the reference camera to predict the color and brightness parameters of each camera. Compared with using RAW domain statistical information to predict color and brightness parameters, this solution consumes less power and produces more accurate prediction results.

[0084] 3. This solution does not require pre-calibration of the parameter mapping relationship between multiple cameras, nor does it rely on additional light source classification algorithms. It directly uses the intermediate image processed by the reference camera to predict color and brightness parameters. This is simple and convenient, improving the feasibility and generalization of the solution.

[0085] 4. The electronic device uses the intermediate image processed by the reference camera to predict the color and brightness parameters. The calculation is fast and the predicted color and brightness parameters can be sent to each camera in real time, allowing it to process the image according to the color and brightness parameters. The parameters can take effect in real time, and the speed at which the color and brightness of the output images of each camera and the reference camera remain consistent can be guaranteed.

[0086] In some embodiments, the processing steps S101-S106 for one preview image frame can be applied to each preview image frame. That is, after the electronic device activates the second camera, each frame of raw image data captured by the second camera is processed using the above method.

[0087] In other embodiments, the electronic device can also detect the stability of the image captured by the second camera, and adjust the frequency of using the above method to process the image based on the stability. Specifically, if the electronic device detects that the image captured by the second camera is relatively stable, it can use the color brightness parameters or similar parameters used when processing the previous frame image to continue processing the current frame image without completely using the above method to redetermine the color brightness parameters; if the electronic device detects that the image captured by the second camera is unstable, it can use the above method to redetermine the color brightness parameters and use the color brightness parameters to further process the current frame image. Among them, whether the image captured by the second camera of the electronic device is stable can be detected by a motion detection algorithm, and the judgment criterion can be to check whether the moving distance of the electronic device is less than a threshold. If so, it is considered stable, otherwise it is considered unstable. This embodiment can ensure that the color brightness of the image output by the second camera is consistent with the color brightness of the image output by the reference camera, and can also reduce the power consumption of the electronic device as much as possible.

[0088] The above embodiment is described through the specific processing of the following two image frames:

[0089] After S106, the electronic device may capture third raw image data through the second camera, and the movement distance of the electronic device when capturing the third raw image data and the second raw image data is less than a threshold; perform RAW domain processing on the third raw image data, or perform RAW domain processing and partial color and brightness processing on the third raw image data to obtain a second image to be processed; perform further color and brightness processing on the second image to be processed using the first parameters to obtain a second preview image; and display the second preview image. In some embodiments, the electronic device may also smooth the first parameters to obtain new parameters, and use the new parameters to perform color and brightness processing on the second image to be processed to obtain a second preview image. Using the first parameters or similar parameters can ensure the continuity and smoothness of the preview effect and also reduce device power consumption.

[0090] After S106, the electronic device can collect fourth original image data through the second camera, and the movement distance of the electronic device is greater than the threshold when collecting the fourth original image data and collecting the second original image data; perform RAW domain processing on the fourth original image data, or perform RAW domain processing and partial color and brightness processing on the fourth original image data to obtain a third image to be processed; collect fifth original image data through the first camera, perform RAW domain processing and color and brightness processing on the fifth original image data to obtain a second reference image, and the fifth original data and the fourth original data are collected at the same time or the difference between the collection times does not exceed the threshold; determine the second parameter, and the second parameter is obtained based on the second reference image and the third image to be processed; use the second parameter to further perform color and brightness processing on the third image to be processed to obtain a third preview image; and display the third preview image.

[0091] In the processing of the above two image frames, the specific implementation of RAW domain processing and color brightness processing can be referred to the relevant description above.

[0092] In some implementations, after S106 , the electronic device may further switch the camera. Figure 1 The method shown may further include the following steps:

[0093] S107: The electronic device receives a user operation to switch from the second camera to the fourth camera.

[0094] The fourth camera and the second camera are different cameras. The fourth camera can be any camera in the electronic device that can be used to display images, such as a main camera, a telephoto camera, a wide-angle camera, a front camera, etc.

[0095] The user operation of switching the second camera to the fourth camera can be implemented as a user operation on the display screen of the electronic device, such as clicking the control to switch the camera in the user interface, or as a voice user operation or a gesture, etc., which is not limited here.

[0096] S108: The electronic device collects sixth original image data through the first camera, performs RAW domain processing and color and brightness processing on the sixth original image data, and obtains a third reference image. For the specific implementation of S108, refer to S102.

[0097] In step S109, the electronic device responds to the user operation in step S107 by activating the fourth camera and capturing seventh raw image data via the fourth camera. If the seventh raw image data and the sixth raw image data are captured at the same time or the difference between the capture times does not exceed a threshold, the electronic device performs RAW domain processing on the seventh raw image data, or performs RAW domain processing and partial color and brightness processing on the seventh raw image data, to obtain a fourth image to be processed. For the specific implementation of step S109, reference may be made to step S103.

[0098] S110: The electronic device determines a third parameter.

[0099] The third parameter can be determined based on the third reference image and the fourth image to be processed. Furthermore, the third parameter can be determined based on third metadata generated when the first camera captures the sixth raw image data and / or fourth metadata generated when the fourth camera captures the seventh raw image data. For the specific implementation of S110, refer to S104.

[0100] S111: The electronic device performs further color and brightness processing on the fourth image to be processed using the third parameter to obtain a fourth preview image. For the specific implementation of S111, refer to S105.

[0101] S112: The electronic device displays a fourth preview image. For a specific implementation of S112, reference may be made to S106.

[0102] Through the above-mentioned method provided in this application, it can be seen that when an electronic device uses any camera to capture an image, the image generated by the reference camera is used to perform color and brightness processing on the image of that camera. This ensures that the color and brightness of the image output by each camera are consistent with the output of the reference camera. At the moment when the electronic device switches cameras and in the steady state after the switch, the color and brightness of the preview image displayed by the electronic device can be kept consistent with those of the reference camera, thus solving the problem of inconsistent color and brightness of the preview image caused by differences between different cameras.

[0103] The above method provided in the embodiment of the present application can be applied to any shooting scene, such as photo shooting scene, video recording scene, and shooting animated pictures scene.

[0104] In some embodiments, the second camera and the first camera in the above method can also be the same camera. If they are the same camera, after receiving a user operation to activate the second camera, the electronic device can capture raw image data through the second camera, perform RAW domain processing and color and brightness processing on the raw image data, obtain a preview image, and display the preview image. Because the second camera is the reference camera itself, it can output the preview image according to its original processing logic without the need for comparison with other cameras.

[0105] In some embodiments of the present application, the reference camera is not fixed and can be changed according to different usage scenarios. The usage scenario here can refer to the environment in which the user is shooting, such as daytime, nighttime, mountain scenery, seascape, etc. The camera that can capture more accurate colors and brightness, more realistic conditions, or better meet the user's aesthetic needs in different environments may not be the same. Therefore, the reference camera can be switched in different scenarios to provide the user with a better shooting experience.

[0106] For example, in Figure 1 The illustrated method may further include the following steps: the electronic device receives a user operation to activate a sixth camera; acquires eighth raw image data via a fifth camera, performs RAW domain processing and color and brightness processing on the eighth raw image data to obtain a fourth reference image; activates a sixth camera, acquires ninth raw image data via the sixth camera, wherein the ninth raw image data and the eighth raw data are acquired at the same time or the difference between the acquisition times does not exceed a threshold, performs RAW domain processing on the ninth raw image data, or performs RAW domain processing and partial color and brightness processing on the ninth raw image data to obtain a fifth image to be processed; determines fourth parameters, the fourth parameters being obtained based on the fourth reference image and the fifth image to be processed; performs further color and brightness processing on the fifth image to be processed using the fourth parameters to obtain a fifth preview image; and displays the fifth preview image. The fifth camera is a reference camera different from the first camera. The fifth camera may be determined based on the current scene.

[0107] In the above embodiment, the process of outputting a preview image by using another camera as a reference camera can be referred to Figure 1 The specific implementation of S101-S106.

[0108] Equipment provided by this application

[0109] refer to Figure 2 , Figure 2 The hardware structure diagram of the electronic device 100 provided in the embodiment of the present application. The electronic device 100 may be the electronic device mentioned above, which is used to perform Figure 1 The shooting method shown.

[0110] like Figure 2As shown, the electronic device 100 may include: the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0111] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0112] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0113] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0114] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0115] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0116] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0117] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0118] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0119] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0120] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0121] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0122] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0123] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0124] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0125] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0126] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0127] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0128] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0129] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0130] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0131] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0132] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0133] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device can include one or N display screens 194, where N is a positive integer greater than one.

[0134] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0135] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0136] Camera 193 is used to capture still images or videos. The lens generates an optical image of an object and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then converted into a digital image signal and transmitted to the ISP. The ISP outputs the digital image signal to the DSP for processing.

[0137] In an embodiment of the present application, the number of cameras 193 can be multiple, for example, it can include a main camera, a telephoto camera and a wide-angle camera, and can also include a front camera, and can also include a multispectral sensor, etc.

[0138] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0139] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0140] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0141] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0142] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0143] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH according to the operating principle, single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC) according to the potential level of the storage cell, and universal flash storage (UFS) and embedded multi-media card (eMMC) according to the storage specification.

[0144] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0145] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0146] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0147] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0148] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0149] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0150] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0151] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0152] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0153] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0154] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0155] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0156] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0157] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0158] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0159] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0160] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0161] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0162] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0163] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0164] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0165] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0166] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0167] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0168] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0169] In an embodiment of the present application, the internal memory 121 is used to store a program for implementing the shooting method provided by the present application on the electronic device side, and the processor 110 is used to call the instructions contained in the program to trigger the electronic device 100 to execute each step of the method.

[0170] Specifically, information input devices such as the display screen 194 and the microphone 170C can be used to receive various user operations, the camera 193 can be used to collect raw image data, and the ISP, DSP, etc. can be used to further process the raw image data.

[0171] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, a mobile operating system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0172] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0173] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, a mobile operating system is divided into four layers: the application layer, the framework layer / core services layer, the underlying libraries and runtime, and the kernel layer.

[0174] The application layer can include a series of application packages.

[0175] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0176] The program framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The program framework layer includes some predefined functions.

[0177] like Figure 3 As shown, the program framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0178] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0179] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0180] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0181] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0182] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0183] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0184] The runtime can refer to all code libraries, frameworks, and other components required for a program to run. For example, for the C language, the runtime includes a series of function libraries required for C programs to run. For the Java language, in addition to the core libraries, the runtime also includes the virtual machine required for Java programs to run. These core libraries include the functions required by the Java language.

[0185] The underlying library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0186] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0187] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0188] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0189] A 2D graphics engine is a drawing engine for 2D drawings.

[0190] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0191] The following describes the workflow of the electronic device software and hardware in conjunction with capturing a photo scene.

[0192] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a still image or video through the camera 193.

[0193] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0194] The present application also provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program, and the processor may be used to call the computer program in the memory to enable the electronic device to execute the method executed on the electronic device side in any of the above embodiments.

[0195] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the electronic device side in any of the above embodiments.

[0196] In one possible design, the chip system further includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0197] The chip system can be composed of chips, or can include chips and other discrete devices.

[0198] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0199] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0200] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0201] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method executed by the electronic device side in any of the above embodiments.

[0202] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device side in any of the above embodiments.

[0203] The various implementation modes of this application can be combined arbitrarily to achieve different technical effects.

[0204] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0205] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0206] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0207] The terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0208] In short, the above description is only an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of this application should be included in the scope of protection of this application.

Claims

1. A shooting method, characterized in that: The method is applied to an electronic device, the electronic device including a first camera and a second camera, and the method includes: receiving a user operation of starting the second camera; collecting first original image data through the first camera, performing RAW domain processing and color brightness processing on the first original image data to obtain a first reference image; Starting the second camera, capturing second raw image data through the second camera, where the second raw image data and the first raw image data are captured at the same time or the difference in the capture times does not exceed a threshold, performing RAW domain processing on the second raw image data, or performing RAW domain processing and partial color and brightness processing on the second raw image data, to obtain a first image to be processed; determining a first parameter, where the first parameter is obtained according to the first reference image and the first image to be processed; performing further color and brightness processing on the first image to be processed using the first parameters to obtain a first preview image; The first preview image is displayed.

2. The method according to claim 1, characterized in that The method further comprises: generating first metadata when the first original image data is collected; The first parameter is also obtained according to the first metadata.

3. The method according to claim 1, characterized in that The method further comprises: generating second metadata when the second original image data is collected; The first parameter is also obtained according to the second metadata.

4. The method according to claim 1, wherein The determining of the first parameter specifically includes: The first reference image and the first image to be processed are input into a neural network, and a first parameter is output through the neural network.

5. The method according to claim 4, characterized in that The neural network is stored in the neural network processor NPU of the electronic device.

6. The method according to claim 1, characterized in that The first parameter includes any one or more of the following: a color correction matrix CCM, a color lookup table LUT, a dynamic range correction DRC, a global tone mapping GTM, a color gamut mapping, or a local tone mapping LTM relationship.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The second camera captures third original image data, and when the third original image data and the second original image data are captured, the movement distance of the electronic device is less than a threshold; performing RAW domain processing on the third original image data, or performing RAW domain processing and partial color and brightness processing on the third original image data, to obtain a second image to be processed; performing further color and brightness processing on the second image to be processed using the first parameter to obtain a second preview image; The second preview image is displayed.

8. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The second camera captures fourth original image data, and when the fourth original image data and the second original image data are captured, the movement distance of the electronic device is greater than a threshold; performing RAW domain processing on the fourth original image data, or performing RAW domain processing and partial color and brightness processing on the fourth original image data, to obtain a third image to be processed; capturing fifth original image data through the first camera, performing RAW domain processing and color brightness processing on the fifth original image data to obtain a second reference image, wherein the fifth original image data and the fourth original image data are captured at the same time or the difference between the capture times does not exceed a threshold; determining a second parameter, where the second parameter is obtained based on the second reference image and the third image to be processed; performing further color and brightness processing on the third image to be processed using the second parameter to obtain a third preview image; The third preview image is displayed.

9. The method according to any one of claims 1 to 6, characterized in that The user operation of starting the second camera specifically includes: A user operation of starting a camera application in the electronic device; Alternatively, the user operation is used to switch the third camera to the second camera, and the electronic device further includes the third camera.

10. The method according to any one of claims 1 to 6, characterized in that The electronic device further includes a fourth camera. After displaying the first preview image, the method further includes: receiving a user operation of switching the second camera to the fourth camera; collecting sixth original image data through the first camera, performing RAW domain processing and color brightness processing on the sixth original image data to obtain a third reference image; activating the fourth camera, capturing seventh raw image data through the fourth camera, where the seventh raw image data and the sixth raw image data are captured at the same time or the difference between the capture times does not exceed a threshold, performing RAW domain processing on the seventh raw image data, or performing RAW domain processing and partial color and brightness processing on the seventh raw image data, to obtain a fourth image to be processed; determining a third parameter, where the third parameter is obtained based on the third reference image and the fourth image to be processed; performing further color and brightness processing on the fourth image to be processed using the third parameter to obtain a fourth preview image; The fourth preview image is displayed.

11. The method according to any one of claims 1 to 6, characterized in that: The electronic device further includes a fifth camera and a sixth camera, the fifth camera and the first camera being different cameras, and the method further includes: receiving a user operation of activating the sixth camera; collecting eighth original image data through the fifth camera, performing RAW domain processing and color brightness processing on the eighth original image data to obtain a fourth reference image; activating the sixth camera and capturing ninth raw image data through the sixth camera, where the ninth raw image data and the eighth raw image data are captured at the same time or the difference between the capture times does not exceed a threshold, performing RAW domain processing on the ninth raw image data, or performing RAW domain processing and partial color and brightness processing on the ninth raw image data, to obtain a fifth image to be processed; determining a fourth parameter, where the fourth parameter is obtained based on the fourth reference image and the fifth image to be processed; performing further color and brightness processing on the fifth image to be processed using the fourth parameter to obtain a fifth preview image; The fifth preview image is displayed.

12. An electronic device, characterized in that: include: a first camera, a second camera, a memory, and one or more processors; The first camera, the second camera, and the memory are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-11.

13. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 11.

14. A chip system, comprising at least one processor and a memory, wherein the memory is used to store program instructions and data, and the processor is used to call the program instructions and data to implement the method according to any one of claims 1 to 11.

Images