Shooting method and electronic equipment

Abstract

The invention discloses a shooting method and electronic equipment. When the electronic equipment uses any camera to shoot the image, the image generated by the reference camera is used for performing color and brightness processing on the image of the camera, so that the color and the brightness of the image output by each camera can be ensured to be consistent with those output by the reference camera. According to the scheme, consistent color and brightness feelings can be provided for a user, after the user switches the camera of the electronic equipment, the user cannot perceive that the color and the brightness of the preview image jump, and good shooting experience can be achieved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a photographing method and an electronic device. Background Art

[0002] Due to the limited size of electronic devices such as mobile phones and tablets, the size of camera modules (camera compact modules, CCM) will not be too large, so multiple camera modules are usually set to meet the shooting needs of different scenes, such as main camera, wide-angle camera, telephoto camera, etc. There are differences between different camera modules. For example, the spectral response of the color filter array (CFA) in the sensor is different, resulting in different colors and brightness of the images output by different camera modules. Summary of the invention

[0003] The present application provides a shooting method and an electronic device, which can provide users with a consistent color and brightness experience.

[0004] In a first aspect, a shooting method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, and the method may include: receiving a user operation to start the second camera; acquiring first original image data through the first camera, performing RAW domain processing and color brightness processing on the first original image data to obtain a first reference image; starting the second camera, acquiring second original image data through the second camera, wherein the second original data and the first original data are acquired at the same time or the difference between the acquisition times does not exceed a threshold, performing RAW domain processing on the second original image data, or performing RAW domain processing and partial color brightness processing on the second original image data to obtain a first image to be processed; determining a first parameter, wherein the first parameter is obtained based on the first reference image and the first image to be processed; performing further color brightness processing on the first image to be processed using the first parameter to obtain a first preview image; and displaying the first preview image.

[0005] By implementing the method of the first aspect, the color brightness of the image output by the second camera can be synchronized with the first camera. The method uses the intermediate image processed by the first camera to predict the parameters required by the second camera, without the need for calibration of mapping relationships or statistical information, and with lower power consumption. In addition, the scheme can achieve real-time effectiveness of parameters, and the speed at which the color brightness of the images output by the second camera and the first camera remain consistent can be guaranteed.

[0006] In combination with the first aspect, in some embodiments, the method may further include: generating first metadata when the first original image data is acquired; wherein the first parameter is also obtained based on the first metadata.

[0007] In combination with the first aspect, in some embodiments, the method may further include: generating second metadata when the second original image data is acquired; wherein the first parameter is also obtained based on the second metadata.

[0008] In the above two implementations, the metadata, as a priori information, can enable the electronic device to output the first parameter that is more suitable for the second camera.

[0009] In conjunction with the first aspect, in some implementations, the electronic device may input the first reference image and the first image to be processed into a neural network, and output the first parameter through the neural network. The neural network can speed up the acquisition of the first parameter, thereby improving the efficiency of the present application.

[0010] In combination with the previous embodiment, the neural network is stored in a neural network processor NPU of the electronic device.

[0011] In combination with the first aspect, in some embodiments, the first parameter includes any one or more of the following: a color correction matrix CCM, a color lookup table LUT, a dynamic range correction DRC, a global tone mapping GTM, a color gamut mapping, or a local tone mapping LTM relationship.

[0012] In combination with the first aspect, in some implementations, the electronic device may further detect the stability of the image captured by the second camera, and adjust the frequency of processing the image using the above method according to the stability.

[0013] In some embodiments, after the method provided in the first aspect, the electronic device can also collect the third original image data through the second camera, and the moving distance of the electronic device is less than the threshold when collecting the third original image data and collecting the second original image data; perform RAW domain processing on the third original image data, or perform RAW domain processing and partial color brightness processing on the third original image data to obtain a second image to be processed; use the first parameter to further perform color brightness processing on the second image to be processed to obtain a second preview image; display the second preview image. In some embodiments, the electronic device can also perform smoothing processing on the first parameter to obtain a new parameter, and use the new parameter to perform color brightness processing on the second image to be processed to obtain a second preview image. Using the first parameter or similar parameters can ensure the continuity and smoothness of the preview effect, and can also reduce the power consumption of the device. This embodiment can ensure that the color brightness of the image output by the second camera is consistent with the color brightness of the image output by the first camera, and can also reduce the power consumption of the electronic device as much as possible.

[0014] In some embodiments, after the method provided in the first aspect, the electronic device can collect fourth original image data through the second camera, and the movement distance of the electronic device is greater than a threshold when collecting the fourth original image data and collecting the second original image data; perform RAW domain processing on the fourth original image data, or perform RAW domain processing and partial color brightness processing on the fourth original image data to obtain a third image to be processed; collect fifth original image data through the first camera, perform RAW domain processing and color brightness processing on the fifth original image data to obtain a second reference image, and the fifth original data and the fourth original data are collected at the same time or the difference between the collection times does not exceed a threshold; determine a second parameter, and the second parameter is obtained based on the second reference image and the third image to be processed; use the second parameter to further perform color brightness processing on the third image to be processed to obtain a third preview image; and display the third preview image.

[0015] In combination with the first aspect, in some embodiments, the user operation of starting the second camera specifically includes any one of the following: a user operation of starting a camera application in the electronic device; or a user operation for switching the third camera to the second camera, and the electronic device also includes a third camera.

[0016] In combination with the first aspect, in some embodiments, the electronic device can also switch cameras. Specifically, the electronic device also includes a fourth camera. After displaying the first preview image, the method can also include: receiving a user operation to switch the second camera to the fourth camera; collecting the sixth original image data through the first camera, performing RAW domain processing and color brightness processing on the sixth original image data to obtain a third reference image; starting the fourth camera, collecting the seventh original image data through the fourth camera, the seventh original data and the sixth original data are collected at the same time or the difference between the collection time does not exceed the threshold, performing RAW domain processing on the seventh original image data, or performing RAW domain processing and partial color brightness processing on the seventh original image data to obtain a fourth image to be processed; determining a third parameter, the third parameter is obtained based on the third reference image and the fourth image to be processed; using the third parameter to further perform color brightness processing on the fourth image to be processed to obtain a fourth preview image; displaying the fourth preview image.

[0017] It can be seen from the above implementation that when the electronic device uses different cameras to capture images, the image generated by the first camera is used to perform color and brightness processing on the image of the camera, so that the color and brightness of the image output by each camera can be guaranteed to be consistent with the output of the first camera. At the moment when the electronic device switches the camera and in the steady state process after the switch, the color and brightness of the preview image displayed by the electronic device can be kept consistent with the first camera, thus solving the problem of inconsistent color and brightness of the preview image caused by the differences between different cameras.

[0018] In combination with the first aspect, in some embodiments, the electronic device can use different reference cameras in different scenarios. The electronic device also includes a fifth camera and a sixth camera, and the fifth camera and the first camera are different cameras. The method may also include: receiving a user operation to start the sixth camera; collecting the eighth original image data through the fifth camera, performing RAW domain processing and color brightness processing on the eighth original image data to obtain a fourth reference image; starting the sixth camera, collecting the ninth original image data through the sixth camera, the ninth original data and the eighth original data are collected at the same time or the difference between the collection time does not exceed the threshold, performing RAW domain processing on the ninth original image data, or performing RAW domain processing and partial color brightness processing on the ninth original image data to obtain a fifth image to be processed; determining a fourth parameter, the fourth parameter is obtained according to the fourth reference image and the fifth image to be processed; using the fourth parameter to further perform color brightness processing on the fifth image to be processed to obtain a fifth preview image; displaying the fifth preview image.

[0019] In a second aspect, an electronic device is provided, comprising: a first camera, a second camera, a memory, and one or more processors; the first camera, the second camera, and the memory are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method provided in the first aspect or any one of the embodiments of the first aspect.

[0020] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions so that the electronic device executes the method provided in the first aspect or any one embodiment of the first aspect.

[0021] According to a fourth aspect, a computer-readable storage medium is provided, comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method provided in the first aspect or any one of the embodiments of the first aspect.

[0022] According to a fifth aspect, a computer program product comprising instructions is provided. When the computer program product is run on an electronic device, the electronic device executes the method provided in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a shooting method provided in an embodiment of the present application;

[0024] Figure 2 A hardware structure block diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 3 The software architecture of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The structure and working principle of the camera module

[0027] The camera module may mainly include: a lens, a photosensitive element (sensor), and an infrared filter (IR filter). The photosensitive element may also be called an image sensor. In some embodiments, the camera module may also include a base bracket, a voice coil motor (VCM), a flexible printed circuit (FPC), a printed circuit board (PCB), a driver integrated circuit (driver IC), etc.

[0028] The lens includes one or more lenses, which are used to receive light signals and focus them on the photosensitive element.

[0029] The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element is essentially a semiconductor chip, which mainly includes modules such as a photosensitive area array (Bayer array or pixel array), timing control, analog signal processing, and analog-to-digital conversion. There are hundreds of thousands to millions of photodiodes on the surface of the photosensitive element. When the photodiodes are exposed to light, they will generate charges, convert the light into electrical signals, and then convert them internally into digital image signals. Each pixel of the photosensitive element can only sense one of the following colors: red (red, R), green (green, G), and blue (blue, B). Therefore, the data stored in each pixel is monochrome. The digital image signal converted from the light source signal collected by the photosensitive element is called RAWData. RAW Data is an unprocessed and uncompressed format, and can also be called raw image data.

[0030] The file that records the original information of the photosensitive element in the electronic device can be called a RAW file, which records some metadata generated by shooting, such as ISO settings, shutter speed, aperture value, white balance, shooting mode (such as night scene mode, face mode, large aperture mode, movie mode, etc.), face frame information, etc.

[0031] The infrared filter is located between the lens and the photosensitive element. It is used to filter out near-infrared light in the light entering the lens. This is because the human eye cannot see infrared light, but the photosensitive element can sense infrared light, so the infrared light in the light needs to be filtered out to make the image closer to what the human eye sees.

[0032] The photosensitive element transmits the converted digital image signal to the image signal processor (ISP), which processes the digital image signal using a series of digital image processing algorithms. After that, the ISP outputs the processed digital image signal to the digital signal processor (DSP) for processing.

[0033] ISP can be set up in two ways: internal and external. External means that the ISP is set up separately outside the application processor (AP) of the electronic device, and internal means that the ISP is embedded inside the AP. Multiple camera modules can share one ISP. Multiple camera modules can share one DSP.

[0034] During the imaging process of the camera module, the ISP is responsible for receiving the raw image data output by the photosensitive element and performing a series of digital image algorithm processing on the raw image data. These processing may include any one or more of the following:

[0035] RAW domain processing refers to compensating for defects in the original image caused by hardware. RAW domain processing specifically includes but is not limited to any one or more of the following: black level compensation (BLC), lens shading correction (LSC), bad pixel correction (BPC), RAW domain noise reduction (RAW denoise), color interpolation (demosaic), etc.

[0036] RGB domain processing refers to the process of converting RAW images into RGB images and then making some corrections to the RGB images. RGB domain processing may include but is not limited to any one or more of the following: automatic white balance (AWB), RGB noise reduction (NR color), crosstalk (X-tasing), color correction (colorcorrection), wide dynamic range (WDR), gamma enhancement (gamma), color space conversion (color space matrix) such as RGB to YUV conversion, etc.

[0037] YUV domain processing refers to converting RGB images into YUV format, performing color noise reduction, image sharpening, color correction and other operations on the image in the YUV domain, and then synthesizing multiple frames to improve the image tolerance or reduce noise. YUV domain processing specifically includes but is not limited to any one or more of the following: color noise removal (color denoise), edge enhancement (edge ​​enhancement), brightness contrast control (contract brightness), hue saturation control (huesaturation), etc.

[0038] After the above processing, the ISP outputs data in YUV or RGB format to the DSP, which encodes the data and outputs the processed image. The output image can be stored in a cache or other storage location.

[0039] Among the above-mentioned processes, RGB domain processing and YUV domain processing involve processing the color and brightness of the image, and thus can be referred to as color and brightness processing of the image. In other words, color and brightness processing can include the above-mentioned RGB domain processing and / or YUV domain processing.

[0040] For the sake of simplicity, the camera module will be referred to as the camera in the following text.

[0041] The electronic device provided in this application may include multiple cameras, which can meet the shooting needs of the electronic device in different scenarios. For example, the electronic device may include a rear main camera, a telephoto camera and a wide-angle camera, and may also include a front camera, and may also include image sensors such as multispectral sensors, CMY sensors, RGGB sensors, RYYB sensors, and RGBW sensors. Among them, the main camera can be used to meet general rear shooting needs, the telephoto camera can meet telephoto shooting needs, and the wide-angle camera can meet large depth of field shooting needs. The images taken by the main camera, telephoto camera, wide-angle camera and front camera can all be displayed to the user, and the images taken by the multispectral sensor generally do not need to be displayed to the user.

[0042] Multispectral sensor is also a kind of camera, which can be composed of two parts: optical head and electronic components. The optical head includes filters, camera lenses and detectors. The optical head uses several independent area array CCDs to obtain data in frames, and the spectral selection and spectral resolution are guaranteed by filters. Multispectral sensors can collect one-way images with accurate colors, and their resolution is low. Multispectral sensors can include single-point multispectral sensors, multi-point multispectral sensors, etc.

[0043] After an electronic device switches cameras, due to differences between different cameras, such as hardware (e.g., inconsistent spectral responses of infrared filters) or software (e.g., exposure strategies of photosensitive elements), the image displayed on the preview interface may have different colors and brightness, affecting the user experience. The differences between different cameras may be differences between different models of cameras produced by different manufacturers, or between different models of cameras produced by the same manufacturer, or between different batches of the same model of cameras produced by the same manufacturer, or between the same model of cameras produced by different manufacturers.

[0044] To solve the above problems, the present application provides a shooting method and an electronic device. When the electronic device uses any camera to shoot an image, the image generated by the reference camera is used to process the color and brightness of the image of the camera, so that the color and brightness of the image output by each camera can be guaranteed to be consistent with the output of the reference camera. This solution can ensure that when the electronic device uses any camera, there will be no significant difference in the color and brightness of the preview image, and can provide users with a consistent color and brightness experience. In this way, when the user switches the camera of the electronic device, the user will not perceive the jump in the color and brightness of the preview image, and can have a good shooting experience.

[0045] The shooting method provided by this application

[0046] Figure 1 The flowchart of the shooting method provided by the embodiment of the present application is exemplarily shown. The method is applied to an electronic device including multiple cameras, the multiple cameras including at least a first camera and a second camera, the first camera being the reference camera in the present application.

[0047] like Figure 1 As shown, the method may include the following steps:

[0048] S101: The electronic device receives a user operation to start a second camera.

[0049] The user operation of starting the second camera can be implemented in any of the following ways:

[0050] 1. The user operation is a user operation of starting a camera application or a shooting function of an application in the electronic device.

[0051] Camera applications may include system camera applications in electronic devices, and may also include third-party camera applications, etc. Some applications integrate the camera function, such as WeChat, Taobao and other applications. When these applications start the camera function, they can call the camera of the electronic device.

[0052] In the first case, the second camera may be the default camera for a camera application or a shooting function in an application. The default camera may be the main camera on the rear of the electronic device or the front camera of the electronic device. For example, the default camera of some selfie applications may be the front camera.

[0053] 2. The user operation is a user operation of switching the camera after starting a camera application or a shooting function of an application in the electronic device. For example, the user operation is a user operation for switching the third camera to the second camera. The electronic device also includes a third camera.

[0054] In the second case, the third camera and the second camera are different cameras. The third camera or the second camera can be any camera in the electronic device that can be used to display images, such as a main camera, a telephoto camera, a wide-angle camera, a front camera, etc.

[0055] The user operation in S101 of the present application can be implemented as a user operation on the display screen of the electronic device, such as clicking on the icon of a camera application on the desktop or clicking on a shooting function control. It can also be implemented as a voice user operation or a gesture, etc., which is not limited here.

[0056] S102: The electronic device collects first original image data through a first camera, and performs RAW domain processing and color brightness processing on the first original image data to obtain a first reference image.

[0057] The first camera is a reference camera, which can be set by default by the electronic device, for example, it can be set in advance before the electronic device leaves the factory, or the user can independently select one of multiple cameras as a reference camera. The reference camera can be a camera among the cameras of the electronic device, which captures images with more accurate color and brightness, more realistic, or more in line with the user's aesthetic needs. The reference camera can be any one of the multiple cameras of the electronic device, such as a main camera, a telephoto camera, a wide-angle camera, a front camera, or any one of a multispectral sensor. The first camera and the second camera are different cameras.

[0058] In an embodiment of the present application, the first camera may be turned on by default after the electronic device starts a camera application or a shooting function of an application. If the electronic device receives the first user operation in S101, the electronic device may start the first camera and the second camera at the same time in response to the user operation. If the electronic device receives the second user operation in S101, the electronic device has already started the first camera when starting the camera application or the shooting function of an application in the electronic device, and may start the second camera in response to the user operation.

[0059] The first original image data collected by the electronic device through the first camera refers to RAWData collected by the first camera.

[0060] The process of the electronic device performing RAW domain processing and color brightness processing on the first original image can refer to the introduction of RAW domain processing and color brightness processing in the previous text. The first reference image is obtained through color brightness processing, so it contains reference information of color and brightness when the first camera processes the image, and the first reference image also contains reference information of color and brightness related to the content currently captured by the first camera.

[0061] In some implementations, when the electronic device captures the first original image data through the first camera, it may also generate first metadata when the first original image data is captured.

[0062] S103, the electronic device responds to the user operation in S101, starts the second camera, collects second original image data through the second camera, the second original data and the first original data are collected at the same time or the difference between the collection times does not exceed the threshold, performs RAW domain processing on the second original image data, or performs RAW domain processing and partial color brightness processing on the second original image data to obtain a first image to be processed.

[0063] The second original image data collected by the electronic device through the second camera refers to RAWData collected by the second camera.

[0064] The second original image data and the first original image data are collected at the same time or the difference between the collection times does not exceed the threshold. If the collection period and collection frequency of the second camera and the first camera are the same, the two original image data can be collected at the same time. If the collection period and collection frequency of the second camera and the first camera are different, the first original data can be the original image data collected by the first camera at the closest time to the collection time of the second original data, either before or after it. Based on the above definition, the second original image data and the first original image data are collected at a similar or same time, and both are collected in the same environment. If the first camera and the second camera are both rear cameras, the content captured by the two is the same or similar.

[0065] The process of the electronic device performing RAW domain processing on the second original image can refer to the introduction of RAW domain processing in the previous article. The partial color brightness processing performed by the electronic device on the second original image data may include some operations in the RGB domain processing and YUV domain processing mentioned above, such as automatic white balance, color correction and other operations, but of course, it may not include automatic white balance, color correction and other operations. The actual processing content of the partial color brightness processing here is related to the position of the neural network mentioned later in the entire image processing process. If the partial color brightness processing here performs more operations, then the color brightness processing after the neural network will be less. If the partial color brightness processing here performs fewer operations, then the color brightness processing after the neural network will be more. The color brightness processing before and after the neural network constitutes a complete brightness processing of the image. The specific operation of this partial color brightness processing can be set by the R&D personnel as needed.

[0066] The content of the first to-be-processed image obtained by processing is the same as or similar to the content of the first reference image.

[0067] In some implementations, when the electronic device captures the second original image data through the second camera, it may also generate second metadata when the second original image data is captured.

[0068] S104: The electronic device determines a first parameter.

[0069] In some embodiments, the electronic device may determine the first parameter based on the first reference image and the first image to be processed. The electronic device may output a parameter suitable for the first image to be processed that is close to or equal to the color brightness of the first reference image based on the color brightness difference between the first reference image and the first image to be processed, i.e., the first parameter. The first reference image and the first image to be processed have the same or similar contents, and the first parameter determined by the electronic device is obtained based on the same or similar image contents, and is therefore suitable for the first image to be processed, and the first parameter is close to or equal to the color brightness of the first reference image.

[0070] In some embodiments, in addition to the first reference image and the first image to be processed, the electronic device may also determine the first parameter based on the first metadata and / or the second metadata. The metadata, as prior information, allows the electronic device to output the first parameter that is more suitable for the second camera and the current shooting scene.

[0071] In some embodiments, the electronic device may use a neural network to determine the first parameter. Specifically, the electronic device may use the data used to determine the first parameter as input to the neural network, and output the first parameter through the neural network. The data used to determine the first parameter may include a first reference image and a first image to be processed, and may also include first metadata and / or second metadata. The neural network may be stored in a neural-network processing unit (NPU) of the electronic device.

[0072] In some embodiments, the electronic device may pre-process the data used to determine the first parameter so that it meets the input requirements of the neural network. For example, the electronic device may perform operations such as translation and cropping on the first reference image to reduce the field of view (FOV) difference with the first image to be processed. For another example, the electronic device may process the first reference image and the first image to be processed into a small resolution image (i.e., a tiny image) of a preset size, or process these data into a preset number of channels, etc.

[0073] The neural network can be obtained by pre-training. When the types of data used to determine the first parameter are different, different neural networks may correspond to them. For example, if the training data may include a reference image and an image to be processed, and the standard parameters corresponding to the image to be processed having a color brightness effect similar to that of the reference image. For another example, the training data may include a reference image and an image to be processed, and also include metadata of the two, and the standard parameters corresponding to the image to be processed having a color brightness effect similar to that of the reference image.

[0074] The first parameter is a parameter related to color and brightness, which may include but is not limited to any one or more of the following: color correction matrix (CCM), color lookup table (LUT), dynamic range calibration (DRC), global tone mapping (GTM), gamut mapping or local tone mapping (LTM). Relationship. The parameter categories included in the first parameter can be pre-defined by equipment developers.

[0075] The number of parameters included in the first parameter is related to the color brightness processing performed by the electronic device in S104. The color brightness processing performed by the electronic device in S103 and the color brightness processing performed in S105 constitute a complete color brightness processing. If no color brightness processing is performed in S103, the first parameter includes all parameters required for the partial color brightness processing (i.e., the color brightness processing to be performed in S105). If partial color brightness processing is performed in S103, the first parameter includes the parameters required for the remaining partial color brightness processing (i.e., the color brightness processing to be performed in S105).

[0076] S105: The electronic device performs further color and brightness processing on the first image to be processed using the first parameter to obtain a first preview image.

[0077] The electronic device may use the first parameter to perform further color brightness processing on the first image to be processed through the ISP to obtain a first preview image. The color brightness processing may include the color brightness processing not performed on the second original image data in S103 above. If the electronic device in S103 only performs RAW domain processing on the second original image data, then it is necessary to perform complete color brightness processing on the second original image data here, such as the RGB domain processing and YUV domain processing described above. If the electronic device in S103 only performs RAW domain processing and partial color brightness processing on the second original image data, then it is only necessary to perform the remaining unperformed color brightness processing on the second original image data here.

[0078] In a specific implementation, the first parameter may be transferred to the ISP via a hardware abstraction layer (HAL), and then the ISP uses the first parameter to perform color and brightness processing on the first image to be processed.

[0079] S106: The electronic device displays a first preview image.

[0080] After the electronic device generates the first preview image, the first preview image may be handed over to the application layer, and the camera application or an application providing a shooting function started in S101 in the application layer is responsible for processing. For example, the electronic device may display a preview interface through the camera application or application, and display the first preview image in the preview area of ​​the preview interface.

[0081] The above S101-S106 are the processing process for a frame of preview image, through which the electronic device can display a frame of preview image captured by the second camera on the display screen. In addition, the display method has at least the following technical effects:

[0082] 1. The color brightness parameters of each camera are predicted by using the intermediate image processed by the reference camera. The color brightness of the image output by each camera can be synchronized with the reference camera, which can improve the accuracy of synchronization. The color brightness parameters are obtained according to the actual image content and shooting style of each camera, which can better handle the consistency of color brightness of different frames.

[0083] 2. This solution uses the intermediate image processed by the reference camera to predict the color and brightness parameters of each camera. Compared with using RAW domain statistical information to predict color and brightness parameters, this solution consumes less power and has more accurate prediction results.

[0084] 3. This solution does not require the pre-calibration of the parameter mapping relationship between multiple cameras, nor does it rely on additional light source classification algorithms. It directly uses the intermediate image processed by the reference camera to predict the color brightness parameters. It is simple and convenient, and improves the feasibility and generalization of the solution.

[0085] 4. The electronic device uses the intermediate image processed by the reference camera to predict the color brightness parameters. The calculation is fast and the predicted color brightness parameters can be sent to each camera in real time, so that it can process the image according to the color brightness parameters. The parameters can take effect in real time, and the speed at which the color brightness of the images output by each camera and the reference camera remain consistent can be guaranteed.

[0086] In some implementations, the processing process for one frame of preview image described in S101-S106 can be applied to each frame of preview image. That is, after the electronic device starts the second camera, each frame of raw image data collected by the second camera is processed using the above method.

[0087] In other embodiments, the electronic device can also detect the stability of the image captured by the second camera, and adjust the frequency of using the above method to process the image according to the stability. Specifically, if the electronic device detects that the image captured by the second camera is relatively stable, the color brightness parameters or similar parameters used in processing the previous frame image can be used to continue processing the current frame image without completely using the above method to redetermine the color brightness parameters; if the electronic device detects that the image captured by the second camera is unstable, the color brightness parameters can be redetermined using the above method, and the color brightness parameters can be used to further process the current frame image. Among them, whether the image captured by the second camera is stable can be detected by a motion detection algorithm, and the criterion for judgment can be to check whether the moving distance of the electronic device is less than a threshold value. If so, it is considered stable, otherwise it is considered unstable. This implementation can ensure that the color brightness of the image output by the second camera is consistent with the color brightness of the image output by the reference camera, and can also reduce the power consumption of the electronic device as much as possible.

[0088] The above implementation is described by the specific processing of the following two image frames:

[0089] After S106, the electronic device can collect the third original image data through the second camera, and the moving distance of the electronic device is less than the threshold when collecting the third original image data and collecting the second original image data; perform RAW domain processing on the third original image data, or perform RAW domain processing and partial color brightness processing on the third original image data to obtain a second image to be processed; use the first parameter to further perform color brightness processing on the second image to be processed to obtain a second preview image; display the second preview image. In some embodiments, the electronic device can also perform smoothing processing on the first parameter to obtain a new parameter, and use the new parameter to perform color brightness processing on the second image to be processed to obtain a second preview image. Using the first parameter or similar parameters can ensure the continuity and smoothness of the preview effect, and can also reduce the power consumption of the device.

[0090] After S106, the electronic device can collect fourth original image data through the second camera, and the moving distance of the electronic device is greater than the threshold when collecting the fourth original image data and collecting the second original image data; perform RAW domain processing on the fourth original image data, or perform RAW domain processing and partial color brightness processing on the fourth original image data to obtain a third image to be processed; collect fifth original image data through the first camera, perform RAW domain processing and color brightness processing on the fifth original image data to obtain a second reference image, and the fifth original data and the fourth original data are collected at the same time or the difference between the collection times does not exceed the threshold; determine the second parameter, and the second parameter is obtained based on the second reference image and the third image to be processed; use the second parameter to further perform color brightness processing on the third image to be processed to obtain a third preview image; display the third preview image.

[0091] In the process of processing the above two image frames, the specific implementation of RAW domain processing and color brightness processing can refer to the relevant description above.

[0092] In some implementations, after S106 , the electronic device may further switch the camera. Figure 1 The method shown may also include the following steps:

[0093] S107: The electronic device receives a user operation to switch the second camera to the fourth camera.

[0094] The fourth camera and the second camera are different cameras. The fourth camera can be any camera in the electronic device that can be used to display images, such as a main camera, a telephoto camera, a wide-angle camera, a front camera, etc.

[0095] The user operation of switching the second camera to the fourth camera can be implemented as a user operation on the display screen of the electronic device, such as clicking the control of switching cameras in the user interface. It can also be implemented as a voice user operation or a gesture, etc., which is not limited here.

[0096] S108, the electronic device collects sixth original image data through the first camera, performs RAW domain processing and color brightness processing on the sixth original image data, and obtains a third reference image. The specific implementation of S108 can refer to S102.

[0097] S109, the electronic device responds to the user operation in S107, starts the fourth camera, collects the seventh original image data through the fourth camera, the seventh original data and the sixth original data are collected at the same time or the difference between the collection times does not exceed the threshold, performs RAW domain processing on the seventh original image data, or performs RAW domain processing and partial color brightness processing on the seventh original image data to obtain a fourth image to be processed. The specific implementation of S109 can refer to S103.

[0098] S110: The electronic device determines a third parameter.

[0099] The third parameter may be determined based on the third reference image and the fourth image to be processed, and further, the third parameter may be determined based on the third metadata generated when the first camera captures the sixth original image data, and / or the fourth metadata generated when the fourth camera captures the seventh original image data. For the specific implementation of S110, reference may be made to S104.

[0100] S111, the electronic device performs further color and brightness processing on the fourth image to be processed using the third parameter to obtain a fourth preview image. For the specific implementation of S111, reference may be made to S105.

[0101] S112, the electronic device displays a fourth preview image. For the specific implementation of S112, reference may be made to S106.

[0102] Through the above method provided in the present application, it can be known that when an electronic device uses any camera to capture an image, the image generated by the reference camera is used to perform color and brightness processing on the image of the camera, so that the color and brightness of the image output by each camera can be guaranteed to be consistent with the output of the reference camera. At the moment when the electronic device switches the camera and in the steady state process after the switch, the color and brightness of the preview image displayed by the electronic device can be kept consistent with the reference camera, thus solving the problem of inconsistent color and brightness of the preview image caused by the differences between different cameras.

[0103] The above method provided in the embodiment of the present application can be applied to any shooting scene, such as photo shooting scene, video recording scene, and dynamic picture shooting scene.

[0104] In some embodiments, the second camera and the first camera in the above method may also be the same camera. If the two are the same camera, after receiving the user operation to start the second camera, the electronic device can collect the original image data through the second camera, perform RAW domain processing and color brightness processing on the original image data, obtain a preview image, and display the preview image. Since the second camera is the reference camera itself, the second camera can output the preview image according to the original processing logic without comparing with other cameras.

[0105] In some embodiments of the present application, the reference camera is not fixed and can be changed according to different usage scenarios. The usage scenario here can refer to the environment in which the user is shooting, such as day, night, mountain view, sea view, etc. The camera that can capture more accurate color and brightness, more restore the real situation or better meet the user's aesthetic needs in different environments may not be the same, so the reference camera can be switched in different scenarios to give the user a better shooting experience.

[0106] For example, in Figure 1 The method shown may also include the following steps: the electronic device receives a user operation to start the sixth camera; the eighth original image data is collected through the fifth camera, and the eighth original image data is processed in the RAW domain and color brightness to obtain a fourth reference image; the sixth camera is started, and the ninth original image data is collected through the sixth camera, and the ninth original image data and the eighth original data are collected at the same time or the difference between the collection time does not exceed the threshold, and the ninth original image data is processed in the RAW domain, or the ninth original image data is processed in the RAW domain and partially processed in color brightness to obtain a fifth image to be processed; the fourth parameter is determined, and the fourth parameter is obtained according to the fourth reference image and the fifth image to be processed; the fifth image to be processed is further processed in color brightness using the fourth parameter to obtain a fifth preview image; and the fifth preview image is displayed. Among them, the fifth camera is another reference camera different from the first camera. The fifth camera can be determined according to the current scene.

[0107] In the above implementation, the process of outputting a preview image by using another camera as a reference camera can be referred to as follows. Figure 1 The specific implementation of S101-S106.

[0108] Equipment provided by this application

[0109] refer to Figure 2 , Figure 2 The hardware structure diagram of the electronic device 100 provided in the embodiment of the present application. The electronic device 100 may be the electronic device mentioned above, which is used to execute Figure 1 The shooting method shown.

[0110] like Figure 2As shown, the electronic device 100 may include: the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0111] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0112] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0113] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0114] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0115] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0116] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0117] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0118] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0119] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0120] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0121] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0122] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0123] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0124] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0125] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0126] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0127] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0128] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0129] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0130] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0131] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0132] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0133] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microled, micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0134] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0135] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0136] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then converts the electrical signal into a digital image signal and transmits it to the ISP. The ISP outputs the digital image signal to the DSP for processing.

[0137] In the embodiment of the present application, the number of cameras 193 can be multiple, for example, it can include a main camera, a telephoto camera and a wide-angle camera, and can also include a front camera and a multi-spectral sensor.

[0138] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0139] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0140] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0141] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0142] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0143] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; can be divided into universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0144] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0145] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0146] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0147] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0148] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0149] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0150] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0151] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0152] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0153] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0154] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0155] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0156] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0157] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0158] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0159] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0160] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0161] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0162] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0163] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0164] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0165] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0166] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0167] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0168] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0169] In an embodiment of the present application, the internal memory 121 is used to store a program for implementing the shooting method provided by the present application on the electronic device side, and the processor 110 is used to call the instructions contained in the program to trigger the electronic device 100 to execute each step of the method.

[0170] Specifically, information input devices such as the display screen 194 and the microphone 170C can be used to receive various user operations, the camera 193 can be used to collect raw image data, and the ISP, DSP, etc. can be used to further process the raw image data.

[0171] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes a mobile operating system with a layered architecture as an example to exemplify the software structure of the electronic device 100.

[0172] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0173] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the mobile operating system is divided into four layers, from top to bottom: application layer, program framework layer / core service layer, underlying library and runtime, and kernel layer.

[0174] The application layer can include a series of application packages.

[0175] like Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0176] The program framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The program framework layer includes some predefined functions.

[0177] like Figure 3 As shown, the program framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0178] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0179] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0180] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0181] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0182] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0183] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0184] The runtime can refer to all code libraries, frameworks, etc. required for the program to run. For example, for the C language, the runtime includes a series of function libraries required for the C program to run. For the Java language, in addition to the core library, the runtime also includes the virtual machine required for the Java program to run. The above core library may include the function functions that the Java language needs to call.

[0185] The underlying library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0186] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0187] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0188] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0189] A 2D graphics engine is a drawing engine for 2D drawings.

[0190] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0191] The following is an example of the workflow of the electronic device software and hardware in conjunction with capturing a photo scene.

[0192] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0193] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0194] The present application also provides an electronic device, which may include: a memory and a processor. The memory may be used to store a computer program; the processor may be used to call the computer program in the memory so that the electronic device executes the method executed by the electronic device side in any of the above embodiments.

[0195] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the electronic device side in any of the above embodiments.

[0196] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0197] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0198] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0199] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0200] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0201] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute the method executed by the electronic device side in any of the above embodiments.

[0202] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device side in any of the above embodiments.

[0203] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0204] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0205] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0206] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0207] The terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0208] In short, the above is only an embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

1. A shooting method, characterized in that: The method is applied to an electronic device, the electronic device includes a first camera and a second camera, and the method includes: receiving a user operation of starting the second camera; The first camera collects first original image data, performs RAW domain processing and color brightness processing on the first original image data, and obtains a first reference image; The second camera is started, and second original image data is collected through the second camera, wherein the second original data and the first original data are collected at the same time or the difference between the collection times does not exceed a threshold, and the second original image data is processed in the RAW domain, or the second original image data is processed in the RAW domain and partially processed in color and brightness, to obtain a first image to be processed; Determining a first parameter, where the first parameter is obtained according to the first reference image and the first image to be processed; Performing further color and brightness processing on the first image to be processed using the first parameter to obtain a first preview image; The first preview image is displayed.

2. The method according to claim 1, characterized in that: The method further comprises: generating first metadata when the first original image data is collected; The first parameter is also obtained according to the first metadata.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: generating second metadata when the second original image data is acquired; The first parameter is also obtained according to the second metadata.

4. The method according to any one of claims 1 to 3, characterized in that: The obtaining of the first parameter specifically includes: The first reference image and the first image to be processed are input into a neural network, and a first parameter is output through the neural network.

5. The method according to claim 4, characterized in that The neural network is stored in the neural network processor NPU of the electronic device.

6. The method according to any one of claims 1 to 5, characterized in that: The first parameter includes any one or more of the following: a color correction matrix CCM, a color lookup table LUT, a dynamic range correction DRC, a global tone mapping GTM, a color gamut mapping or a local tone mapping LTM relationship.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The second camera collects third original image data, and when collecting the third original image data and collecting the second original image data, the moving distance of the electronic device is less than a threshold; Performing RAW domain processing on the third original image data, or performing RAW domain processing and partial color brightness processing on the third original image data, to obtain a second image to be processed; Performing further color and brightness processing on the second image to be processed using the first parameter to obtain a second preview image; The second preview image is displayed.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The fourth original image data is collected by the second camera, and when the fourth original image data and the second original image data are collected, the moving distance of the electronic device is greater than a threshold; Performing RAW domain processing on the fourth original image data, or performing RAW domain processing and partial color brightness processing on the fourth original image data, to obtain a third image to be processed; The fifth original image data is collected by the first camera, and the fifth original image data is processed in the RAW domain and the color brightness is processed to obtain a second reference image, wherein the fifth original data and the fourth original data are collected at the same time or the difference between the collection times does not exceed a threshold; determining a second parameter, where the second parameter is obtained according to the second reference image and the third image to be processed; Performing further color and brightness processing on the third image to be processed using the second parameter to obtain a third preview image; The third preview image is displayed.

9. The method according to any one of claims 1 to 8, characterized in that: The user operation of starting the second camera specifically includes: A user operation of starting a camera application in the electronic device; Alternatively, the user operation is used to switch the third camera to the second camera, and the electronic device also includes the third camera.

10. The method according to any one of claims 1 to 9, characterized in that: The electronic device further includes a fourth camera, and after displaying the first preview image, the method further includes: receiving a user operation of switching the second camera to the fourth camera; Acquire sixth original image data through the first camera, perform RAW domain processing and color brightness processing on the sixth original image data, and obtain a third reference image; The fourth camera is started, and seventh original image data is collected through the fourth camera, wherein the seventh original image data and the sixth original image data are collected at the same time or the difference between the collection times does not exceed a threshold, and the seventh original image data is processed in the RAW domain, or the seventh original image data is processed in the RAW domain and partially processed in color and brightness, to obtain a fourth image to be processed; Determining a third parameter, wherein the third parameter is obtained according to the third reference image and the fourth image to be processed; Performing further color and brightness processing on the fourth image to be processed using the third parameter to obtain a fourth preview image; The fourth preview image is displayed.

11. The method according to any one of claims 1 to 10, characterized in that: The electronic device further includes a fifth camera and a sixth camera, the fifth camera and the first camera are different cameras, and the method further includes: Receiving a user operation of starting the sixth camera; Acquire eighth original image data through the fifth camera, perform RAW domain processing and color brightness processing on the eighth original image data, and obtain a fourth reference image; The sixth camera is started, and ninth original image data is collected through the sixth camera, wherein the ninth original image data and the eighth original image data are collected at the same time or the difference between the collection times does not exceed a threshold, and the ninth original image data is processed in the RAW domain, or the ninth original image data is processed in the RAW domain and partially processed in color and brightness, to obtain a fifth image to be processed; determining a fourth parameter, wherein the fourth parameter is obtained according to the fourth reference image and the fifth image to be processed; Performing further color and brightness processing on the fifth image to be processed using the fourth parameter to obtain a fifth preview image; The fifth preview image is displayed.

12. An electronic device, characterized in that: include: A first camera, a second camera, a memory, and one or more processors; The first camera, the second camera, and the memory are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-11.

13. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method as claimed in any one of claims 1 to 11.

14. A chip system, comprising at least one processor and a memory, wherein the memory is used to store program instructions and data, and the processor is used to call the program instructions and data to implement the method as described in any one of claims 1-11.

Images