Image processing method of camera application and related device

Abstract

The application provides an image processing method of a camera application and related equipment, and relates to the technical field of image processing. The image processing method comprises the following steps: starting a first camera application, wherein the resolution of the camera application issued by a camera of an electronic device is a first resolution; obtaining a first preview image in a first time period, wherein, in the first time period, the ISO value sensed by the camera is a first ISO value, the first ISO value is less than a preset ISO threshold, the resolution of the first preview image is a second resolution, and the second resolution is equal to the first resolution; obtaining a second preview image in a second time period, wherein, in the second time period, the ISO value sensed by the camera is a second ISO value, the second ISO value is greater than or equal to the preset ISO threshold, the resolution of the second preview image is a third resolution, and the third resolution is less than the first resolution; and adjusting the resolution of the preview image based on the resolution issued by the camera and the ISO value sensed by the camera, so that the resolution of the preview image is adapted to the ambient light brightness.

Description

Technical Field

[0001] This application relates to the field of electronic information technology, specifically to an image processing method and related equipment for camera applications. Background Technology

[0002] When an electronic device launches its camera app, it can convert the raw image captured by the camera into a preview image and display it on the camera app's interface. However, in some situations, such as when the ambient light is bright, the preview image has a lower resolution, resulting in lower image clarity. Summary of the Invention

[0003] This application provides an image processing method and related equipment for camera applications, aiming to solve the problem of low image clarity when the ambient light is strong.

[0004] The first aspect of this application provides an image processing method for a camera application, applied to an electronic device. The method includes: launching a first camera application and displaying a first interface; displaying a first preview image in the first interface during a first time period, wherein the ambient light intensity during the first time period is a first brightness, the resolution sent by the first camera application to the camera of the electronic device is a first resolution, and the resolution of the first preview image is a second resolution; and displaying a second preview image in the first interface during a second time period, wherein the ambient light intensity during the second time period is a second brightness, and the resolution of the second preview image is a third resolution. The method is based on the condition that the first resolution is greater than the third resolution, and the second brightness is less than the first brightness, meaning the second resolution is greater than the third resolution.

[0005] In this embodiment, based on the ambient light brightness changing from a first brightness level to a second brightness level, and the first resolution being greater than the third resolution, the resolution of the preview image displayed by the first camera application correspondingly changes from the second resolution to the third resolution. That is, as the ambient light brightness changes, the resolution of the preview image changes accordingly. As the ambient light brightness increases, the resolution of the preview image increases accordingly. Therefore, when the ambient light brightness is high, the resolution of the preview image is higher, resulting in higher clarity.

[0006] In one implementation, the method further includes: closing a first camera application, launching a second camera application, and displaying a second interface. A third preview image is displayed during a third time period. The third preview image is displayed on the second interface. During the third time period, the ambient light brightness is either a first brightness or a second brightness. The resolution sent by the second camera application to the camera of the electronic device is a fourth resolution. Based on the fact that the fourth resolution is less than the first resolution, the resolution of the third preview image is the third resolution.

[0007] In this embodiment, based on the ambient light brightness being either a first brightness or a second brightness, and the fourth resolution being less than the first resolution, the resolution of the preview image displayed by the second camera application is correspondingly less than the resolution of the preview image displayed by the first camera application. That is, when the ambient light brightness is the same, the resolution of the preview image changes accordingly as the resolution sent by the camera application to the camera changes. As the resolution sent by the camera application to the camera increases, the resolution of the preview image increases accordingly. Therefore, when the resolution sent by the camera application to the camera is higher, the resolution of the preview image is higher, and thus the clarity of the preview image is also higher.

[0008] In another implementation, the resolution of the first preview image is the second resolution, based on the fact that the first resolution is greater than the third resolution and the ISO value is less than a preset ISO threshold.

[0009] In another implementation, the resolution of the second preview image is the third resolution, based on the fact that the first resolution is greater than the third resolution and the ISO value is greater than or equal to a preset sensitivity threshold.

[0010] In another implementation, the resolution of the third preview image is the third resolution, based on the fact that the fourth resolution is less than the third resolution and the ISO value is less than a preset sensitivity threshold.

[0011] In another implementation, the resolution of the third preview image is the third resolution, based on the fact that the fourth resolution is less than the third resolution and the ISO value is greater than or equal to a preset sensitivity threshold.

[0012] In another implementation, before displaying the first preview image in the first time period, the method further includes: a camera application sending a streaming request to a camera driver via the camera API of an electronic device, the streaming request including a first resolution. Based on the streaming request, the camera driver configures the preview stream and creates channels, and determines the initial operating mode of the color filter array (CFA) device, the initial operating mode including a boxed mode or an unboxed mode; the first resolution is the resolution of the original image, and the CFA device is used to convert the original image into a preview image; based on the CFA device operating mode being boxed, the resolution of the preview image displayed by the first camera application is a third resolution; based on the CFA device operating mode being unboxed, the resolution of the preview image displayed by the first camera application is a second resolution.

[0013] In another implementation, determining the initial operating mode of the CFA device includes: determining the initial operating mode of the CFA device to be a non-packing mode based on a first resolution being greater than a third resolution and an ISO value being less than a preset sensitivity threshold; and determining the initial operating mode of the CFA device to be a packing mode based on a first resolution being less than or equal to a third resolution, or an ISO value being greater than or equal to a preset sensitivity threshold.

[0014] In another implementation, the method further includes: based on successful streaming, the camera application sends a frame processing request to the camera driver via the camera API, the frame processing request including a first resolution and the original image. Based on the frame processing request, the camera driver converts the original image into a preview image.

[0015] In another implementation, converting the original image into a preview image includes: determining a preset operating mode for the CFA device, where the preset operating mode includes either a boxed mode or an unboxed mode; determining the operating mode of the CFA device based on the preset operating mode and a reference operating mode; the reference operating mode being the operating mode of the CFA device when processing the previous frame processing request, or the initial operating mode; setting the operating mode of the CFA device; and driving the CFA device to convert the original image into a preview image.

[0016] In another implementation, determining the operating mode of the CFA device based on a preset operating mode and a reference operating mode includes: determining the operating mode of the CFA device as the reference operating mode if the preset operating mode and the reference operating mode are the same; and determining the operating mode of the CFA device as the preset operating mode if the preset operating mode and the reference operating mode are different.

[0017] In another implementation, setting the operating mode of the CFA device includes: based on the CFA device's operating mode being determined to be a preset operating mode, performing a seamless switch via a pre-configured register or register group to switch the reference operating mode to the preset operating mode.

[0018] A second aspect of this application provides an electronic device, which includes a memory, a processor, a camera, a sensor module, and a display screen. The camera is used to capture still images or videos. The sensor module includes a color filter array (CFA) device and an ambient light sensor. The CFA device operates in either a boxed mode or an unboxed mode. The ambient light sensor is used to sense ambient light brightness. The display screen is used to display an interface. The memory is used to store instructions. The processor is used to execute the instructions stored in the memory, causing the electronic device to perform the image processing method of the camera application according to the embodiments of this application.

[0019] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform an image processing method for a camera application according to an embodiment of this application.

[0020] The technical effects brought about by the second and third aspects of the embodiments of this application can be found in the relevant description of the image processing method for camera applications in the first aspect above, and will not be repeated here. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the software structure of an electronic device provided as an example.

[0022] Figure 2 This is an example illustrating the conversion of a raw image into a box-size image.

[0023] Figure 3 This is an example illustrating the conversion of a raw image into a non-boxed size image.

[0024] Figure 4 This is a schematic diagram of the software structure of an electronic device provided in one embodiment of this application.

[0025] Figure 5 This is a timing diagram of an image processing method for a camera application provided in one embodiment of this application.

[0026] Figure 6 yes Figure 5 Flowchart of the sub-steps in step S508.

[0027] Figure 7 yes Figure 5 Flowchart of the sub-step in step S514.

[0028] Figure 8 yes Figure 7 Flowchart of the sub-steps in step S702.

[0029] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0030] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0031] It should also be noted that the methods disclosed in the embodiments of this application or the methods shown in the flowcharts include one or more steps for implementing the method. Without departing from the scope of the claims, the execution order of multiple steps can be interchanged, and some steps can also be deleted.

[0032] Figure 1 This is a schematic diagram of the software structure of an electronic device provided as an example.

[0033] like Figure 1 As shown, the software system of an electronic device is divided into three layers, from top to bottom: the application (APP) layer, the framework layer, and the hardware abstraction layer (HAL).

[0034] The APP layer includes camera applications, which can include system camera applications or third-party camera applications.

[0035] The Framework layer provides Application Programming Interfaces (APIs) for the various applications in the App layer, such as the Camera API. The Framework layer includes a Camera Service module, which is used to implement calls to the Camera API.

[0036] HAL includes camera drivers, which are used to drive the various components (such as sensors) of a camera (also called a webcam) in an electronic device to perform various camera functions and data processing, such as previewing, taking pictures, or recording videos.

[0037] The following example uses a camera application preview scene, combined with... Figure 1 The implementation process of the preview function is explained.

[0038] When the camera application launches, it sends a resolution request to the camera driver via the Camera API. After receiving the resolution request, the camera driver sends an initial resolution to the camera application via the Camera API, based on at least one initial resolution reported by the sensor. The camera application then determines a target resolution based on this initial resolution; the target resolution is the maximum resolution that is the same as or close to the camera application's output image size.

[0039] In some cases, when at least one initial resolution does not have a resolution that is the same as or close to the output image size of the camera application, the target resolution may not be able to be determined, causing the camera application to be unable to output a preview image.

[0040] After determining the target resolution, the camera application sends a streaming request to the camera driver via the Camera API. This request requests configuration of the preview stream. Upon receiving the streaming request, the camera driver configures the preview stream and creates a pipeline (e.g., a channel). The preview stream includes at least two preview images, and the pipeline is used to stream-encapsulate the processing of these two preview images, resulting in a single preview stream. After streaming is complete, the camera driver sends the streaming result to the camera application via the Camera API, indicating whether the streaming was successful or failed.

[0041] When the streaming result is successful, the camera application sends multiple frame processing requests to the camera driver via the Camera API. Each frame processing request includes one original image frame with the target resolution. The frame processing request requests the conversion of the original image into a corresponding preview image. For each frame processing request, the camera driver, after receiving the request, converts the original image into a preview image and sends the preview image to the camera application via the Camera API. After receiving the preview image, the camera application displays it on the preview interface.

[0042] The camera driver converts the raw image into a preview image by driving the sensor. The sensor includes a color filter array (CFA) device, such as a quadrature color filter array (Quadra CFA) device.

[0043] The following explanation uses the Quadra CFA device as an example to illustrate the working principle of how a camera driver converts a raw image into a preview image.

[0044] Quadra CFA devices typically operate in either binning mode or full-size mode. In low-light scenes (also known as dark scenes), the camera driver sets the Quadra CFA device to binning mode. In binning mode, the Quadra CFA device converts the original image into a binning-size image, that is, it merges four adjacent pixels in the original image into one pixel to obtain a binning-size image. Therefore, the photosensitive area of ​​each pixel in the binning-size image is four times that of each pixel in the original image, resulting in a brighter preview image displayed on the camera application's preview screen. Binning mode improves the brightness of the image output by the camera application by merging pixels. Since a higher number of pixels in an image results in a higher image resolution, the resolution of the binning-size image (also known as the binning resolution) is lower than the target resolution of the original image.

[0045] In scenarios with normal ambient light (also known as normal scenarios), the camera driver sets the Quadra CFA device to Fullsize mode. Since the camera's Image Signal Processor (ISP) channel only allows the transmission of images with a Bayer pixel arrangement, and the original image has a non-Bayer pixel arrangement, in Fullsize mode, the Quadra CFA device converts the pixel arrangement of the original image to a Bayer pixel arrangement, thus providing an unpacked (also known as full-size) image that can be transmitted on the ISP channel. The number of pixels in the Fullsize image is equal to the number of pixels in the original image; therefore, the resolution of the Fullsize image (also known as unpacked resolution or full-size resolution) is equal to the target resolution of the original image. For example, a 16MP (or M, megapixel) Quadra CFA device activates Binning mode in low-light scenes, converting the original 16M image into a 4M Binningsize image. Since the photosensitive area of ​​each pixel in the Binningsize image is four times that of each pixel in the original image, the Binningsize image has significantly higher brightness. In normal scenes, the 16MP Quadra CFA device activates Fullsize mode, converting the pixel arrangement of the original 16M image into a Bayer-style pixel arrangement, thus providing a 16M Fullsize image with a Bayer-style pixel arrangement.

[0046] Quadra CFA devices can include 4-cell 1 devices, 9-cell 1 devices, or 16-cell 1 devices. Taking a 4-cell 1 device as an example... Figure 2 and Figure 3 As shown, in low-light scenes, the camera driver sets the 4-cell 1 device to Binning mode. The 4-cell 1 device performs internal compositing processing using a Complementary Metal Oxide Semiconductor (CMOS) device to convert the original image into a Binning-size image. In normal scenes, the camera driver sets the 4-cell 1 device to Fullsize mode. The 4-cell 1 device uses Remosaic technology to convert the original image into a Fullsize image with a Bayer pixel arrangement. Remosaic technology includes both software and hardware methods; the software method uses the Remosaic algorithm, while the hardware method involves internal processing within the device.

[0047] It's understandable that in low ambient light, users perceive image sensitivity more readily than image resolution. In this situation, the Binning Size image provided by the Quadra CFA device has high brightness, meeting the user's brightness requirements. Conversely, in normal or strong ambient light, users perceive image resolution more readily than image sensitivity. If the Quadra CFA device remains in Binning Mode in this case, the lower resolution of the Binning Size image will fail to meet the user's resolution needs, negatively impacting the user experience. Therefore, in normal or strong ambient light, the camera driver sets the Quadra CFA device's operating mode to Fullsize Mode. In this mode, the Fullsize image provided by the Quadra CFA device has high resolution, meeting the user's resolution requirements.

[0048] However, in certain situations, such as when the camera application is a third-party camera application, the chip platform allows the QuadraCFA device to report an initial resolution less than or equal to the binning resolution. This results in the target resolution of the original image being less than or equal to the binning resolution, causing the maximum output image size displayed by the camera application to be the binning size. Under normal or strong ambient light conditions, the lower resolution of the image displayed by the camera application leads to lower image sharpness, affecting the output image quality.

[0049] For example, the 9MP Quadra CFA device itself can support providing images with a resolution of 9M, but due to limitations of the chip platform, camera applications can only display images with a maximum resolution of 2M, resulting in lower image clarity.

[0050] Based on this, embodiments of this application provide an image processing method and related equipment for camera applications, aiming to enable the Quadra CFA device to adapt its working mode to changes in ambient light brightness even when limited by the chip platform, outputting full-size images in normal scenes and binning-size images in dark scenes, thereby improving the image output effect of camera applications.

[0051] The following is combined with Figure 4 and Figure 5 The image processing method for camera applications provided in the embodiments of this application will be described in detail.

[0052] Figure 4 This is a schematic diagram of the software structure of an electronic device provided in one embodiment of this application.

[0053] Compared to Figure 1 ,exist Figure 4 In the software system of the illustrated electronic device, the Camera API includes a resolution customization interface. This interface is used to obtain at least one initial resolution reported by the camera driver and at least one customized resolution from a resolution profile. The initial resolution is less than or equal to the binning resolution, and the customized resolution is greater than the binning resolution.

[0054] Resolution configuration files are pre-configured files in the system or pushed to the cloud, such as resolution-customized templates (e.g., XML templates) for third-party camera apps and their pages. These configuration files can store the camera app's version number, product name, package name, page information, camera identifier, and streaming configuration information. Streaming configuration information can include shooting modes and customized resolutions. Shooting modes can be photo, video, or preview modes.

[0055] For example, the resolution configuration file is:

[0056] <CustomCameraSetting version="7.10.22.00"product="XXX">

[0057] <CustomCameraSetting pkg="com.xxxxxx.mm"

[0058] activity="com.xxxxxx.mm.plugin.scanner.ui.BaseScanUI">

[0059] <StreamConfigurations id="0">

[0060] <StreamConfiguration content="34 3840 2400OUTPUT" / >

[0061] <StreamConfiguration content="35 3840 2400OUTPUT" / >

[0062] <StreamConfiguration content="33 3840 2400OUTPUT" / >

[0063]

[0064]

[0065] Where version represents the version number of the camera application, product represents the product name, pkg represents the package name of the camera application, activity represents the camera application page, id represents the camera identifier, content represents the stream configuration information, 33 / 34 / 35 represents the shooting mode, and 3840 2400 is the customized resolution corresponding to the shooting mode.

[0066] The camera application parses the resolution configuration file by calling a resolution customization interface. For example:

[0067] EXPORT_API void*xxCameraServiceEx::setStreamConfigs(void*param)

[0068] {

[0069] ...

[0070] tinyxml2::XMLElement*pkgNameListElement=rootElement->FirstChildElement

[0071] ("CustomCameraSetting");

[0072] ...

[0073] }

[0074] Tinyxml2 is an XML parser used to parse XML files, and CustomCameraSetting is a resolution configuration file.

[0075] After obtaining the initial resolution and the custom resolution, the camera application determines the target resolution based on these two values. Even when the initial resolution reported by the Quadra CFA device is less than or equal to the bin resolution due to chip platform limitations, the camera application can still obtain a custom resolution greater than the bin resolution, thus enabling the target resolution to be determined as a resolution greater than the bin resolution.

[0076] In addition, Figure 4 In the software system of the illustrated electronic device, the streaming request issued by the camera application includes the target resolution. The camera driver includes a first mode adaptation module and a frame processing module. The first mode adaptation module is used to determine the initial operating mode of the Quadra CFA device based on the sensitivity value and the target resolution during the streaming process.

[0077] The frame processing module includes a mode selection module, which in turn includes a second mode adaptation module. The second mode adaptation module is used to determine the preset operating mode of the Quadra CFA device based on the photosensitivity value and the target resolution during the frame processing request process.

[0078] The mode selection module is used to determine the working mode of the Quadra CFA device when processing the current frame processing request, based on the preset working mode and the working mode of the Quadra CFA device when processing the previous frame processing request, or based on the preset working mode and the initial working mode, after obtaining the preset working mode determined by the second mode adaptation module.

[0079] The frame processing module is used to set the working mode of the Quadra CFA device after the working mode is determined by the acquisition mode selection module, and then drive the Quadra CFA device to convert the original image into a preview image.

[0080] For example, during the streaming process, the camera driver calls the `FindBestSensorMode` function to execute the functionality of the first mode adaptation module. During frame processing request handling, the camera driver calls the `OnExecuteProcessRequest` function to execute the functionality of the frame processing module. The `OnExecuteProcessRequest` function nests the `OnSelectSensorMode` function, which the camera driver calls to execute the functionality of the mode selection module. The `OnSelectSensorMode` function also nests the `FindBestSensorMode` function, which the camera driver calls to execute the functionality of the second mode adaptation module.

[0081] Specifically, during the distribution process, the camera driver calls the FindestSensorMode function to perform the following operations:

[0082] Query the camera app's custom field (e.g., ThirdAppCustomSupport). When the camera app's custom field indicates that the camera app supports mode selection, if the target resolution is greater than the binning resolution and the current ISO value is less than the preset ISO threshold, then the initial operating mode is determined to be Fullsize mode. If the target resolution is less than or equal to the binning resolution, or the current ISO value is greater than or equal to the preset ISO threshold, then the initial operating mode is determined to be Binning mode.

[0083] After determining the initial operating mode, an initial operating mode indicator (e.g., the SensorMode value) is returned. The initial operating mode indicator indicates whether the initial operating mode is Fullsize mode or Binning mode. For example, the initial operating mode indicator can be set to 1 or 0. When the initial operating mode indicator is set to 1, it indicates that the initial operating mode is Fullsize mode. When the initial operating mode indicator is set to 0, it indicates that the initial operating mode is Binning mode.

[0084] The camera application-specific fields can be stored in the sensor driver file (e.g., the Sensor XML driver file). When the camera driver calls the `FindBestSensorMode` function, it retrieves these fields from the sensor driver file. These fields indicate whether the camera application supports mode selection. For example, the fields can be set to `True` or `False`. When a field is set to `True`, it indicates that the camera application supports mode selection, meaning the Quadra CFA device can be set to either Fullsize mode or Binning mode. When a field is set to `False`, it indicates that the camera application does not support mode selection, meaning the Quadra CFA device operates in a single mode (Fullsize mode or Binning mode).

[0085] ISO (sensitivity) values ​​measure how sensitive a camera's sensor is to ambient light. When the current ISO value is greater than or equal to a preset ISO threshold, the camera's sensor is more sensitive to ambient light, indicating that the current ambient light level is low. When the current ISO value is less than the preset ISO threshold, the camera's sensor is less sensitive to ambient light, indicating that the current ambient light level is normal or relatively high. The preset ISO threshold can be set as needed, for example, a preset ISO threshold of 700.

[0086] When the streaming result is successful, the camera application sends a first frame processing request (also known as the initial frame processing request) to the camera driver via the Camera API. This request includes the target resolution and the original image. Upon receiving the first frame processing request, the camera driver calls the OnExecuteProcessRequest function to execute the frame processing module's functionality, thereby converting the original image into a preview image. Specifically, the camera driver calls the OnExecuteProcessRequest function to perform the following operations:

[0087] (1) Call the FindestSensorMode function.

[0088] Query the camera application's custom fields. When the camera application's custom fields indicate that the camera application supports mode selection, if the target resolution is greater than the binning resolution and the current ISO value is less than the preset ISO threshold, then the preset working mode is determined to be Fullsize mode. If the target resolution is less than or equal to the binning resolution, or the current ISO value is greater than or equal to the preset ISO threshold, then the preset working mode is determined to be Binning mode.

[0089] After determining the preset operating mode, the `FindBestSensorMode` function returns a preset operating mode indicator. This indicator specifies whether the preset operating mode is Fullsize mode or Binning mode. For example, the preset operating mode indicator can be set to 1 or 0. When the preset operating mode indicator is set to 1, it indicates that the preset operating mode is Fullsize mode. When the preset operating mode indicator is set to 0, it indicates that the preset operating mode is Binning mode.

[0090] (2) Call the OnselectSensorMode function.

[0091] After obtaining the preset operating mode indicator, it is determined whether the preset operating mode indicator is the same as the initial operating mode indicator. The initial operating mode indicator can be stored in the sensor driver file. When the camera driver calls the OnselectSensorMode function, it retrieves the initial operating mode indicator from the sensor driver file. If the preset operating mode indicator is the same as the initial operating mode indicator, the operating mode of the Quadra CFA device is determined to be the initial operating mode. If the preset operating mode indicator is different from the initial operating mode indicator, the operating mode of the Quadra CFA device is determined to be the preset operating mode.

[0092] After determining the operating mode, the OnselectSensorMode function returns an operating mode indicator. This indicator specifies whether the operating mode is Fullsize mode or Binning mode. For example, the indicator can be set to 1 or 0. A setting of 1 indicates Fullsize mode, while a setting of 0 indicates Binning mode.

[0093] (3) The function of executing the OnExecuteProcessRequest function.

[0094] After acquiring the operating mode indicator, the operating mode of the Quadra CFA device is set to either Fullsize mode or Binning mode according to the indicator. When the operating mode indicator indicates that the Quadra CFA device is in the initial operating mode, the initial operating mode is maintained. When the operating mode indicator indicates that the Quadra CFA device is in the preset operating mode, the initial operating mode is switched to the preset operating mode.

[0095] When switching the Quadra CFA device from its initial operating mode to a preset operating mode, a seamless switch can be performed using pre-configured registers or register groups. This reduces switching latency and helps avoid long output delays caused by stuttering during mode switching. The pre-configured registers or register groups can be stored in the sensor driver file. For example, the sensor driver file might be:

[0096] <customsettingsinfo>

[0097] <customsettings>

[0098] <sensormodefrom> 1< / sensormodefrom>

[0099] <ressettings>

[0100] reg1,reg2,reg3

[0101] < / ressettings>

[0102] < / customsettings>

[0103] < / customsettingsinfo>

[0104] Among them, sensorModeFrom is the initial working mode indicator, and reg1, reg2, and reg3 are pre-configured registers or register groups.

[0105] When the Quadra CFA device operates in Fullsize mode, it converts the pixel arrangement of the original image into a Bayer-style pixel arrangement, resulting in a Fullsize image. When the Quadra CFA device operates in Binning mode, it merges four adjacent pixels of the original image into one pixel, resulting in a Binning-size image.

[0106] After acquiring a full-size or binary-size image, the camera driver sends the full-size or binary-size image to the camera application via the Camera API. Upon receiving the full-size or binary-size image, the camera application displays it on the preview screen.

[0107] After displaying the preview image corresponding to the first frame processing request on the preview interface, the camera application sends a second frame processing request (also known as a non-first frame processing request) to the camera driver via the Camera API. The second frame processing request includes the target resolution and the original image. After receiving the second frame processing request, the camera driver calls the OnExecuteProcessRequest function to execute the functions of the frame processing module, thereby converting the original image into a preview image.

[0108] Compared to the process of handling the first frame processing request, the OnselectSensorMode function performs differently when the camera driver calls the OnExecuteProcessRequest function to handle the second frame processing request, as detailed below:

[0109] After obtaining the preset operating mode indicator, it is determined whether the preset operating mode indicator is the same as the operating mode indicator used when processing the previous frame processing request. The operating mode indicator used when processing the previous frame processing request can be stored in the sensor driver file. When the camera driver calls the OnselectSensorMode function, it retrieves the operating mode indicator used when processing the previous frame processing request from the sensor driver file. If the preset operating mode indicator is the same as the operating mode indicator used when processing the previous frame processing request, the operating mode of the Quadra CFA device is determined to be the operating mode used when processing the previous frame processing request. If the preset operating mode indicator is different from the operating mode indicator used when processing the previous frame processing request, the operating mode of the Quadra CFA device is determined to be the preset operating mode.

[0110] It is understandable that after displaying a preview image corresponding to a second frame processing request on the preview interface, the camera application sends the next second frame processing request to the camera driver via the Camera API. The camera driver processes the first frame processing request and at least one second frame processing request in sequence, and then the camera application displays the preview image corresponding to the first frame processing request and at least one preview image corresponding to at least one second frame processing request in sequence on the preview interface.

[0111] Figure 5 This is a timing diagram of an image processing method for a camera application provided in one embodiment of this application.

[0112] Please refer to them together. Figure 4 and Figure 5 The image processing methods for camera applications include the following steps:

[0113] S501, camera application launched.

[0114] S502, the camera application sends a resolution request to the Camera Service module.

[0115] In S503, the Camera Service module, based on the resolution request, obtains the initial resolution reported by the camera driver and the customized resolution from the resolution configuration file by calling the resolution customization interface, and generates a resolution list.

[0116] The resolution list includes at least one initial resolution and at least one custom resolution. The initial resolution is less than or equal to the packing resolution, and the custom resolution is greater than the packing resolution.

[0117] Taking a 9MP Quadra CFA device as an example, assume the packing resolution is 2M. Due to limitations of the chip platform, the initial resolution reported by the camera driver is 1M and 2M. When the Camera Service module receives the initial resolution reported by the camera driver, it parses the customized resolutions from the resolution configuration file as 3M, 5M, 7M, and 9M. Therefore, the Camera Service module generates a resolution list based on the initial resolution and the customized resolutions, resulting in a resolution list of {1M, 2M, 3M, 5M, 7M, 9M}.

[0118] S504, the Camera Service module sends a list of resolutions to the camera application.

[0119] The S505 camera application determines the target resolution based on a resolution list.

[0120] Continuing with the example of a 9MP Quadra CFA device, assume the resolution list is {1M, 2M, 3M, 5M, 7M, 9M}. When the output image size of the camera application is 9M, the target resolution determined by the camera application based on the resolution list is 9M. When the output image size of the camera application is 6M, the target resolution determined by the camera application based on the resolution list is 5M.

[0121] S506, the camera application sends a streaming request to the Camera Service module.

[0122] The distribution request includes the target resolution.

[0123] S507, the Camera Service module sends a streaming request to the camera driver.

[0124] The S508 camera driver configures the preview stream and creates channels based on the distribution request, and calls the FindestSensorMode function to determine the initial operating mode of the Quadra CFA device.

[0125] S509, the camera driver sets the initial operating mode of the Quadra CFA device based on the initial operating mode indicator returned by the FindestSensorMode function.

[0126] S510, the camera driver sends the distribution result to the Camera Service module.

[0127] The distribution result includes whether the distribution was successful or failed.

[0128] S511, the Camera Service module sends the distribution results to the camera application.

[0129] S512, when the distribution result is successful, the camera application sends the first frame processing request to the Camera Service module.

[0130] The first frame processing request (first frame processing request) includes the target resolution and the original image, with the resolution of the original image being the target resolution.

[0131] S513, the Camera Service module sends the first frame processing request to the camera driver.

[0132] S514, the camera driver calls the OnExecuteProcessRequest function based on the first frame processing request to convert the raw image in the first frame processing request into a preview image.

[0133] S515, the camera driver sends preview images to the Camera Service module.

[0134] S516, the Camera Service module sends preview images to the camera application.

[0135] S517: The camera app displays the preview image corresponding to the first frame processing request on the camera app's preview interface.

[0136] S518, the camera application sends a second frame processing request to the Camera Service module.

[0137] The second frame processing request (not the first frame processing request) includes the target resolution and the original image, with the resolution of the original image being the target resolution.

[0138] S519, the Camera Service module sends a second frame processing request to the camera driver.

[0139] In the S520, the camera driver calls the OnExecuteProcessRequest function based on the second frame processing request to convert the original image in the second frame processing request into a preview image.

[0140] S521, the camera driver sends a preview image to the Camera Service module.

[0141] S522, the Camera Service module sends preview images to the camera application.

[0142] S523, the camera app displays the preview image corresponding to the second frame processing request on the camera app's preview interface.

[0143] In step S508 above, please refer to Figure 6The FindestSensorMode function is called to determine the initial operating mode of the Quadra CFA device, including the following sub-steps:

[0144] S601, query camera application custom fields.

[0145] S602, based on the camera application's custom field indicating that the camera application supports mode selection, obtains the target resolution from the distribution request and obtains the current ISO value by driving the camera's sensor.

[0146] S603 determines the initial working mode as Fullsize mode based on the fact that the target resolution is greater than the packing resolution and the current sensitivity value is less than the preset sensitivity threshold.

[0147] S604 determines the initial working mode as Binning mode based on the target resolution being less than or equal to the packing resolution, or the current sensitivity value being greater than or equal to the preset sensitivity threshold.

[0148] S605, returns to the initial operating mode indicator.

[0149] It is understandable that camera application customization fields are typically set to indicate whether the camera application supports mode selection. In other words, in some cases, step S601 above may not be necessary.

[0150] In step S514 above, please refer to Figure 7 The OnExecuteProcessRequest function is called to convert the original image in the first frame processing request into a preview image, including the following sub-steps:

[0151] S701 calls the FindestSensorMode function to determine the preset operating mode of the Quadra CFA device.

[0152] It is understandable that the specific implementation method of step S701 is the same as... Figure 6 The process shown is roughly the same, so it will not be repeated here.

[0153] S702 calls the OnselectSensorMode function to determine the operating mode of the Quadra CFA device.

[0154] S703 sets the operating mode of the Quadra CFA device based on the operating mode indicator returned by the OnselectSensorMode function.

[0155] The S704, based on the operating mode of the Quadra CFA device, drives the Quadra CFA device to convert the original image in the first frame processing request into a preview image.

[0156] In step S702 above, please refer to Figure 8 The OnselectSensorMode function is called to determine the operating mode of the Quadra CFA device, including the following sub-steps:

[0157] S801, obtain the preset operating mode indicator returned by the FindestSensorMode function and the initial operating mode indicator from the sensor driver file.

[0158] S802, determine whether the preset working mode indicator is the same as the initial working mode indicator.

[0159] S803, based on the fact that the preset operating mode indicator is the same as the initial operating mode indicator, determines that the operating mode of the Quadra CFA device is the initial operating mode.

[0160] S804 determines the operating mode of the Quadra CFA device to be the preset operating mode based on the difference between the preset operating mode indicator and the initial operating mode indicator.

[0161] S805, Return to operating mode indicator.

[0162] The specific implementation method of step S520 above is as follows Figure 7 The processes shown are largely the same; the difference lies in the specific implementation of step S702 (e.g., ...). Figure 8 In the process shown, the "initial working mode indicator" is replaced with the "working mode indicator when processing the previous frame processing request".

[0163] In the above embodiments, when the resolution reported by the camera driver is less than or equal to the binning resolution due to limitations of the chip platform, the Camera Service module obtains a customized resolution greater than the binning resolution by calling the resolution customization interface. This allows the camera application to determine the target resolution based on the initial resolution and the customized resolution. Therefore, the camera driver can set the operating mode of the Quadra CFA device based on the target resolution and the ISO value. For example, when the target resolution is greater than the binning resolution and the ambient light is normal or strong, the Quadra CFA device's operating mode is set to Fullsize mode to meet the user's image resolution requirements. When the target resolution is less than or equal to the binning resolution, or the ambient light is weak, the Quadra CFA device's operating mode is set to Binning mode to meet the user's image brightness requirements.

[0164] It is understood that for problems such as stretched preview images output by camera applications, or preview images not being displayed due to the failure to select a target resolution that matches the output image size of the camera application, the image processing method for camera applications provided in this application embodiment can solve the above problems and enable the images provided by the camera application to meet the user's needs.

[0165] Furthermore, the above embodiments only take the preview scene of the camera application as an example, but the image processing method of the camera application is not limited to the preview scene. The image processing method of the camera application is also applicable in the shooting scene or video recording scene.

[0166] The image processing methods for camera applications provided in the embodiments of this application will be described below in conjunction with different scenarios.

[0167] Scenario 1: The ambient light is normal or strong, and the target resolution determined by the camera application is greater than the packing resolution.

[0168] The image processing method for the camera application includes: launching a first camera application and displaying a first interface; displaying a first preview image during a first time period; the first preview image being displayed on the first interface; during the first time period, the ambient light brightness being a first brightness; the target resolution determined by the first camera application being a first resolution; and the resolution of the first preview image being a second resolution, which is equal to the first resolution.

[0169] In the first scenario, "first brightness" indicates normal or strong ambient light, and "first resolution" is greater than the package resolution. Based on the ambient light brightness being the first brightness and the target resolution being the first resolution, even if the chip platform allows the camera driver to report a resolution less than or equal to the package resolution, the camera driver can still set the CFA device's operating mode to Fullsize mode. Therefore, the first camera application outputs a Fullsize image. The first preview image is a Fullsize image.

[0170] Scenario 2: The ambient light is weak, and the target resolution determined by the camera application is greater than the packing resolution.

[0171] The image processing method for the camera application includes: launching a first camera application and displaying a first interface; displaying a second preview image during a second time period, wherein the second preview image is displayed on the first interface; during the second time period, the ambient light brightness is a second brightness, which is less than the first brightness; the target resolution determined by the first camera application is the first resolution; and the resolution of the second preview image is a third resolution, which is less than the first resolution.

[0172] In the second scenario, the first brightness indicates normal or strong ambient light, while the second brightness indicates weak ambient light. The first resolution is greater than the packing resolution. Based on the second brightness level for ambient light and the first resolution for the target, the camera driver sets the CFA device's operating mode to Binning mode, thus the first camera application outputs a Binningsize image. The second preview image is the Binningsize image.

[0173] Scenario 3: Ambient light is normal or strong, and the target resolution determined by the camera application is less than or equal to the packing resolution.

[0174] The image processing method of the camera application includes: launching a second camera application and displaying a second interface; displaying a third preview image during a third time period; the third preview image is displayed on the second interface; during the third time period, the ambient light brightness is a first brightness; the target resolution determined by the second camera application is a fourth resolution, which is less than the first resolution; and the resolution of the third preview image is the third resolution.

[0175] In the third scenario, the first brightness indicates normal or strong ambient light, and the fourth resolution is less than or equal to the packing resolution. Based on the first brightness (ambient light) and the fourth resolution (target resolution), the camera driver sets the CFA device's operating mode to Binning mode, thus the second camera application outputs a Binning size image. The third preview image is the Binning size image.

[0176] Fourth scenario: The ambient light is weak, and the target resolution determined by the camera application is less than or equal to the packing resolution.

[0177] The image processing method for the camera application includes: launching a second camera application and displaying a second interface; displaying a third preview image during a third time period; the third preview image being displayed on the second interface; during the third time period, the ambient light brightness is a second brightness; the target resolution determined by the second camera application is a fourth resolution, which is less than the first resolution; and the resolution of the third preview image is the third resolution.

[0178] In the fourth scene, the first brightness indicates normal or strong ambient light, the second brightness indicates weak ambient light, and the fourth resolution is less than or equal to the binning resolution. Based on the second brightness level for ambient light and the fourth resolution for the target, the camera driver sets the CFA device's operating mode to Binning mode, thus the second camera application outputs a Binning size image. The third preview image is also a Binning size image.

[0179] The electronic device provided in the embodiments of this application will be described below.

[0180] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application.

[0181] See also Figure 9 The electronic device 100 includes a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a CFA device 180M, etc.

[0182] The processor 110 can execute instructions stored in the internal memory 121, causing the electronic device 100 to perform the image processing method for the camera application provided in this application embodiment.

[0183] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0184] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0185] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0186] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0187] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0188] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0189] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0190] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0191] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0192] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0193] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0194] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0195] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0196] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0197] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0198] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0199] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0200] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0201] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including Wireless Local Area Network (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0202] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. Wireless communication technology may include Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. GNSS can include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the Beidou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or the Satellite Based Augmentation System (SBAS).

[0203] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0204] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light-Emitting Diode (AMOLED), a Flexible Light-Emitting Diode (FLED), a Miniature LED, a MicroLED, a Micro-OLED, a Quantum Dot Light-Emitting Diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0205] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0206] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0207] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0208] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0209] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0210] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0211] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0212] Internal memory 121 can be used to store executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phone book, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0213] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0214] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0215] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0216] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0217] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0218] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0219] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0220] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0221] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0222] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0223] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and is applicable to screen orientation switching, pedometers, and other applications.

[0224] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0225] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0226] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0227] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0228] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0229] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0230] For details on the working principle of the 180M CFA device, please refer to the relevant description of the Quadra CFA device; it will not be repeated here.

[0231] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0232] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0233] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0234] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0235] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0236] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the image processing method of the camera application according to this application.

[0237] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.

[0238] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. An image processing method for camera applications, applied to electronic devices, characterized in that, The method includes: The first camera application is launched, and the resolution sent by the first camera application to the camera of the electronic device is the first resolution. The first resolution is determined based on the initial resolution reported by the camera driver and the customized resolution from the resolution configuration file. The initial resolution is less than or equal to the boxed resolution, and the customized resolution is greater than the boxed resolution. A first preview image is acquired in a first time period. During the first time period, the ISO value sensed by the camera is a first ISO value, which is less than a preset sensitivity threshold. The resolution of the first preview image is a second resolution, which is equal to the first resolution. A second preview image is acquired during a second time period. During this second time period, the ISO value sensed by the camera is a second ISO value, which is greater than or equal to the preset sensitivity threshold. The resolution of the second preview image is a third resolution, which is less than the first resolution.

2. The image processing method for a camera application as described in claim 1, characterized in that, The method further includes: The first camera application is closed, and the second camera application is launched. The resolution sent by the second camera application to the camera is a fourth resolution, which is determined based on the initial resolution and the customized resolution; the fourth resolution is less than or equal to the third resolution. A third preview image is acquired during a third time period, and the resolution of the third preview image is the third resolution during the third time period.

3. The image processing method for camera applications as described in claim 2, characterized in that, In the third time period, the resolution of the third preview image is the third resolution, including: In the third time period, the ISO value sensed by the camera is a third ISO value, which is less than the preset sensitivity threshold, and the resolution of the third preview image is the third resolution; or, During the third time period, the ISO value sensed by the camera is the fourth ISO value, which is greater than or equal to the preset sensitivity threshold, and the resolution of the third preview image is the third resolution.

4. The image processing method for a camera application as described in any one of claims 1 to 3, characterized in that, The first resolution is greater than the packing resolution, and the third resolution is equal to the packing resolution.

5. The image processing method for a camera application as described in any one of claims 1 to 3, characterized in that, The resolution sent by the first camera application to the camera of the electronic device is a first resolution, including: The first camera application sends a streaming request to the camera driver via the camera API of the electronic device, the streaming request including the first resolution.

6. The image processing method for a camera application as described in claim 5, characterized in that, The method further includes: The camera driver configures the preview stream and creates channels based on the streaming request, and determines the initial operating mode of the color filter array (CFA) device; the initial operating mode includes a boxed mode or an unboxed mode, and the CFA device is used to convert the raw image captured by the camera into a preview image.

7. The image processing method for a camera application as described in claim 6, characterized in that, The camera driver determines the initial operating mode of the CFA device, including: The camera driver queries custom fields for the camera application. When the camera application customization field indicates that the first camera application supports mode selection, the camera driver determines the initial operating mode of the CFA device based on the first resolution and the ISO value sensed by the camera.

8. The image processing method for a camera application as described in claim 7, characterized in that, The camera driver determines the initial operating mode of the CFA device based on the first resolution and the ISO value sensed by the camera, including: The camera driver determines the initial operating mode of the CFA device to be the non-packing mode based on the fact that the first resolution is greater than the packing resolution and the ISO value sensed by the camera is less than the preset sensitivity threshold. The camera driver determines the initial operating mode of the CFA device to be the packing mode based on the first resolution being less than or equal to the packing resolution, or the ISO value sensed by the camera being greater than or equal to the preset sensitivity threshold.

9. The image processing method for a camera application as described in any one of claims 6 to 8, characterized in that, The method further includes: Based on the successful allocation of the stream, the first camera application sends a frame processing request to the camera driver through the camera API. The frame processing request includes the first resolution and the original image captured by the camera. Based on the frame processing request, the camera driver controls the CFA device to convert the original image into a preview image.

10. The image processing method for a camera application as described in claim 9, characterized in that, The camera driver controls the CFA device to convert the original image into a preview image, including: The camera driver determines a preset operating mode for the CFA device, wherein the preset operating mode is either the boxed mode or the unboxed mode. The camera driver determines the operating mode of the CFA device based on the preset operating mode and the reference operating mode. The reference operating mode is the operating mode of the CFA device when processing the previous frame processing request, or the initial operating mode. The camera driver sets the operating mode of the CFA device; The camera driver controls the CFA device to convert the original image into a preview image.

11. The image processing method for a camera application as described in claim 10, characterized in that, The camera driver determines the operating mode of the CFA device based on the preset operating mode and the reference operating mode, including: The camera driver determines the operating mode of the CFA device to be the reference operating mode based on the fact that the preset operating mode and the reference operating mode are the same. The camera driver determines the operating mode of the CFA device to be the preset operating mode based on the difference between the preset operating mode and the reference operating mode.

12. The image processing method for a camera application as described in claim 10 or 11, characterized in that, The camera driver sets the operating mode of the CFA device, including: The camera driver, based on the operating mode of the CFA device, is the preset operating mode, and performs seamless switching through a pre-configured register or register group to switch the reference operating mode to the preset operating mode.

13. An electronic device, characterized in that, The electronic device includes a memory, a processor, a camera, a sensor module, and a display screen. The camera is used to capture still images or videos. The sensor module includes a color filter array (CFA) device and an ambient light sensor. The CFA device operates in either a boxed mode or an unboxed mode. The ambient light sensor is used to sense ambient light brightness. The display screen is used to display an interface. The memory is used to store instructions. The processor is used to execute the instructions stored in the memory, causing the electronic device to perform the image processing method for a camera application as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform an image processing method for a camera application as described in any one of claims 1 to 12.

Images