Image processing method, shooting method, electronic equipment and readable storage medium

Abstract

The invention discloses an image processing method, a photographing method, electronic equipment and a readable storage medium. The image processing method is used for processing at least one image frame obtained by an image sensor in the image shooting process of the electronic equipment so as to preview and display the processed at least one image frame, and then a user can shoot according to the previewed and displayed at least one image frame. The method comprises the following steps: obtaining a preview data stream according to an image signal processing parameter; the preview data stream comprises a plurality of first image frames; the preview data stream is used for previewing an image shooting picture; for at least one first image frame in the plurality of first image frames, determining a region of interest in each first image frame; obtaining statistical data of the region of interest and statistical data of the region of the first image frame; and updating image signal processing parameters according to the statistical data of the region of interest and the statistical data of the region of the first image frame. According to the embodiment of the invention, the image shooting effect can be optimized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, a shooting method, an electronic device, and a readable storage medium. Background Technology

[0002] With the development of computer technology, various types of electronic devices have universally acquired camera functions, and as camera pixel counts increase, photo quality and related technologies continue to improve. When taking photos with electronic devices, the raw image signal obtained by the image sensor needs to be processed to generate an image that conforms to human visual habits. As camera hardware technology continues to improve, image processing technology also needs to be continuously improved to ensure the best possible image presentation. Summary of the Invention

[0003] This application provides an image processing method, a shooting method, an electronic device, and a readable storage medium, which can avoid improving the visual harmony of the interface while providing diverse display methods. The technical solution is as follows:

[0004] In a first aspect, an image processing method is provided for processing at least one image frame obtained by an image sensor during the process of an electronic device capturing an image, comprising: obtaining a preview data stream according to image signal processing parameters; the preview data stream including a plurality of first image frames; the preview data stream being used to preview the captured image; determining a region of interest in each of the plurality of first image frames for at least one of the first image frames; obtaining statistical data of the region of interest and statistical data of the regions of the first image frames; updating the image signal processing parameters according to the statistical data of the region of interest and the statistical data of the regions of the first image frames, wherein the updated image signal processing parameters are used to update the preview data stream or generate the captured image.

[0005] In this embodiment, the image sensor generates raw image data based on the received analog signal and inputs it into the processing algorithm flow module. The processing algorithm flow module determines image signal processing parameters based on the raw image data or the ambient light detected by the sensor. The image signal processing parameters can be determined by an automatic adjustment algorithm (such as the 3A algorithm) used by the processing algorithm flow module. The automatic adjustment algorithm can include at least one of an autofocus algorithm, an automatic exposure algorithm, and an automatic white balance algorithm. When obtaining the initial preview data stream based on the image signal processing parameters, the parameters can be determined based on factors such as ambient light detected by the electronic device's sensor. The processing algorithm flow module processes the raw image data using the image signal processing parameters to generate the preview data stream.

[0006] In this embodiment, the field of view of the image sensor is greater than or equal to the field of view of the original image data, the field of view of the original image data is greater than or equal to the field of view of the first image frame, and the field of view of the first image frame is greater than or equal to the field of view of the region of interest.

[0007] When the entire region of the first image frame is the region of interest, the field of view of the first image frame is equal to the field of view of the region of interest. When a portion of the first image frame is the region of interest, the field of view of the first image frame is greater than the field of view of the region of interest.

[0008] Because the field of view of the analog signal that the image sensor can detect may be significantly larger than the range that the user wants to capture, the user may zoom in on the captured image when taking photos or videos. As a result, the field of view in the preview data stream may be significantly smaller than the field of view of the original image data from the image sensor. In this case, if the image signal processing parameters are determined using the original image data or the analog signal detected by the image sensor, the image signal processing parameters in the large field of view may differ significantly from the optimal image signal processing parameters in the small field of view, leading to poor preview and poor image capture results.

[0009] The method provided in this application provides that statistical data is obtained from image frames in the preview data stream, image signal processing parameters are adjusted (or updated) based on the statistical data, and the preview data stream is obtained based on the updated image signal processing parameters. This avoids situations where the automatic adjustment algorithm is inaccurate due to a large difference between the field of view of the image detected by the image sensor and the field of view of the preview data stream, thus affecting the image effect.

[0010] In one implementation, the preview data stream is obtained based on raw image data input from an image sensor, wherein the field of view of the raw image data is greater than the field of view of the preview data stream.

[0011] Because the field of view of the original image data is larger than that of the preview data stream, the image signal processing parameters calculated by the 3A algorithm may not be compatible with the image under the field of view of the preview data stream. The method described above can correct the image signal processing parameters and improve the image processing effect before shooting, even when the field of view of the preview data stream is significantly smaller than that of the original image data.

[0012] It should be noted that when the field of view of the preview data stream is roughly the same as that of the original image data, statistical data can be determined based on the region of interest in the preview data stream, and then the image signal processing parameters can be adjusted (or updated) based on the statistical data of the region of interest. Therefore, after a user captures an image or video, if they zoom in on the image frames to view the region of interest, the zoomed-in image will also present a good visual effect.

[0013] In one implementation, the preview data stream is output from the post-processing end of the processing algorithm module of the image signal processor. Alternatively, the field of view of the first image frame in the preview data stream is consistent with the field of view of the captured image or video.

[0014] In another possible implementation, the processing algorithm flow module can be a module in the image signal processor used to execute the processing algorithm flow of the image signal.

[0015] In different implementations, the processing algorithm flow module may have different structures. For example, the processing algorithm flow module may include a front-end and a back-end, where the raw image data is processed by the front-end and back-end respectively and converted into a preview data stream. Alternatively, the processing algorithm flow module may include a front-end, a back-end, and a post-processing end.

[0016] In one embodiment, obtaining the preview data stream based on image signal processing parameters includes: obtaining raw image data from an image sensor based on image signal processing parameters; the raw image data includes multiple raw image frames; the first image frame corresponds to the raw image frame; and processing the raw image data sequentially through the front end of the processing algorithm flow module, the back end of the image signal processing algorithm flow module, and the post-processing end of the image signal processing algorithm flow module to obtain the preview data stream.

[0017] The first image frame mentioned above corresponds to the original image frame. That is to say, in the preview data stream, the sequence of the first image frame is obtained by processing the sequence of the original image frames in the original image data through the processing algorithm flow module.

[0018] In the above method, the first image frame can be obtained by locally magnifying the corresponding original image frame; that is, the field of view of the first image frame is smaller than that of the original image frame. After processing by the front-end, back-end, and post-processing end of the processing algorithm module, the field of view of the first image frame changes compared to the field of view of the original image frame, and the field of view of the first image frame becomes the field of view that the user wants to capture. Since the field of view of the first image frame in the preview data stream is the field of view that the user wants to capture, the updated image processing parameters are adapted to the field of view that the user wants to capture. Statistical data is determined based on the first image frame, the image processing parameters are updated based on the statistical data, and then the updated image processing parameters are used to update the preview data stream, so that the preview data stream can present the best shooting effect under the field of view that the user wants to capture.

[0019] In one embodiment, the image processing method further includes: obtaining a front-end data stream output by the front-end of the processing algorithm flow module; the front-end data stream includes a plurality of second image frames; obtaining statistical data of all regions of the second image frames, wherein the statistical data of all regions of the second image frames is used to update the image signal processing parameters.

[0020] Compared to the preview data stream, the field of view of the front-end data stream is closer to that of the original image data generated by the image sensor. By using the above method, the image signal processing parameters are updated based on the front-end data stream, so that the image signal processing parameters are adapted to the field of view that the image sensor can capture.

[0021] In one implementation, the region of interest is the region where the target object is located.

[0022] Using the above method, electronic devices can determine image signal processing parameters for the specific target object being photographed when capturing images, thereby improving the image processing effect of the specific target object.

[0023] In one implementation, the target object is a face.

[0024] In this embodiment, the face can be further defined as a human face. A human face can be detected using a face detection algorithm. Since there is various prior information about the face, such as prior white point values, the white point can be estimated based on this prior information, improving the accuracy of AWB calculation.

[0025] In one embodiment, the image processing method further includes: converting the first image frame to a linear domain to obtain a linear domain-mapped image; the linear domain-mapped image is used to obtain statistical data of the region of interest and statistical data of the region of the first image frame.

[0026] During the processing of raw image data in the algorithm flow module, various non-linear processing operations may be used on the raw image frames, causing the statistical data of the first image frame in the preview data stream to change compared to the captured environment. If the statistical data is directly determined using the first image frame and then the image signal processing parameters are adjusted based on the statistical data, the preview data stream may not present a satisfactory visual effect. The method described above transforms the first image frame into the linear domain, obtaining statistical data in the linear domain. This eliminates the factors causing changes in statistical data due to non-linear processing of the first image frame in the preceding processing stages, thus improving the accuracy of the statistical data.

[0027] In one implementation, the statistical data includes grayscale data and white balance data.

[0028] The white balance data mentioned above are statistical data used to determine white balance parameters, such as the ratio of the number of red pixels to the number of green pixels, and / or the ratio of the number of blue or green pixels to the number of green pixels.

[0029] In one implementation, the statistical data includes a grayscale histogram, and the image signal processing parameters include exposure parameters. Updating the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame includes: calculating the grayscale mean based on the grayscale histogram of the region of interest; determining the exposure amount based on the grayscale mean; and adjusting the exposure amount based on the grayscale histogram of the first image frame to obtain the exposure parameters.

[0030] The above method allows for updating exposure parameters using the grayscale histogram of the preview data stream, thus optimizing the exposure effect. Furthermore, by calculating statistical data and updating exposure parameters separately for regions of interest, even better exposure results can be achieved for these regions within the preview data stream. For example, if the preview data stream includes a human body and the region of interest is the area containing the face, then the method provided in this application embodiment can present better human body and face shooting effects in the preview data stream.

[0031] In one implementation, the statistical data includes white balance statistical data, the image signal processing parameters include white balance parameters, and updating the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame includes: estimating the white point of the region of interest or the white point of the first image frame based on the white balance statistical data of the region of interest and the white balance statistical data of the first image frame; correcting the estimated white point based on the white point of the region of interest or the white point of the first image frame to obtain the white balance parameters.

[0032] In this embodiment, the processing algorithm module can determine the pixel corresponding to the estimated white point in the first image frame, and then estimate the current color temperature based on the color information of the white point's pixel. Since the color of a white object varies under different color temperatures (e.g., it appears bluish at high color temperatures and reddish at low color temperatures), the current color temperature value can be inferred by analyzing the color of the white point. After estimating the color temperature, the processing algorithm module determines the white balance parameters based on this color temperature value, and adjusts the color balance in the new preview data stream according to the white balance parameters, ensuring that the white object still appears white in the image. When adjusting the color balance parameters, the processing algorithm module can adjust the gain values ​​of the red (R), green (G), and blue (B) channels to achieve color adjustment.

[0033] By using the above method, the white balance parameters can be corrected (or updated or adjusted) by utilizing the estimated white point information of the preview data stream. The colors presented in the preview data stream can be closer to the colors in the actual environment, thus obtaining a better white balance effect.

[0034] Secondly, embodiments of this application also provide a method for taking a picture, including: obtaining a preview data stream based on updated image processing parameters, wherein the updated image processing parameters are the updated image processing parameters provided by any embodiment of this application; the preview data stream includes a plurality of first image frames; and in response to a shooting command, obtaining a captured image and / or video based on the first image frames.

[0035] In this embodiment, the captured image can be a single image, an image frame in an animation, or an image frame in a short video. After the electronic device updates the preview data stream according to the updated image processing parameters, the first image frame in the preview data stream is displayed on the preview interface. The user can operate on the preview interface and issue a shooting command. In response to the shooting command, the electronic device can obtain at least one first image frame in the preview data stream and generate the captured image and / or the captured video based on at least one first image frame.

[0036] Through the above methods, the embodiments of this application can achieve better shooting results.

[0037] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory;

[0038] The memory is used to store a program for an electronic device to perform the method provided in any embodiment of the present application, and to store data related to implementing the method provided in any embodiment of the present application;

[0039] The processor is configured to execute programs stored in memory.

[0040] Optionally, there may be one or more processors and one or more memories.

[0041] Alternatively, the memory can be integrated with the processor, or the memory can be set up separately from the processor.

[0042] The processing device in the third aspect above can be a chip. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. The memory can be integrated into the processor or located outside the processor and exist independently.

[0043] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0044] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to perform the method described in the first aspect.

[0045] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform any of the possible implementations of the first aspect.

[0046] Sixthly, embodiments of this application also provide a processor, including: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive signals through the input circuit and transmit signals through the output circuit, causing the processor to execute the method in any of the embodiments of the first aspect described above.

[0047] In specific implementation, the processor can be a chip, the input circuit can be input pins, the output circuit can be output pins, and the processing circuit can be transistors, gate circuits, flip-flops, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to and transmitted by a transmitter. Furthermore, the input circuit and the output circuit can be the same circuit, which is used as both the input circuit and the output circuit at different times. This application does not limit the specific implementation of the processor and various circuits.

[0048] The technical effects achieved by the second, third, fourth, fifth, and sixth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of a software architecture according to an embodiment of this application;

[0051] Figure 3 This is a schematic diagram illustrating an application scenario according to an embodiment of this application;

[0052] Figure 4 yes Figure 3 The corresponding interface diagram;

[0053] Figure 5 This is a schematic diagram of an ISP according to an embodiment of this application;

[0054] Figure 6 This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0055] Figure 7 This is a schematic diagram illustrating an image capture effect according to an embodiment of this application;

[0056] Figure 8 This is a schematic diagram illustrating another image capture effect according to an embodiment of this application;

[0057] Figure 9 This is a schematic diagram of an ISP structure in one example of this application;

[0058] Figure 10 This is a schematic diagram of a grayscale histogram of an example of this application;

[0059] Figure 11 This is a schematic diagram of an image processing apparatus according to an embodiment of this application;

[0060] Figure 12 This is a schematic diagram of the structure of a shooting device according to an embodiment of this application. Detailed Implementation

[0061] In the following description, specific details such as particular system architectures and technologies are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0062] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0063] It should be understood that "one or more" as mentioned in this application refers to one, two, or more, and "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0064] Furthermore, to facilitate a clear description of the technical solution of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0065] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0066] The display method provided in this application can be applied to electronic devices. These electronic devices can be mobile phones, tablets, wearable devices, digital cameras, in-vehicle devices, augmented reality (AR) devices, virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), laptops, etc., and this application does not limit the specific application to these devices.

[0067] First, the possible structures of the electronic devices in the embodiments of this application will be introduced.

[0068] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0069] See Figure 1 The electronic device 1000 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0070] It should be noted that, Figure 10 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may include more than Figure 10 The components shown may include more or fewer components, or the electronic device 1000 may include... Figure 10 The components shown may be a combination of certain components, or the electronic device 1000 may include... Figure 10 The components shown are sub-components of certain components. For example, Figure 10 The proximity sensor 180G shown is optional. Figure 10 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0071] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0072] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0073] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0074] Figure 1 The connection relationships between the modules shown are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of various connection methods described in the above embodiments.

[0075] Electronic device 1000 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] The display screen 194 can be used to display images or videos. In some embodiments, the electronic device 1000 may include one or N display screens 194, where N is a positive integer greater than 1.

[0077] Electronic device 1000 can achieve shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0078] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0079] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard red-green-blue (RGB) formats, luminance, and chrominance (YUV). In some embodiments, the electronic device 1000 may include one or N cameras 193, where N is a positive integer greater than 1.

[0080] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 1000 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0081] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. Thus, electronic device 1000 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0082] An NPU (Neural Processing Unit) is a processor that borrows from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, to rapidly process input information and continuously learn. NPUs can enable intelligent cognitive functions in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0083] Electronic device 1000 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0084] The distance sensor 180F is used to measure distance. The electronic device 1000 can measure distance via infrared or laser. In some embodiments, such as in a shooting scenario, the electronic device 1000 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0085] Button 190 includes a power button and volume buttons. Motor 191 can generate vibration when the electronic device receives information. Motor 191 can be used to display notification information.

[0086] exist Figure 1 Based on the electronic device shown, the user can control the camera 193 to capture images or videos via button 190 or display screen 194.

[0087] Figure 1 The electronic device shown can be configured with a certain software architecture. Figure 2 This is a schematic diagram of an architecture (including a software system and some hardware) applied in an embodiment of this application. Figure 2 As shown, the architecture of an electronic device is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the driver layer, and the hardware layer.

[0088] like Figure 2 As shown, the application layer includes the camera and gallery. This is understandable. Figure 2 The examples shown are only a portion of the applications; in fact, the application layer can include other applications as well, and this application does not limit this. For example, the application layer may also include applications such as messaging, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and payment program.

[0089] like Figure 2 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer includes multiple camera devices. The camera algorithm library includes post-processing algorithm modules and decision-making modules.

[0090] It should be understood that the decision-making module can also be placed in other layers. As one possible implementation, the decision-making module can be placed in the application layer or the application framework layer.

[0091] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. For example... Figure 2 As shown, the driver layer includes camera device drivers, digital signal processor drivers, and graphics processor drivers, etc.

[0092] The hardware layer includes sensors, an image signal processor, a digital signal processor, and a graphics processor. The sensors include multiple sensors, a time-of-flight (TOF) camera, and a multispectral sensor. The image signal processor includes a front-end, a back-end, and a post-processing unit. The steps performed by the front-end, back-end, and post-processing unit can be implemented through software units or modules within the image signal processor.

[0093] For example, a user can tap the camera application to launch the camera and take a picture. When the user taps the camera to take a picture, the shooting command is sent to the camera hardware abstraction layer (HAL) through the camera access interface. The HAL responds to the shooting command by calling the camera device driver and the camera algorithm library. The decision module in the camera algorithm library determines the shooting mode (e.g., mode 1, mode 2, or mode 3) based on the zoom level and ambient light, and sends the configured parameters (including sensor output method, parameter configurations for each ISP module, and parameter configurations for the post-processing algorithm module) to the HAL. The HAL then sends the parameters configured by the decision module to the camera device driver. The camera device driver sends the configuration parameters sent by the HAL to the hardware layer; for example, it sends the sensor output method to the sensor and the parameter configurations for each ISP module to the image signal processor. The sensor outputs the image based on the sensor output method. The image signal processor performs corresponding processing based on the parameter configurations of each ISP module. The camera algorithm library is also used to send digital signals to the digital signal processor driver in the driver layer, so that the digital signal processor driver can call the digital signal processor in the hardware layer for digital signal processing. The digital signal processor (DSP) can return the processed digital signals to the camera algorithm library via its driver. The camera algorithm library also sends digital signals to the image signal processor (Image Signal Processor) driver in the driver layer, enabling the Image Signal Processor driver to invoke the graphics processor (GPU) in the hardware layer for digital signal processing. The GPU can then return the processed image data to the camera algorithm library via its driver.

[0094] Additionally, the image output from the image signal processor can be sent to the camera device driver. The camera device driver can then send the image output from the image signal processor to the camera hardware abstraction layer. The camera hardware abstraction layer can then send the image to the post-processing algorithm module for further processing, or it can send the image to the camera access interface. The camera access interface can then send the image returned by the camera hardware abstraction layer to the camera.

[0095] The image processing method provided in this application can be applied to scenarios where electronic devices capture images or videos. The following provides examples of scenarios where this application embodiment can be applied.

[0096] Figure 3 This is a schematic diagram of a possible scenario involved in an embodiment of this application. When the electronic device is a mobile phone, the user may take a picture to obtain an image. The field of view (FOV), also referred to as the viewing angle in this embodiment, is an important parameter of a mobile phone lens, referring to the range of vision that the lens can cover or capture. FOV describes the maximum range that the lens can capture, that is, the area included in the visible image of the subject formed on the focal plane through the lens. During the image capture process, zooming may be necessary to change the field of view or range of the image. Zooming refers to changing the viewing angle and range of the captured image by adjusting the focal length of the lens. The shorter the focal length, the larger the field of view, and the wider the range of the captured image. Figure 3 As shown in (a), with a larger field of view, the captured image includes more content. The longer the focal length, the smaller the angle of view, and the narrower the area of ​​the captured image, but distant objects can be magnified more clearly. Figure 3 As shown in (b), when the field of view is small, there is less content in the captured image, and the content in the image is magnified.

[0097] In most cases, electronic devices are equipped with optical fixed-focus lenses. An optical fixed-focus lens, or simply a fixed-focus lens, refers to a lens with a fixed focal length. Because an optical fixed-focus lens can only operate at a fixed focal length, its angle of view is also fixed; the distance at which the angle of view can be changed by adjusting the lens itself. For example, with a 50mm optical fixed-focus lens, the angle of view remains constant regardless of when or how it is used. Therefore, in electronic devices using optical fixed-focus lenses, zooming is generally achieved through digital zoom.

[0098] Digital zoom refers to the process of enlarging the area of ​​each pixel in an image using a processor within an electronic device. The method is similar to using image processing software to enlarge the area of ​​an image, but the image processing software runs internally within the electronic device. Specifically, digital zoom uses software to analyze the colors surrounding existing pixels in the image and, relying on algorithms in the image processing software, inserts new pixels based on these colors. It enlarges a portion of the pixels originally captured by the image sensor using interpolation, thus magnifying the image to the full size of the image. In reality, digital zoom does not change the focal length of the electronic device's optical fixed-focus lens.

[0099] In the shooting of electronic devices, such as Figure 3 When viewing the image shown in (a), the preview interface effect can be referenced for example. Figure 4 As shown in (a) in the figure, in Figure 4In the preview interface shown in (a), the magnification information is displayed as 1X. In this embodiment, the magnification can be used to represent magnification factor; for example, 1X represents magnification by one time, and 3X represents magnification by three times. Users can zoom in by performing a two-finger zoom or a tap operation. In response to the two-finger zoom or tap operation, the electronic device zooms in on the preview screen. Figure 3 The image shown in (a) is magnified to... Figure 3 The image shown in (b) is captured by an electronic device. Figure 3 When viewing the image shown in (b), the preview interface effect can be referenced for example. Figure 4 As shown in (b) of the diagram, in Figure 4 In the preview interface shown in (b), the multiplier information is not displayed as 3X.

[0100] In Figure 3 In similar scenarios, electronic devices can also be digital cameras, desktop computers, laptops, fixed terminals such as fixed cabinets, smart wearable devices, handheld computers, electronic monitoring equipment, or servers, or dedicated cameras or webcams connected to these devices.

[0101] In various electronic devices, ISPs are used for high-quality image processing and optimization, playing a crucial role in the electronic device's image capture process. ISP, or Image Signal Processor, is generally used to process the raw image data output from the image sensor. When processing the raw image, the ISP may use the 3A algorithm to obtain image signal processing parameters, and then use these parameters to process the raw image to generate a preview data stream.

[0102] The aforementioned 3A algorithms may include an automatic exposure (AE) algorithm, an automatic white balance (AWB) algorithm, and an automatic focus (AF) algorithm. The automatic exposure algorithm is used to determine exposure parameters. The automatic white balance algorithm is used to determine white balance parameters. The automatic focus algorithm is used to determine focus parameters. Correspondingly, image signal processing parameters may include at least one of the exposure parameters, white balance parameters, and focus parameters.

[0103] In this embodiment of the application, the image signal processing parameters can be generated by any one of the AE algorithm, AWB algorithm, and AF algorithm.

[0104] Figure 5This diagram illustrates a possible processing algorithm flow module structure of an ISP. The ISP can be equivalent to the processor configured in the processing algorithm flow module of other embodiments, and may include multiple image processing algorithms. During image capture, the lens projects light signals within its field of view (equivalent to the acquired data in other embodiments of this application) onto the photosensitive area of ​​the image sensor. The image sensor performs photoelectric conversion on the light signals and inputs raw images in Bayer format to the ISP front end (FE) at certain time intervals. Multiple raw image frames input to the ISP front end constitute a data stream of raw image data. The ISP FE can be the first processing node after the image enters the algorithm flow. The FE can be mainly responsible for the preliminary processing operations of the image signal. The FE stage receives raw image data from the input device and can perform preliminary processing and conversion on the raw image data in the raw domain, and then pass the processed front-end data stream to the subsequent ISP processing module. The image processing operations performed by the ISP FE may include image cropping (to remove unwanted parts or adjust image size), format conversion, etc. In the front-end data stream output by the ISP front-end, a statistics submodule can be configured to perform statistics on the front-end data stream, obtain statistical data, and adjust (or update or correct) the image signal processing parameters of the 3A algorithm based on the statistical data.

[0105] The front-end data stream output by the Image Front End (FE) of an ISP can be stored in the memory of an electronic device. The back-end (BE) of the ISP can retrieve at least one second image frame from the memory of the electronic device and process the data output by the FE. The memory of the aforementioned electronic device may include double data rate synchronous dynamic random access memory (DDR SDRAM). Compared to the FE of the ISP, the BE of the ISP can perform more complex image processing operations, such as noise reduction, color correction, automatic exposure, and automatic white balance, resulting in clearer, more natural image frames that conform to human visual perception. The data output by the BE can be referred to as the back-end data stream.

[0106] The back-end data stream output by the ISP's BE is stored in the electronic device's DDR synchronous dynamic random access memory. The ISP's post-processing end (PE) retrieves the back-end data stream from the DDR synchronous dynamic random access memory and performs further processing on at least one third image frame in the back-end data stream in the YUV domain to obtain the preview data stream.

[0107] exist Figure 5 In the processing algorithm flow module shown, a tiny pipe can be configured at the back end of the module. The ISP performs face detection from the tiny stream (tiny stream) output by the BE node through the tiny pipe. The tiny stream can be the back-end data stream output by the BE node, which can be used for face detection, portrait segmentation, and optical flow information calculation. The resolution of the tiny stream is low, which can be lower than the original image data obtained by the image sensor. The original image data can also include multiple video frames. For example, the resolution of the original image data can be 4096×3072, while the resolution of the tiny stream may be 720×540. The image size of the tiny stream can be smaller than the image size of the original image data, meaning that the FOV of the tiny stream can be smaller than the FOV of the original image data. When performing face detection based on the tiny stream, the FOV of the image frames in the tiny stream may still be larger than the actual preview data stream. If the 3A algorithm determines the white balance parameters based on the face detection results of the tiny stream, it may lead to exposure or white balance abnormalities. Similarly, if the 3A algorithm determines the exposure parameters based on the FOV of the original image data, it may lead to exposure abnormalities.

[0108] To address issues such as abnormal exposure or white balance, this application proposes an image processing method. In addition... Figures 2 to 4 In addition to the mobile phone shooting scenarios shown, the embodiments of this application may also involve the following scenarios, which are not limited in the embodiments of this application.

[0109] (1) Video surveillance.

[0110] In video surveillance scenarios, ISP algorithms can improve image clarity and contrast by processing image signals captured by surveillance cameras, enhancing nighttime monitoring effects and ensuring the quality of surveillance footage.

[0111] (2) Intelligent recognition.

[0112] In applications such as intelligent driving, ISP algorithms can identify environmental videos captured by cameras, perceive objects in the environment, and help vehicles understand the surrounding roads and traffic conditions in real time, thereby enabling safe driving.

[0113] (3) Intelligent detection.

[0114] In production scenarios, ISP algorithms can perform quality inspections on product images captured by cameras on the production line, identify products that may have quality problems, and assist in intelligent production.

[0115] In the aforementioned application scenarios, as well as other potential applications, cameras are increasingly being integrated with other electronic devices, and their applications are becoming more widespread. Simultaneously, due to continuous advancements in lens technology, the optical performance of lenses has been significantly improved, enabling them to capture a wider field of view (FOV) and thus increasing the lens's FOV. At the same time, the image sensor size of electronic devices is gradually increasing; larger sensors can capture more light and detail, thereby improving image sharpness and dynamic range, supporting a wider FOV. Furthermore, electronic devices utilize improved image processing algorithms, including color correction, noise reduction, and sharpening algorithms. These algorithms not only improve image quality but also, through computation and processing, can simulate a wider FOV effect.

[0116] However, in the scenarios described above and other possible scenarios, when users take photos or videos with electronic devices, they may not need the entire field of view to be in the image. Therefore, users may adjust the captured image to narrow the field of view. For example, from Figure 3 The image shown in (a) is adjusted to Figure 3 The image shown in (b) is an example. When the 3A algorithm called by the ISP module performs automatic exposure, automatic focus, and automatic white balance, it can determine the image signal processing parameters based on the data obtained by the ISP module from processing the raw image data. The image signal processing parameters may include at least one of the parameters for automatically adjusting exposure, focus, and white balance. The statistical data used by the ISP algorithm to calculate the image signal processing parameters generally comes from the front-end data stream output by the ISP front-end FE after processing the raw image data in the RAW domain, while the preview data stream is generated by the ISP post-processing end after processing the back-end data in the YUV domain. When the FOV of the front-end data stream and the preview data stream differs greatly, it will cause inaccuracies in the AE and AWB algorithms, which may lead to abnormal exposure or white balance.

[0117] To solve the above problems, Figures 1 to 4 Based on the electronic device, software architecture, and scenario shown, embodiments of this application provide an image processing method that can avoid the situation where the FOV of the image detected by the image sensor and the FOV of the preview data stream are too different, leading to inaccuracies in the 3A algorithm and affecting the image quality. The image processing method provided in embodiments of this application includes, as follows: Figure 6 The steps are shown.

[0118] Step S61: The image sensor converts the acquired data into raw image data according to the initial image signal processing parameters. The raw image data includes multiple raw image frames.

[0119] In this embodiment, when the user activates the camera of the electronic device, the image sensor can be activated and continuously acquire data. The acquired data can be light passing through the lens and illuminating the photosensitive module of the image sensor, causing the image sensor to receive an analog signal. After acquiring the data, the photosensitive module of the image sensor performs digital-to-analog conversion on the light signal, converting the analog signal of the acquired data into an electrical signal to obtain the original image data.

[0120] Therefore, step S61 may include: the image sensor obtains acquired data according to the initial image signal processing parameters, the acquired data is an analog signal, the image sensor converts the acquired data into raw image data according to the initial image signal processing parameters, the raw image data is a digital signal, and the raw image data includes multiple raw image frames.

[0121] For example, the image sensor can be a CCD or CMOS, etc., and the embodiments of this application are not limited thereto.

[0122] In possible implementations, the initial image signal processing parameters may include at least one of initial exposure parameters, initial white balance parameters, and initial focus parameters. The initial exposure parameters may be parameters calculated by an automatic exposure algorithm, such as initial exposure time and / or initial aperture value. The initial aperture value may further include the ISO sensitivity as specified by the International Organization for Standardization (ISO).

[0123] When obtaining initial exposure parameters, the automatic exposure algorithm determines the required exposure based on the ambient light information sensed by the electronic device's image sensor, thus generating the initial exposure parameters. Without pressing the shutter button, the aperture value in the initial exposure parameters can be visually displayed by the electronic device through a preview data stream. Multiple first image frames in the preview data stream are refreshed and displayed on the preview interface, allowing the user to intuitively view the exposure effect. When the shutter button is pressed to take a picture, the exposure time in the initial exposure parameters can be displayed by the electronic device through the captured image or video.

[0124] The initial white balance parameters mentioned above can be determined by an automatic white balance algorithm based on the white points within the field of view (FOV) detected by the photosensitive element of the electronic device. For example, the white balance parameters may include color temperature information that needs to be adjusted.

[0125] In another possible implementation, the initial image signal processing parameters can be calculated using the 3A algorithm employed by the ISP, based on statistical data from either the original image frame or the second image frame. The second image frame can be statistical data from the front-end data stream output by the ISP's front-end. The size of the second image frame can be either the size of the original image frame output by the image sensor or the default image frame size of the ISP's front-end. If the ISP's front-end is configured with a default image frame size, the ISP's front-end can adjust the size of the original image frame to match the front-end's default size.

[0126] For example, the image frames included in the raw image data can be in Bayer format, which is image data in the raw field. Bayer format can be an image sensor format used for color images.

[0127] Step S62: The image sensor inputs the raw image data into the ISP's processing algorithm module.

[0128] In this embodiment, the processing algorithm module of the ISP can be referred to as an image signal processing pipeline, which can be a software module set in the image signal processor. The ISP processing algorithm module can be used to perform a series of processing on the image to optimize image quality and improve camera performance. The input data of the ISP processing algorithm module can be a digital signal stream of raw image data, and the output data of the ISP processing algorithm module can be a preview data stream.

[0129] For example, the ISP's processing algorithm flow module can adopt the following approach: Figure 4 The structure shown includes a front-end, a back-end, and a post-processing end. When the image sensor inputs raw image data into the processing algorithm module, the front-end of the processing algorithm module receives the raw image data. Specifically, the front-end of the processing algorithm module obtains the online raw image data stream from the image sensor; that is, after the image sensor generates the raw image data, it directly inputs the raw image data into the front-end, which then performs a series of processes on the raw image data stream in the raw domain.

[0130] Step S63: The ISP's processing algorithm module processes the original image data to obtain a preview data stream; the preview data stream includes multiple first image frames; the first image frames correspond to the original image frames, and the first image frames are obtained by locally magnifying the corresponding original image frames.

[0131] The preview data stream is used by the user to preview the image capture scene. The electronic device displays a preview interface based on the preview data stream, and the electronic device pre-presents the image effects that can be captured in the preview interface so that the user can determine the timing of shooting based on the pre-presented image effects in the preview interface. For example, the first image frame of the preview data stream is used to display the image capture scene to the user on the screen of the electronic device. The user can control the camera of the electronic device to capture video or images based on the content and effects displayed in the preview data stream. When displaying the preview data stream, the electronic device captures the preview data stream according to the shooting command received from the user to obtain the captured video or image.

[0132] In step S63, the original image data is a data stream of digital signals converted from acquired analog data. The original image data is processed sequentially by the front end of the processing algorithm flow module, the back end of the image signal processing algorithm flow module, and the post-processing end of the image signal processing algorithm flow module to obtain the preview data stream output by the post-processing end.

[0133] The front-end of the processing algorithm module can perform a series of operations on the original image data to obtain processed original image data, i.e., the front-end data stream. The front-end data stream can include multiple second image frames, the order of which corresponds to the order of the original image frames in the original image data. For example, the original image frames in the original image data may sequentially include: image frame 1, image frame 2, image frame 3… image frame N. The front-end processes the image frames in the original image data sequentially to obtain the front-end data stream, which includes the following second image frames sequentially: image frame 11, image frame 12, image frame 13… image frame 1N. Specifically, image frame 11 is obtained after processing image frame 1, image frame 12 is obtained after processing image frame 2, image frame 13 is obtained after processing image frame 3, and so on.

[0134] Alternatively, the second image frame corresponds to the original image frame, and the number of second image frames is less than the number of original image frames. When processing the original image frames, the front end can select original image frames for processing at intervals. The original image frames in the original image data include, in sequence: image frame 1, image frame 2, image frame 3... image frame N. The front end processes image frames 1, image frame 3, image frame 5... to obtain the front end data stream. The second image frames included in the front end data stream are, in sequence: image frame 11, image frame 13, image frame 15... and so on.

[0135] In different implementations, the processing operations performed on the image frame by the front-end, back-end, and post-processing end may differ.

[0136] For example, the front end can be used to perform at least one of the following processes on the raw image data: cropping, depigmentation, format conversion, or noise removal.

[0137] For example, the front end can be used to crop or perform basic format conversion on the raw image data.

[0138] The front-end data stream generated by the processing algorithm flow module is input into the DDR of the electronic device. The back-end of the processing algorithm flow module can obtain the front-end data stream frame by frame from the DDR and process it frame by frame. In other words, the image frames processed by the back-end are offline image frames. Compared to the front-end of the processing algorithm flow module, the back-end of the processing algorithm flow module performs a series of more complex processing operations on the front-end data stream.

[0139] For example, the backend performs at least one of the following processing operations on the image frame: color correction, contrast adjustment, sharpening, image enhancement, white balance processing, gamma correction, etc. Furthermore, the backend of the processing algorithm module may also perform certain analysis algorithms on the frontend data stream, such as edge detection and feature extraction.

[0140] After processing the front-end data stream, the back-end of the algorithm flow module obtains the back-end data stream. The back-end data stream may include multiple third image frames. The order of the third image frames in the back-end data stream corresponds to the order of the second image frames in the front-end data stream. The number of third image frames may be equal to or less than the number of second image frames.

[0141] After the backend generates the backend data stream, it stores the backend data stream in the DDR of the electronic device. The post-processing end of the processing algorithm module obtains the backend data stream from the DDR, performs further processing on the backend data stream, and obtains the preview data stream.

[0142] For example, the post-processor can be used to perform pre-preview optimization on the backend data stream.

[0143] For example, the post-processing end performs at least one of the following processing operations on the image frame: image quality optimization, color management, format conversion, and special effects processing. The image quality optimization may include at least one of noise removal, improving image sharpness, contrast, and color saturation, to make the image more realistic and detailed. The color management may include further calibration and management of the image's colors to ensure accurate color reproduction that conforms to human visual perception. The format conversion may include converting a red-green-blue (RGB) format image to a YUV format image, or converting a first YUV format image to a second YUV format image. The first YUV format and the second YUV format can be different YUV formats.

[0144] In this embodiment, the processing performed sequentially on the input data stream by the front-end, back-end, and post-processing end can include processing of the same type. For example, the front-end and back-end can perform noise reduction processing on the input data stream to different degrees or with different precision.

[0145] In other possible implementations, the algorithm processing module may include a front-end and a back-end.

[0146] The content included in the first image frame may be a part of the content included in the original image frame. In a possible implementation, the first image frame may be obtained by the electronic device performing a local magnification of the original image frame in response to a user's operation command to magnify a part of the original image frame.

[0147] Step S64: For at least one of the plurality of first image frames, determine the region of interest (ROI) in each of the first image frames.

[0148] In step S64, the electronic device can process each first image frame in the preview data stream to obtain the region of interest for each first image frame. This improves the adjustment effect of image signal processing parameters.

[0149] Alternatively, the electronic device can process a portion of all the first image frames in the preview data stream at regular time intervals. For example, if the first image frames in the preview data stream are sequentially: image frame 11, image frame 12, image frame 13, image frame 14... image frame 1N, processing can be performed on image frames 12, 14, 16... to obtain the regions of interest for the first image frames 12, 14, 16, etc. This reduces the amount of data processing and saves resource consumption.

[0150] In this embodiment, a region of interest (ROI) detection module can be set after the processing algorithm module. When the first image frame is displayed in the preview interface, the first image frame can be detected synchronously to obtain the ROI. The ROI can be the area in the first image frame where the target object appears. The target object can be, for example, a human body, a face, a part of a human body, one of the facial features, or an area in the first image frame that may have abnormal exposure or white balance.

[0151] In other possible implementations, the target object could be an object with a defined category, such as a still life object with a defined category.

[0152] For example, the areas that may have abnormal exposure or white balance can be areas with abnormal lighting in the first image frame.

[0153] For example, a user uses an electronic device to film a stage scene, where spotlights are set up to create a spotlight effect. The image sensor's field of view (FOV) captures the range and content of the image, which may vary depending on the field of view (FOV). Figure 7 As shown in (a) in the diagram. Figure 7 In the image shown in (a), the person may be on a stage, and there may be a spotlight above the stage. The spotlight shines on the person, and the area around the person is brighter. In the area not illuminated by the spotlight, there may be a black background. In the Beijing area, the light is weaker.

[0154] exist Figure 7 Based on the image shown in (a), the target object can be a person and / or a person's face. When the user zooms in on the target object in the image, the electronic device responds to the user's zooming operation by reducing the camera's shooting range. The range and content of the image displayed in the preview interface can be as follows: Figure 7 As shown in (b) of the diagram. That is to say, Figure 7 The image shown in (b) is a first image frame in the preview data stream, while Figure 7 The image shown in (a) is a raw image frame from the raw image data. Because in Figure 7 In the image shown in (a), the overall brightness is low. To avoid the image being too dark, the automatic exposure algorithm may increase the exposure to enhance the brightness of the obtained image. Therefore, the automatic exposure algorithm adjusts the exposure according to the image's brightness. Figure 7 The exposure parameters generated for image (a) may cause overexposure in the area where the person is located, which may result in an image appearing as shown in the preview interface. Figure 7 The overexposed and washed-out effect is shown in (b) above. Therefore, areas that may be overexposed can be designated as regions of interest, for example, in... Figure 7In the image shown, the area illuminated by the spotlight can be designated as the region of interest.

[0155] At the same time, Figure 7 In the image shown in (a), there is a mixed color temperature. That is, when a human body is presented under a spotlight, the color temperature of the area where the human body is located in the image is different from the color temperature of the background. If we consider... Figure 7 If the white balance parameter corresponding to (a) in the diagram is adjusted based on the field of view, then when responding to user input and zooming in on the area containing the face or body, it may cause a color deviation in the face or body (e.g., appearing yellowish compared to the actual face or body). Figure 8 As shown in (a) above. Therefore, regions where white balance anomalies may exist can be designated as regions of interest, for example, in... Figure 7 In the image shown, the area illuminated by the spotlight can be designated as the region of interest.

[0156] In another possible implementation, when photographing people, users are generally concerned with whether the person's face or the person as a whole is overexposed, rather than whether the background is overexposed. Therefore, in Figure 7 In the image shown in (b), the target object is a human body or face, and the area of ​​the face in the first image frame can be used as the region of interest.

[0157] In one possible implementation, the electronic device can determine the region of interest in the first image frame using an object detection algorithm or other image detection algorithms.

[0158] Step S65: Obtain statistical data of the region of interest and statistical data of the region of the first image frame.

[0159] In this embodiment of the application, the statistical data of the region of interest may include statistical data of the pixels in the region of interest, such as grayscale statistical data of the pixels in the region of interest, color value statistical data of the pixels in the region of interest, and / or white point statistical data of the pixels in the region of interest.

[0160] The statistical items included in the statistical data of the region of the first image frame may be the same as or partially the same as the statistical items included in the statistical data of the region of interest. The region of the first image frame may include the region of interest of the first image frame, as well as the region outside the region of interest of the first image frame.

[0161] In possible implementations, the region of interest can include different types of regions of interest; for example, the region of interest can include the face region and the human body region.

[0162] For example, statistics can be data related to exposure parameters, white balance parameters, and / or focus parameters. For exposure parameters, statistics could include the average grayscale value. For white balance parameters, statistics could include the ratio of red pixels to green pixels, and / or the ratio of blue pixels to green pixels.

[0163] The region of the first image frame can also include the region outside the region of interest of the first image frame.

[0164] Step S66: Update the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame.

[0165] In this embodiment of the application, updating image signal processing parameters may include updating exposure parameters and / or white balance parameters.

[0166] The updating of image signal processing parameters in step S66 may include fine-tuning the image signal processing parameters. That is, within a preset adjustment range, the image signal processing parameters are updated based on statistical data of the region of interest and statistical data of the region of the first image frame. Alternatively, step S66 may include: obtaining first adjustment data based on statistical data of the region of interest and statistical data of the region of the first image frame, and using the first adjustment data to update the image signal processing parameters.

[0167] The adjustment range of the first adjustment data mentioned above is within the preset range threshold.

[0168] In a possible implementation, updating the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame may include inputting the statistical data of the region of interest and the statistical data of the region of the first image frame into the 3A algorithm to obtain the image signal processing parameters updated by the 3A algorithm.

[0169] Step S67: Obtain the updated preview data stream or captured image based on the updated image signal processing parameters.

[0170] In this embodiment, the preview screen in the preview interface is updated according to the updated image signal processing parameters, or the captured image is obtained according to the updated image signal processing parameters in response to the user clicking the capture button.

[0171] In possible implementations, obtaining the preview data stream or the captured image based on the updated image signal processing parameters may include refreshing the preview data stream according to the refresh time interval of the image frames in the preview data stream, and obtaining the captured image based on the latest updated image signal processing parameters in response to the user clicking the capture button.

[0172] In one possible implementation, in step S67, the electronic device can adjust the preview data stream or the captured image based on all the updated image signal processing parameters; alternatively, the electronic device can adjust the preview data stream or the captured image based on some of the updated image signal processing parameters. That is, in one possible implementation, the preview interface generated by the electronic device based on the preview data stream refreshes its display based on the updated image signal processing parameters after the update. In another possible implementation, the preview interface generated by the electronic device based on the preview data stream does not refresh its display based on the updated image signal processing parameters after the update; instead, upon receiving a shooting command, the updated image signal processing parameters are reflected in the captured image or video.

[0173] As can be seen, by using the method provided in this application embodiment, statistical data is obtained from the image frames in the preview data stream, the image signal processing parameters are adjusted (or updated) based on the statistical data, and the preview data stream is obtained based on the updated image signal processing parameters. This can avoid the situation where the 3A algorithm is inaccurate due to the large difference between the FOV of the image detected by the image sensor and the FOV of the preview data stream, thus affecting the image effect.

[0174] exist Figure 7 In the stage scene shown in (a), there is a mixed color temperature, meaning that the area where the human figure is located has a different color temperature than the background area due to the spotlight illumination. Meanwhile, in Figure 7 In the stage scene shown in (a), the grayscale information of the area where the human body is located and the area where the background is located differs significantly. If the automatic exposure algorithm and automatic white balance algorithm in the 3A algorithm are based on... Figure 7 Statistical data of the image with a large FOV shown in (a) are used to calculate image signal processing parameters, and these parameters are then applied to the image. Figure 7 When using a smaller FOV image as shown in (b) of the diagram, a series of problems may arise. For example, reference errors may occur. Figure 7 The face shown in (b) is overexposed, as... Figure 8 The image (a) shows phenomena such as facial skin tone distortion. The method provided in this application embodiment enables statistical analysis of the preview data stream to obtain statistical data, and then determines image signal processing parameters based on the statistical data. Adjustment Figure 7The overexposure shown in (b) is... Figure 8 The color cast and other issues shown in (a) are addressed by adjusting the image to achieve a smaller FOV, resulting in more consistent exposure and color adjustments. Figure 8 As shown in (b) of the diagram.

[0175] In possible implementations, the processing algorithm flow of the image signal processor is mainly executed by the processing algorithm flow module. Furthermore, in different implementations, the processing algorithm flow module may include a front-end and a back-end, or it may include a front-end, a back-end, and a post-processing end. Simultaneously, in different implementations, the image processing operations performed by the front-end and back-end of the processing algorithm flow module may differ, or the image processing operations performed by the front-end, back-end, and post-processing end of the processing algorithm flow module may differ. The following combines... Figure 9 This application describes a processing algorithm flow module in one of its embodiments.

[0176] exist Figure 9 In the illustrated embodiment, the processing algorithm flow module includes a front-end, a back-end, and a post-processing end. The front-end receives raw image data from the image sensor and outputs a front-end data stream. The back-end processes the front-end data stream and outputs a back-end data stream. The post-processing end processes the back-end data stream and outputs a preview data stream.

[0177] For example, the front end performs at least one of the following processing operations on each raw image frame in the raw image data.

[0178] Operation (1), Defect removal processing

[0179] A dead pixel refers to a pixel on an image sensor or display that fails to display color or brightness correctly. The pixel corresponding to a dead pixel may differ significantly from surrounding pixels. For example, a white pixel may appear in a completely dark environment, or a black pixel may appear in a bright environment. Dead pixel removal is a repair operation targeting abnormal pixels in an image or display.

[0180] Operation (2), raw domain noise reduction processing

[0181] The RAW domain can refer to the image directly output from the image sensor, including the initial, unprocessed raw image frame. Image noise refers to unwanted or redundant interference information present in the image data. RAW domain denoising processing can include denoising the image frames in the RAW domain, removing noise from them.

[0182] Operation (3), black level correction

[0183] Black level correction refers to processing the raw image data output by the image sensor to eliminate or reduce image signal offset caused by factors such as dark current and analog-to-digital conversion threshold, so that the image data reaches a uniform reference level in the black area.

[0184] Operation (4), Optical Shadow Correction

[0185] Optical shading correction, also known as lens shading correction, addresses the uneven brightness of an image caused by the optical characteristics of a lens. Lens shading typically manifests as a brighter center and darker edges, creating vignetting. This phenomenon is caused by a combination of factors, including the lens's convex lens principle, light refraction, and the attenuation of light collected by pixels at the sensor's edges. The main purpose of optical shading correction is to adjust the brightness values ​​of the image data to achieve a uniform brightness level across the entire field of view.

[0186] Operation (5), Automatic White Balance Correction

[0187] Automatic white balance correction is used to automatically adjust the proportions of the three primary colors—red, green, and blue—based on the color temperature of the shooting environment in order to restore the true colors of the subject.

[0188] Through at least one of the above operations (1) to (5), the front end generates a front end data stream and outputs the front end data stream to the back end. For each second image frame in the front end data stream, the back end performs at least one of the following operations.

[0189] Operation (6), color interpolation processing

[0190] Color interpolation can be used to estimate the color value of unknown pixels by using the color values ​​of known neighboring pixels, so as to regenerate an image with higher resolution from the original image.

[0191] Operation (7), color correction processing

[0192] Color correction is used to restore and optimize the colors of images or videos, making them more realistic, vivid, and stylistically consistent. After color correction, image frames are converted from the raw domain to the RGB domain.

[0193] Operation (8), Global tone mapping processing

[0194] Global tone mapping maps the brightness of a high dynamic range scene to the limited dynamic range suitable for the display device while preserving image detail and color. It addresses the loss of detail and reduced contrast in images captured in low-light environments due to insufficient brightness. By enhancing overall image brightness, global tone mapping can fully preserve image detail, avoiding overexposure and improper detail processing.

[0195] Operation (9), Gamma correction processing

[0196] Gamma correction can be used to adjust the brightness and contrast of an image to make the brightness and contrast of the image frame more consistent with the visual characteristics of the human eye.

[0197] Through at least one of the above operations (6) to (9), the backend generates a backend data stream and inputs the backend data stream into the post-processing end. The post-processing end performs at least one of the following operations on each image frame in the backend data stream.

[0198] Operation (10) Image distortion correction

[0199] Image distortion correction refers to correcting distortions in an image caused by factors such as camera lens, so that objects in the image retain their true shape and size.

[0200] Operation (11) Noise Treatment

[0201] Operation (11) may include noise reduction processing of image noise in the brightness and color dimensions.

[0202] Operation (12) Brightness Processing

[0203] Brightness processing refers to adjusting the brightness of an image frame.

[0204] Operation (13) Color Processing

[0205] Color processing refers to adjusting the colors of an image frame.

[0206] Operation (14) Image scaling

[0207] Image scaling refers to adjusting the size of an image frame.

[0208] By performing at least one of the above operations (10) to (14), the post-processing end can process the back-end data stream to obtain a preview data stream.

[0209] One possible implementation is to perform statistical analysis on the front-end data stream to obtain statistical data, and then calculate the initial image signal processing parameters based on this data. After determining the initial image signal processing parameters, the statistical data from the front-end data stream continues to be obtained in real time, and the image signal processing parameters are continuously updated based on the real-time statistical data.

[0210] In another possible implementation, the processing algorithm flow module may include a front-end and a back-end. The front-end may be used to perform the operations (1) to (9) described above, and the back-end may be used to perform the operations (10) to (14) described above.

[0211] The method provided in the embodiments of this application can... Figure 9 The processing algorithm shown includes a module for identifying and statistically analyzing the preview data stream. This module identifies the preview data stream, obtains statistical data based on it, and then corrects the image signal processing parameters based on this statistical data. When calculating statistical data based on the preview data stream, statistical data can be calculated separately for all regions and regions of interest in the first image frame of the preview data stream. The following section describes how to determine the regions of interest in the first image frame.

[0212] In one possible implementation, target objects that are prone to exposure, focus, or white balance abnormalities can be identified, and the region of the identified target object in the first image frame can be set as the region of interest.

[0213] In possible implementations, the region of interest is the area containing at least one target object. For example, in Figure 8 In the image shown in (b), the electronic device can target a face and perform face recognition operation on at least one first image frame to obtain a face bounding box, where region 81 of the face bounding box is a region of interest. When the face bounding box is determined to be a region of interest, a label for the face can be assigned to the region of interest.

[0214] In other possible implementations, other objects can be used as target objects, such as the human body or a part of the human body (e.g., head, hand, etc.). When determining the region of interest, human body recognition or human body part recognition operations can be performed on at least one first image frame to obtain a human body bounding box or a human body part bounding box. The region of the human body bounding box or the region of the human body part bounding box is the region of interest.

[0215] In other possible implementations, the region of interest (ROI) can include areas containing multiple different types of target objects. For example, the target objects can include faces, the human body, or parts of the human body. After recognizing the target object, the electronic device can add labels corresponding to the target object to the ROI for each type of target object; for example, adding a face label to a face bounding box and a hand label to a hand bounding box.

[0216] In other possible implementations, the region of interest (ROI) can be determined by the electronic device based on the user's selection instructions. In the preview interface of a photograph, the user can click anywhere in the image. The electronic device then displays a bounding box indicating the ROI at the location of the click, and designates the area containing this bounding box as the ROI.

[0217] In this embodiment, statistics are performed on both the region of interest and the region of the first image frame to determine statistical data for local regions of interest and statistical data for all regions in the first image frame. When calculating the statistical data, the electronic device may first perform statistical analysis on the region of the first image frame, then determine the region of interest in the first image frame, and then perform statistical analysis on the region of interest. Alternatively, the electronic device may first determine the region of interest in the first image frame, and then perform statistical analysis on both the region of the first image frame and the region of interest. Next, the method for obtaining the statistical data for the region of interest or the statistical data for the region of the first image frame will be described.

[0218] Because the original image frames undergo multiple nonlinear transformations, such as gamma correction and color correction, during the processing of the original image data in the algorithm flow module, an inverse gamma (invGamma) and an inverse color correction matrix (invCCM) transformation are applied to the first image frame in the preview data stream when obtaining statistical data. This transforms the first image frame to the linear domain. Then, statistical data is obtained from the linearly transformed first image frame. The inverse gamma transform is used to restore the gamma-corrected image to a state closer to the original linear light. The inverse color correction transform is used to restore the color-corrected image to its original linear color state.

[0219] In the embodiments of this application, statistical data may include grayscale histograms (also known as image histograms) and white balance statistics.

[0220] In other possible implementations, the grayscale histogram can also be the grayscale mean.

[0221] Among them, grayscale histograms can be used to statistically analyze the distribution of pixel values ​​in an image. By counting the number of times each grayscale level (also known as brightness level) appears in the image, a grayscale histogram graphically represents the number of pixels at each brightness level and shows the distribution of pixels in the image.

[0222] In this embodiment of the application, the grayscale histogram can be a bar chart. The grayscale histogram divides the grayscale level or brightness level of the image into a series of continuous intervals and counts the number of pixels in each interval. Figure 10 This is an exemplary grayscale histogram diagram of this application. The grayscale histogram may include a horizontal axis and a vertical axis. The horizontal axis represents the pixel value or the range of pixel values, and the vertical axis represents the number of pixels in the first image frame.

[0223] Based on the statistical data including the grayscale histogram, the image signal processing parameters are updated according to the obtained grayscale histogram of the region of interest and the grayscale histogram of the region of the first image frame, including: updating the exposure parameters according to the grayscale histogram of the region of interest and the grayscale histogram of the region of the first image frame.

[0224] For example, an electronic device can send grayscale histogram data to the automatic detection algorithm's execution module. The automatic detection algorithm then adjusts the exposure parameters based on the grayscale histogram data, and these parameters may include the exposure level determined by the automatic exposure algorithm. The automatic exposure algorithm's execution module can calculate the average grayscale value of the region of interest based on its grayscale histogram, and obtain the exposure level using spot metering. Then, the automatic exposure algorithm's execution module can calculate the average grayscale value of a region in the first image frame using its grayscale histogram, and use this average grayscale value to help calculate the overexposed area, thereby adjusting the exposure level of the automatic exposure algorithm.

[0225] The aforementioned spot metering refers to metering a specific point in the first image frame.

[0226] White balance statistics can include the ratio between the red (R) channel and the green (G) channel (R / G) (also known as the ratio of the number of red pixels to the number of green pixels), and the ratio between the blue (B) channel and the green channel (B / G) (also known as the ratio of the number of blue pixels to the number of green pixels). The main function of white balance is to ensure that white objects appear white in an image under different lighting conditions, thus restoring the image's normal colors. For a specific light source, its color temperature is fixed. By measuring the R / G and B / G values ​​in an image, the color temperature of the light source can be determined. Under the same color temperature, the R / G and B / G values ​​for gray tones are generally within a relatively defined range. Therefore, as long as the R / G and B / G values ​​in the current image can be determined, the color temperature value of the current light source can be obtained by looking up a table, and the white balance parameters can be adjusted accordingly.

[0227] For example, the automatic white balance algorithm's execution module can re-estimate the white points in the first image frame based on white balance statistics, and determine the white balance parameters based on the white points.

[0228] For example, the region of interest is the face. The automatic exposure algorithm's operating module can determine the exposure parameters based on prior data of the face and the grayscale mean. The automatic white balance algorithm's operating module includes a face automatic white balance unit, which can determine the white balance parameters based on prior data of the face and estimated white point information.

[0229] In this application example, prior data for the face can be determined based on face samples from a publicly available face database.

[0230] When photographing the human body, the After Effects (AE) algorithm typically uses mean metering to determine the automatic exposure. With a large field of view (FOV), the area occupied by the human body may be small, resulting in a lower average grayscale value in the original image frame. In this case, the AE algorithm module might increase the exposure, leading to overexposure of the area containing the human body. When performing automatic white balance (AWB), it also uses the original image frame with a large FOV to determine the estimated white point and uses this estimate to determine the automatic white balance parameters. This can cause a significant deviation between the automatic white balance parameters and the actual image of the human body, making it difficult for the preview interface and the captured image to meet the visual needs of the average person.

[0231] When shooting other target objects, it is also possible that the AE and AWB algorithms use original image frames with a large FOV to determine the automatic exposure and automatic white balance parameters, which may cause the area where the target object is located to be overexposed or have abnormal white balance.

[0232] The image processing method provided in this application embodiment can determine exposure parameters and white balance parameters based on the preview data stream when a user takes an image using an electronic device. When the FOV of the original image frame obtained by the electronic device is large, while the FOV of the preview data stream is small, the problem of exposure abnormality or white balance abnormality caused by the large difference between the FOV of the original image frame and the preview data stream is improved.

[0233] This application also provides a method for taking pictures, such as... Figure 12 As shown, the steps include the following.

[0234] Step S121: Obtain a preview data stream based on image signal processing parameters; the preview data stream includes multiple first image frames; the preview data stream is used to preview the captured image.

[0235] Step S122: For at least one of the plurality of first image frames, determine the region of interest in each of the first image frames.

[0236] Step S123: Obtain statistical data of the region of interest and statistical data of the region of the first image frame.

[0237] Step S124: Update the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame. The updated image signal processing parameters are used to update the preview data stream.

[0238] Step S125: Update the preview data stream based on the updated image signal processing parameters.

[0239] Step S126: Display the preview interface based on the updated preview data stream.

[0240] Step S127: In response to a shooting command for shooting operation on the preview interface, obtain at least one first image frame in the preview data stream.

[0241] Step S128: Generate the captured image or video based on at least one first image frame.

[0242] In other possible implementations, the preview interface can be updated based on the updated preview data stream. Alternatively, the updated preview data stream may not be reflected in the preview interface, but the captured image or video can be generated only based on the updated preview data stream after the shooting command is received.

[0243] In other possible implementations, the image processing method provided in any embodiment of this application can be used to update the image signal processing parameters, and then the updated image signal processing parameters can be used to obtain an image or video.

[0244] This application also provides an image processing apparatus, such as... Figure 11 As shown, it includes: a preview data stream acquisition module 111, a region of interest module 112, a statistical data module 113, and an image signal processing parameter update module 114.

[0245] The preview data stream acquisition module 111 is used to obtain a preview data stream based on image signal processing parameters; the preview data stream includes multiple first image frames; the preview data stream is used to preview the image capture screen.

[0246] The region of interest module 112 is used to determine a region of interest in each of the plurality of first image frames for at least one of the first image frames.

[0247] The statistics module 113 is used to obtain statistics on the region of interest and statistics on the region of the first image frame.

[0248] The image signal processing parameter update module 114 is used to update the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region of the first image frame.

[0249] The image processing apparatus provided in this application embodiment can be used to perform, for example... Figure 6 And the steps of the method provided in any related embodiments.

[0250] This application also provides an imaging device capable of capturing images based on image processing parameters updated by any image processing method provided in this application. In one embodiment, the imaging device includes a preview data stream acquisition module and an imaging module.

[0251] The preview data stream acquisition module is used to obtain the preview data stream according to the updated image processing parameters, wherein the updated image processing parameters are the updated image processing parameters provided by any embodiment of this application.

[0252] A shooting module is used for the preview data stream including a plurality of first image frames; in response to a shooting command, it obtains the captured image and / or video based on the first image frames.

[0253] The imaging device provided in this application embodiment can execute the imaging method or image processing method provided in any embodiment of this application.

[0254] This application also provides an electronic device having the image processing apparatus provided in any embodiment of this application.

[0255] This application also provides an electronic device having the imaging device provided in any embodiment of this application.

[0256] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0257] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0258] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0259] This application also provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0260] This application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0261] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0262] The above-described embodiments are optional embodiments provided by this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: The preview data stream is obtained based on the image signal processing parameters; The preview data stream includes multiple first image frames; The preview data stream is used to preview the captured image. For at least one of the plurality of first image frames, determine the region of interest in each of the first image frames; Obtain statistical data of the region of interest, and statistical data of the region of the first image frame; Based on the statistical data of the region of interest and the statistical data of the region of the first image frame, the image signal processing parameters are updated. The updated image signal processing parameters are used to update the preview data stream or generate the captured image.

2. The method of claim 1, wherein, The preview data stream is obtained based on the raw image data input from the image sensor, and the field of view of the raw image data is greater than the field of view of the preview data stream.

3. The method according to claim 1 or 2, characterized in that, The preview data stream is output from the post-processing end of the image signal processor's processing algorithm module.

4. The method according to any one of claims 1 to 3, characterized in that, The step of obtaining the preview data stream based on image signal processing parameters includes: Based on image signal processing parameters, raw image data from the image sensor is obtained; the raw image data includes multiple raw image frames; the first image frame corresponds to the raw image frame, and the first image frame is obtained by locally magnifying the corresponding raw image frame; The original image data is processed sequentially through the front end of the processing algorithm flow module, the back end of the image signal processing algorithm flow module, and the post-processing end of the image signal processing algorithm flow module to obtain the preview data stream.

5. The method of claim 4, wherein, The method further includes: Obtain the front-end data stream output by the processing algorithm flow module; the front-end data stream includes multiple second image frames; Statistical data for the entire region of the second image frame is obtained, and the statistical data for the entire region of the second image frame is used to update the image signal processing parameters.

6. The method according to any one of claims 1 to 5, characterized in that, The region of interest is the area where the target object is located.

7. The method of claim 6, wherein, The target object is the face.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The first image frame is converted to the linear domain to obtain a linearly mapped image; the linearly mapped image is used to obtain statistical data of the region of interest, as well as statistical data of the region of the first image frame.

9. The method according to any one of claims 1-8, characterized in that, The statistical data includes grayscale data and white balance data.

10. The method according to any one of claims 1-9, characterized in that, The statistical data includes a grayscale histogram, and the image signal processing parameters include exposure parameters. Updating the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region in the first image frame includes: Calculate the mean gray level based on the gray level histogram of the region of interest; The exposure amount is determined based on the average grayscale value; Based on the grayscale histogram of the first image frame, the exposure amount is adjusted to obtain the exposure parameters.

11. The method according to any one of claims 1-10, characterized in that, The statistical data includes white balance statistical data, and the image signal processing parameters include white balance parameters. Updating the image signal processing parameters based on the statistical data of the region of interest and the statistical data of the region in the first image frame includes: Based on the white balance statistics of the region of interest and the white balance statistics of the first image frame, estimate the white point of the region of interest or the white point of the first image frame; Based on the white points in the region of interest or the white points in the first image frame, the estimated white points are corrected to obtain the white balance parameters.

12. A photographing method, characterized by, include: A preview data stream is obtained based on the updated image processing parameters, wherein the updated image processing parameters are the image processing parameters updated by any one of claims 1-11; the preview data stream includes a plurality of first image frames; In response to a shooting command, the captured image and / or video are obtained based on the first image frame.

13. An electronic device, comprising: The electronic device includes: a processor and a memory; The memory is used to store a program for the electronic device to perform the method as described in any one of claims 1-12, and to store data related to implementing the method as described in any one of claims 1-12; The processor is configured to execute programs stored in the memory.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-12.

Images