Flash lamp calibration method and device, terminal and computer readable storage medium

Abstract

The embodiment of the invention discloses a flash lamp calibration method and device, a terminal and a computer readable storage medium. The flash lamp calibration efficiency can be improved. Comprising the following steps: sending a flash lamp calibration instruction to a micro-processing unit in a camera subsystem in an inter-core communication mode through an application processor in an application subsystem; reading at least one driving current gear under the condition that the micro-processing unit receives the flash lamp calibration instruction; and calling an image signal processing module to carry out flashing photographing based on the at least one driving current gear, carrying out analysis processing on statistical data corresponding to an image of each flashing photographing through a preset calibration algorithm in the image signal processing module, and determining at least one calibration result corresponding to the at least one driving current gear.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a flashlight calibration method, device, terminal and computer-readable storage medium. Background Art

[0002] Currently, mobile phone cameras generally have multiple cameras. The flash on the phone may have 10 to 50 drive current levels for each camera. For each drive current level of each camera, the flash parameters can be calibrated in the production line environment or debugging environment, which is also called flash calibration. The current flash calibration method requires that for each drive current level, calibration photography instructions need to be sent layer by layer from the terminal application layer to the underlying hardware, and then the calibration results for each drive current level are obtained. This will cause the flash calibration to take a long time, thereby reducing the efficiency of flash calibration. Summary of the invention

[0003] The embodiments of the present application are intended to provide a flashlight calibration method, device, terminal and computer-readable storage medium, which can improve the efficiency of flashlight calibration.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a flash calibration method, which is applied to a terminal, wherein the terminal includes an application subsystem and a camera subsystem; the camera subsystem includes a microprocessing unit; and the method includes:

[0006] Sending a flashlight calibration instruction to the microprocessing unit by means of inter-core communication via an application processor in the application subsystem;

[0007] When the microprocessing unit receives the flash lamp calibration instruction, reading at least one driving current gear;

[0008] An image signal processing module is called to perform flash photography based on at least one driving current gear, and statistical data corresponding to the image taken each time the flash is photographed is analyzed and processed through a preset calibration algorithm in the image signal processing module to determine at least one calibration result corresponding to the at least one driving current gear; the statistical data is obtained by performing image information statistics on the image taken each time the flash is photographed through the statistical module of the camera subsystem.

[0009] In a second aspect, an embodiment of the present application provides a flash calibration device, which is applied to a terminal, wherein the terminal includes an application subsystem and a camera subsystem; the camera subsystem includes a microprocessing unit; and the device includes:

[0010] An instruction sending module, used for sending a flashlight calibration instruction to the microprocessing unit through an application processor in the application subsystem in an inter-core communication manner;

[0011] A reading module, used for reading at least one driving current gear when the microprocessing unit receives the flash lamp calibration instruction;

[0012] A calibration module is used to call the image signal processing module to perform flash photography based on at least one driving current gear, and analyze and process the statistical data corresponding to the image of each flash photography through a preset calibration algorithm in the image signal processing module to determine at least one calibration result corresponding to the at least one driving current gear; the statistical data is obtained by performing image information statistics on the image of each flash photography through the statistical module of the camera subsystem.

[0013] In a third aspect, an embodiment of the present application provides a terminal, including a memory and a processor; wherein:

[0014] The memory is used to store executable instructions;

[0015] The processor is used to implement the flash light calibration method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the flashlight calibration method provided in the embodiment of the present application.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the flash calibration method provided in the embodiment of the present application.

[0018] The embodiment of the present application provides a flash calibration method, device, terminal and computer-readable storage medium. A microprocessing unit is provided in the camera subsystem of the terminal. After the application processor sends the flash calibration instruction to the microprocessing unit in the form of inter-core communication, the microprocessing unit reads at least one drive current gear to be calibrated for traversal calibration, and obtains at least one calibration result corresponding to at least one drive current gear. The embodiment of the present application realizes that the calibration process of the flash is completed by the microprocessing unit inside the camera subsystem to control the image signal processing module, that is, the calibration of at least one drive current gear is automatically completed inside the camera subsystem, so that in the calibration process corresponding to the flash calibration instruction, the flash calibration under the traversal drive current gear is realized, thereby reducing the interaction between each drive current gear calibration and the upper-layer application, speeding up the calibration process, thereby shortening the calibration time, and improving the flash calibration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of an optional flow chart of a flashlight calibration method provided in an embodiment of the present application;

[0020] Figure 2 An optional structural diagram of a terminal hardware system provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an optional flow chart of a flashlight calibration method provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of an optional flow chart of a flashlight calibration method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of an optional flow chart for traversing at least one driving current gear for flash lamp calibration provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of an optional flow chart of a flashlight calibration method provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of a process of skipping some ISP hardware module processing in a flash calibration mode provided in an embodiment of the present application;

[0026] Figure 8 A schematic diagram of an image display process based on skipping some ISP hardware module processing provided in an embodiment of the present application;

[0027] Fig. 9 An optional structural schematic diagram of a flashlight calibration device provided in an embodiment of the present application;

[0028] Fig.10 An optional structural schematic diagram of a flashlight calibration device provided in an embodiment of the present application;

[0029] Fig.11 An optional structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0034] The nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are applicable to the following interpretations:

[0035] The image signal processing system (ISP) is mainly used to process the output signal of the front-end image sensor to match the image sensors of different manufacturers.

[0036] 5A: Auto Exposure (AE), Auto Focus (AF), Auto While Balance (AWB), Auto Lens Shading Correction (ALSC) and Auto Anti-Flicker (ATF).

[0037] Statistics (Statistics, Stats): refers to the data obtained when the image data stream passes through the 5A statistics module.

[0038] Micro Control Unit (MCU)

[0039] Application Processor Computing Processing Unit (ACPU)

[0040] RAW image: RAW image can be understood as the raw data that the image sensor converts the captured light source signal into a digital signal. RAW files record the original information of the digital camera sensor, and also record some metadata (such as ISO settings, shutter speed, aperture value, white balance, etc.) generated by the camera.

[0041] Flash is an artificial light source used in photography. When the camera shutter is pressed, it usually illuminates the scene for 1 / 1000s to 1 / 2000s. Flash is mostly used for instant lighting in dark places, and is also used to fill in the light of the subject in bright places. Types of flash include LED, Dual LED, Xennon, etc.

[0042] For LED and Dual LED types, according to Newton's three-color principle, white light can be obtained by mixing the red, green and yellow light-emitting diodes. The red, green and yellow light intensities can be adjusted by the parameters rg_ratio, bg_ratio and flux, so that the flash LED can emit appropriate white light. For different image cameras and different gears of LED drive current, the above parameters of LED are different, so it is necessary to combine the camera, traverse all drive current gears, and calibrate the most appropriate calibration parameters for each drive current gear.

[0043] The current flash calibration method requires the terminal to be placed in an appropriate calibration environment (such as a dark box). For each flash control current of a camera on the terminal, it is necessary to trigger the pre-installed flash calibration function App to send a photo command (such as a Single Flash photo command). The AE algorithm is used to analyze parameters such as several pictures affected by the light during the flash and the AE statistical values ​​of the pictures, and the optimal calibration parameters rg_ratio, bg_ratio, and flux of the Flash under this current when the AE exposure converges are obtained, which are used as the correction values ​​that can be used by the Flash in subsequent normal photo taking.

[0044] In the current flash calibration method, each flashing of the driving current level requires sending the photo command from the APP layer by layer to the terminal's underlying hardware, and then uploading the calibration results layer by layer. For current mobile phone cameras, there are generally multiple cameras, and each camera may correspond to 10 to 50 driving current levels. The current flash calibration method will make the calibration time-consuming and reduce the efficiency of flash calibration. Furthermore, the camera system of mobile phones using heterogeneous platforms has multiple processors, and the entire path takes longer, further reducing the efficiency of flash calibration.

[0045] In addition, in the current flash calibration method, a photo is selected and stored when calibrating each drive current. Since the image sensor imaging and the image file generation need to go through multiple software and hardware links, such as image effect processing, image format conversion, etc.; in fact, the flash calibration result and whether it is successful can be completely based on the RAW image, which has nothing to do with the generation of the final image file, thus further reducing the efficiency of flash calibration.

[0046] The embodiments of the present application provide a flashlight calibration method, device, terminal, and computer-readable storage medium, which can improve the efficiency of flashlight calibration. Figure 1 , Figure 1 An optional flow chart of the flashlight calibration method provided in the embodiment of the present application is as follows:

[0047] S101 , sending a flashlight calibration instruction to a microprocessor unit via an application processor in an application subsystem in an inter-core communication manner.

[0048] The flash calibration method of the embodiment of the present application is applied to a terminal, the terminal includes a flash and a camera, and the system architecture of the terminal includes an application subsystem and a camera subsystem. The application subsystem and the camera subsystem are two independent subsystems, the application subsystem includes an application processor, the application subsystem is used to run the operating system of the terminal and the software applications on the terminal, and provide management and services for the camera subsystem and other subsystems on the terminal.

[0049] In the embodiment of the present application, when the flashlight is calibrated, the terminal is placed in a dark environment. For example, the terminal is placed in a special calibration box or dark box, and a preset application for flashlight debugging is running in the application processor. Electronic devices outside the calibration box or dark box, such as a debugging computer, can control the preset application on the mobile phone by wired or wireless means, such as the engineering mode application for flashlight debugging to issue the flashlight calibration instruction, thereby realizing the application processor to issue the flashlight calibration instruction.

[0050] In some embodiments, for example, such as a preset model application, the application processor can receive calibration instructions sent externally through the preset model application, such as receiving an instruction to start flash calibration for a camera of the terminal sent by an electronic device outside the darkbox via wired or wireless means.

[0051] The embodiment of the present application adds a microprocessing unit (i.e., MCU) to the camera subsystem to serve and manage the core of the camera subsystem, the image signal processing system ISP. In other words, the MCU is used as a coprocessor to take over the control of the ISP by the application subsystem in the current flash calibration process. Since the camera subsystem and the application subsystem belong to different subsystems, the MCU and the ACPU in the application subsystem need to interact with each other through a message mailbox (Mailbox) in the form of inter-core communication.

[0052] In some embodiments, the terminal hardware system structure diagram of the embodiment of the present application can be as follows: Figure 2 As shown. It can be seen that under the overall control of the ACPU in the application subsystem (AP-Sys), the embodiment of the present application can control each module in the ISP through the MCU. Figure 2 The Camera heterogeneous system shown can send a flash calibration instruction and at least one driving current level from the ACPU side, and implement the main logic of traversing each driving current level, flashing, calibration algorithm scheduling, etc. on the MCU side, so as to automatically implement multiple flashes inside the camera subsystem (Camera-Sys) to complete the flash calibration under each driving current level, avoiding the time-consuming calibration of starting the photo-taking process for each driving current level separately under multiple driving current levels, and improving the flash calibration efficiency.

[0053] In some embodiments, based on Figure 1 ,like Figure 3 As shown, S101 can be implemented by executing S1011-S1013 as follows:

[0054] S1011. When a preset industrial model application in the application processor receives a calibration instruction sent from the outside, a preset calibration flag is sent to a first function node in the hardware abstraction layer of the terminal.

[0055] In the embodiment of the present application, the preset calibration flag represents sending a flashlight calibration instruction. The preset model application can obtain the preset calibration flag by setting the value of the calibration flag to the preset value representing sending a flashlight calibration instruction to send the preset calibration flag to notify the terminal software system level below the application layer to start the flashlight calibration process.

[0056] Exemplarily, the calibration flag may be meta.flash.cali.mode; the preset working mode application sends the parameter meta.flash.cali.mode=1, which indicates sending a flash calibration instruction.

[0057] In an embodiment of the present application, the hierarchical structure of the terminal's software system may include, from bottom to top: a firmware layer (Firmware layer), a kernel layer (Kernel layer), a hardware abstraction layer (Camera HAL layer), a framework layer (Framework layer) and an application layer (APP layer). Among them, the firmware layer serves the hardware system of the terminal. In some embodiments, a driver layer may also be set between the firmware layer and the hardware system. The specific selection is made according to the actual situation, and the embodiment of the present application is not limited. The preset industrial model application runs in the application layer. The preset industrial model application sends the preset calibration flag to the first functional node in the hardware abstraction layer. Among them, the first functional node is used to receive the flash calibration instruction issued by the upper layer and send it to the microprocessor unit layer by layer.

[0058] S1012: Convert the preset calibration flag into a flashlight calibration instruction through the first functional node, and send the flashlight calibration instruction to the microprocessing unit in an inter-core communication manner.

[0059] In an embodiment of the present application, when the first functional node receives a preset calibration flag, it can determine that the flash calibration process needs to be started according to the preset calibration flag. The first functional node converts the preset calibration flag into a flash calibration instruction, and sends the flash calibration instruction to the microprocessing unit in the form of a message through the functional interface of the kernel layer of the terminal and using the message mailbox (Mailbox) of inter-core communication.

[0060] S102: When the microprocessor receives a flash lamp calibration instruction, read at least one driving current gear.

[0061] In the embodiment of the present application, the microprocessing unit reads at least one driving current gear from the shared storage space when receiving the flash lamp calibration instruction sent by the application processor in the inter-core communication manner.

[0062] In some embodiments, the first functional node of the hardware abstraction layer is also used to read at least one driving current gear corresponding to the camera on the terminal from a preset initialization file, such as a tunning file, during the initialization phase of the terminal, and write it into a shared storage space corresponding to the application processor and the microprocessor unit.

[0063] Exemplarily, in the initialization phase of the terminal, at least one driving current gear is written into the shared storage space corresponding to the application processor and the microprocessing unit through the first functional node calling the functional interface of the kernel layer of the terminal. The first functional node sends information of the shared storage space, such as address, size, type, etc., to the microprocessing unit. In this way, the first functional node calls the inter-core communication interface of the kernel layer of the terminal, and sends the flashlight calibration instruction to the microprocessing unit in the form of an inter-core communication message. When the microprocessing unit receives the inter-core communication message, it can read at least one driving current gear from the shared storage space corresponding to the application processor and the microprocessing unit.

[0064] S103, calling the image signal processing module to perform flash photography based on at least one driving current gear, and analyzing and processing the statistical data corresponding to the image of each flash photography through a preset calibration algorithm in the image signal processing module, to determine at least one calibration result corresponding to at least one driving current gear.

[0065] In the embodiment of the present application, the image signal processing module is the ISP firmware module corresponding to the ISP hardware system in the firmware layer of the terminal software system hierarchical architecture. The statistical data of the image is obtained by performing image information statistics on each image taken with flash photography by the statistical module of the camera subsystem; wherein the statistical module is a hardware module in the camera subsystem.

[0066] In the embodiment of the present application, in the flash calibration mode, the terminal is placed in a dark calibration environment, and the flash is controlled to flash under a certain fixed drive current through the flash shooting process, and the AE algorithm is controlled to analyze one or more images affected by the light during the flash and the AE statistical value of the image and other parameters, and the best calibration parameters of the Flash under this drive current when the AE exposure converges are obtained as the calibration result corresponding to the drive current. In this way, when the flash works normally under this drive current in the subsequent period, the calibration result can be read as the correction value.

[0067] In the embodiment of the present application, the image output by the image sensor will flow into the statistical module, and the statistical module will perform statistical processing on the data of each input image to obtain statistical data of each image, such as AE Stats data. The image signal processing module inputs the AE Stats data and other relevant parameters of each image into a preset calibration algorithm, such as an AE algorithm. The preset calibration algorithm calculates the exposure parameters of each image, such as image exposure time and exposure gain, and determines whether the calibration algorithm result of the corresponding drive current gear is converged and stable. In the case of non-convergence, the exposure parameters of each image are provided to the image sensor and the ISP hardware system, and the next photo is taken based on the exposure parameters; in the case of convergence, the calibration parameters of the flash of the drive current gear are generated by the preset calibration algorithm as the calibration result of the drive current gear.

[0068] In some embodiments, Figure 4 As shown, S103 can be implemented by executing the process of S1031-S1033 as follows:

[0069] S1031. Using a microprocessor unit, call a preset calibration algorithm in an image signal processing module to obtain at least one driving current gear.

[0070] In an embodiment of the present application, a preset calibration algorithm is pre-implemented in the image signal processing module. Exemplarily, the image signal processing module includes an algorithm library, such as an AE algorithm lib, and the AE algorithm lib includes a preset calibration algorithm. The microprocessing unit includes a control module (Control SW), and the control module can send information such as address, size, and type of the shared storage space to the preset calibration algorithm in the image signal processing module, and parse the information through the algorithm library in the preset calibration algorithm. Then, the MCU side control software can obtain at least one driving current gear obtained by parsing through the algorithm interface, such as the gear sequence number Index_1~Index_N and its corresponding current value. In this way, the preset calibration algorithm in the image signal processing module can obtain at least one driving current gear; and the microprocessing unit can also obtain at least one driving current gear, which is used to traverse each driving current gear, perform flash shooting control and calibration algorithm analysis control, so as to complete the flash calibration process for at least one driving current gear.

[0071] S1032. For each driving current gear of at least one driving current gear, determine the flashing parameter corresponding to each driving current gear through a preset calibration algorithm.

[0072] In an embodiment of the present application, for each driving current gear in at least one driving current gear, the image signal processing module determines the flashing parameter corresponding to each driving current gear through a preset calibration algorithm.

[0073] In some embodiments, the preset calibration algorithm in the image signal processing module can generate flash control parameters such as flash duration, current intensity, etc. as flash parameters corresponding to each driving current gear according to the current value of each driving current gear. The specific selection is made according to the actual situation, and the embodiments of the present application are not limited.

[0074] S1033. Based on the flash parameters, control the flash and the image sensor to perform at least one flash photo. Through a preset calibration algorithm, analyze and process the statistical data corresponding to the image of each flash photo to obtain a calibration result corresponding to each driving current level, thereby obtaining at least one calibration result by traversing at least one driving current level.

[0075] In the embodiment of the present application, the image signal processing module controls the flash to flash at least once based on the flash parameters corresponding to each driving current gear. The flash can be turned on for several frames or dozens of frames each time. At least one photo can be taken during each flash. Through each photo in at least one photo, an image of each flash photo is obtained. Exemplarily, the flash on time, that is, the flash duration, can be 100ms or 300ms. During this period, the flash will be on all the time. Multiple photos can be taken during this period to output multiple images.

[0076] In the embodiment of the present application, each image of the flash photo is sent to the statistical module, and the statistical module outputs the statistical data corresponding to the image to the image signal processing module. The image signal processing module will calculate the exposure parameters corresponding to the image, such as exposure time, exposure gain, etc., according to the statistical data corresponding to the image through a preset calibration algorithm, and at the same time determine whether the algorithm result converges. If it is determined that it has not converged, the exposure parameters of the image will be applied to the next photo, and so on, until it is determined that the algorithm result converges or reaches the preset number of photos. The flash calibration of each driving current gear is completed to obtain the calibration result corresponding to each driving current gear. Exemplarily, for a driving current gear of 10mA, when the preset calibration algorithm reaches convergence when the flash photo is taken to the third image, the preset calibration algorithm can be used to perform a comprehensive analysis based on the third image to obtain the calibration parameters of the flash at the driving current gear of 10mA as the calibration result corresponding to the driving current gear of 10mA.

[0077] In an embodiment of the present application, the microprocessing unit controls the image signal processing module to traverse each driving current gear in at least one driving current gear, performs flash lamp calibration according to the above process, and obtains at least one calibration result corresponding to at least one driving current gear.

[0078] In some embodiments, the image signal processing module controls the flash and the image sensor to take flash photos based on the flash parameters to obtain an image taken each time; the statistical module of the camera subsystem performs image information statistics on the image taken each time, and determines the statistical data corresponding to the image taken each time; through a preset calibration algorithm, analyzes the statistical data to determine the exposure parameters corresponding to the image taken each time, and determines whether the algorithm results corresponding to each driving current gear are converged; when convergence is determined, the calibration parameters corresponding to each driving current gear are determined through the preset calibration algorithm as the calibration results corresponding to each driving current gear; when non-convergence is determined, the flash and the image sensor are controlled to take the next flash photo based on the calibrated photo parameters, the flash parameters and the exposure parameters, until the calibration parameters corresponding to each driving current gear are determined, thereby obtaining the calibration results corresponding to each driving current gear.

[0079] In some embodiments, when the number of photographs reaches a preset threshold of the number of photographs and the algorithm result corresponding to the driving current level has not converged, the calibration result corresponding to the driving current level is determined as calibration failure.

[0080] In the embodiment of the present application, for a certain driving current gear, according to the above flash calibration process, the number of photos taken reaches the preset photo number threshold, but the algorithm result of the preset calibration algorithm has not reached convergence, indicating that the flash takes photos abnormally at this driving current gear. Therefore, the calibration result corresponding to this driving current gear is determined as calibration failure.

[0081] In some embodiments, when the number of photographs taken at a driving current level reaches a threshold and the algorithm result corresponding to the driving current level has not converged, at least one retry can be performed. If convergence cannot be achieved in at least one retry, the calibration result corresponding to the driving current level is determined to be a calibration failure.

[0082] In some embodiments, when the preset working model application in the application processor issues the flash calibration instruction, it will also issue the calibrated photographing parameters corresponding to the flash calibration instruction. Here, the calibrated photographing parameters include sampling at least one configuration parameter of the camera corresponding to the flash calibration instruction for photographing, such as focal length, resolution, etc. The preset working model application issues the flash calibration instruction and the calibrated photographing parameters to the framework layer of the terminal. The framework layer continues to send the flash calibration instruction and the calibrated photographing parameters downward layer by layer along with the image frame configuration request (Online Request) until the flash calibration instruction and the calibrated photographing parameters are sent to the hardware abstraction layer of the terminal. Here, the Online Request is triggered by an image acquisition instruction, such as a flash calibration instruction or a photographing instruction, and is used to configure the relevant parameters of image frame acquisition and the storage space (buffer) of the image frame.

[0083] In some embodiments, the calibrated photographing parameters are sent to the data transmission pipelines (Pipelines) of the hardware abstraction layer. Through the data transmission pipelines, the calibrated photographing parameters are sent to the microprocessing unit in the camera subsystem. In this way, the microprocessing unit can control the flash and the image sensor to perform flash photography based on the calibrated photographing parameters and the flash parameters, and obtain an image of each flash photography. That is to say, in the above-mentioned flash photography process, the flash is flashed according to the flash parameters, and the photo is taken according to the calibrated photographing parameters, so as to obtain an image of each flash photography.

[0084] Exemplarily, the preset calibration algorithm is the AE algorithm in the image signal processing module. In combination with S102 and S103, the method of traversing at least one driving current gear to calibrate the flash lamp in the embodiment of the present application can be as follows: Figure 5As shown, including S0-S9, as follows:

[0085] S0, the control module (Control SW) in the MCU parses the shared storage space through the AE algorithm interface to obtain at least one driving current gear, including Index_1 to Index_N.

[0086] S1. For each drive current gear Index, along with the camera image stream, the MCU prepares the above data, such as mapping the current gear, scheduling the AE algorithm through the algorithm control module (AE CTRL) in the image signal processing module, and setting the flash parameters and calibration shooting parameters to the AE algorithm Lib according to a specific timing. The image signal processing module decides to output turning on / off the flash, the flash duration, etc.

[0087] S2. MCU calls the flash control module (FlashCtrl) to flash at an appropriate time according to the above flash parameters.

[0088] S3, FlashCtrl calls the flash driver (Flash Driver), which is slightly driven with a current corresponding to the driving current level to light up the flash, such as an LED. The flash can be turned on for several frames or dozens of frames.

[0089] S4. The flash lamp emits white light when driven by a specific current.

[0090] S5. The flash light illuminates the object in the darkbox, the inside of the box, etc.

[0091] S6. The object reflects light to the image sensor, generates an image at a specific frame rate, and outputs it to the ISP hardware system corresponding to the image signal processing module.

[0092] S7. On the one hand, the image flows to the hardware statistics module (AE Statistics) in the ISP hardware system, generates AEStats statistics and stores them in the memory buffer, and on the other hand, continues to flow to various processing modules in the ISP system.

[0093] S8, MCU schedules the AE algorithm library in the image signal processing module, and inputs the AE Stats data and other related parameters generated by each frame of the image into the AE algorithm in sequence. The AE algorithm calculates the exposure parameters, such as image exposure time, exposure gain, etc., and determines whether the corresponding drive current gear Flash calibration algorithm is converged and stable. If the calibration convergence state is not reached, the flash will continue to be turned on, and the MCU will set the exposure parameters calculated this time to the image sensor and ISP hardware, and act on the next frame of the image. Similarly, the exposure parameters are continued to be calculated for the next frame of the image, and it is determined again whether the corresponding drive current gear Flash calibration algorithm has converged successfully. Within a few frames or dozens of frames, if the Flash calibration algorithm converges successfully, the calibration of this gear is completed, and the AE algorithm generates calibration parameters as the calibration result of the corresponding drive current gear, and outputs the Flash shutdown control flag at the same time. The MCU turns off the Flash flash according to this flag.

[0094] If a certain drive current gear fails to reach the convergence of the calibration algorithm during calibration, it will be retried n times, where n is an integer greater than 1. For example, if it fails to converge after 3 retries, the calibration failure is determined as the calibration result of the corresponding drive current gear.

[0095] When a drive current level is calibrated successfully, the MCU will traverse the flash calibration of the next drive current level in sequence according to the recorded drive current level information, using the process of S1-S8, until the flash calibration of all drive current levels is completed.

[0096] S9. After all drive current gears are calibrated, the MCU obtains the gear calibration results of all drive currents, that is, at least one calibration result, from the AE algorithm side through the algorithm interface.

[0097] It should be noted that in the case where multiple cameras are installed on the terminal, such as wide-angle, ultra-wide, telephoto, ultra-micro, etc., the flash calibration methods corresponding to different cameras are consistent with the above process description and will not be repeated here.

[0098] It can be understood that the embodiment of the present application realizes that the calibration process of the flash is completed by the microprocessing unit inside the camera subsystem controlling the image signal processing module, that is, the calibration of at least one driving current level is automatically completed inside the camera subsystem, so that in the calibration process corresponding to a flash calibration instruction, the flash calibration under the driving current level is traversed, thereby reducing the interaction between each driving current level calibration and the upper-level application, speeding up the calibration process, shortening the calibration time, and improving the flash calibration efficiency.

[0099] In some embodiments, when the terminal determines at least one calibration result corresponding to at least one driving current gear through the above method, it will upload at least one calibration result at one time to write it into the preset engineering file and notify the upper layer application. Figure 1 ,like Figure 6 As shown below:

[0100] S104. Send at least one calibration result to the kernel layer of the terminal through the image signal processing module.

[0101] S105. Send at least one calibration result to the hardware abstraction layer of the terminal through the kernel layer in an inter-core communication manner.

[0102] S106. Write at least one calibration result into a preset engineering file through a second function node in the hardware abstraction layer.

[0103] In some embodiments, the microprocessing unit controls the image signal processing module to perform flash photography based on at least one driving current gear, and analyzes at least one image obtained by the flash photography through a preset calibration algorithm to obtain at least one calibration result corresponding to at least one driving current gear. The image signal processing module sends at least one calibration result to the kernel layer of the terminal, and then sends at least one calibration result to each functional node (Node) in the hardware abstraction layer of the terminal through the inter-core communication mechanism of the kernel layer. The second functional node in the hardware abstraction layer writes at least one calibration result into a preset engineering file. In this way, the terminal can perform flash photography with corresponding calibration results at different driving current gears according to the calibration results recorded in the preset engineering file.

[0104] In some embodiments, the image signal processing module may write at least one calibration result into the upload data queue UpMetaBuffer, and send it to the relevant modules in the application subsystem through a predetermined periodic reporting mechanism.

[0105] In some embodiments, at least one calibration result may also be parsed through a hardware abstraction layer to determine at least one reported data corresponding to at least one drive current gear; and the at least one reported data may be sent to an application processor.

[0106] Here, at least one reported data corresponds to at least one driving current gear, and the reported data corresponding to each driving current gear includes at least a calibration result flag, which is used to indicate whether the corresponding driving current gear is successfully calibrated or failed. The hardware abstraction layer parses at least one calibration result. When the calibration result is a calibration failure, the corresponding reported data is a calibration failure. When the calibration result is a calibration parameter, the corresponding reported data is a calibration success. In this way, the upper layer application can be notified whether each driving current gear has successfully completed the flash calibration.

[0107] In some embodiments, the reported data corresponding to each driving current gear may also include the reason for the calibration failure, and / or the original image output by the imaging front-end module, etc., which will be explained in subsequent embodiments.

[0108] It can be understood that by reporting at least one calibration result at one time, compared with the current flash calibration solution, the interaction between the bottom layer and the upper layer can be further reduced, thereby improving the flash calibration efficiency.

[0109] In some embodiments, the ISP hardware system mainly includes an image sensor, an image preprocessing module, a front end (FE) module, a middle end (ME) module, a back end BE module, and a display subsystem (DSS). During preview, the image data flows through the above modules in sequence and is sent to the DSS for display. In the process of controlling the flashlight and the image sensor to take flash photos based on the flash parameters, the image signal processing module can control the flashlight to flash based on the flash parameters and form an image through the image sensor. In the imaging process, the original image is obtained through the imaging front end module, and the imaging middle end module and / or back end module are not processed on the original image, that is, the processing process of at least one of the imaging middle end module and the back end module is skipped, and the original image output by the front end module is used as the image for each flash photo.

[0110] For example, the ISP hardware module skipping strategy in the flash calibration mode can be as follows: Figure 7 As shown. In the flash calibration mode, when taking photos with a flash in a darkroom environment, the flash calibration is mainly done with the help of the AE algorithm. The AE algorithm mainly outputs exposure parameters, which are specifically set in the front-end (FE) module in the image sensor and ISP. It is insensitive to the secondary and tertiary modules such as the mid-end module and / or back-end module after FE. Therefore, in the flash calibration mode, the hardware processing of the ME module and / or BE module is skipped (Bypass), the time consumption of image processing is reduced, the frame rate can be improved, and the flash calibration process can be greatly accelerated. In the case of multi-gear and multi-camera calibration, the calibration time on the production line is reduced, the calibration cost is greatly saved, and the flash calibration efficiency is improved.

[0111] In some embodiments, when the image signal processing module performs image analysis processing on the statistical data corresponding to each flash photo through a preset calibration algorithm, it can also skip the analysis and processing process of at least one preset algorithm function in the preset calibration algorithm to determine at least one calibration result. Here, the at least one preset algorithm function includes an algorithm function without image performance and quality requirements in a dark environment.

[0112] For example, when the flashlight is calibrated in a dark environment, the processing functions of AWB and / or ATF in the 5A algorithm can be temporarily skipped (Bypass). The underlying MCU bypasses the hardware processing corresponding to the AWB and / or ATF part in the 5A algorithm to complete the flashlight calibration under the corresponding drive current gear.

[0113] It is understandable that skipping at least one preset image analysis function can further improve the frame rate, reduce the calibration time consumption, and improve the flash calibration efficiency.

[0114] In some embodiments, since the preview data of image acquisition generally comes from the YUV image output by the BE module or the ME module, when the imaging process of the BE module and / or the ME module is skipped during the imaging process of flash photography, the preview image can be obtained by the following method:

[0115] At least one original image corresponding to each driving current gear is sent to an application processor; through the application processor, at least one original image corresponding to each driving current gear is downsampled and format converted to determine at least one preview image corresponding to each driving current gear; at least one preview image is sent to a display module of the terminal, and at least one preview image corresponding to each driving current gear is displayed through the display module.

[0116] In the embodiment of the present application, the flash calibration mode is generally used for factory settings of production line products, which is performed in a dark environment and has no image quality requirements for the screen preview image. After each frame of the image sensor is output, the FE module will cache the original image (Raw image) and store it in the set Buffer. Therefore, if Figure 8 As shown, the Raw image can be sent to the application processor (ACPU) in the application subsystem, and converted into YUV format after downsampling by the application processor. In this way, the image data can be sent to the rotating buffer for display and sent to the DSS display without going through the hardware.

[0117] In some embodiments, the image signal processing module can also detect at least one of the image output time and the blank time between frames of the image sensor; and adjust the image output frame rate of the image sensor according to at least one of the image output time and the blank time between frames.

[0118] In some embodiments, when the preset calibration algorithm in the image signal processing module determines that the calibration result corresponding to a certain drive current gear is a calibration failure, it can further determine the failure reason information of the calibration failure; and send the failure reason information corresponding to the drive current gear to. Exemplarily, the Lib library of the AE algorithm will analyze the cause of the convergence failure and give possible failure reasons. For example, when the flash is controlled, there is actually no flash, the reason may be that the flash is out of power or faulty; or when the flash is flashing, the brightness of the flash is abnormal, resulting in the failure to achieve the expected flash effect, which may be that the drive current does not reach the actual gear, the drive is insufficient; or when the flash is flashing, some gears are missed, etc. Further, the failure reason can be given in the form of an error code ErrCode. For example, different ErrCodes correspond to the calibration environment that does not meet the requirements (not in dark conditions), the device is damaged, and so on. The image signal processing module reports the calibration result of the calibration failure corresponding to the drive current gear and the failure reason information to the application processor layer by layer. In this way, the user can locate the fault of the flash calibration failure and screen the faulty equipment according to the failure reason, further improving the efficiency of the flash calibration.

[0119] In the present application embodiment, based on Figure 7 and Figure 8 In the ISP processing flow, the image signal processing module can detect the image output time (Rdout) and / or the blank time between frames (V-blank) of the image sensor, and adaptively improve the image output frame rate according to different sensors, so that the time used for flash current calibration at each level is reduced, thereby reducing the calibration time under all levels, thereby further improving the flash calibration efficiency.

[0120] In some embodiments, based on the software hierarchy structure of the terminal, the flashlight calibration process of the embodiment of the present application can be as follows: Fig. 9 As shown below:

[0121] S71. The preset working model application of the application layer starts the flash calibration mode.

[0122] In S71, the preset model application sets the camera to the flash calibration mode, initiates the Single Flash photo process, and simultaneously sends the preset calibration flag meta.flash.cali.mode=1 to enable the flash calibration function.

[0123] S72. The framework layer sends an image frame configuration request (Camera OnlineRequest) according to a preset calibration flag.

[0124] In S72, the framework layer reads the calibration shooting parameters sent from the application layer and the preset calibration flag meta.flash.cali.mode. The framework layer sends the above parameters to the next layer along with the OnLine Request of each frame image.

[0125] S73. The UseCase layer continues to send down calibration photography parameters and preset calibration flags.

[0126] In S73, the EngineerCapture function interface of the UseCase layer sends the received calibration shooting parameters and the preset calibration flag meta.flash.cali.mode as control parameters (CtrlMeta) to the first function node, the second function node and other service nodes of the hardware abstraction layer. Here, the first function node may be an automatic exposure node (AeNode), the second function node may be an engineering node (EngineerNode), and other service nodes may include service node 1 and service node 2.

[0127] S74. The hardware abstraction layer (Camera HAL) converts the preset calibration flag into a flash calibration instruction, and calls the interface of the kernel encapsulation layer to send it to the kernel layer.

[0128] In S74, the hardware abstraction layer includes multiple function nodes and data transmission pipelines. Pipelines implement camera preview resources and parameter construction based on the parameters in the OnLine Request sent from the upper layer, and send it to the lower kernel layer in the form of a request.

[0129] During the initialization phase of the terminal, the first functional node (AeNode) loads at least one driving current level required for flash calibration from the tuning file, stores this resource in the ACPU and MCU shared buffer, and sends this buffer information such as address, size, and type to the MCU side.

[0130] During the operation of the terminal, AeNode converts the meta.flash.cali.mode sent by the upper layer into a Cmd instruction and sends it to the MCU in the form of a message for inter-core communication.

[0131] In S74, the kernel encapsulation layer is an encapsulation layer for interaction between the upper layer module and the kernel, and is used to provide a calling interface to the upper layer module as a medium for message sending. The upper layer module, such as AeNode, can store information such as at least one drive current gear in a shared buffer, and send it to the kernel layer inter-core communication message mailbox by calling the functional interface implemented in the kernel encapsulation layer.

[0132] S75. The kernel encapsulation layer transmits the parameters in the OnLine Request to the kernel layer through the interface.

[0133] S76. The kernel layer uses the communication information received from the upper layer to realize the interaction between the ACPU and the MCU cores through the message mailbox of the inter-core communication.

[0134] S77, the MCU controls the image signal processing module to receive the message from the inter-core communication forwarded by the MCU, parses the message content to obtain the flash calibration instruction, and responds to the flash calibration instruction.

[0135] In S77, after the MCU receives the flash calibration instruction, during the current image preview stream operation, the algorithm control module (AE CTRL) in the image signal processing module is called to set the software function of the AE algorithm in the image signal processing module to obtain at least one driving current level. For each driving current level, the specific parameters of the Flash flashing, such as the flashing duration, current intensity, etc., are obtained from the AE algorithm to control the Flash flashing.

[0136] For each driving current gear in at least one driving current gear, the MCU obtains specific parameters of the flash flash from the AE algorithm, such as the flash duration, current intensity, etc., to control the flash flash. When each flash ends, the AE algorithm stores the optimal configuration parameters of the flash under each current gear as calibration parameters.

[0137] The algorithm library (AE Lib) in the image signal processing module is used to generate AE control parameters, flash parameters and calibration results. The algorithm library generates flash control parameters, such as flash duration, current intensity, etc. as flash parameters according to the current value of each drive current gear and the calibrated shooting parameters. After each flash, the algorithm library comprehensively analyzes the image affected by the flash light, obtains the analysis result corresponding to the image, and stores it in the form of numerical parameters. In addition, for each frame of image, after various parameters are input, the algorithm library generates parameters related to the exposure of the next frame of image, such as exposure time and exposure gain. In this way, the AE algorithm analyzes the parameters such as the AE statistical value of several pictures affected by the light during the flash and the pictures during the flash, and realizes multiple exposure parameters, and obtains the best calibration parameters of the Flash under this gear drive current when the AE exposure converges, which is the calibration parameter for normal operation during this Flash period, that is, a successful calibration result. Alternatively, when the AE exposure does not converge, the calibration result under this gear drive current is determined to be a calibration failure.

[0138] In S76, the flashing process mainly involves the AE algorithm and the ISP FE stage, so the hardware pipelines such as ME and BE on the ISP can be bypassed. In this way, the calibration process can be accelerated by increasing the frame rate and reducing the duration of the calibration process.

[0139] S78, the MCU controls at least one drive current gear to complete calibration, and writes at least one calibration result into the reporting data space (UpMetaBuffer) to send to the upper layer, that is, the kernel layer for reporting.

[0140] Here, it should be noted that for the flash calibration process with preview requirements, the RAW image can also be stored in UpMetaBuffer and reported together. And / or, the failure reason of the AE algorithm for the drive current gear analysis of the failed calibration can be stored in UpMetaBuffer and reported together.

[0141] S79. The kernel layer receives communication information from the next layer, that is, at least one calibration result; the inter-core interaction between the MCU and the ACPU is realized with the help of the message mailbox for inter-core communication, and at least one calibration result is sent to the hardware abstraction layer through the kernel encapsulation layer through the pre-implemented functional interface in the kernel encapsulation layer.

[0142] S80: The hardware abstraction layer writes at least one calibration result into a preset engineering file, parses at least one calibration result to obtain at least one reporting data, and continues to report to the application layer.

[0143] In S80, the source node (SourceNode) of the hardware abstraction layer distributes at least one calibration result to each service node in the hardware abstraction layer.

[0144] The data node (MetaNode) of the hardware abstraction layer parses at least one calibration result, obtains the calibration parameters corresponding to each drive current gear, or the result of calibration failure, and puts it into the data storage table as calibration data (Metadata). Moreover, the MetaNode determines at least one reporting data corresponding to at least one drive current gear based on whether each drive current gear corresponds to a calibration parameter or a calibration failure result.

[0145] The engineering node (EngineerNode) of the hardware abstraction layer obtains the calibration result corresponding to each drive current gear from the data storage table, and writes it into the preset engineering file for subsequent loading and configuration when the terminal is started. In addition, the second function node calls the use case interface to report at least one reporting data, that is, the success or failure flag corresponding to each drive current gear.

[0146] S81. The data transmission pipelines (Pipelines) of the hardware abstraction layer implement the reporting of at least one reported data, or at least one reported data and at least one Raw image, and report them to the use case domain layer.

[0147] S82. The use case domain layer reports at least one reporting data to the framework layer.

[0148] S83. The framework layer determines the response data corresponding to the Online Request according to the at least one reported data, and reports the at least one reported data to the application layer.

[0149] The preset tool model application of the application layer receives at least one reported data, and displays whether each drive current gear is calibrated successfully or failed in a preset display interface according to the calibration result flag in the at least one reported data, and / or writes at least one reported data into a result record file.

[0150] It can be understood that through the calibration method in the embodiment of the present application, in a photo command process, after the underlying MCU receives the calibration instructions in sequence, it can implement multiple flashes to complete the flash calibration under each current level, and report the calibration results at one time, avoiding the time-consuming calibration of separate photos under multiple levels, thereby improving the efficiency of flash calibration.

[0151] It is understandable that the AE algorithm analyzes and reports different types of flash failures, such as failure to meet the calibration environment conditions, flash damage, abnormal flash brightness, insufficient current driving capability, missed gear detection, etc. In addition to giving calibration failures during the calibration phase, it can also give potential causes of abnormal problems, which facilitates rapid positioning of the production line, repairs faults, and improves overall product yield.

[0152] The embodiment of the present application also provides a flash calibration device, which is applied to the terminal of the embodiment of the present application, the terminal including an application subsystem and a camera subsystem; the camera subsystem includes a microprocessing unit. Fig.10 This is a schematic diagram of the structure of the flash light calibration device provided in the embodiment of the present application. Fig.10 As shown, the flashlight calibration device 1 includes: an instruction issuing module 11, a reading module 12 and a calibration module 13, wherein:

[0153] The instruction sending module 11 is used to send the flash calibration instruction to the microprocessing unit in the camera subsystem through the application processor in the application subsystem in an inter-core communication manner;

[0154] The reading 12 is used to read at least one driving current gear when the microprocessing unit receives the flash lamp calibration instruction;

[0155] The calibration module 13 is used to call the image signal processing module to perform flash photography based on at least one driving current gear, and analyze and process the statistical data corresponding to the image of each flash photography through a preset calibration algorithm in the image signal processing module to determine at least one calibration result corresponding to the at least one driving current gear; the statistical data is obtained by performing image information statistics on the image of each flash photography through the statistical module of the camera subsystem.

[0156] In some embodiments, the instruction issuing module 11 is also used to send a preset calibration flag to the first functional node in the hardware abstraction layer of the terminal when the preset industrial model application in the application processor receives a calibration instruction sent externally; through the first functional node, the preset calibration flag is converted into the flash calibration instruction, and the flash calibration instruction is sent to the microprocessing unit in the form of inter-core communication.

[0157] In some embodiments, the instruction issuing module 11 is also used to send the preset calibration flag to the framework layer of the terminal through the preset working model application; and send the preset calibration flag to the first functional node in the hardware abstraction layer through the framework layer.

[0158] In some embodiments, the instruction issuing module 11 is also used to call the inter-core communication interface of the kernel layer of the terminal through the first functional node, and send the flash calibration instruction to the microprocessing unit in the form of an inter-core communication message, so that the microprocessing unit reads the at least one drive current gear from the shared storage space corresponding to the application processor and the microprocessing unit when receiving the inter-core communication message.

[0159] In some embodiments, the flash light calibration device 1 also includes an initialization module, which is used to call the functional interface of the kernel layer of the terminal through the first functional node during the initialization stage of the terminal, and write the at least one driving current gear into the shared storage space corresponding to the application processor and the microprocessing unit.

[0160] In some embodiments, the calibration module 13 is also used to call the preset calibration algorithm in the image signal processing module through the microprocessing unit to obtain the at least one driving current gear; for each of the at least one driving current gear, determine the flash parameters corresponding to each driving current gear through the preset calibration algorithm; based on the flash parameters, control the flash and the image sensor to perform at least one flash photo, and analyze and process the statistical data corresponding to the image of each flash photo through the preset calibration algorithm to obtain the calibration result corresponding to each driving current gear, thereby obtaining the at least one calibration result by traversing the at least one driving current gear.

[0161] In some embodiments, the calibration module 13 is also used to control the flash and the image sensor to take flash photos based on the flash parameters to obtain the image of each flash photo; through the statistical module of the camera subsystem, the image information of the image of each flash photo is counted to determine the statistical data corresponding to the image of each flash photo; through the preset calibration algorithm, the statistical data is analyzed to determine the exposure parameters corresponding to the image of each flash photo, and determine whether the algorithm results corresponding to each driving current gear are converged; when convergence is determined, the calibration parameters corresponding to each driving current gear are determined through the preset calibration algorithm as the calibration results corresponding to each driving current gear; when non-convergence is determined, the flash and the image sensor are controlled to take the next flash photo based on the calibrated photo parameters, the flash parameters and the exposure parameters, until the calibration parameters corresponding to each driving current gear are determined, thereby obtaining the calibration results corresponding to each driving current gear.

[0162] In some embodiments, the calibration module 13 is further used to determine the calibration result corresponding to the driving current gear as calibration failure when the number of photographs reaches a preset threshold number of photographs and the algorithm result corresponding to the driving current gear has not converged.

[0163] In some embodiments, the instruction issuing module 11 is further used to send the calibration photography parameters corresponding to the flash calibration instruction to the framework layer of the terminal through the preset working model application; send the calibration photography parameters to the data transmission pipeline in the hardware abstraction layer of the terminal through the framework layer; send the calibration photography parameters to the microprocessing unit through the data transmission pipeline;

[0164] In some embodiments, the calibration module 13 is further used to control the flash and the image sensor to perform flash photography based on the calibrated photography parameters and the flash parameters to obtain an image of each flash photography.

[0165] In some embodiments, the flash calibration device 1 also includes a result uploading module; the result uploading module is used to send the at least one calibration result to the kernel layer of the terminal through the image signal processing module; through the kernel layer, the at least one calibration result is sent to the hardware abstraction layer of the terminal in an inter-core communication manner; through the second functional node in the hardware abstraction layer, the at least one calibration result is written into a preset project file.

[0166] In some embodiments, the result upload module is further used to parse the at least one calibration result through the hardware abstraction layer, determine at least one reporting data corresponding to the at least one driving current gear; and send the at least one reporting data to the application processor.

[0167] In some embodiments, the calibration module 13 is also used to control the flash to flash based on the flash parameters, and to form an image through the image sensor. During the imaging process, the original image is obtained through the imaging front-end module, and the original image is not processed by the imaging mid-end module and / or back-end module, and the original image is used as the image taken each time the flash is photographed.

[0168] In some embodiments, the calibration module 13 is also used to skip the analysis and processing process of at least one preset algorithm function in the preset calibration algorithm when performing image analysis processing on the statistical data corresponding to the image taken each time the flash photo is taken through the preset calibration algorithm, so as to obtain the calibration result corresponding to each driving current gear.

[0169] In some embodiments, the result uploading module is also used to send at least one original image corresponding to each driving current gear to the application processor; through the application processor, downsample and format convert the at least one original image corresponding to each driving current gear to determine at least one preview image corresponding to each driving current gear; send the at least one preview image to the display module of the terminal, and display the at least one preview image corresponding to each driving current gear through the display module.

[0170] In some embodiments, the flash calibration device 1 also includes a frame rate adjustment module; the frame rate adjustment module is used to detect at least one of the image output time and the inter-frame blank time of the image sensor through the image signal processing module; and adjust the image output frame rate of the image sensor according to at least one of the image output time and the inter-frame blank time.

[0171] In some embodiments, the calibration module 13 is further used to determine the failure cause corresponding to the calibration result through the preset calibration algorithm when the calibration result is a calibration failure; the result reporting module is further used to send the failure cause to the application processor.

[0172] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0173] The embodiment of the present application also provides a terminal, Fig.11 This is an optional structural diagram of a terminal provided in an embodiment of the present application. Fig.11 As shown, the terminal 3 includes: a memory 32 and a processor 33. The memory 32 and the processor 33 are connected via a communication bus 34; the memory 32 is used to store executable instructions; the processor 33 is used to execute the executable instructions stored in the memory 32 to implement the flash calibration method provided in the embodiment of the present application.

[0174] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein executable instructions are stored. When the executable instructions are executed by the above-mentioned processor, the above-mentioned processor will execute the flash light calibration method provided in the embodiment of the present application.

[0175] In some embodiments, the computer-readable storage medium (i.e., readable storage medium) may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0176] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0177] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0178] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0179] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0180] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0182] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0183] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A flashlight calibration method, characterized in that: Applied to a terminal, the terminal includes an application subsystem and a camera subsystem; the camera subsystem includes a microprocessing unit; the method includes: Sending a flashlight calibration instruction to the microprocessing unit by means of inter-core communication via an application processor in the application subsystem; When the microprocessing unit receives the flash lamp calibration instruction, reading at least one driving current gear; An image signal processing module is called to perform flash photography based on at least one driving current gear, and statistical data corresponding to the image taken each time the flash is photographed is analyzed and processed through a preset calibration algorithm in the image signal processing module to determine at least one calibration result corresponding to the at least one driving current gear; the statistical data is obtained by performing image information statistics on the image taken each time the flash is photographed through the statistical module of the camera subsystem.

2. The method according to claim 1, characterized in that The step of sending the flashlight calibration instruction to the microprocessing unit by means of inter-core communication through the application processor in the application subsystem includes: When the preset industrial model application in the application processor receives a calibration instruction sent from the outside, a preset calibration flag is sent to a first function node in the hardware abstraction layer of the terminal; The preset calibration flag is converted into the flashlight calibration instruction through the first functional node, and the flashlight calibration instruction is sent to the microprocessing unit in an inter-core communication manner.

3. The method according to claim 2, characterized in that The sending of a preset calibration flag to a first functional node in a hardware abstraction layer of the terminal includes: By applying the preset working model, the preset calibration flag is sent to the framework layer of the terminal; The preset calibration flag is sent to the first function node in the hardware abstraction layer through the framework layer.

4. The method according to claim 2 or 3, characterized in that: The step of sending the flashlight calibration instruction to the microprocessing unit in an inter-core communication manner includes: The inter-core communication interface of the kernel layer of the terminal is called through the first functional node, and the flash calibration instruction is sent to the microprocessing unit in the form of an inter-core communication message, so that the microprocessing unit reads the at least one driving current gear from the shared storage space corresponding to the application processor and the microprocessing unit when receiving the inter-core communication message.

5. The method according to claim 4, characterized in that The method further comprises: During the initialization phase of the terminal, the functional interface of the kernel layer of the terminal is called through the first functional node to write the at least one driving current gear into the shared storage space corresponding to the application processor and the micro-processing unit.

6. The method according to any one of claims 1 to 3 or claim 5, characterized in that: The calling of the image signal processing module to perform flash photography based on at least one driving current gear, and analyzing and processing statistical data corresponding to each flash photography image through a preset calibration algorithm in the image signal processing module, to determine at least one calibration result corresponding to the at least one driving current gear, includes: By means of the microprocessing unit, a preset calibration algorithm in the image signal processing module is called to obtain the at least one driving current gear; For each of the at least one driving current gear, determining the lightning parameter corresponding to each driving current gear by using the preset calibration algorithm; Based on the flash parameters, the flash and the image sensor are controlled to perform at least one flash photo, and the statistical data corresponding to the image of each flash photo is analyzed and processed through the preset calibration algorithm to obtain the calibration result corresponding to each driving current gear, thereby obtaining the at least one calibration result by traversing the at least one driving current gear.

7. The method according to claim 6, characterized in that Based on the flash parameters, the flash lamp and the image sensor are controlled to perform at least one flash photo, and statistical data corresponding to the image of each flash photo are analyzed and processed through the preset calibration algorithm to obtain the calibration result corresponding to each driving current gear, including: Based on the flash parameters, control the flashlight and the image sensor to perform flash photography, and obtain an image of each flash photography; By using the statistical module of the camera subsystem, image information statistics are performed on the image taken each time when the flash is struck, and statistical data corresponding to the image taken each time when the flash is struck are determined; By using the preset calibration algorithm, analyzing according to the statistical data, determining the exposure parameters corresponding to the image taken each time with the flash, and determining whether the algorithm results corresponding to each driving current gear are converged; When convergence is determined, the calibration parameters corresponding to each driving current gear are determined by the preset calibration algorithm as the calibration results corresponding to each driving current gear; When it is determined that no convergence has occurred, the flash and the image sensor are controlled to perform the next flash photography based on the calibrated photography parameters, the flash parameters and the exposure parameters until the calibration parameters corresponding to each driving current level are determined, thereby obtaining the calibration results corresponding to each driving current level.

8. The method according to claim 7, characterized in that The method further comprises: When the number of photographing times reaches a preset threshold of the number of photographing times and the algorithm result corresponding to the driving current gear has not converged, the calibration result corresponding to the driving current gear is determined as calibration failure.

9. The method according to claim 7 or 8, characterized in that: The method of controlling the flashlight and the image sensor to perform flash photography based on the flash parameters to obtain an image of each flash photography comprises: Sending the calibration photographing parameters corresponding to the flash calibration instruction to the framework layer of the terminal through the preset tool mode application in the application processor; Sending the calibration photographing parameters to a data transmission pipeline in a hardware abstraction layer of the terminal through the framework layer; Sending the calibration photographing parameters to the microprocessing unit through the data transmission pipeline; Based on the calibrated photographing parameters and the flash parameters, the flash and the image sensor are controlled to perform flash photographing to obtain an image of each flash photographing.

10. The method according to any one of claims 1-3, 5, 7, and 8, characterized in that: The method further comprises: Sending the at least one calibration result to the kernel layer of the terminal through the image signal processing module; Sending the at least one calibration result to the hardware abstraction layer of the terminal through the kernel layer in an inter-core communication manner; The at least one calibration result is written into a preset engineering file through a second function node in the hardware abstraction layer.

11. The method according to claim 10, characterized in that The method further comprises: Parsing the at least one calibration result through the hardware abstraction layer to determine at least one reporting data corresponding to the at least one driving current gear; The at least one reporting data is sent to the application processor.

12. The method according to claim 7, characterized in that The method of controlling the flashlight and the image sensor to perform flash photography based on the flash parameters to obtain an image of each flash photography comprises: Based on the flash parameters, the flash is controlled to flash, and imaging is performed through the image sensor. During the imaging process, the original image is obtained through the imaging front-end module, and the imaging mid-end module and / or back-end module are not processed for the original image. The original image is used as the image for each flash photo.

13. The method according to claim 6, characterized in that The preset calibration algorithm is used to analyze and process the statistical data corresponding to each image taken with flash, and obtain the calibration result corresponding to each driving current gear, including: When performing image analysis and processing on the statistical data corresponding to the image taken each time the flash is photographed using the preset calibration algorithm, the analysis and processing process of at least one preset algorithm function in the preset calibration algorithm is skipped to obtain the calibration result corresponding to each driving current gear.

14. The method according to claim 12, characterized in that The method further comprises: Sending at least one original image corresponding to each driving current gear to the application processor; By means of the application processor, downsampling processing and format conversion are performed on at least one original image corresponding to each driving current gear, so as to determine at least one preview image corresponding to each driving current gear; The at least one preview image is sent to a display module of the terminal, and the at least one preview image corresponding to each driving current gear is displayed through the display module.

15. The method according to claim 7 or 12, characterized in that: The method further comprises: Detecting at least one of an image output time and an inter-frame blank time of the image sensor by the image signal processing module; The image output frame rate of the image sensor is adjusted according to at least one of the image output time and the inter-frame blank time.

16. The method according to claim 8 or 11, characterized in that: The method further comprises: When the calibration result corresponding to the driving current gear is a calibration failure, determining the failure cause information through the preset calibration algorithm; The failure reason information corresponding to the driving current gear is sent to the application processor.

17. A flashlight calibration device, characterized in that: Applied to a terminal, the terminal includes an application subsystem and a camera subsystem; the camera subsystem includes a microprocessing unit; the device includes: An instruction sending module, used for sending a flashlight calibration instruction to the microprocessing unit through an application processor in the application subsystem in an inter-core communication manner; A reading module, used for reading at least one driving current gear when the microprocessing unit receives the flash lamp calibration instruction; A calibration module is used to call the image signal processing module to perform flash photography based on at least one driving current gear, and analyze and process the statistical data corresponding to the image of each flash photography through a preset calibration algorithm in the image signal processing module to determine at least one calibration result corresponding to the at least one driving current gear; the statistical data is obtained by performing image information statistics on the image of each flash photography through the statistical module of the camera subsystem.

18. A terminal, characterized in that: include: A memory and a processor; wherein, The memory is used to store executable instructions; The processor is configured to implement the method according to any one of claims 1 to 16 when executing the executable instructions stored in the memory.

19. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method according to any one of claims 1 to 16.

Images