Photographing method and apparatus, electronic device, and medium

Abstract

The application discloses a photographing method and device, electronic equipment and medium, and belongs to the technical field of communication. The electronic equipment comprises an image sensor, a front image signal processor (ISP) and an application processor, the image sensor is connected with the front ISP, and the front ISP is connected with the application processor. The method comprises the following steps: collecting original image data through the image sensor in the case that a photographing instruction is received; performing first image processing on the original image data by using an AI image processing algorithm through the front ISP to obtain first image data; and performing second image processing on the first image data through the application processor, and outputting target image data.

Description

Technical Field

[0001] This application belongs to the field of communication technology, specifically relating to a shooting method, device, electronic device, and medium. Background Technology

[0002] With the widespread use of mobile phones and other electronic devices and their convenience in photography, users are increasingly relying on them for daily shooting, leading to a growing demand for advanced shooting functions. The shooting quality and experience of electronic devices are closely related to the processing power of their image systems; therefore, improving these processing capabilities is of paramount importance.

[0003] Currently, electronic devices are primarily developed based on application processors (APs), and imaging systems rely on artificial intelligence (AI) image processing algorithms to improve the post-capture image processing results. However, limitations in the computing power and system power consumption of the AP platform itself prevent the deployment of some computationally demanding AI image processing algorithms, ultimately affecting the overall image processing quality. Summary of the Invention

[0004] The purpose of this application is to provide a shooting method, apparatus, electronic device, and medium that can solve the problem of poor overall processing effect of existing shooting processing schemes based on application processors (APs).

[0005] In a first aspect, embodiments of this application provide a shooting method executed by an electronic device, the electronic device including an image sensor, a front-facing image signal processor (ISP), and an application processor, wherein the image sensor is connected to the ISP, and the ISP is connected to the application processor; the method includes:

[0006] Upon receiving a shooting instruction, raw image data is acquired through the image sensor;

[0007] The first image data is obtained by using an artificial intelligence (AI) image processing algorithm on the original image data through the front-end ISP.

[0008] The application processor performs second image processing on the first image data to output target image data.

[0009] Secondly, embodiments of this application provide an electronic device, including: N image sensors, a selection module, a front-end image signal processor (ISP), and an application processor, wherein the image sensors are connected to the front-end ISP, and the front-end ISP is connected to the application processor;

[0010] The front-end ISP is equipped with an ASIC chip and a MIPI switching bus network; the N first input terminals of the selection module are connected one-to-one with the N image sensors, the second terminal of the selection module is connected to the MIPI switching bus network, the MIPI switching bus network is connected to the application processor; the MIPI switching bus network is also communicatively connected to the ASIC chip.

[0011] The MIPI switching bus network is used to determine whether to transmit the raw image data to the ASIC chip for processing based on the shooting scene and shooting parameters.

[0012] Thirdly, embodiments of this application provide a shooting device, installed in an electronic device, including an image sensor, a front-facing image signal processor (ISP), and an application processor. The image sensor is connected to the ISP, and the ISP is also connected to the application processor.

[0013] The image sensor is used to acquire raw image data upon receiving a shooting command;

[0014] The front-end ISP is used to perform a first image processing on the original image data using an AI image processing algorithm to obtain the first image data.

[0015] The application processor is used to perform second image processing on the first image data and output target image data.

[0016] Fourthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the shooting method as described in the first aspect.

[0017] Fifthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the shooting method as described in the first aspect.

[0018] In a sixth aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the shooting method as described in the first aspect.

[0019] In a seventh aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the shooting method as described in the first aspect.

[0020] In this embodiment, the electronic device includes an image sensor, a front-end image signal processor (ISP), and an application processor. The image sensor is connected to the ISP, and the ISP is also connected to the application processor. Upon receiving a shooting command, the image sensor acquires raw image data. The ISP performs a first image processing on the raw image data using an AI image processing algorithm to obtain first image data. The application processor then performs a second image processing on the first image data to output target image data. Thus, by using an external ISP to deploy AI image processing algorithms, the initial processing of the captured image data can be accelerated. Further image processing of the image data after initial ISP processing by the application processor effectively improves image capture quality and image processing performance. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the multimedia processing system of the electronic device provided in the embodiments of this application;

[0022] Figure 2 This is one of the flowcharts of the shooting method provided in the embodiments of this application;

[0023] Figure 3 This is the second flowchart of the shooting method provided in the embodiments of this application;

[0024] Figure 4 This is a framework structure diagram of the high-performance dedicated multimedia chip for the electronic device provided in the embodiments of this application;

[0025] Figure 5 This is a structural diagram of a multimedia chip with a multi-layer MIPI switching bus network provided in the embodiments of this application.

[0026] Figure 6 This is a structural diagram of the electronic device provided in the embodiments of this application;

[0027] Figure 7 This is a hardware structure diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0029] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0030] To make the embodiments of this application clearer, the relevant technical knowledge involved in the embodiments of this application will be further described in detail below, and the purpose and improvement points of the embodiments of this application will be briefly introduced, as follows:

[0031] AP: Application Processor;

[0032] ISP: Image Signal Processor;

[0033] PRE_ISP: Front-end ISP processor, which processes image data output by the camera, i.e., the image sensor, in front of the AP multimedia processing unit;

[0034] ORB: Oriented Fast and Rotated BRIEF, is a commonly used image feature detection algorithm. The ORB algorithm consists of two parts: feature point extraction and feature point description.

[0035] NPU: Neural Processing Unit.

[0036] Currently, multimedia systems, especially video / image capture functions, are crucial for electronic devices such as smartphones, tablets, and in-vehicle displays. The quality of the camera function—the effect and user experience of captured photos—is closely related to the capabilities of the electronic device's multimedia system. This is especially true given the increasing computational demands of current AI algorithms; therefore, improving the processing power of multimedia systems is currently key.

[0037] Current electronic devices are developed based on application processors (APs). The rapid development of AI processing algorithms has significantly improved the image and video processing capabilities of mobile phones, especially in image detail processing. AI algorithms offer more intelligent processing results compared to traditional computer vision (CV) algorithms, and their results are more in line with human processing habits and requirements. However, AI processing algorithms place higher demands on the computing power and performance of mobile phone processors. Currently, limitations in the computing power and system power consumption of the AP platform itself prevent the deployment of some high-performance image processing AI algorithms, leading to a decline in overall image processing quality and a lower image success rate. Furthermore, due to the limitations of shared chip memory and bus resources among multiple processing units on the AP platform, AI algorithms take longer to run on the AP platform under heavy user loads, resulting in greater overall photo-taking latency and impacting user experience.

[0038] To address the technical problems existing in the prior art, this application proposes a system solution that involves attaching a high-performance dedicated front-end ISP chip to the application processor (AP) side, such as... Figure 1 As shown, in terms of system architecture, this solution adopts a front-end PRE architecture, meaning that the external multimedia chip, i.e., the front-end PRE_ISP chip, is located in front of the AP, between the image sensor and the AP, and processes image data in the raw RAW domain. Specifically, image AI processing algorithms and image preprocessing algorithms (such as image frame selection and ORB fast registration preprocessing algorithms) can be superimposed on the front-end ISP chip to improve shooting effects. In terms of system interface, this embodiment can use a flexible multi-layer Mobile Industry Processor Interface (MIPI) switching bus network to realize the switching of working modes and timing control of the external AI acceleration chip in different scenarios. This embodiment also implements the image processing enhancement algorithm in the multimedia chip using a dedicated ASIC circuit to reduce processing latency and improve system performance.

[0039] The following description, in conjunction with the accompanying drawings, details the shooting method and multimedia chip architecture of the electronic device provided in this application through specific embodiments and application scenarios.

[0040] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the structure of the multimedia processing system of the electronic device provided in the embodiments of this application. Figure 2 This is a flowchart illustrating a shooting method provided in an embodiment of this application. The method is executed by an electronic device, such as... Figure 1As shown, the electronic device includes an image sensor 101, a front-facing image signal processor (ISP) 102, and an application processor 103. The image sensor 101 is connected to the ISP 102, and the ISP 102 is also connected to the application processor 103. Figure 2 As shown, the method includes the following steps:

[0041] Step 201: Upon receiving a shooting instruction, acquire raw image data through the image sensor 101.

[0042] In this embodiment, the image sensor 101 can be understood as the camera activated during shooting; the front-facing ISP 102 is an external image processing chip; and the application processor 103 is an AP chip.

[0043] Specifically, upon receiving a shooting command, the camera, i.e., the image sensor 101, enters the working state and acquires image data. Specifically, it can acquire multiple frames of image data, which are then further processed by the front-end ISP 102 for frame selection, registration, and even image compositing, ultimately forming image data with enhanced shooting effects. The shooting command can be a photo capture command or a video recording command. It should be noted that the image data acquired by the image sensor 101 is the raw image data, i.e., RAW domain data without any image processing.

[0044] In addition, the raw image data acquired by the image sensor 101 can be transmitted to the front-end ISP 102 through a relevant data interface such as the MIPI interface.

[0045] Step 202: The original image data is processed by the pre-amplifier ISP102 using an artificial intelligence (AI) image processing algorithm to obtain the first image data.

[0046] In this embodiment of the application, in order to accelerate the real-time processing of captured image data, a portion of the AI ​​image processing algorithms can be deployed on the front-end ISP 102 to reduce the burden on the application processor 103 while improving the image processing capability of the captured image data and increasing processing efficiency.

[0047] Specifically, in this step, after receiving the raw image data transmitted from the image sensor 101, the front-end ISP 102 can use the AI ​​image processing algorithm deployed on it to perform first image processing on the raw image data, that is, preliminary image data, and obtain first image data after processing. The first image processing can be such as RAW domain noise reduction, dynamic range enhancement such as HDR processing, image synthesis, etc.

[0048] Step 203: Perform second image processing on the first image data through the application processor 103 to output target image data.

[0049] Specifically, additional image processing algorithms, such as image synthesis, image quality enhancement, and filter AI image processing algorithms, can be superimposed on the AP platform, i.e., the application processor 103. In specific applications, the corresponding AI image processing algorithms can be used to process the captured image data in combination with the actual shooting scenario.

[0050] In this step, after receiving the pre-processed image data output by the front-end ISP 102, the application processor 103 can apply the corresponding AI image processing algorithm to perform further image processing on the processed image data, such as image quality enhancement, image compositing, special filters, etc.

[0051] After the application processor 103 performs the second image processing, it outputs the final processed target image data, which can be sent to the display screen 104 for display, so that the user can preview the shooting effect after image processing.

[0052] In some embodiments, prior to step 202, the method further includes:

[0053] The pre-processor ISP102 performs frame selection processing on the original image data using a preset frame selection strategy, and uses an image feature detection algorithm to perform registration processing on the selected image frames.

[0054] Step 202 includes:

[0055] The registered image data is processed by the pre-amplifier ISP102 to obtain the first image data. The first image processing includes at least one of noise reduction, dynamic range enhancement, depigmentation, and image compositing.

[0056] Step 203 includes:

[0057] The application processor 103 performs second image processing on the first image data and outputs the target image data.

[0058] In some embodiments, preprocessing algorithms such as frame selection and registration, as well as AI image processing algorithms other than the aforementioned preprocessing algorithms such as frame selection and registration, can be superimposed on the front-end ISP 102, such as one or more of the following algorithms: AI noise reduction, High Dynamic Range Imaging (HDR), AI de-mosaic, and AI image synthesis. Correspondingly, other image processing algorithms, such as AI image processing algorithms for image quality enhancement, blurring, and brightening, can be superimposed on the AP platform, i.e., the application processor 103.

[0059] Therefore, the registered image data can be processed by the front-end ISP 102 to perform the first image processing, namely the first step of image enhancement processing, such as noise reduction, HDR, de-mosaic, and image compositing, to obtain the first image data. The first image data is then transmitted to the application processor 103 for the second image processing, namely the second step of image enhancement processing, such as image compositing, image quality enhancement, blurring, and brightening, to obtain the final target image data.

[0060] This implementation method can improve the overall computing power and processing performance of the system by overlaying image preprocessing and image AI enhancement algorithms on the front-facing ISP102, thereby improving the image processing effect and shooting efficiency during shooting.

[0061] Optionally, such as Figure 4 As shown, the front-end ISP 102 includes an application-specific integrated circuit (ASIC) chip 110; the ASIC chip 110 includes a pre-processing unit 111 and a processing unit 112, the image sensor 101 is connected to the pre-processing unit 111, the pre-processing unit 111 is connected to the processing unit 112, and the processing unit 112 is connected to the application processor 103.

[0062] The first image data is obtained by performing a first image processing on the original image data using an AI image processing algorithm through the front-end ISP102, including:

[0063] The pre-processing unit 111 performs frame selection processing on the original image data using the preset frame selection strategy, and uses an image feature detection algorithm to perform registration processing on the selected image frames.

[0064] The image enhancement processing unit 112 performs image enhancement processing on the registered image data to obtain the first image data, wherein the image enhancement processing includes at least one of noise reduction, dynamic range enhancement, de-mosaic, and image compositing.

[0065] In some embodiments, such as Figure 4As shown, the front-facing ISP 102 can be implemented using a dedicated ASIC chip 110. The ASIC chip 110 integrates a pre-processing unit 111 and a processing unit 112. The pre-processing unit 111 is implemented using dedicated ASIC acceleration hardware circuitry, while the processing unit 112 is implemented using a dedicated NPU acceleration unit. Both are implemented using dedicated hardware circuitry, which accelerates the processing of captured images. The pre-processing unit 111 can also be called the image acceleration pre-processing unit, and the processing unit 112 can also be called a dedicated AI accelerator.

[0066] Specifically, the pre-processing unit 111 can perform frame selection processing on the original image data according to a preset frame selection strategy, and registration processing on the selected image frames can be performed using image feature detection algorithms such as ORB. The accelerated processing unit 112 can perform image enhancement processing on the registered image data, such as AI noise reduction, AI dynamic range enhancement, AI de-mosaicing, and AI image compositing. The processed data is then output to the application processor 103 for a second image enhancement process, such as image quality enhancement, to obtain the final target image data.

[0067] The preset frame selection strategy provides a strategy for selecting a reference frame and capturing frames, so that the original image data can be processed by selecting frames according to the preset frame selection strategy, so as to select a number of image frames from the multiple original image frames captured by the camera for subsequent processing, such as selecting the 1st, 3rd and 5th frames according to the preset frame selection strategy.

[0068] The above-mentioned registration process of selected image frames using algorithms such as ORB can be achieved by using image feature detection algorithms such as ORB to perform image registration processing on several selected image frames. Specifically, the selected image frames can be aligned in spatial position so that the same features in these images are in the same position as much as possible.

[0069] It should be noted that the above frame selection and registration processing can be collectively referred to as preprocessing. In specific implementation, frame selection, registration and other preprocessing algorithms can be superimposed on the shooting preprocessing unit 111 of the front ISP 102. Thus, the shooting preprocessing unit 111 can perform frame selection and registration and other preprocessing on the raw image data acquired by the image sensor 101, thereby reducing the image processing pressure on the application processor 103, i.e., the AP platform.

[0070] In addition, the image data preprocessed by the pre-ISP 102 can be transmitted to the application processor 103 for further image enhancement processing through relevant data interfaces such as the MIPI interface.

[0071] It should be noted that in some embodiments, such as Figure 4As shown, the front-end ISP102 also includes a storage unit 113, which may be an integrated high-performance on-chip dedicated dynamic random access memory (DRAM) storage unit, or a high-performance double data rate synchronous dynamic random access memory (DDR) storage unit.

[0072] The pre-processing unit 111 is connected to the storage unit 113, and the storage unit 113 is also connected to the shooting acceleration processing unit 112.

[0073] Specifically, the image data processed by the pre-processing unit 111 can be written into the storage unit 113 for caching, and the image acceleration processing unit 112 can access the storage unit 113 when needed to read the image data from the storage unit 113 for further processing.

[0074] This enables an in-memory computing architecture based on a front-end ISP chip. By integrating a high-performance on-chip dedicated memory unit on the dedicated ASIC chip 110, the latency of AI accelerator reading and writing memory access can be reduced, thereby improving AI processing performance.

[0075] Optionally, the step of performing frame selection processing on the original image data using a preset frame selection strategy includes:

[0076] Determine the reference frame in the original image data;

[0077] Using the reference frame as a reference, M image frames are captured forward and / or backward from the original image data, wherein the selected image frames include the reference frame and the M image frames, and M is an integer greater than 1.

[0078] Specifically, the preset frame selection strategy may include a set number of frames to capture, a preset reference frame selection method, and a frame capture strategy that captures image frames forward and / or backward based on the reference frame. For example, the preset frame selection strategy stipulates that the first frame is the reference frame, a total of 3 frames are captured, and 3 frames are captured forward from the reference frame. Thus, the reference frame in the original image data can be determined according to the preset frame selection strategy, and several frames can be captured forward / backward from the reference frame according to the frame capture strategy. It should be noted that when forward frame capture is used, the front-facing ISP chip can continuously capture image data output by the camera and store historically acquired image data. When selecting frames, several frames can be captured from the historically acquired image data based on the current reference frame.

[0079] It should be noted that the above-mentioned frame capture strategy based on capturing image frames forward and / or backward from the reference frame can be specifically implemented in some embodiments as follows: capturing image frames forward from the reference frame, such as capturing 3 frames forward from the reference frame; in other embodiments, capturing image frames backward from the reference frame, such as capturing 3 frames backward from the reference frame; and in still other embodiments, capturing image frames forward and backward from the reference frame can be combined, that is, these two frame capture strategies can be combined, such as capturing 3 frames forward and backward from the reference frame respectively.

[0080] This implementation method enables frame selection processing using a preset frame selection strategy, ensuring the effectiveness of image enhancement processing based on the selected frames.

[0081] The following is combined Figure 3 The illustrated embodiment is used to describe one implementation method of this application:

[0082] This implementation proposes a system solution that integrates a high-performance front-end ISP multimedia photography AI acceleration chip with the AP. In terms of system architecture, this solution adopts a front-end processing architecture, meaning the external multimedia chip is placed before the AP and performs AI processing on the image data in the RAW domain. Figure 3 As shown, the specific steps include the following:

[0083] 301. The camera acquires image / video data according to the configured frame rate and outputs it to the PRE_ISP external chip through a high-performance MIPI interface;

[0084] 302. After receiving the image RAW data, the dedicated front-end ISP multimedia chip performs frame selection processing according to the preset frame selection strategy (including the number of frames captured and forward / backward selection based on the reference frame).

[0085] 303. For the selected data frames, perform image registration processing using the ORB algorithm;

[0086] 304. Output the processing results to a dedicated AI acceleration hardware unit for further processing, such as RAW domain noise reduction, HDR processing, and image synthesis.

[0087] 305. After completing the post-processing of the image, the dedicated front-end ISP multimedia chip transmits the image data and feature images back to the AP through different MIPIs.

[0088] 306. After receiving the image data output by the dedicated front-end ISP multimedia chip, the AP platform overlays the corresponding image processing algorithm, such as the image quality enhancement algorithm, and writes the image data into the camera storage unit.

[0089] This implementation method effectively improves the image capture effect and performance of electronic devices, especially mobile phones, by using an external high-performance dedicated AI acceleration chip, namely the front-end ISP chip, and deploying a high-performance AI image capture acceleration algorithm.

[0090] Optionally, before the pre-processing unit 111 performs frame selection processing on the original image data using a preset frame selection strategy, the method further includes:

[0091] The original image data is subjected to pixel correction processing by the pre-processing unit 111, wherein the pixel correction includes at least one of bad pixel correction and black level correction;

[0092] The step of performing frame selection processing on the original image data by the shooting preprocessing unit 111 using a preset frame selection strategy includes:

[0093] The pre-processing unit 111 performs frame selection processing on the corrected original image data using the preset frame selection strategy.

[0094] In some embodiments, the camera preprocessing unit 111 can first perform pixel correction processing on the raw image data acquired by the camera, such as performing bad pixel correction and black level correction, so as to obtain complete raw image data without bad pixels, and then perform frame selection, registration and other processing on the corrected image data.

[0095] Specifically, Defective Pixel Correction (DPC) primarily addresses pixel defects caused by manufacturing processes. These defective pixels differ significantly from surrounding pixels, potentially appearing as white spots in complete darkness or black spots in bright light. Defective pixels are categorized into static and dynamic defects; the former remains constant, while the latter varies depending on factors such as exposure and temperature. The DPC process involves two steps: defect detection and correction. Detection typically involves comparing the difference between the center pixel and its neighboring pixels; if the difference exceeds a preset threshold, it is identified as a defective pixel. Correction then uses methods such as interpolation replacement to repair these defective pixels.

[0096] Black Level Correction (BLC) addresses image distortion in dark areas caused by insufficient sensor accuracy or thermal radiation from the circuitry. Even when the lens module is completely blocked and no light enters, the sensor may still display non-zero voltage data. This data does not represent actual image information but is caused by electrons excited by sensor inaccuracies or thermal radiation from the circuitry. Black Level Correction corrects these inaccurate dark area data, ensuring the accuracy of image dark areas and thus improving image quality.

[0097] The following is combined Figure 4 The high-performance dedicated multimedia chip framework shown is used to illustrate Embodiment 2 of this application:

[0098] This implementation proposes a scheme to implement image AI processing using hardware circuitry within a dedicated multimedia chip, reducing AI algorithm processing latency and power consumption. The main process is as follows:

[0099] 401. The camera sensor transmits the image RAW domain data to a dedicated front-end ISP multimedia chip via the MIPI interface; among which, Figure 4 Both the first and second interfaces are MIPI interfaces.

[0100] 402. Dedicated front-end ISP multimedia chip, which performs image preprocessing through dedicated ASIC acceleration hardware circuitry, including a front-end basic ISP hardware circuit unit, to complete dead pixel correction, black level correction, etc.

[0101] 403. Dedicated front-end ISP multimedia chip, which performs AI noise reduction, AI de-mosaicing, and AI dynamic range enhancement on images through a dedicated NPU acceleration unit;

[0102] 404. Dedicated front-end ISP multimedia chip, by integrating a high-performance on-chip dedicated DRAM memory unit, reduces the latency of AI accelerator reading and writing memory access, and improves AI processing performance;

[0103] 405. The dedicated multimedia chip transmits the processed data back to the AP via the MIPI interface. Among other things, Figure 4 The third and fourth interfaces are both MIPI interfaces.

[0104] In this implementation, by employing dedicated ASIC circuits, AI accelerators, and DDR memory, it becomes possible to deploy high-performance AI models on processors of electronic devices such as mobile phones, effectively improving the energy efficiency and latency performance of electronic devices, especially mobile phone multimedia processing.

[0105] Optionally, such as Figure 5 As shown, the front-end ISP 102 is equipped with an ASIC chip 110 and a MIPI switching bus network 120; the number of image sensors 101 is N; the electronic device also includes a selection module 105, the N first terminals of the selection module 105 are connected to the N image sensors 101 one by one, the second terminal of the selection module 105 is connected to the MIPI switching bus network 120, the MIPI switching bus network 120 is connected to the application processor 103; the MIPI switching bus network 120 is also communicatively connected to the ASIC chip 110.

[0106] Before performing the first image processing on the original image data using an AI image processing algorithm via the pre-amplifier ISP102, the method further includes:

[0107] Based on the shooting scene, the control selection module 105 selects at least one of the N image sensors 101 as the target image sensor;

[0108] Raw image data is acquired using the target image sensor;

[0109] When the ASIC chip 110 is activated based on the shooting scene and shooting parameters, the original image data is transmitted to the ASIC chip 110 via the MIPI switching bus network 120.

[0110] The first image processing step, which involves using an AI image processing algorithm on the pre-amplifier ISP102 to process the original image data, includes:

[0111] The original image data is processed by using an AI image processing algorithm through an ASIC chip 110.

[0112] Specifically, in some embodiments, a flexible multi-layer MIPI switching bus network is proposed to realize the switching of working modes and timing control of external AI acceleration chips in different scenarios. This allows for flexible function switching in different scenarios, achieving the optimal combination of overall camera performance, power consumption, and effect. The main system block diagram is as follows: Figure 5 As shown.

[0113] The electronic device may include multiple image sensors 101, i.e., equipped with multiple camera sensors, such as Sensor0, Sensor1, Sensor2, and Sensor3, specifically a front-facing camera, a rear-facing camera, an infrared camera, and a rear ultra-wide-angle camera. The multiple sensors are controlled to be enabled or disabled by a selection module 105. Specifically, the target sensor to be enabled can be flexibly selected according to the actual usage scenario requirements. For example, two sensors can be flexibly selected as inputs to a photography AI acceleration external chip, such as selecting the rear main camera and the rear ultra-wide-angle camera.

[0114] The MIPI switching bus network 120 supports physical layer pass-through and dynamic switching functions, meaning it can directly pass raw image data to the application processor 103, and also switch raw image data between the ASIC chip 110 and the application processor 103. The camera system can select whether to enable the external AI acceleration chip for image processing, i.e., whether to enable the ASIC chip 110 to process the raw image data, based on shooting parameters such as the shooting scene and the focal length used by the activated camera. When enabled, the MIPI switching bus network 120 sends the raw image data to the ASIC chip 110 for processing, such as frame selection and registration.

[0115] The MIPI switching bus network 120 also supports data link layer bypass and dynamic switching functions. The AI ​​acceleration add-on chip can continuously capture image data output by the sensor and perform forward frame capture processing based on the reference frame to improve the photo-taking effect. At the same time, the image data is output to the AP for preview.

[0116] The MIPI switching bus network 120 also supports the use of a dedicated MIPI channel and supports custom data frame packaging. The output ghost image and meta information, i.e. intermediate processing results, are sent to the AP for processing, which can ensure high transmission efficiency.

[0117] Optionally, the ASIC chip 110 is in a sleep state when it is not turned on.

[0118] In some embodiments, when the ASIC chip 110 is not turned on, that is, when the AI ​​acceleration add-on chip is not turned on, the ASIC chip 110 can be put into a sleep state. Specifically, the shooting preprocessing unit 111 and the shooting acceleration processing unit 112 of the add-on ASIC chip 110 are in a power-off state, thereby achieving the purpose of reducing system power consumption.

[0119] Optionally, the method further includes:

[0120] If the ASIC chip 110 is not turned on based on the shooting scene and shooting parameters, the raw image data is transmitted to the application processor 103 through the MIPI switching bus network 120, and the application processor 103 processes the raw image data.

[0121] When the ASIC chip 110 is not enabled based on the shooting scene and the shooting parameters such as the focal length used by the enabled camera, that is, when the external chip's shooting function is not enabled, the MIPI switching bus network 120 can directly output the image data output by the sensor to the application processor 103, i.e., AP, through the physical layer bypass interface, and the AP will complete the image processing.

[0122] In this way, the flexible multi-layer MIPI switching bus network can realize the switching of working modes and timing control of external AI acceleration chips, thereby flexibly realizing the switching of functions in different scenarios and achieving the overall optimization of camera performance, power consumption and effect.

[0123] Optionally, when the ASIC chip 110 is activated based on the shooting scene and shooting parameters, transmitting the original image data to the ASIC chip 110 via the MIPI switching bus network 120 includes:

[0124] If the ASIC chip 110 is not currently enabled, but it is determined that the ASIC chip 110 needs to be enabled based on the shooting scene and shooting parameters, the target image sensor is kept in the startup state, and the original image data is transmitted to the ASIC chip 110 through the MIPI switching bus network 120.

[0125] In other words, the external chip system can support seamless switching. When the next frame of the image needs to activate the AI ​​acceleration external chip, the target image sensor can remain powered on without turning off the camera, thus achieving a seamless switch to the AI ​​acceleration external chip for image processing.

[0126] The following is combined Figure 5 The multimedia dedicated chip system framework shown is equipped with a multi-layer MIPI switching bus network, which is used to describe Embodiment 3 of this application:

[0127] This implementation proposes a flexible multi-layer MIPI switching bus network to enable switching of the working mode and timing control of external AI acceleration chips in different scenarios. This allows for flexible function switching in different scenarios, resulting in the optimal combination of overall camera performance, power consumption, and effect. The main details are as follows:

[0128] 1. The system hardware circuit design can flexibly select two sensors as inputs for the photography AI acceleration external chip according to the specific product usage scenario requirements, such as selecting the rear main camera and the rear ultra-wide camera;

[0129] 2. The external chip's internal MIPI switching network supports physical layer bypass and dynamic switching functions. The camera system can choose whether to enable the AI ​​acceleration external chip for photography based on the scene, focal length, etc. When the chip's photography function is not enabled, the image data output by the sensor is output to the AP through the physical layer bypass interface, and the AP completes the image processing. Furthermore, the system supports seamless switching, meaning that when the next frame of the image requires the AI ​​acceleration external chip to be enabled, the camera does not need to be turned off, and the system seamlessly switches to the AI ​​acceleration external chip for image processing. At the same time, when the AI ​​acceleration external chip is not enabled, the image preprocessing and AI acceleration processing units of the external chip are in a powered-off state, resulting in the lowest system power consumption.

[0130] 3. The external chip's internal MIPI switching network supports data link layer bypass and dynamic switching functions. The AI-accelerated external chip can continuously capture image data output by the sensor and perform forward frame capture processing based on the reference frame to improve the shooting effect. At the same time, the image data is output to the AP for preview.

[0131] 4. The external chip's internal MIPI switching network supports the use of a dedicated MIPI channel and supports custom data frame packaging, sending the output ghost image and meta information to the AP for processing.

[0132] In this implementation, we adopt a flexible multi-layer MIPI switching bus network for the system interface to enable switching of the working mode and timing control of the external AI acceleration chip in different scenarios, so as to flexibly achieve the optimal combination of overall camera performance, power consumption and effect by switching functions in different scenarios.

[0133] It should be noted that in practical applications, more hardware accelerators, DSPs and other dedicated processor units can be added on top of the PRE architecture to improve the system's processing flexibility.

[0134] The shooting method in this embodiment is executed by an electronic device, which includes an image sensor, a front-end image signal processor (ISP), and an application processor. The image sensor is connected to the ISP, and the ISP is also connected to the application processor. Upon receiving a shooting command, the image sensor acquires raw image data. The ISP then performs a first image processing on the raw image data using an AI image processing algorithm to obtain first image data. The application processor then performs a second image processing on the first image data to output target image data. By using an external ISP to deploy AI image processing algorithms, the initial processing of the captured image data can be accelerated. Further image processing of the image data after initial ISP processing by the application processor effectively improves image capture quality and image processing performance.

[0135] This application primarily addresses image capture acceleration. Specifically, it utilizes a high-performance NPU hardware circuit and AI image processing algorithms to synthesize multiple images into a single image, enhancing the image capture effect. Simultaneously, in the image preprocessing module, based on a reference frame, it can capture consecutive images forward or backward, and employs the ORB algorithm to perform registration processing on the captured images, reducing the impact of camera shake. Regarding the system interface, we adopt a flexible multi-layer MIPI switching bus network to enable switching of the working mode and timing control of external AI acceleration chips in different scenarios.

[0136] Please see Figure 5 , Figure 5 This is a structural diagram of an electronic device with a multi-layer MIPI switching bus network deployed in a multimedia chip, as provided in an embodiment of this application. Figure 5 As shown, the electronic device includes: N image sensors 101, a selection module 105, a front-end image signal processor ISP 102 and an application processor 103. The image sensors 101 are connected to the front-end ISP 102, and the front-end ISP 102 is also connected to the application processor 103.

[0137] The front-end ISP 102 is equipped with an ASIC chip 110 and a MIPI switching bus network 120; the N first terminals of the selection module 105 are connected to the N image sensors 101 one by one, the second terminal of the selection module 105 is connected to the MIPI switching bus network 120, the MIPI switching bus network 120 is connected to the application processor 103; the MIPI switching bus network 120 is also communicatively connected to the ASIC chip 110.

[0138] The MIPI switching bus network 120 is used to determine whether to transmit the raw image data to the ASIC chip 110 for processing based on the shooting scene and shooting parameters.

[0139] This embodiment serves as a structural embodiment of an electronic device corresponding to the aforementioned method embodiment. For a detailed description of this embodiment, please refer to the relevant descriptions in the aforementioned embodiments. This electronic device can achieve the same technical effects as the aforementioned embodiments. To avoid repetition, it will not be described again here.

[0140] The shooting method provided in this application can be executed by a shooting device. This application uses a shooting device executing the shooting method as an example to illustrate the shooting device provided in this application.

[0141] Please see Figure 1 , Figure 1This is a schematic diagram of the imaging device provided in an embodiment of this application. It is installed in an electronic device and includes an image sensor 101, a front-facing ISP 102, and an application processor 103. The image sensor 101 is connected to the front-facing ISP 102, and the front-facing ISP 102 is also connected to the application processor 103.

[0142] Image sensor 101 is used to acquire raw image data upon receiving a shooting command;

[0143] The front-end ISP102 is used to perform first image processing on the original image data using artificial intelligence (AI) image processing algorithms to obtain first image data;

[0144] Application processor 103 is used to perform image enhancement processing on the first image data and output target image data.

[0145] Optionally, such as Figure 4 As shown, the front-end ISP 102 includes an application-specific integrated circuit (ASIC) chip 110; the ASIC chip 110 includes a pre-processing unit 111 and a processing unit 112, the image sensor 101 is connected to the pre-processing unit 111, the pre-processing unit 111 is connected to the processing unit 112, and the processing unit 112 is connected to the application processor 103.

[0146] The pre-processing unit 111 is used to perform frame selection processing on the original image data using a preset frame selection strategy, and to perform registration processing on the selected image frames using an image feature detection algorithm.

[0147] The image acceleration processing unit 112 is used to perform image enhancement processing on the registered image data to obtain first image data, wherein the image enhancement processing includes at least one of noise reduction, dynamic range enhancement, de-mosaic, and image compositing.

[0148] Optionally, the pre-processing unit 111 is further configured to perform pixel correction processing on the original image data, wherein the pixel correction includes at least one of bad pixel correction and black level correction.

[0149] Optionally, the front-end ISP102 is used for:

[0150] Determine the reference frame in the original image data;

[0151] Using the reference frame as a reference, M image frames are captured forward and / or backward from the original image data, wherein the selected image frames include the reference frame and the M image frames, and M is an integer greater than 1.

[0152] Optionally, such as Figure 5As shown, the front-end ISP 102 is equipped with an ASIC chip 110 and a MIPI switching bus network 120; the number of image sensors 101 is N; the shooting device also includes a selection module 105, the N first terminals of the selection module 105 are connected to the N image sensors 101 one by one, the second terminal of the selection module 105 is connected to the MIPI switching bus network 120, the MIPI switching bus network 120 is connected to the application processor 103; the MIPI switching bus network 120 is also communicatively connected to the ASIC chip 110.

[0153] The imaging device also includes:

[0154] Application processor 103 is configured to control selection module 105 to select at least one of the N image sensors as the target image sensor according to the shooting scene;

[0155] The target image sensor is used to acquire raw image data;

[0156] MIPI switching bus network 120 is used to transmit the original image data to ASIC chip 110 when ASIC chip 110 is turned on based on the shooting scene and shooting parameters.

[0157] The ASIC chip 110 is used to perform first image processing on the original image data using an AI image processing algorithm.

[0158] Optionally, the shooting device further includes:

[0159] The MIPI switching bus network 120 is also used to transmit the raw image data to the application processor 103 when it is determined, based on the shooting scene and shooting parameters, that the ASIC chip 110 is not to be turned on.

[0160] Application processor 103 is used to process the raw image data.

[0161] Optionally, the ASIC chip 110 is in a sleep state when it is not turned on.

[0162] Optionally, if the ASIC chip is not currently enabled, but it is determined that the ASIC chip needs to be enabled based on the shooting scene and shooting parameters, the target image sensor remains enabled.

[0163] The shooting device is installed in an electronic device, including an image sensor, a front-end image signal processor (ISP), and an application processor. The image sensor is connected to the ISP, and the ISP is also connected to the application processor. The image sensor is used to acquire raw image data upon receiving a shooting command. The ISP uses an AI image processing algorithm to perform a first image processing on the raw image data to obtain first image data. The application processor performs a second image processing on the first image data to output target image data. Thus, by using an external ISP to deploy AI image processing algorithms, the initial processing of the captured image data can be accelerated through the ISP. Further image processing of the pre-processed image data by the application processor effectively improves image capture quality and image processing performance.

[0164] The shooting device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope.

[0165] The shooting device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0166] The imaging device provided in this application embodiment can achieve... Figure 2 or Figure 3 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0167] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described shooting method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0168] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0169] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0170] The electronic device 100 includes, but is not limited to, components such as: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710. The electronic device also includes an image sensor, a front-end image signal processor (ISP), and an application processor. The image sensor is connected to the ISP, and the ISP is connected to the application processor.

[0171] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0172] The image sensor is used to acquire raw image data upon receiving a shooting command;

[0173] The front-end ISP is used to perform first image processing on the original image data using an artificial intelligence (AI) image processing algorithm to obtain first image data.

[0174] The application processor is used to perform second image processing on the first image data and output target image data.

[0175] Optionally, the pre-processor ISP is configured to include an application-specific integrated circuit (ASIC) chip; the ASIC chip includes an image preprocessing unit and an image acceleration processing unit, the image sensor is connected to the image preprocessing unit, the image preprocessing unit is connected to the image acceleration processing unit, and the image acceleration processing unit is connected to the application processor;

[0176] The shooting preprocessing unit is used to perform first image processing on the original image data using an AI image processing algorithm to obtain first image data;

[0177] The shooting acceleration processing unit is further configured to perform image enhancement processing on the registered image data to obtain the first image data, wherein the image enhancement processing includes at least one of noise reduction, dynamic range enhancement, de-mosaic, and image compositing.

[0178] Optionally, the pre-processing unit is further configured to perform pixel correction processing on the original image data, wherein the pixel correction includes at least one of bad pixel correction and black level correction;

[0179] The shooting preprocessing unit is used to perform frame selection processing on the corrected original image data using the preset frame selection strategy.

[0180] Optionally, the front-end ISP is further configured to:

[0181] Determine the reference frame in the original image data;

[0182] Using the reference frame as a reference, M image frames are captured forward and / or backward from the original image data, wherein the selected image frames include the reference frame and the M image frames, and M is an integer greater than 1.

[0183] Optionally, the front-end ISP is equipped with an ASIC chip and a MIPI switching bus network; the number of image sensors is N, and the electronic device further includes a selection module, wherein the N first terminals of the selection module are connected to the N image sensors one by one, the second terminal of the selection module is connected to the MIPI switching bus network, the MIPI switching bus network is connected to the application processor, and the MIPI switching bus network is also communicatively connected to the ASIC chip;

[0184] The processor 710 is configured to control the selection module to select at least one of the N image sensors as the target image sensor according to the shooting scene;

[0185] The target image sensor is used to acquire raw image data;

[0186] The MIPI switching bus network is used to transmit the original image data to the ASIC chip when the ASIC chip is activated based on the shooting scene and shooting parameters.

[0187] The ASIC chip is used to perform first image processing on the original image data using an AI image processing algorithm.

[0188] Optionally, the MIPI switching bus network is further configured to transmit the raw image data to the application processor when it is determined, based on the shooting scene and shooting parameters, that the ASIC chip is not to be turned on, and to process the raw image data through the application processor.

[0189] Optionally, the ASIC chip is in a sleep state when it is not turned on.

[0190] Optionally, the processor 710 is further configured to keep the target image sensor in an active state when the ASIC chip is not currently enabled, but it is determined from the shooting scene and shooting parameters that the ASIC chip needs to be enabled.

[0191] The MIPI switching bus network is used to transmit the raw image data to the ASIC chip.

[0192] It should be understood that, in this embodiment, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0193] The memory 709 can be used to store software programs and various data. The memory 709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0194] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 710.

[0195] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described shooting method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0196] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0197] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described shooting method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0198] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0199] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described shooting method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0200] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0201] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0202] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A shooting method, characterized in that, Performed by an electronic device, the electronic device including an image sensor, a front-end image signal processor (ISP), and an application processor, wherein the image sensor is connected to the ISP and the ISP is also connected to the application processor; The front-end ISP is equipped with a dedicated integrated circuit (ASIC) chip and a mobile industry processor interface (MIPI) switching bus network. The method includes: Upon receiving a shooting instruction, raw image data is acquired through the target image sensor in the image sensor; If it is determined from the shooting scene and shooting parameters that the ASIC chip should not be turned on, the following operations are performed: The raw image data is transmitted to the application processor via the MIPI switching bus network. The original image data is processed by the application processor; If the ASIC chip is activated based on the shooting scene and shooting parameters, the following operations are performed: The original image data is transmitted to the ASIC chip via the MIPI switching bus network. The ASIC chip employs an artificial intelligence (AI) image processing algorithm to perform first image processing on the original image data to obtain first image data. The application processor performs second image processing on the first image data to output target image data. When the ASIC chip is activated based on the shooting scene and shooting parameters, the raw image data is transmitted to the ASIC chip via the MIPI switching bus network, including: If the ASIC chip is not currently enabled, but it is determined that the ASIC chip needs to be enabled based on the shooting scene and shooting parameters, the target image sensor is kept in the startup state, and the raw image data is transmitted to the ASIC chip through the MIPI switching bus network.

2. The method according to claim 1, characterized in that, The ASIC chip includes an image preprocessing unit and an image acceleration processing unit. The image sensor is connected to the image preprocessing unit, the image preprocessing unit is connected to the image acceleration processing unit, and the image acceleration processing unit is connected to the application processor. The first image data is obtained by performing a first image processing on the original image data using an AI image processing algorithm through the ASIC chip, including: The shooting preprocessing unit uses a preset frame selection strategy to perform frame selection processing on the original image data, and uses an image feature detection algorithm to perform registration processing on the selected image frames; The image enhancement processing unit performs image enhancement processing on the registered image data to obtain the first image data. The image enhancement processing includes at least one of noise reduction, dynamic range enhancement, depigmentation, and image compositing.

3. The method according to claim 2, characterized in that, Before the frame selection processing of the original image data by the shooting preprocessing unit using a preset frame selection strategy, the method further includes: The original image data is subjected to pixel correction processing by the shooting preprocessing unit, wherein the pixel correction includes at least one of bad pixel correction and black level correction; The step of performing frame selection processing on the original image data by the shooting preprocessing unit using a preset frame selection strategy includes: The shooting preprocessing unit uses the preset frame selection strategy to perform frame selection processing on the corrected original image data.

4. The method according to claim 2, characterized in that, The step of performing frame selection processing on the original image data using a preset frame selection strategy includes: Determine the reference frame in the original image data; Using the reference frame as a reference, M image frames are captured forward and / or backward from the original image data, wherein the selected image frames include the reference frame and the M image frames, and M is an integer greater than 1.

5. The method according to claim 1, characterized in that, The number of image sensors is N. The electronic device also includes a selection module. The N first terminals of the selection module are connected to the N image sensors one by one. The second terminal of the selection module is connected to the MIPI switching bus network. The MIPI switching bus network is connected to the application processor. The MIPI switching bus network is also communicatively connected to the ASIC chip. The method further includes: Based on the shooting scene, the selection module is controlled to select at least one of the N image sensors as the target image sensor.

6. An electronic device, characterized in that, include: N image sensors, a selection module, a front-end image signal processor (ISP), and an application processor, wherein the image sensors are connected to the front-end ISP, and the front-end ISP is also connected to the application processor; The front-end ISP is equipped with an ASIC chip and a MIPI switching bus network. The N first terminals of the selection module are connected one-to-one with the N image sensors, the second terminal of the selection module is connected to the MIPI switching bus network, and the MIPI switching bus network is connected to the application processor. The MIPI switching bus network is also communicatively connected to the ASIC chip. The MIPI switching bus network is used to determine whether to transmit the raw image data to the ASIC chip for processing based on the shooting scene and shooting parameters. The raw image data is acquired by the target image sensor among the N image sensors; If the electronic device determines, based on the shooting scene and shooting parameters, that the ASIC chip should not be turned on, it performs the following operations: The raw image data is transmitted to the application processor via the MIPI switching bus network. The original image data is processed by the application processor; When the electronic device determines to activate the ASIC chip based on the shooting scene and shooting parameters, it performs the following operations: The original image data is transmitted to the ASIC chip via the MIPI switching bus network. The ASIC chip employs an artificial intelligence (AI) image processing algorithm to perform first image processing on the original image data to obtain first image data. The application processor performs second image processing on the first image data to output target image data. When the ASIC chip is activated based on the shooting scene and shooting parameters, the raw image data is transmitted to the ASIC chip via the MIPI switching bus network, including: If the ASIC chip is not currently enabled, but it is determined that the ASIC chip needs to be enabled based on the shooting scene and shooting parameters, the target image sensor is kept in the startup state, and the raw image data is transmitted to the ASIC chip through the MIPI switching bus network.

7. A shooting device, characterized in that, An electronic device is provided, including an image sensor, a front-end image signal processor (ISP), and an application processor, wherein the image sensor is connected to the ISP, and the ISP is also connected to the application processor; The front-end ISP is equipped with a dedicated integrated circuit (ASIC) chip and a MIPI switching bus network; wherein... The target image sensor in the image sensor is used to acquire raw image data upon receiving a shooting command; If the ASIC chip is not activated based on the shooting scene and shooting parameters, the MIPI switching bus network is used to pass the raw image data through to the application processor; the application processor is used to process the raw image data. When the ASIC chip is activated based on the shooting scene and shooting parameters, the MIPI switching bus network is used to transmit the original image data to the ASIC chip; the ASIC chip is used to perform a first image processing on the original image data using an AI image processing algorithm to obtain first image data; the application processor is used to perform a second image processing on the first image data to output target image data. The step of transmitting the raw image data to the ASIC chip includes: when the ASIC chip is not currently enabled, but it is determined from the shooting scene and shooting parameters that the ASIC chip needs to be enabled, keeping the target image sensor in the startup state, and transmitting the raw image data to the ASIC chip through the MIPI switching bus network.

8. The shooting device according to claim 7, characterized in that, The number of image sensors is N. The electronic device also includes a selection module. The N first terminals of the selection module are connected to the N image sensors one by one. The second terminal of the selection module is connected to the MIPI switching bus network. The MIPI switching bus network is connected to the application processor. The MIPI switching bus network is also communicatively connected to the ASIC chip. The application processor is also configured to control the selection module to select at least one of the N image sensors as the target image sensor based on the shooting scene.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the shooting method as described in any one of claims 1 to 5.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the shooting method as described in any one of claims 1 to 5.

Images