Image processing method, chip, system chip, equipment, medium and chip system

Abstract

The invention relates to an image processing method, a chip, a system chip, equipment, a medium and a chip system, and relates to the technical field of image processing.The method comprises the steps that first image data shot by a camera are obtained, the first image data are preprocessed to obtain second image data, the second image data are stored in a first storage space, and the second image data are stored in a second storage space; and performing image enhancement processing on the third image data to obtain fourth image data, storing the fourth image data in the second storage space, and performing image post-processing on the fifth image data to obtain target image data. The image processing flow is decoupled into multiple sections, so that the image processing performance in each section of image processing flow is improved, and the image processing performance is integrally improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, chip, system chip, device, medium and chip system. Background Art

[0002] With the rapid development of image processing technology, image signal processors (ISPs) have gradually shifted from basic signal processing to intelligent, multi-core, and computational processing, becoming the core of mobile phone imaging systems. Current ISPs focus on multi-camera collaboration, AI enhancement, dynamic range enhancement, and true color reproduction. As camera processing performance requirements increase, ISPs are increasingly required to achieve real-time computational photography, high throughput, and high energy efficiency. Improving ISP processing performance is a pressing issue in this field.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides an image processing method, chip, system chip, device, medium and chip system.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0006] Acquire first image data captured by a camera;

[0007] preprocessing the first image data to obtain second image data;

[0008] storing the second image data in the first storage space;

[0009] performing image enhancement processing on the third image data to obtain fourth image data; the third image data is generated based on the second image data;

[0010] storing the fourth image data in the second storage space;

[0011] Image post-processing is performed on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0012] In one embodiment of the present disclosure, obtaining first image data captured by a camera includes:

[0013] A plurality of first image data captured by a plurality of cameras is acquired.

[0014] In one embodiment of the present disclosure, obtaining first image data captured by a camera includes:

[0015] The first image data is acquired through the camera and / or the third storage space; the first image data captured by the camera is stored in the third storage space.

[0016] In one embodiment of the present disclosure, obtaining first image data captured by a camera includes:

[0017] Acquire multiple first image data obtained by multiple exposure shooting with a camera.

[0018] In one embodiment of the present disclosure, the method further includes:

[0019] Calling a preset first image enhancement algorithm to perform image enhancement processing on the second image data to obtain third image data; and / or,

[0020] A preset second image enhancement algorithm is called to perform image enhancement processing on the fourth image data to obtain fifth image data.

[0021] In one embodiment of the present disclosure, the first image enhancement algorithm includes an image alignment algorithm.

[0022] In one embodiment of the present disclosure, the method further includes:

[0023] Performing image data statistics on the first image data to obtain image data statistical information; the image data statistical information includes: at least one of automatic white balance, automatic focus, and automatic exposure;

[0024] The camera's shooting parameters are controlled based on the image data statistics.

[0025] In one embodiment of the present disclosure, the fifth image data adopts a color gamut space of a RAW domain, an RGB domain, or a YUV domain.

[0026] In one embodiment of the present disclosure, performing image enhancement processing on the third image data to obtain fourth image data includes:

[0027] The plurality of third image data are subjected to image fusion processing to obtain fourth image data; the plurality of third image data correspond one-to-one to the plurality of first image data.

[0028] In one embodiment of the present disclosure, the second image data and / or the fourth image data is image data presented in an image pyramid.

[0029] In one embodiment of the present disclosure, preprocessing the first image data to obtain the second image data includes:

[0030] The first image data is subjected to at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference flow separation processing to obtain second image data.

[0031] In one embodiment of the present disclosure, performing image enhancement processing on the third image data to obtain fourth image data includes:

[0032] Perform at least one of tone mapping processing, gamma correction processing, demosaicing processing, RAW domain noise reduction processing, HDR fusion processing, and AI noise reduction processing on the third image data to obtain fourth image data.

[0033] In one embodiment of the present disclosure, performing image post-processing on the fifth image data to obtain target image data includes:

[0034] At least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing is performed on the fifth image data to obtain target image data.

[0035] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0036] An acquisition module, configured to acquire first image data captured by a camera;

[0037] a preprocessing module, configured to preprocess the first image data to obtain second image data;

[0038] A first storage module, configured to store the second image data into the first storage space;

[0039] an enhancement module, configured to perform image enhancement processing on the third image data to obtain fourth image data; the third image data is generated based on the second image data;

[0040] A second storage module, configured to store the fourth image data into a second storage space;

[0041] The post-processing module is used to perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0042] In one embodiment of the present disclosure, the acquisition module includes:

[0043] The first acquisition unit is used to acquire a plurality of first image data captured by a plurality of cameras.

[0044] In one embodiment of the present disclosure, the acquisition module includes:

[0045] The second acquisition unit is used to acquire the first image data through the camera and / or the third storage space; the third storage space stores the first image data captured by the camera.

[0046] In one embodiment of the present disclosure, the acquisition module includes:

[0047] The third acquisition unit is used to acquire a plurality of first image data obtained by multiple exposure shooting of the camera.

[0048] In one embodiment of the present disclosure, the apparatus further comprises:

[0049] a calling module, configured to call a preset first image enhancement algorithm to perform image enhancement processing on the second image data to obtain third image data; and / or,

[0050] A preset second image enhancement algorithm is called to perform image enhancement processing on the fourth image data to obtain fifth image data.

[0051] In one embodiment of the present disclosure, the first image enhancement algorithm includes an image alignment algorithm.

[0052] In one embodiment of the present disclosure, the apparatus further comprises:

[0053] a statistics module for performing image data statistics on the first image data to obtain image data statistical information; the image data statistical information includes at least one of automatic white balance, automatic focus, and automatic exposure;

[0054] The control module is used to control the shooting parameters of the camera according to the image data statistical information.

[0055] In one embodiment of the present disclosure, the fifth image data adopts a color gamut space of a RAW domain, an RGB domain, or a YUV domain.

[0056] In one embodiment of the present disclosure, the enhancement module includes:

[0057] The first enhancement unit performs image fusion processing on the plurality of third image data to obtain fourth image data; the plurality of third image data corresponds one-to-one to the plurality of first image data.

[0058] In one embodiment of the present disclosure, the second image data and / or the fourth image data is image data presented in an image pyramid.

[0059] In one embodiment of the present disclosure, the pre-processing module includes:

[0060] The preprocessor is used to perform at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference flow processing on the first image data to obtain second image data.

[0061] In one embodiment of the present disclosure, the enhancement module includes:

[0062] The second enhancement unit performs at least one of tone mapping processing, gamma correction processing, demosaicing processing, RAW domain noise reduction processing, HDR fusion processing, and AI noise reduction processing on the third image data to obtain fourth image data.

[0063] In one embodiment of the present disclosure, the post-processing module includes:

[0064] At least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing is performed on the fifth image data to obtain target image data.

[0065] According to a third aspect of the present disclosure, there is provided an image processing chip, comprising: an image pre-processing module, an image enhancement module, and an image post-processing module;

[0066] An image preprocessing module is used to preprocess the first image data captured by the camera to obtain second image data, and store the second image data in the first storage space;

[0067] An image enhancement module is configured to call the third image data in the first storage space and perform image enhancement processing on the third image data to obtain fourth image data, and store the fourth image data in the second storage space; the third image data is generated based on the second image data;

[0068] The image post-processing module is used to call the fifth image data in the second storage space and perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0069] In one embodiment of the present disclosure, the image preprocessing module is configured to perform at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference stream processing on first image data captured by the camera to obtain second image data, and store the second image data in the first storage space;

[0070] an image enhancement module, configured to retrieve the third image data from the first storage space and perform at least one of tone mapping, gamma correction, demosaicing, RAW-domain noise reduction, HDR fusion, and AI noise reduction on the third image data to obtain fourth image data, and store the fourth image data in the second storage space; the third image data being generated based on the second image data;

[0071] An image post-processing module is used to call the fifth image data in the second storage space and perform at least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0072] According to a fourth aspect of the present disclosure, a system chip is provided, including an image processing chip.

[0073] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0074] processor;

[0075] a memory for storing processor-executable instructions;

[0076] The processor is configured to implement the steps of any one of the image processing methods of the first aspect above.

[0077] According to a sixth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, which, when instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute any one of the image processing methods of the first aspect described above.

[0078] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0079] The present disclosure obtains first image data captured by a camera, pre-processes the first image data to obtain second image data, stores the second image data in a first storage space, performs image enhancement processing on the third image data to obtain fourth image data, stores the fourth image data in a second storage space, and performs image post-processing on the fifth image data to obtain target image data. The image processing process is decoupled into multiple segments, thereby improving the image processing performance of each segment and thereby improving the overall image processing performance.

[0080] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0082] Figure 1 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 1 .

[0083] Figure 2 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 2 .

[0084] Figure 3 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 3 .

[0085] Figure 4 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 4 .

[0086] Figure 5 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 5 .

[0087] Figure 6 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 6 .

[0088] Figure 7 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 7 .

[0089] Figure 8 The figure is a schematic diagram showing the structure of an image processing chip according to an exemplary embodiment of the present disclosure.

[0090] Figure 9 is a block diagram showing an image processing apparatus according to an exemplary embodiment of the present disclosure.

[0091] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0092] Figure 11 It is a block diagram of a chip system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0093] Some exemplary embodiments of the present disclosure will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but may be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, descriptions of features known in the art may be omitted for clarity and brevity.

[0094] The following exemplary embodiments of the present disclosure do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0095] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0096] Figure 1 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 1 The image processing method can be used in electronic devices with image capture functions, including but not limited to terminals such as smart phones, computers, smart tablets, cameras, and camcorders.

[0097] like Figure 1 As shown, the following steps are included.

[0098] S110: Acquire first image data captured by a camera.

[0099] In an exemplary embodiment of the present disclosure, the camera may include an imaging element having an image acquisition capability. Among them, the camera may include an infrared camera, a complementary metal oxide semiconductor (COMS) camera, a charge coupled device (CCD) camera, a thermal imaging camera, a panoramic camera, a high-speed camera, and a 3D camera. It should be noted that the camera of the present disclosure can be configured on any electronic device with image processing capabilities. The first image data may be an image acquired by the camera without being processed by other modules. Exemplarily, the first image data may be image data in RAW format.

[0100] In an exemplary embodiment of the present disclosure, an image signal processor (ISP) may obtain first image data captured by a camera. Specifically, the ISP may obtain the first image data captured by the camera through a mobile industry processor interface (MIPI), storage space, USB, high-definition multimedia interface (HDMI) interface, Wi-Fi, and Bluetooth. It should be noted that the ISP may also obtain the first image data through storage space.

[0101] S120: Preprocess the first image data to obtain second image data.

[0102] In exemplary embodiments of the present disclosure, preprocessing the first image data can be a processing step performed on the first image data before formal processing, and can be a series of adjustments and optimizations to the original image. Preprocessing the first image data can improve data quality, enhance feature expression, and adapt to the requirements of subsequent image processing algorithms, and is the first stage of image processing. Preprocessing the first image data can include performing at least one of bad pixel correction, image data statistics processing, black level correction, pixel binning, face detection, and frame difference processing on the first image data to obtain second image data. Bad pixel correction can include monitoring and correcting pixels in the image to improve image quality; image data statistics processing can include obtaining image parameters, which can include, for example, image sharpness, brightness, and color balance. Black level correction can include eliminating pixel value deviations caused by charges generated by thermal excitation. Pixel binning can include merging signals from adjacent pixels in the image to improve image quality. Face detection can include identifying a bounding box around a face in the image. Frame difference streaming processing can include efficient target detection and resource optimization in dynamic scenes. It should be noted that this disclosure does not specifically limit the algorithms used in bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference streaming processing.

[0103] In an exemplary embodiment of the present disclosure, after the ISP acquires the first image data, the ISP may be used to perform the above-mentioned preprocessing on the first image data to obtain the second image data.

[0104] S130: Store the second image data in the first storage space.

[0105] In an exemplary embodiment of the present disclosure, the first storage space may be part of the storage space of a memory that implements the above-mentioned image processing method. For example, the first storage space may be part of the storage space of a double data rate synchronous dynamic random access memory (DDR). The first storage space may be provided inside the ISP or may be a designated storage space located outside the ISP. In the exemplary embodiments of the present disclosure, the location where the first storage space is provided is not specifically limited.

[0106] In an exemplary embodiment of the present disclosure, after obtaining the second image data, the ISP may store the second image data in the first storage space so that other functional modules of the ISP can perform other image processing on the second image data.

[0107] S140, performing image enhancement processing on the third image data to obtain fourth image data; the third image data is generated based on the second image data.

[0108] In exemplary embodiments of the present disclosure, the second image data may be identical to the third image data. Enhancement processing of the third image data may be performed after image preprocessing and is the second stage of image processing. This may involve adjusting the image's visual attributes or structural features to improve image quality, highlight key information, and / or adapt to algorithm requirements. Image enhancement can improve the image's visual quality, make it more adaptable to complex processing environments, and prepare for the third stage of image processing. Image enhancement processing may include performing at least one of tone mapping, gamma correction, demosaicing, RAW-domain noise reduction, HDR fusion, and AI-based noise reduction on the third image data to generate fourth image data. Tone mapping may include adjusting the image's hue based on varying light intensities. Gamma correction may include adjusting the image's brightness curve to better align with human perception. Demosaicing may include interpolating single color elements in the image into a full-color image. RAW-domain noise reduction may include reducing artifacts in a RAW-domain image, where the RAW-domain image may include unprocessed raw image data. HDR fusion may combine multiple images with different exposures into a single HDR image. AI noise reduction processing includes using artificial intelligence technology to reduce the noise of the image. It should be noted that this disclosure does not specifically limit the algorithm used in the above processing.

[0109] In an exemplary embodiment of the present disclosure, the ISP-related modules may perform image enhancement processing on the third image data. It should be noted that, if the third image data is not obtained by processing the first image data, the ISP may perform image enhancement processing on the third image data while pre-processing the first image data.

[0110] S150: Store the fourth image data in the second storage space.

[0111] In an exemplary embodiment of the present disclosure, the second storage space and the first storage space can be the same storage space, or they can be different storage spaces of the same storage module. The ISP can store the obtained fourth image data in the second storage space. By storing the fourth image data in the second storage space through the ISP, other functional modules of the ISP can obtain the fourth image data for image post-processing. It should be noted that the second storage space can be set inside the ISP, or it can be a designated storage space located outside the ISP. In the exemplary embodiment of the present disclosure, there is no specific limitation on the location of the first storage space.

[0112] S160 , performing image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0113] In an exemplary embodiment of the present disclosure, the fourth image data may be the same as the fifth image data. Image post-processing may include post-processing (PE), wherein image post-processing of the fifth image data may be a key step in optimizing the output image data after completing the core image analysis task of the fifth image data. Image post-processing may be the third image processing stage following pre-processing and image enhancement processing. Image post-processing can convert the intermediate results generated by the algorithm into a deliverable form that meets the actual needs of the application. Image post-processing can improve image accuracy, enhance image usability, and improve image processing efficiency. Image post-processing may also include processing images in a video. The target image data may include image data for display or encoded storage after image processing is completed.

[0114] In an exemplary embodiment of the present disclosure, image post-processing is performed on the fifth image data to obtain target image data, including: performing at least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing on the fifth image data through an image post-processing module to obtain the target image data. Among them, electronic image stabilization processing may include compensating for physical movement of the camera or device through a software algorithm. Contrast adjustment may include enhancing the difference between light and dark by expanding the pixel brightness range. Color restoration processing may include correcting color cast and optimizing color performance. Appetite noise reduction processing may include suppressing noise using time dimension information. It should be noted that the present disclosure does not specifically limit the algorithms used in the above-mentioned processing.

[0115] In an exemplary embodiment of the present disclosure, the ISP-related module may obtain fifth image data from the second storage space and perform image enhancement processing on the fifth image data. It should be noted that, if the fifth image data is not obtained by processing the first image data, the ISP may perform image post-processing on the fifth image data while pre-processing the first image data. If the fifth image data is not obtained by processing the third image data, the ISP may perform image post-processing on the fifth image data while performing image enhancement processing on the third image data.

[0116] In exemplary embodiments of the present disclosure, the second image data and / or the fourth image data are presented as image data using an image pyramid. An image pyramid is a multi-scale representation method that generates a series of images with gradually decreasing resolutions by downsampling and reconstructing the original image at different levels. Storing the second image data or the fourth image data in the form of an image pyramid facilitates image enhancement processing of the third image data and / or image post-processing of the fifth image data.

[0117] The present disclosure obtains first image data captured by a camera, preprocesses the first image data to obtain second image data, stores the second image data in a first storage space, performs image enhancement processing on the third image data to obtain fourth image data, stores the fourth image data in a second storage space, and performs image post-processing on the fifth image data to obtain target image data. By storing the second image data in the first storage space and the fourth image data in the second storage space, and then performing subsequent processing on the second image data stored in the first storage space and the fourth image data stored in the second storage space, image processing is decoupled into multiple image processing stages, including pre-processing, image enhancement, and post-processing. This fully unleashes the image processing performance of each processing stage. By improving pre-processing performance, the problem of camera performance limitations caused by poor pre-processing performance is resolved, indirectly unleashing camera performance. Furthermore, since unprocessed images are stored in the storage space between multiple image processing stages, it is possible to insert other image enhancement algorithms and image processing algorithms between image processing stages. Users can independently set image enhancement algorithms and image processing algorithms to process images stored in the storage space, improving the scalability and autonomy of image processing.

[0118] Figure 2 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 2 . Figure 2 Steps S220 to S250 are Figure 1 The steps S120 to S150 in FIG. 1 correspond to each other and are not repeated here. Figure 2 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0119] S210: Acquire a plurality of first image data captured by a plurality of cameras.

[0120] In an exemplary embodiment of the present disclosure, the multiple cameras may include a camera fixed on an electronic device and a camera that can be movably installed on an electronic device. The multiple cameras may also include a camera connected to the ISP via a MIPI port and a camera connected to the ISP via a non-MIPI port. It should be noted that each camera is connected to the ISP via a channel corresponding to each camera, and multiple cameras can acquire images at the same time, and then transmit the images to the ISP through the images corresponding to each camera to achieve multi-camera concurrency.

[0121] In an exemplary embodiment of the present disclosure, by setting up multiple cameras and configuring a corresponding transmission channel for each camera, the frequency of each camera can be increased, and the computing density within a unit week can be maximized based on each transmission channel, thereby fully releasing the highest performance of the camera and achieving high energy efficiency and faster response.

[0122] In exemplary embodiments of the present disclosure, the ISP's processing capabilities can be set based on the camera's processing capabilities. The ISP's processing capabilities include its image processing performance, while the camera's processing capabilities include its ability to acquire and / or initially process images. Consequently, as camera performance improves, the processing capabilities of images transmitted by the camera are enhanced, thereby avoiding image processing delays caused by unprocessed images transmitted by the camera.

[0123] In an exemplary embodiment of the present disclosure, obtaining first image data captured by a camera includes: obtaining multiple first image data captured by the camera using multiple exposures. The multiple exposures may include high dynamic range (HDR) exposures, which can obtain multiple first image data at different brightness levels, thereby retaining more details in both bright and dark areas of the image. This can improve the quality of the processed image during further image processing. It should be noted that the transmission channel corresponding to each camera can support the simultaneous transmission of multiple images captured at different exposures.

[0124] The present disclosure obtains multiple first image data obtained by multiple cameras, and transmits the multiple first image data through the channels corresponding to each camera respectively, thereby enriching the shooting function, and performing processing based on the multiple first image data to improve the quality of the processed image.

[0125] Figure 3 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 3 . Figure 3 Steps S320 to S350 are Figure 1 The steps S120 to S150 in FIG. 1 correspond to each other and are not repeated here. Figure 3 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0126] S310, obtaining first image data through a camera and / or a third storage space; the third storage space stores the first image data captured by the camera.

[0127] In an exemplary embodiment of the present disclosure, the third storage space may be the same as the first storage space and the second storage space. The third storage space may be a different storage space that belongs to the same storage module as the first storage space and / or the second storage space. Furthermore, the third storage space may also be a different storage space that belongs to a different storage module as the first storage space and / or the second storage space. The third storage space may store images captured by a camera that is actively connected to the electronic device. The camera that is actively connected to the electronic device may store the captured images in the third storage space via the network, Bluetooth, USB, or the like.

[0128] In an exemplary embodiment of the present disclosure, the ISP can simultaneously process the first image data acquired by multiple cameras configured by the electronic device, wherein the cameras configured by the electronic device send the first image data to the ISP through the MIPI port. The ISP can process the first image data stored only by the third storage space, wherein the ISP can acquire the first image data stored in the third storage space through a pre-set connection relationship with the third storage space, and process the first image data. The ISP can also simultaneously process the first image data acquired by multiple cameras configured by the electronic device and the first image data stored in the third storage space. It should be noted that the multiple cameras configured by the electronic device can also store the acquired first image data in the third storage space.

[0129] In an exemplary embodiment, the camera may include one or more cameras fixedly configured in the electronic device, and the camera may send the first image data to the ISP through a MIPI port.

[0130] In an exemplary embodiment, the camera may include multiple fixed electronic devices and multiple external cameras. The camera and the ISP are in a MIPI offline state. The camera sends the captured first image data to a storage space for storage. The ISP retrieves the first image data from the storage space at a specified location. It should be noted that the storage space in this embodiment includes a first storage space and a third storage space.

[0131] In an exemplary embodiment, the camera may include multiple fixed electronic devices and multiple external cameras. Some of the multiple cameras transmit first image data to the ISP via a MIPI port. Some of the multiple cameras transmit the captured first image data to a storage space for storage. The ISP retrieves the first image data from the storage space at a designated location.

[0132] The present disclosure processes the first image data obtained by the camera and / or the third storage space to finally obtain the target image data, thereby expanding the user's photo-taking methods, and improving the efficiency of image processing by parallel processing of the first image data obtained by the camera and the first image data obtained by the third storage space.

[0133] Figure 4 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 4 . Figure 4 Steps S410, S420, S430, S450, S460 and S470 are Figure 1 The steps S110 to S150 in FIG. 1 correspond to each other and are not repeated here. Figure 4 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0134] S440: Call a preset first image enhancement algorithm to perform image enhancement processing on the second image data to obtain third image data.

[0135] In an exemplary embodiment of the present disclosure, calling a preset first image enhancement algorithm to perform image enhancement processing on the second image data may be image processing outside the pre-processing of the first stage, the image enhancement processing of the second stage, and the post-processing of the third stage. Calling the preset first image enhancement algorithm to perform image enhancement processing on the second image data can enable a third party to independently preset an image enhancement algorithm to further enhance the image outside the above-mentioned multiple stages. The scalability of the ISP is improved. And further image enhancement processing can further improve the quality of the processed image. The preset first image enhancement algorithm can be deployed on a processor outside the ISP or in other modules. The processor outside the ISP or other modules may include an XPU. The first image enhancement algorithm may include a RAW domain soft algorithm or an AI algorithm, wherein the first image enhancement algorithm can be set independently by the user, and the present disclosure does not limit it.

[0136] In an exemplary embodiment of the present disclosure, the third image data can adopt the color gamut space of the RAW domain. After calling the preset first image enhancement algorithm to process the second image data to obtain the third image data, the third image data can be stored in the first storage space so that the third image data can be subsequently enhanced to obtain the fourth image data.

[0137] In an exemplary embodiment of the present disclosure, the first image enhancement algorithm includes an image alignment algorithm, wherein the image alignment algorithm can spatially align multiple images through geometric transformations so that corresponding pixels are geometrically aligned for subsequent processing. The above embodiments of the present disclosure disclose that the present disclosure uses multiple cameras and each camera uses multiple exposures to acquire images. After calling the image alignment algorithm, the multiple second image data can be better integrated, reducing the ghosting problem.

[0138] The present disclosure utilizes a preset first image enhancement algorithm to perform image enhancement processing on a second image to obtain third image data. This allows users to independently preset the first image enhancement algorithm, meeting their differentiated needs and enhancing the scalability of the image processing module. Furthermore, utilizing the first image enhancement algorithm to perform image enhancement processing on the second image can enhance the image quality of the processed third image data, further enhancing the image quality of the obtained target image data.

[0139] Figure 5 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 5 . Figure 5 Steps S510, S520, S530, S540, S550, and S570 are Figure 1 The steps S110 to S150 in FIG. 1 correspond to each other and are not repeated here. Figure 5 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0140] S560: Call a preset second image enhancement algorithm to perform image enhancement processing on the fourth image data to obtain fifth image data.

[0141] In an exemplary embodiment of the present disclosure, calling a preset second image enhancement algorithm to perform image enhancement processing on the fourth image data may be image processing outside the pre-processing of the first stage, the image enhancement processing of the second stage, and the post-processing of the third stage. Calling the preset second image enhancement algorithm to perform image enhancement processing on the fourth image data can enable a third party to independently preset an image enhancement algorithm to further enhance the image outside the above-mentioned multiple stages. The scalability of the ISP is improved. And further image enhancement processing can further improve the quality of the processed image. The preset second image enhancement algorithm can be deployed on a processor outside the ISP or in other modules. The processor outside the ISP or other modules may include an XPU. The second image enhancement algorithm may include a RAW domain soft algorithm or an AI algorithm, wherein the second image enhancement algorithm can be set independently by the user, and the present disclosure does not limit it.

[0142] In an exemplary embodiment of the present disclosure, the fifth image data adopts a color space of a RAW domain, an RGB domain, or a YUV domain. Correspondingly, the second image enhancement algorithm may include image enhancement algorithms corresponding to the above color spaces, respectively.

[0143] In an exemplary embodiment of the present disclosure, both the third image data and the fourth image data may utilize a color gamut space in a RAW domain, an RGB domain, or a YUV domain. Performing image enhancement processing on the third image data to obtain the fourth image data may include identifying the third image data, determining a color gamut of the third image data, and, after determining the color gamut of the third image data, processing the third image data based on an image enhancement algorithm corresponding to the color gamut corresponding to the current third image data.

[0144] In an exemplary embodiment, the third image data can be processed by an image enhancement module in the ISP. The image enhancement module can sequentially set an algorithm for RAW domain image enhancement, an algorithm for RGB domain image enhancement, and an algorithm for YUV domain image enhancement. After the color gamut of the third image data is identified, image enhancement processing is performed using image enhancement algorithms corresponding to different color gamuts based on the identified color gamut of the third image data.

[0145] It should be noted that after the fourth image data is stored in the second storage space, a processor other than the ISP in the electronic device can also call the fourth image data in the second storage space and process the fourth image data based on a RAW domain soft algorithm or an AI algorithm to obtain fifth image data. The fourth image data originally in the RAW domain can be converted into a color gamut space of the RGB domain or the YUV domain based on the RAW domain soft algorithm or the AI ​​algorithm. The processor other than the ISP in the electronic device can include a CPU, a GPU, an FPGA, and an XPU.

[0146] In an exemplary embodiment of the present disclosure, after invoking a preset second image enhancement algorithm to process the fourth image data to obtain fifth image data, the fifth image data can be stored in a second storage space so that subsequent image post-processing can be performed on the fifth image data to obtain target image data. It should be noted that the second image enhancement algorithm can be the same as the first image enhancement algorithm.

[0147] The present disclosure utilizes a preset second image enhancement algorithm to enhance the fourth image to obtain fifth image data. This allows users to customize the second image enhancement algorithm, thus meeting their differentiated needs. Furthermore, utilizing the second image enhancement algorithm to enhance the fourth image can enhance the image quality of the processed fifth image data, further enhancing the image quality of the obtained target image data.

[0148] Figure 6This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 6 . Figure 6 Steps S610, and S640 to S680 are Figure 1 The steps S110 to S150 in FIG. 1 correspond to each other and are not repeated here. Figure 6 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0149] S620: Perform image data statistics on the first image data to obtain image data statistical information; the image data statistical information includes at least one of automatic white balance, automatic focus, and automatic exposure.

[0150] In an exemplary embodiment of the present disclosure, the image data statistical information may include image parameter information required by the 3A algorithm. The 3A algorithm may include 3A algorithms, which may refer to auto focus, auto exposure, and auto white balance. The 3A algorithm can ensure the quality of images acquired by the camera in different scenarios. Exemplarily, the image data statistical information includes at least one of auto white balance, auto focus, and auto exposure.

[0151] S630: Control the shooting parameters of the camera according to the image data statistical information.

[0152] In an exemplary embodiment of the present disclosure, controlling the shooting parameters of the camera may include a gyroscope, an accelerometer, a proximity sensor, an ambient light sensor, a laser radar (LiDAR), a ToF sensor, a focus motor (VCM), and an anti-shake motor (OIS).

[0153] In an exemplary embodiment of the present disclosure, the ISP can perform image data statistics on the first image data to obtain image data statistical information and then control the shooting parameters. After the shooting parameters are controlled, the adjusted camera can acquire multiple first image data again and then transmit the acquired multiple different first image data to the ISP according to the corresponding channel of the camera.

[0154] The present invention performs image data statistics on the first image data to obtain image data statistical information, and then controls the shooting parameters of the camera according to the image data statistical information, thereby improving the image quality of the first image data obtained by the camera, and further improving the image quality of the target image data.

[0155] Figure 7 This is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure. Figure 7 . Figure 7 Steps S720, S730, S750, and S760 are Figure 1 The steps S120, S130, S150 and S160 correspond to each other, and S710 corresponds to S210, which will not be repeated here. Figure 7 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0156] S740, performing image fusion processing on the plurality of third image data to obtain fourth image data; the plurality of third image data corresponds one-to-one to the plurality of first image data.

[0157] In an exemplary embodiment of the present disclosure, the first image data may be image data acquired by cameras of different configurations, and the third image data corresponds to multiple first image data. Therefore, in an exemplary embodiment of the present disclosure, the third image data may be acquired by cameras of different configurations. Multiple cameras have been described in detail in the above embodiments and will not be further explained here.

[0158] In exemplary embodiments of the present disclosure, since the second image data acquired from different cameras can all be stored in the first storage space, the third image data derived from the second image data can be called within the first storage space to complete image fusion processing. Image fusion processing can be a technique for integrating multiple or multi-source image information into a single image, and the present disclosure does not specifically limit the algorithm used to complete image fusion processing.

[0159] The present disclosure processes first image data acquired by multiple cameras respectively to obtain third image data stored in a first storage space, and then performs image fusion processing on the third image data to obtain fourth image data, thereby achieving the fusion of image data from different sources and improving the image quality of the generated image data.

[0160] Figure 8 FIG. 1 is a schematic diagram showing the structure of an image processing chip according to an exemplary embodiment of the present disclosure. Figure 8 As shown, the image processing chip 80 includes an image pre-processing module 801, an image enhancement module 802 and an image post-processing module 803;

[0161] The image preprocessing module 801 is used to preprocess the first image data captured by the camera 70 to obtain second image data, and store the second image data in the first storage space 910;

[0162] The image enhancement module 802 is configured to retrieve the third image data from the first storage space 910 and perform image enhancement processing on the third image data to obtain fourth image data, and store the fourth image data in the second storage space 920; the third image data is generated based on the second image data;

[0163] The image post-processing module 803 is configured to call the fifth image data in the second storage space 920 and perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0164] In an exemplary embodiment of the present disclosure, the image processing chip 80 is connected to the first storage space 910, the second storage space 920 and the camera 70, wherein the steps implemented by each module of the image processing chip have been described in detail in the above embodiment and will not be repeated here.

[0165] In an exemplary embodiment of the present disclosure, the image pre-processing module 801 is configured to perform at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference stream processing on the first image data captured by the camera 70 to obtain second image data, and store the second image data in the first storage space 910;

[0166] The image enhancement module 802 is configured to retrieve the third image data from the first storage space 910 and perform at least one of tone mapping, gamma correction, demosaicing, RAW-domain noise reduction, HDR fusion, and AI noise reduction on the third image data to obtain fourth image data, and store the fourth image data in the second storage space 920; the third image data is generated based on the second image data;

[0167] The image post-processing module 803 is used to call the fifth image data in the second storage space 920 and perform at least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0168] To further explain the present disclosure, the present disclosure also provides an exemplary embodiment, in which the electronic device includes a smartphone. In this exemplary embodiment, multiple native cameras and / or external cameras of the smartphone respectively acquire multiple first image data using HDR multi-exposure. The native cameras transmit the multiple first image data to the ISP via the MIPI interface, and the external cameras transmit the first image data to the ISP via other means. The ISP performs at least one of bad pixel correction, image data statistics, black level correction, pixel merging, face detection, and frame difference stream processing on the first image data to obtain second image data, where the second image data is in the RAW domain. Simultaneously, the ISP may perform image data statistics on the previously acquired first image data, where the image data statistics include at least one of automatic white balance, autofocus, and auto exposure. The ISP controls the native camera and / or external camera based on the image data statistics and the 3A algorithm. After adjustment, the native camera and / or external camera continues to acquire first image data and then transmits the first image data to the ISP according to the above method.

[0169] The ISP stores the second image data in the form of an image pyramid in the DDR, and then can further process the second image data by calling the RAW domain soft algorithm or AI algorithm deployed on the XPU to obtain the third image data. Among them, the RAW domain soft algorithm or AI algorithm deployed on the XPU can be customized by the user. The ISP calls the image alignment algorithm to align the third image data, and then performs at least one of tone mapping processing, gamma correction processing, de-mosaic processing, RAW domain noise reduction processing, HDR fusion processing, and AI noise reduction processing on the third image data to obtain the fourth image data. At the same time, since the first image data that generates the third image data may come from different cameras, such as native cameras or external cameras. The ISP can call the image fusion algorithm to perform image fusion processing on multiple third image data to obtain the fourth image data.

[0170] After obtaining the fourth image data, the ISP stores the fourth image data in the DDR in the form of an image pyramid. The fourth image data can then be further processed by calling a RAW domain soft algorithm or AI algorithm deployed on the XPU to obtain fifth image data. This further processing may involve image conversion, and the fifth image data can use a color space in the RAW domain, RGB domain, or YUV domain. After obtaining the fifth image data, at least one of electronic image stabilization, contrast adjustment, color restoration, and temporal noise reduction can be performed on the fifth image data to obtain the target image data.

[0171] It should be noted that the acquisition, storage, use, and processing of information or data in the technical solution disclosed herein are in compliance with the relevant provisions of national laws and regulations.

[0172] Figure 9 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure. Figure 9 , the apparatus 900 comprises:

[0173] An acquisition module 901 is configured to acquire first image data captured by a camera;

[0174] A preprocessing module 902 is used to preprocess the first image data to obtain second image data;

[0175] A first storage module 903 is configured to store the second image data in a first storage space;

[0176] The enhancement module 904 is configured to perform image enhancement processing on the third image data to obtain fourth image data; the third image data is generated based on the second image data;

[0177] The second storage module 905 is used to store the fourth image data into the second storage space;

[0178] The post-processing module 906 is configured to perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

[0179] In one embodiment of the present disclosure, the acquisition module 901 includes:

[0180] The first acquisition unit is used to acquire a plurality of first image data captured by a plurality of cameras.

[0181] In one embodiment of the present disclosure, the acquisition module 901 includes:

[0182] The second acquisition unit is used to acquire the first image data through the camera and / or the third storage space; the third storage space stores the first image data captured by the camera.

[0183] In one embodiment of the present disclosure, the acquisition module 901 includes:

[0184] The third acquisition unit is used to acquire a plurality of first image data obtained by multiple exposure shooting of the camera.

[0185] In one embodiment of the present disclosure, the apparatus further comprises:

[0186] a calling module, configured to call a preset first image enhancement algorithm to perform image enhancement processing on the second image data to obtain third image data; and / or,

[0187] A preset second image enhancement algorithm is called to perform image enhancement processing on the fourth image data to obtain fifth image data.

[0188] In one embodiment of the present disclosure, the first image enhancement algorithm includes an image alignment algorithm.

[0189] In one embodiment of the present disclosure, the apparatus further comprises:

[0190] a statistics module, configured to perform image data statistics on the first image data to obtain image data statistical information; the image data statistical information includes at least one of automatic white balance, automatic focus, and automatic exposure;

[0191] The control module is used to control the shooting parameters of the camera according to the image data statistical information.

[0192] In one embodiment of the present disclosure, the fifth image data adopts a color gamut space of a RAW domain, an RGB domain, or a YUV domain.

[0193] In one embodiment of the present disclosure, the enhancement module includes:

[0194] The first enhancement unit is configured to perform image fusion processing on the plurality of third image data through the image enhancement module to obtain fourth image data; the plurality of third image data corresponds one-to-one to the plurality of first image data.

[0195] In one embodiment of the present disclosure, the second image data and / or the fourth image data is image data presented in an image pyramid.

[0196] In one embodiment of the present disclosure, the pre-processing module 902 includes:

[0197] The pre-processing unit is used to perform at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference flow processing on the first image data to obtain second image data.

[0198] In one embodiment of the present disclosure, the enhancement module includes:

[0199] The second enhancement unit is used to perform at least one of tone mapping processing, gamma correction processing, demosaicing processing, RAW domain noise reduction processing, HDR fusion processing, and AI noise reduction processing on the third image data through the image enhancement module to obtain fourth image data.

[0200] In one embodiment of the present disclosure, the post-processing module 906 includes:

[0201] The image post-processing module 906 performs at least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing on the fifth image data to obtain target image data.

[0202] Figure 10 1 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. For example, the device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0203] Reference Figure 10 , the device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 10010 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0204] The processing component 1002 generally controls the overall operation of the device 1000, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1002 may include one or more modules to facilitate interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate interaction between the multimedia component 10010 and the processing component 1002.

[0205] The memory 1004 is configured to store various types of data to support the operations of the device 1000. Examples of such data include instructions for any application or method operating on the device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0206] The power supply component 1006 provides power to the various components of the device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1000.

[0207] The multimedia component 10010 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 10010 includes a front camera and / or a rear camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0208] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC) that is configured to receive external audio signals when the device 1000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0209] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0210] The sensor assembly 1014 includes one or more sensors for providing various aspects of the status assessment of the device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the device 1000, the relative positioning of components, such as the display and keypad of the device 1000. The sensor assembly 1014 can also detect changes in the position of the device 1000 or a component of the device 1000, the presence or absence of user contact with the device 1000, the orientation or acceleration / deceleration of the device 1000, and changes in the temperature of the device 1000. The sensor assembly 1014 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0211] The communication component 1016 is configured to facilitate wired or wireless communication between the apparatus 1000 and other devices. The apparatus 1000 can access a wireless network based on a communication standard, such as Wi-Fi, 3G, 4G, 5G, other communication standards, or a combination thereof. In some embodiments of the present disclosure, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of the present disclosure, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0212] In some embodiments of the present disclosure, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned methods.

[0213] In some embodiments of the present disclosure, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the apparatus 1000 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0214] In some embodiments of the present disclosure, a non-transitory computer-readable storage medium enables a terminal to perform an image processing method when instructions in the storage medium are executed by a processor of a terminal.

[0215] In some embodiments of the present disclosure, a computer program product is further provided, including a computer program / instruction, which implements an image processing method when the computer program / instruction is executed by a processor.

[0216] Figure 11 FIG. 1 is a block diagram of a chip system according to an exemplary embodiment of the present disclosure. Figure 11As shown, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101). Exemplarily, the interface circuit 1102 can read instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the image processing device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, and some embodiments of the present disclosure are not specifically limited to this.

[0217] In some embodiments of the present disclosure, the interface circuit 1102 can obtain data, program instructions and / or information from the internal storage area of ​​the chip system; it can also obtain data, program instructions and / or information from outside the chip system.

[0218] Optionally, the chip system further includes a memory 1103, which is used to store necessary computer programs and data.

[0219] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present application.

[0220] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present application.

[0221] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0222] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquire first image data captured by a camera; preprocessing the first image data to obtain second image data; storing the second image data in the first storage space; performing image enhancement processing on the third image data to obtain fourth image data; the third image data is generated based on the second image data; storing the fourth image data in a second storage space; Perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

2. The method according to claim 1, characterized in that The step of obtaining the first image data captured by the camera includes: Acquire a plurality of first image data captured by a plurality of the cameras.

3. The method according to claim 1 or 2, characterized in that The step of obtaining the first image data captured by the camera includes: The first image data is acquired through the camera and / or the third storage space; the first image data captured by the camera is stored in the third storage space.

4. The method according to claim 1, wherein The step of obtaining the first image data captured by the camera includes: Acquire a plurality of first image data obtained by multiple exposure shooting by the camera.

5. The method according to claim 1, characterized in that The method further comprises: Calling a preset first image enhancement algorithm to perform image enhancement processing on the second image data to obtain the third image data; and / or, A preset second image enhancement algorithm is called to perform image enhancement processing on the fourth image data to obtain the fifth image data.

6. The method according to claim 5, characterized in that The first image enhancement algorithm includes an image alignment algorithm.

7. The method according to claim 1, characterized in that The method further comprises: Performing image data statistics on the first image data to obtain image data statistical information; the image data statistical information includes at least one of automatic white balance, automatic focus, and automatic exposure; The shooting parameters of the camera are controlled according to the image data statistical information.

8. The method according to claim 1, characterized in that The fifth image data uses a color space of a RAW domain, an RGB domain, or a YUV domain.

9. The method according to claim 2, characterized in that The performing image enhancement processing on the third image data to obtain fourth image data includes: Perform image fusion processing on the plurality of third image data to obtain the fourth image data; the plurality of third image data correspond one-to-one to the plurality of first image data.

10. The method according to claim 1, characterized in that The second image data and / or the fourth image data are image data presented in an image pyramid.

11. The method according to claim 1, wherein The preprocessing of the first image data to obtain second image data includes: The first image data is subjected to at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference flow separation processing to obtain the second image data.

12. The method according to claim 1, characterized in that The performing image enhancement processing on the third image data to obtain fourth image data includes: The third image data is subjected to at least one of tone mapping processing, gamma correction processing, demosaicing processing, RAW domain noise reduction processing, HDR fusion processing, and AI noise reduction processing to obtain the fourth image data.

13. The method according to claim 1, wherein The performing image post-processing on the fifth image data to obtain target image data includes: At least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing is performed on the fifth image data to obtain the target image data.

14. An image processing chip, comprising an image pre-processing module, an image enhancement module, and an image post-processing module; The image preprocessing module is used to preprocess the first image data captured by the camera to obtain second image data, and store the second image data in the first storage space; The image enhancement module is configured to call third image data in the first storage space and perform image enhancement processing on the third image data to obtain fourth image data, and store the fourth image data in the second storage space; the third image data is generated based on the second image data; The image post-processing module is used to call the fifth image data in the second storage space and perform image post-processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

15. The chip according to claim 14, characterized in that The image preprocessing module is configured to perform at least one of bad pixel correction processing, image data statistical processing, black level correction processing, pixel merging processing, face detection processing, and frame difference stream processing on the first image data captured by the camera to obtain second image data, and store the second image data in the first storage space; the image enhancement module being configured to retrieve third image data from the first storage space and perform at least one of tone mapping, gamma correction, demosaicing, RAW domain noise reduction, HDR fusion, and AI noise reduction on the third image data to obtain fourth image data, and store the fourth image data in the second storage space; the third image data being generated based on the second image data; The image post-processing module is used to call the fifth image data in the second storage space and perform at least one of electronic image stabilization processing, contrast adjustment processing, color restoration processing, and time domain noise reduction processing on the fifth image data to obtain target image data; the fifth image data is generated based on the fourth image data.

16. A system chip, characterized in that: Comprising the image processing chip as claimed in claim 14 or 15.

17. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the image processing method according to any one of claims 1 to 13. 18 . A non-transitory computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform the steps of the image processing method according to claim 1 .

19. A chip system, characterized in that: The chip system includes a processor and an interface circuit, the processor obtains program instructions through the interface circuit, the program instructions are executed by the processor, and the processor is used to execute the steps of the image processing method according to any one of claims 1 to 13.

Images