Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The image processing method is applied to the image processing device, the image processing device comprises a main control chip and an image processing chip, and the image processing method comprises the following steps: processing at least one frame of first image through the image processing chip under the condition that shooting input is received in a video recording process to obtain a second image, the at least one frame of first image is obtained based on shooting input; transmitting the second image to the main control chip through the image processing chip; and encoding the second image through the main control chip, and storing the encoded second image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image processing method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the continuous development of electronic device photography technology, more and more electronic devices support photographing and video recording functions. At present, electronic devices generally provide a simple photographing function during video recording.

[0003] However, if the electronic device allocates the computing power resources of the main control chip to the video recording function during video recording to avoid frame loss during video recording, the main control chip has no extra computing power resources for additional image processing of the photographed photos, resulting in poor image quality of the photos generated during video recording. If the electronic device allocates the computing power resources of the main control chip to additional image processing of the photographed photos during video recording to improve the image quality of the photos generated during video recording, the main control chip has no extra computing power resources for the video recording function, resulting in frame loss during video recording. Therefore, the computing power resources of the main control chip of the electronic device are difficult to balance photographing and video recording. SUMMARY

[0004] The purpose of the embodiments of the application is to provide an image processing method and device, an electronic device and a storage medium, which can improve the image quality of the photos generated during video recording.

[0005] In a first aspect, the embodiments of the application provide an image processing method applied to an image processing device, the image processing device comprising a main control chip and an image processing chip, and the method comprises the following steps:

[0006] In the case that a photographing input is received during video recording, at least one frame of first image is processed by the image processing chip to obtain a second image, and the at least one frame of first image is obtained based on the photographing input;

[0007] The second image is transmitted to the main control chip by the image processing chip;

[0008] The second image is encoded by the main control chip, and the encoded second image is stored.

[0009] In a second aspect, the embodiments of the application provide an image processing device, which comprises a main control chip and an image processing chip;

[0010] The image processing chip is configured to process at least one frame of first image to obtain a second image in the case that a photographing input is received during video recording, and the at least one frame of first image is obtained based on the photographing input;

[0011] The image processing chip is also configured to transmit the second image to the master chip.

[0012] The master chip is configured to encode the second image and store the encoded second image.

[0013] In a third aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory stores programs or instructions executable on the processor. The programs or instructions, when executed by the processor, implement the steps of the image processing method according to the first aspect.

[0014] In a fourth aspect, a readable storage medium is provided. The readable storage medium stores programs or instructions. The programs or instructions, when executed by a processor, implement the steps of the image processing method according to the first aspect.

[0015] In a fifth aspect, a chip is provided. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute programs or instructions to implement the image processing method according to the first aspect.

[0016] In a sixth aspect, a computer program product is provided. The computer program product is stored in a storage medium. The computer program product is executed by at least one processor to implement the image processing method according to the first aspect.

[0017] In the embodiments of the present application, when a shooting input is received during a video recording process, at least one frame of first image is processed by an image processing chip to obtain a second image. The at least one frame of first image is obtained based on the shooting input. The second image is transmitted to a master chip by the image processing chip. The second image is encoded by the master chip, and the encoded second image is stored. That is, the extra shooting image during the video recording process is processed and encoded by the image processing chip and the master chip to generate a picture, so that the master chip has no extra computing resources to perform extra image processing on the extra shooting image, thereby improving the image quality of the picture generated during the video recording process. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application;

[0019] Figure 2 FIG. 2 is a flowchart of an image processing method according to an embodiment of the present application; Figure One

[0020] Figure 3 FIG. 3 is a flowchart of an image processing method according to an embodiment of the present application; Figure Two ​​

[0021] Figure 4 is a flowchart of an image processing method provided by an embodiment of the present application Figure Three ;

[0022] Figure 5 is a flowchart of an image processing method provided by an embodiment of the present application Figure Four ;

[0023] Figure 6 is a flowchart of an image processing method provided by an embodiment of the present application Figure One ;

[0024] Figure 7 is a flowchart of an image processing method provided by an embodiment of the present application Figure Two ;

[0025] Figure 8 is a structural diagram of an image processing apparatus provided by an embodiment of the present application Figure One ;

[0026] Figure 9 is a structural diagram of an image processing apparatus provided by an embodiment of the present application Figure Two ;

[0027] Figure 10 is a structural diagram of an electronic device provided by an embodiment of the present application Figure Two ;

[0028] Figure 11 is a structural diagram of an electronic device provided by an embodiment of the present application DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in an "or" relationship.

[0031] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0032] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0033] The image processing method provided in this application can be applied to video recording scenarios of electronic devices. Specifically, the image processing method provided in this application can be applied to scenarios where electronic devices take photos during video recording.

[0034] The following is combined with Figure 1 This application describes a scheme for taking photos during video recording using an electronic device, as provided in the embodiments of this application. For example... Figure 1 As shown, the electronic device 10 includes an image sensor 11, an image processing chip 12 connected to the image sensor 11 via a first camera serial interface (CSI) 111, a main control chip 13 connected to the image processing chip 12 via a second CSI 121, and a screen 14 connected to the main control chip. The main control chip 13 includes an image signal processor (ISP) 131, a video encoding module 132 connected to the ISP 131, and an image encoding module 133. The image processing chip 12 includes a buffer 122 and a fusion module 123.

[0035] Combination Figure 1As shown, during video recording, the electronic device 10 acquires at least one frame of image via the image sensor 11 and transmits it to the image processing chip 12 via the first CSI 111. The image processing chip 12 transmits the at least one frame of image cached in the buffer 122 to the main control chip 13 via the second CSI 121, and caches the at least one newly acquired frame of image by the image sensor 11 in the buffer 122. After receiving the at least one frame of image via the second CSI 121, the main control chip 13 processes the at least one frame of image via the ISP 131. For example, the ISP 131 can perform operations such as bad pixel correction, color correction, and single-frame noise reduction on the at least one frame of image, and then perform pixel format conversion on the processed at least one frame of image to obtain the target format, such as the at least one frame of image in YUV format. Here, the YUV color space is composed of a Y "luminance" component and two UV "chrominance" components. Y is called the grayscale component, UV is the chrominance component, U represents the blue component, and V represents the red component. Then, the video encoding module 132 in the main control chip 13 can encode at least one frame of the target format image video to obtain the recorded video, and then transmit the video image to the screen 14 for display.

[0036] If a user needs to take a photo during video recording, they can input a photo into the electronic device. When the electronic device 10 receives the user's photo input during video recording, it retrieves at least one first image frame corresponding to the photo input from the buffer via the image processing chip 12. Then, it performs at least one image processing operation on the first image frame, such as multi-frame noise reduction, high dynamic range imaging, and super-resolution processing, to obtain a second image. This second image is then transmitted to the main control chip 13 via the second CSI 121. After receiving the second image from the second CSI 121 via the main control chip 13, the electronic device encodes the second image via the image encoding module 133 to obtain and store the encoded second image, which is the captured photograph.

[0037] This application provides an image processing method, image processing apparatus, electronic device, and medium. When a shooting input is received during video recording, an image processing chip processes at least one frame of a first image to obtain a second image, where the at least one frame of the first image is obtained based on the shooting input. The image processing chip then transmits the second image to a main control chip. The main control chip encodes and stores the encoded second image. In other words, additional images captured during video recording are processed and encoded by the image processing chip and the main control chip respectively, avoiding the main control chip lacking sufficient computing resources to perform additional image processing on extra captured images. This improves the image quality of the photos generated during video recording.

[0038] The image processing method provided in this application can be executed by an image processing device, which can be an electronic device, or a functional module or entity module within the electronic device that implements the image processing method. The specific implementation can be determined based on actual usage requirements, and this application does not impose any limitations. The following description uses an image processing device as an example of an entity module within an electronic device to illustrate how the image processing device executes the image processing method provided in this application.

[0039] This application provides an image processing method applied to an image processing device, which includes a main control chip and an image processing chip.

[0040] In some embodiments of this application, the aforementioned main control chip can be a System on Chip (SoC), also known as a system-on-a-chip, which is a system composed of multiple integrated circuits with specific functions. It generally includes modules such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Display Processing Unit (DPU), a Digital Signal Processor (DSP), Random Access Memory (RAM), and a Modem. The DPU is used for image compositing and processing; the image processed by the DPU can be directly sent to the screen for display.

[0041] Based on the above, embodiments of this application provide an image processing method, such as... Figure 2 As shown, the image processing method provided in this application embodiment may include the following steps 201 to 203:

[0042] Step 201: When the image processing device receives the shooting input during the recording process, it processes at least one frame of the first image through the image processing chip to obtain the second image.

[0043] In some embodiments of this application, if a user needs to take photos during the recording process, the user can input photos into the image processing device during the recording process.

[0044] In some embodiments of this application, the aforementioned at least one first image is obtained based on the aforementioned shooting input.

[0045] In some embodiments of this application, the above-mentioned shooting input may include any of the following: user click input, swipe input, press input, voice input, gesture input, or other feasible inputs, which are not limited in this application embodiment.

[0046] In some embodiments of this application, the above-mentioned gesture input may include, but is not limited to, at least one of the following: click gesture, swipe gesture, drag gesture, pressure recognition gesture, long press gesture, area change gesture, double press gesture, double tap gesture, specific gesture input or other possible gesture inputs. The specific gesture input form can be determined according to actual needs, and is not limited in some embodiments.

[0047] In some embodiments of this application, the above-mentioned click input can be single-click input, double-click input, or any number of clicks, or it can be long-press input or short-press input. In some embodiments, this is not limited.

[0048] In some embodiments of this application, the above-mentioned sliding input can be a sliding input in any direction, such as sliding up, sliding down, sliding left, or sliding right, etc., and in some embodiments, this is not limited.

[0049] In some embodiments of this application, the image processing device may activate the recording function upon receiving video input to perform a recording operation.

[0050] In some embodiments of this application, the above-mentioned video input may include any of the following: user click input, swipe input, press input, voice input, gesture input, or other feasible input, which is not limited in this application embodiment.

[0051] In some embodiments of this application, the first image may be one frame or multiple frames captured by the image processing device during the recording process. It should be noted that, in these embodiments, the images captured during recording refer to each frame captured via the recording function, and are unrelated to the aforementioned shooting input.

[0052] In some embodiments of this application, combined with Figure 2 ,like Figure 3 As shown, before performing the step 201 above, "processing the first image using an image processing chip to obtain the second image", the image processing method provided in this application further includes the following steps 204a to 204c:

[0053] Step 204a: When the image processing device receives a shooting input during the recording process, it obtains the first timestamp corresponding to the shooting input through the image processing chip.

[0054] In some embodiments of this application, when the image processing device receives the aforementioned shooting input, it can generate a timestamp through the image processing chip. This timestamp is the first timestamp corresponding to the aforementioned shooting input. That is, the first timestamp represents the time when the shooting input was received.

[0055] Step 204b: The image processing device obtains N frames of images from the buffer of the image processing chip through the image processing chip.

[0056] Step 204c: The image processing device uses an image processing chip to determine at least one first image from the N frames of images.

[0057] In some embodiments of this application, the buffer described above stores at least one frame of image captured during the recording process.

[0058] In some embodiments of this application, the aforementioned N frames of images include a third image corresponding to the first timestamp in the aforementioned at least one frame of images, or include the aforementioned third image and images acquired before the aforementioned third image, where N is a positive integer.

[0059] In some embodiments of this application, "the third image corresponding to the first timestamp" can be understood as the image acquired at the time indicated by the first timestamp, which is the third image.

[0060] In some embodiments of this application, the images acquired before the third image are N-1 consecutive frames of images acquired by the image sensor before acquiring the third image.

[0061] In some embodiments of this application, the above-mentioned cache may be Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM), or Direct Rambus RAM (DRRAM), or other caches. This embodiment does not impose specific limitations here.

[0062] It should be noted that, when performing steps 204a to 204c, step 201 can be: the image processing device processes the first image through the image processing chip to obtain the second image.

[0063] In this way, the image processing device obtains the first timestamp corresponding to the shooting input through the image processing chip. It can directly retrieve at least one first image frame from the buffer based on the timestamp, without having to acquire the image based on the current shooting input, thereby reducing image acquisition latency. Therefore, it can effectively reduce the photo-taking latency during video recording.

[0064] In some embodiments of this application, the aforementioned N frames of images include the aforementioned third image and images acquired prior to the aforementioned third image. Exemplarily, in conjunction with... Figure 3 ,like Figure 4 As shown, the step 201 above, "processing at least one frame of the first image using an image processing chip to obtain a second image", can be achieved through the following steps 201a1 to 201a3:

[0065] Step 201a1: The image processing device performs color correction processing on each frame of the N frames of images through the image processing chip to obtain the corrected N frames of images.

[0066] In some embodiments of this application, the image processing device can use an image processing chip to perform color correction processing on each frame of an N-frame image using a color correction matrix. Color correction corrects the color cast of an image; it is a complementary color correction process based on the optical concepts of the three primary colors (Red, Green, Blue, RGB) and the three complementary colors (Cyan, Magenta, Yellow, CMY). The complementary color of a primary color is the sum of the other two primary colors. Among the three primary colors, the sum of red and green is yellow, the sum of green and blue is cyan, and the sum of red and blue is magenta. Color correction of an image ensures that the image's color space is converted to a color space perceptible to the human eye, thereby improving the image's visual effect.

[0067] In some embodiments of this application, before the image processing device performs color correction processing on each frame of the N frames of images through the image processing chip, it can also perform conventional image processing operations such as bad pixel correction processing and de-mosaic processing on each frame of the N frames of images through the image processing chip to improve image quality.

[0068] Step 201a2: The image processing device performs multi-frame noise reduction processing on the corrected N-frame images through the image processing chip to obtain a single frame image.

[0069] In some embodiments of this application, the image processing device performs multi-frame noise reduction processing on the corrected N-frame images by averaging the pixels at the same position on the corrected N-frame images or by weighting the pixels at the same position on the corrected N-frame images through an image processing chip, thereby obtaining a single frame image.

[0070] In some embodiments of this application, the image processing device can add the pixel values ​​of pixels at the same pixel position in the corrected N-frame image using an image processing chip to obtain the accumulated pixel value corresponding to the pixel position, and calculate the ratio between the accumulated pixel value corresponding to the pixel position and N; then, based on the ratio between the accumulated pixel value corresponding to the pixel position and N, a frame image is generated, which is a frame image after multi-frame noise reduction processing of the corrected N-frame image.

[0071] In some embodiments of this application, the image processing device can calculate the weighted value of pixels at the same pixel position in the corrected N-frame image using an image processing chip, and then generate a frame image based on the weighted value corresponding to the pixel position. This frame image is a frame image after multi-frame noise reduction processing of the corrected N-frame image.

[0072] Step 201a3: The image processing device uses an image processing chip to perform super-resolution processing and pixel format conversion processing on the denoised frame image to obtain a second image in the target format.

[0073] In some embodiments of this application, the image processing device can perform super-resolution processing on the aforementioned frame image using an image processing chip and an interpolation-based super-resolution image reconstruction algorithm. For example, nearest neighbor interpolation or bilinear interpolation algorithms can be used; however, this embodiment does not impose specific limitations. Exemplarily, nearest neighbor interpolation assigns the grayscale value of the nearest neighbor pixel to the pixel to be determined among its four neighboring pixels.

[0074] In some embodiments of this application, the image processing device can convert the color space of the aforementioned frame image through an image processing chip, thereby changing the pixel format of the pixels in the aforementioned frame image to obtain a second image in the target format.

[0075] In some embodiments of this application, the target format may be YUV444 format, YUV420 format, etc., and no specific limitation is made here.

[0076] For example, an image processing device can use an image processing chip to convert the color space of the aforementioned frame image from CMY to YUV420, thereby converting the pixel format of the aforementioned frame image to YUV420 format, and obtaining a second image in YUV420 format.

[0077] Thus, the image processing device performs color correction on the image using the image processing chip, improving the visual effect of the image; the image processing device performs multi-frame noise reduction on the image using the image processing chip to obtain a single frame image, which can effectively reduce image noise and thus improve the image quality of the single frame image; the image processing device performs super-resolution processing and pixel format conversion processing on the single frame image obtained after noise reduction using the image processing chip to obtain a second image in the target format, which can improve the resolution of the second image and thus improve the clarity of the second image.

[0078] In some embodiments of this application, the aforementioned N frames of images include N image pairs, each of which includes a long-exposure image and a short-exposure image; for example, the step 201 above, "processing at least one frame of the first image by an image processing chip to obtain a second image", can be implemented by the following steps 201b1 to 201b4:

[0079] Step 201b1: The image processing device performs color correction processing on the long exposure image in the N image pairs through the image processing chip to obtain N corrected long exposure images, and performs color correction processing on the short exposure image in the N image pairs to obtain N corrected short exposure images.

[0080] In some embodiments of this application, the aforementioned long exposure image refers to an image with a relatively long exposure time when the image sensor acquires the image, and the aforementioned short exposure image refers to an image with a relatively short exposure time when the image sensor acquires the image.

[0081] For example, the exposure time corresponding to the long exposure image can be a first preset time. The exposure time corresponding to the short exposure image can be a second preset time, and the second preset time is less than the first preset time; or, the exposure time corresponding to the short exposure image can be within a preset time range, the maximum time within the preset time range is a third preset time, and the third preset time is less than the first preset time.

[0082] For example, the first preset duration can be 10ms, the second preset duration can be 3ms, and the preset duration range can be 1ms to 4ms.

[0083] It should be noted that the color correction process in step 201b1 above can be referred to the description in step 201a1 above. To avoid repetition, this embodiment will not repeat it here.

[0084] Step 201b2: The image processing device performs multi-frame noise reduction processing on the corrected N-frame long exposure images through the image processing chip to obtain a long exposure image, and performs multi-frame noise reduction processing on the corrected N-frame short exposure images to obtain a short exposure image.

[0085] It should be noted that the multi-frame noise reduction process in step 201b2 above can be referred to the description in step 201a2 above. To avoid repetition, this embodiment will not repeat it here.

[0086] Step 201b3: The image processing device uses an image processing chip to fuse a long-exposure image obtained after noise reduction and a short-exposure image obtained after noise reduction to obtain a fourth image.

[0087] In some embodiments of this application, when the image sensor in the image processing device acquires images, it cannot rely on a single frame image to cover the high dynamic range. Therefore, the image processing chip can be used to fuse long exposure images and short exposure images.

[0088] In some embodiments of this application, when the image processing device performs fusion processing on the aforementioned long-exposure image and short-exposure image obtained after denoising, it can first identify the dark areas with poor signal-to-noise ratio in the short-exposure image obtained after denoising, and then determine the target location region of the dark area in the short-exposure image obtained after denoising. Finally, using the long-exposure image obtained after denoising as a reference, the image of the target location region in the long-exposure image obtained after denoising is overlaid on the dark area of ​​the short-exposure image obtained after denoising to obtain a fourth image. By performing fusion processing on the long-exposure image and the short-exposure image obtained after denoising, a fourth image with high dynamic range can be obtained.

[0089] Step 201b4: The image processing device performs super-resolution processing and pixel format conversion processing on the fourth image through the image processing chip to obtain the second image in the target format.

[0090] It should be noted that the fusion processing in step 201b4 above can be referred to the description in step 201a3 above. To avoid repetition, this embodiment will not repeat the description here.

[0091] Thus, the image processing device performs color correction on the image using the image processing chip, improving the visual effect of the image; the image processing device performs multi-frame noise reduction on the image using the image processing chip to obtain a single frame image, which can effectively reduce image noise and thus improve the image quality of the single frame image; the image processing device performs fusion processing on the single frame long exposure image and the single frame short exposure image obtained after noise reduction using the image processing chip, which can generate an image with high dynamic range; the image processing device performs super-resolution processing and pixel format conversion processing on the single frame image obtained after noise reduction using the image processing chip, which can improve the resolution of the second image and thus improve the clarity of the second image.

[0092] Step 202: The image processing device transmits the second image to the main control chip through the image processing chip.

[0093] In some embodiments of this application, the image processing device can transmit the second image to the main control chip via an image processing chip using CSI. CSI stands for Mobile Industry Processor Interface. ， MIPI is an interface specification.

[0094] Step 203: The image processing device encodes the second image through the main control chip and stores the encoded second image.

[0095] In some embodiments of this application, the image processing device can encode the second image using an image encoding algorithm, such as the Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Graphics Interchange Format (GIF), and High Efficiency Image File Format (HEIF), developed by the same expert group. Specific examples are not shown here. For instance, the image processing device encodes the second image using the JPEG algorithm via the main control chip, resulting in a JPEG format for the encoded second image. Alternatively, the image processing device can encode the second image using the PNG algorithm via the main control chip, resulting in a PNG format for the encoded second image.

[0096] In some embodiments of this application, the image processing device can store the encoded second image into the memory of the main control chip via the main control chip.

[0097] It is understandable that the encoded second image is the photo taken during the video recording process, that is, the image corresponding to the above-mentioned input image.

[0098] In the image processing method provided in this application, at least one frame of the first image can be processed by the image processing chip to obtain the second image, and then the second image can be encoded by the main control chip to generate a photograph, which can improve the image quality of the photograph generated during the recording process.

[0099] Furthermore, since the main control chip only needs to encode the second image to generate a photo without performing any other processing on the second image, the main control chip only occupies a small portion of computing resources when generating the photo. This allows the main control chip to allocate more computing resources to the recording function, thereby improving the smoothness and stability of the recording.

[0100] In some embodiments of this application, combined with Figure 3 ,like Figure 5 As shown, prior to step 201 above, the image processing method provided in this application further includes the following steps 205a and 205b:

[0101] Step 205a: The image processing device acquires at least one frame of image during the recording process.

[0102] In some embodiments of this application, the image processing apparatus may acquire at least one frame of image via an image sensor.

[0103] In some embodiments of this application, the image processing device acquires the above-mentioned at least one frame of image through an image sensor, and can transmit the above-mentioned at least one frame of image to the image processing chip through the image sensor using a first CSI.

[0104] Step 205b: The image processing device obtains at least one timestamp corresponding to at least one frame of image through the image processing chip, and stores at least one frame of image and at least one timestamp in the buffer of the image processing chip.

[0105] In some embodiments of this application, each of the at least one timestamp is the timestamp when one of the at least one frames of images was acquired.

[0106] In some embodiments of this application, the image processing apparatus may use an image processing chip to store the at least one frame of image and the timestamp corresponding to each frame of image in the above-mentioned buffer.

[0107] In some embodiments of this application, the image processing device can use an image processing chip to store the at least one frame of image and the timestamp corresponding to each frame of image in the above-mentioned buffer according to the order of timestamps.

[0108] In this way, the image processing device obtains at least one timestamp corresponding to at least one frame of image through the image processing chip, and stores at least one frame of image and at least one timestamp in the buffer of the image processing chip. This enables the image processing device to obtain N frames of image from the buffer of the image processing chip based on the first timestamp corresponding to the shooting input after obtaining the first timestamp, which can effectively reduce the delay in the image acquisition process and thus reduce the shooting delay in the video recording process.

[0109] In some embodiments of this application, the image processing method provided by this application further includes the following steps 206a and 206d:

[0110] Step 206a: The image processing device acquires at least one frame of image during the recording process.

[0111] In some embodiments of this application, the aforementioned at least one frame image includes at least one image group, and each image group includes a long exposure image and a medium exposure image.

[0112] In some embodiments of this application, the aforementioned intermediate exposure image refers to an image in which the exposure time of the image sensor during image acquisition is greater than the exposure time corresponding to the aforementioned short exposure image, but less than the exposure time corresponding to the aforementioned long exposure image. For example, the exposure time corresponding to the aforementioned intermediate exposure image can be a fourth preset time, which is less than the aforementioned first preset time and greater than the aforementioned second preset time, or greater than the aforementioned third preset time.

[0113] For example, when the first preset duration is 10ms, the second preset duration is 3ms, and the third preset duration is 4ms, the fourth preset duration can be 6ms.

[0114] It should be noted that the implementation process of step 207a can refer to the description in step 206a above. To avoid repetition, this embodiment will not repeat it here.

[0115] Step 206b: The image processing device uses an image processing chip to fuse each long exposure image and the corresponding medium exposure image to obtain at least one fused image.

[0116] It should be noted that the fusion processing in step 206b above can be referred to the description in step 201b3 above, and will not be repeated here in this embodiment.

[0117] Step 206c: The image processing device transmits at least one frame of fused image to the main control chip through the image processing chip.

[0118] It should be noted that the image transmission implementation process in step 206c above can be referred to the description in step 202 above, and will not be repeated here in this embodiment.

[0119] Step 206d: The image processing device performs video encoding on at least one frame of the fused image through the main control chip to obtain the recorded video.

[0120] In some embodiments of this application, the image processing device can perform color correction processing and pixel format conversion processing on each frame of the fused image in at least one frame of fused image through the main control chip to obtain at least one frame of fused image in the target format, and then encode the at least one frame of fused image in the target format to obtain a video recording.

[0121] It should be noted that the process of color correction and pixel format conversion for each frame of the above-mentioned at least one frame of fused image can be referred to the description in step 201a3 above. To avoid repetition, this embodiment will not repeat it here.

[0122] In some embodiments of this application, the image processing device can use a main control chip to perform video encoding on at least one frame of the fused image in the target format using video encoding algorithms such as H.263, H.264, Motion Joint Photographic Experts Group (M-JPEG), and Moving Picture Experts Group (MPEG). This embodiment does not impose specific limitations. For example, the image processing device uses the main control chip to perform video encoding on at least one frame of the fused image using the H.263 algorithm, resulting in a recorded video in H.263 format. The image processing device also uses the main control chip to encode a second image using the MPEG algorithm, resulting in a recorded video in MPEG format.

[0123] Thus, the image processing device, through the image processing chip, fuses each long-exposure image and the corresponding medium-exposure image to obtain an image with high dynamic range. The image processing device, through the main control chip, performs color correction and pixel format conversion on each frame of the fused image to obtain at least one frame of the target format fused image, thereby improving the resolution and clarity of the target format fused image. Furthermore, the main control chip performs video encoding on the target format fused image to obtain a recorded video, improving the dynamic range and clarity of the video images in the recorded video, thus enhancing the quality of the recorded video.

[0124] In some embodiments of this application, each of the above image groups further includes a short-exposure image frame, and the image processing method provided in this application further includes the following steps 207a and 207b:

[0125] Step 207a: During the recording process, the image processing device acquires image parameters through the image processing chip.

[0126] In some embodiments of this application, the above image parameters include at least one of the following: image brightness parameters and image motion parameters, wherein the image motion parameters are the motion parameters of moving objects in the image.

[0127] In some embodiments of this application, during the recording process, the image processing device can obtain the brightness of the environment where the image sensor is located through the image processing chip, and then use the brightness of the environment where the image sensor is located as the aforementioned image brightness parameter.

[0128] In some embodiments of this application, during video recording, the image processing device uses an image processing chip to perform inter-frame difference calculations on the pixel values ​​of corresponding pixels in two consecutively acquired frames of images by the image sensor to obtain the aforementioned image motion parameters. For example, if the differences in pixel values ​​in the same area of ​​the two frames are all within a preset range, then the average value of the differences in pixel values ​​in the same area of ​​the two frames is taken as the aforementioned image motion parameters.

[0129] Step 205b: The image processing device adjusts the exposure time when acquiring short-exposure images during the recording process based on the image parameters using the image processing chip.

[0130] In some embodiments of this application, when the exposure time corresponding to the short-exposure image is within a preset time range, the exposure time is negatively correlated with the image parameters. For example, the larger the value of the image parameters, the smaller the exposure time; conversely, the smaller the value of the image parameters, the larger the exposure time.

[0131] It should be noted that when the value of the image motion parameter is larger, reducing the exposure time when collecting short exposure images can avoid motion ghosting in the captured image due to excessive exposure time in moving scenes, thereby improving image quality.

[0132] It should be noted that a higher value for the image brightness parameter allows for a shorter exposure time when creating short-exposure images, which can prevent overexposure and thus improve image quality.

[0133] In this way, during the recording process, the image processing device obtains the image parameters through the image processing chip, and adjusts the exposure time when capturing short-exposure images based on the image parameters, which can effectively avoid motion ghosting and overexposure problems in the image, thereby effectively improving the image quality.

[0134] Below, in conjunction with Figure 6 Taking a mobile phone as an example, the image processing method provided in this application will be illustrated through specific embodiments. In Embodiment 1, the image processing method may include steps 301 to 308.

[0135] 301. The mobile application (APP) responds to the user's tap input on the camera and opens the video preview.

[0136] 302. The mobile phone turns on the image processing chip and transmits at least one frame of image captured by the image sensor as a video preview stream to the main control chip through the image processing chip.

[0137] 303. The image processing chip caches at least one frame of image acquired by the image sensor into the cache queue in the cache DDR through the first CSI.

[0138] 304. The mobile phone displays the video preview stream on the screen after it is processed by the ISP in the main control chip.

[0139] 305. The mobile app responds to the user's recording input and starts recording.

[0140] 306. The mobile phone will transmit the recorded video stream split from the ISP to the video encoding module for encoding.

[0141] 307. In response to the user's shooting input in video recording mode, the mobile phone obtains at least one frame of image from the cache queue of DDR cache through the image processing chip, performs multi-frame noise reduction, super-resolution and other algorithm processing to generate a YUV image and transmits it to the main control chip for subsequent shooting processing.

[0142] 308. After receiving the YUV image processed by the image processing chip through the second CSI, the main control chip sends it directly to the image encoding module to be processed into a photo for storage without going through ISP processing.

[0143] In this way, the mobile phone can use the image processing chip to perform multi-frame noise reduction, super-resolution, and other algorithms on the image to generate a YUV image. Then, when the main control chip encodes the YUV image to generate a photo, on the one hand, it improves the image quality of the photos generated during video recording; on the other hand, the main control chip only needs to encode the YUV image to generate the photo. Therefore, the main control chip uses only a small portion of its computing resources when generating photos, allowing it to allocate more computing resources to the video recording function, thereby improving the smoothness and stability of video recording.

[0144] The following is combined with Figure 7 Taking a mobile phone as an example, the image processing method provided in this application will be illustrated through a specific embodiment two. In embodiment two, the image processing method may include the following steps 401 to 408.

[0145] 401. The mobile app responds to the user's tap input on the camera and opens the video preview.

[0146] 402. The mobile phone turns on the image processing chip, and through the fusion module of the image processing chip, it fuses the long exposure image and the medium exposure image in at least one frame of the image sensor into an HDR image and transmits it to the main control chip as a video recording stream.

[0147] For example, at least one frame of an image includes at least one group of images, and a group of images includes a long exposure image, a medium exposure image, and a short exposure image.

[0148] 403. The image processing chip caches the long exposure image and short exposure image in at least one frame of image into the cache queue in the cache DDR through the first CSI.

[0149] 404. The mobile phone displays the video preview stream on the screen after it is processed by the ISP in the main control chip.

[0150] 405. The mobile app responds to the user's recording input and starts recording.

[0151] 406. The mobile phone will transmit the recorded video stream split from the ISP to the video encoding module for encoding.

[0152] 407. In response to the user's shooting input in video recording mode, the mobile phone obtains at least one frame of image from the cache queue of DDR cache through the image processing chip, performs multi-frame noise reduction, super-resolution and other algorithm processing to generate a YUV image and transmits it to the main control chip for subsequent shooting processing.

[0153] 408. After receiving the YUV image processed by the image processing chip through the second CSI, the main control chip sends it directly to the image encoding module to be processed into a photo for storage without going through ISP processing.

[0154] In this way, by acquiring three types of exposure images through the image sensor, the photo and video can be output separately. The exposure can be adjusted according to the needs of both photos and videos, thus solving the problem that adjusting the exposure parameters of the photos during single-frame recording can lead to deterioration of video noise or flickering brightness. This improves the image quality of both the recorded video and the photos.

[0155] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0156] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0157] Figure 8 This is a schematic diagram of the structure of the image processing apparatus provided in an embodiment of this application. Figure 8 As shown, the image processing device 800 includes: an image processing chip 801 and a main control chip 802; the image processing chip 801 is used to process at least one frame of a first image to obtain a second image when a shooting input is received during video recording, wherein at least one frame of the first image is obtained based on the shooting input; the image processing chip 801 is also used to transmit the second image to the main control chip; the main control chip is used to encode the second image and store the encoded second image.

[0158] In some embodiments of this application, the image processing chip 801 is further configured to: obtain a first timestamp corresponding to the shooting input before processing at least one frame of the first image to obtain a second image; obtain N frames of images from the buffer of the image processing chip 801, wherein the buffer stores at least one frame of images captured during the recording process; determine the N frames of images as at least one frame of the first image; wherein the N frames of images include a third image corresponding to the first timestamp in the at least one frame of images, or include the third image and an image captured before the third image, and N is a positive integer.

[0159] In some embodiments of this application, the N-frame images include a third image and images acquired before the third image; the image processing chip 801 is specifically used for: performing color correction processing on each frame of the N-frame images to obtain the corrected N-frame images; performing multi-frame noise reduction processing on the corrected N-frame images to obtain a single frame image; and performing super-resolution processing and pixel format conversion processing on the single frame image obtained after noise reduction to obtain a second image in the target format.

[0160] In some embodiments of this application, the N-frame image includes N image pairs, each image pair including a long exposure image and a short exposure image; the image processing chip 601 is specifically used for: performing color correction processing on the long exposure image in the N image pairs to obtain corrected N-frame long exposure images, and performing color correction processing on the short exposure image in the N image pairs to obtain corrected N-frame short exposure images; performing multi-frame noise reduction processing on the corrected N-frame long exposure images to obtain a long exposure image, and performing multi-frame noise reduction processing on the corrected N-frame short exposure images to obtain a short exposure image; performing fusion processing on the noise-reduced long exposure image and the noise-reduced short exposure image to obtain a fourth image; and performing super-resolution processing and pixel format conversion processing on the fourth image to obtain a second image in the target format.

[0161] In some embodiments of this application, combined with Figure 8 ,like Figure 9As shown, the image processing device 800 also includes an image sensor; the image sensor 803 is used to acquire at least one frame of image during the recording process before obtaining a second image by processing at least one frame of first image through the image processing chip when receiving shooting input during the recording process; the image processing chip 801 is also used to acquire at least one timestamp corresponding to at least one frame of image, and store at least one frame of image and the at least one timestamp in the buffer of the image processing chip 801.

[0162] In some embodiments of this application, an image sensor 803 is used to acquire at least one frame of image during video recording. The at least one frame of image includes at least one image group, and each image group includes a long exposure image and a medium exposure image. The image processing chip is further used to fuse each long exposure image and the corresponding medium exposure image to obtain at least one fused image. The image processing chip is further used to transmit the at least one fused image to the main control chip. The main control chip is further used to perform video encoding on the at least one fused image to obtain a recorded video.

[0163] In some embodiments of this application, each image group further includes a short-exposure image. The image processing chip 801 is also used to: acquire image parameters during the recording process, the image parameters including at least one of the following: image brightness parameters, image motion parameters, the image motion parameters being the motion parameters of moving objects in the image; and adjust the exposure time when acquiring short-exposure images during the recording process based on the image parameters.

[0164] In the image processing apparatus provided in this application embodiment, when a shooting input is received during video recording, at least one frame of a first image is processed by an image processing chip to obtain a second image, wherein the at least one frame of the first image is obtained based on the shooting input; the second image is transmitted to the main control chip by the image processing chip; the second image is encoded by the main control chip and stored. That is, additional images captured during video recording are processed and encoded by the image processing chip and the main control chip respectively, avoiding the main control chip lacking sufficient computing resources to perform additional image processing on the additional captured images, thus improving the image quality of the photos generated during video recording.

[0165] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device, augmented reality / virtual reality device, robot, wearable device, super mobile personal computer, netbook, or personal digital assistant, etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific device.

[0166] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0167] The image processing apparatus provided in this application embodiment can implement the various processes implemented in the various embodiments of the above image processing method. To avoid repetition, it will not be described again here.

[0168] Optionally, such as Figure 10 As shown, this application embodiment also provides an electronic device 1000, including a processor 1001 and a memory 1002. The memory 1002 stores a program or instructions that can run on the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0169] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0170] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0171] The electronic device 1100 includes, but is not limited to, components such as: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1110.

[0172] Those skilled in the art will understand that the electronic device 1100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0173] The processor 1110 is configured to: when receiving shooting input during video recording, process at least one frame of the first image through an image processing chip to obtain a second image, wherein the at least one frame of the first image is obtained based on the shooting input; transmit the second image to the main control chip through the image processing chip; encode the second image through the main control chip and store the encoded second image.

[0174] In some embodiments of this application, the processor 1110 is further configured to: obtain a first timestamp corresponding to the shooting input by the image processing chip before processing at least one frame of the first image by the image processing chip to obtain the second image; obtain N frames of images from the buffer of the image processing chip by the image processing chip, wherein the buffer stores at least one frame of image captured during the recording process; and determine the N frames of images as the first image by the image processing chip; wherein the N frames of images include a third image corresponding to the first timestamp in at least one frame of image, or include the third image and images captured before the third image, and N is a positive integer.

[0175] In some embodiments of this application, the N-frame images include the third image and images acquired before the third image; the processor 1110 is specifically configured to: perform color correction processing on each frame of the N-frame images using an image processing chip to obtain corrected N-frame images; perform multi-frame noise reduction processing on the corrected N-frame images using an image processing chip to obtain one frame image; and perform super-resolution processing and pixel format conversion processing on the noise-reduced one frame image using an image processing chip to obtain a second image in the target format.

[0176] In some embodiments of this application, the N-frame image includes N image pairs, each image pair including a long exposure image and a short exposure image; the processor 1110 is specifically configured to: perform color correction processing on the long exposure image in the N image pairs using an image processing chip to obtain corrected N-frame long exposure images, and perform color correction processing on the short exposure image in the N image pairs to obtain corrected N-frame short exposure images; perform multi-frame noise reduction processing on the corrected N-frame long exposure images using an image processing chip to obtain a long exposure image, and perform multi-frame noise reduction processing on the corrected N-frame short exposure images to obtain a short exposure image; perform fusion processing on the noise-reduced long exposure image and the noise-reduced short exposure image using an image processing chip to obtain a fourth image; and perform super-resolution processing and pixel format conversion processing on the fourth image using an image processing chip to obtain a second image in the target format.

[0177] In some embodiments of this application, the processor 1110 is further configured to: before processing at least one frame of the first image through the image processing chip to obtain the second image when receiving the shooting input during the recording process, capture at least one frame of the image during the recording process; obtain at least one timestamp corresponding to the at least one frame of the image through the image processing chip, and store the at least one frame of the image and at least one timestamp in the buffer of the image processing chip.

[0178] In some embodiments of this application, the processor 1110 is further configured to: acquire at least one frame of image during recording, the at least one frame of image including at least one image pair, each image pair including a long exposure image and a medium exposure image; perform fusion processing on each long exposure image and the corresponding medium exposure image through an image processing chip to obtain at least one fused image; transmit the at least one fused image to the main control chip through the image processing chip; and perform video encoding on the at least one fused image through the main control chip to obtain a recorded video.

[0179] In some embodiments of this application, each image group further includes a short-exposure image. The processor 911 is also configured to: during the recording process, acquire image parameters during the recording process through an image processing chip, the image parameters including at least one of the following: image brightness parameters, image motion parameters, the image motion parameters being the motion parameters of moving objects in the image; and adjust the exposure duration when acquiring short-exposure images during the recording process based on the image parameters through the image processing chip.

[0180] In the electronic device provided in this application embodiment, when a shooting input is received during video recording, at least one frame of the first image is processed by the image processing chip to obtain a second image, wherein the at least one frame of the first image is obtained based on the shooting input; the second image is transmitted to the main control chip by the image processing chip; the second image is encoded by the main control chip and stored. That is, the additional images captured during the video recording process are generated by the image processing chip and the main control chip performing image processing and encoding operations respectively, avoiding the main control chip lacking extra computing power resources to perform additional image processing on the additional captured images, thus improving the image quality of the photos generated during the video recording process.

[0181] It should be understood that, in this embodiment, the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042. The GPU 11041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1106 may include a display panel 11061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0182] The memory 1109 can be used to store software programs and various data. The memory 1109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0183] Processor 1110 may include one or more processing units; optionally, processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1110.

[0184] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0185] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0186] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0187] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0188] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0189] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0190] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0191] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method applied to an image processing device, characterized by, The image processing device comprises a master chip and an image processing chip, and comprises: In the case that a shooting input is received in a video recording process, at least one frame of first image is processed by the image processing chip to obtain a second image, the at least one frame of first image is obtained based on the shooting input; The second image is transmitted to the master chip by the image processing chip; The second image is encoded by the master chip, and the encoded second image is stored.

2. The method of claim 1, wherein, Before the first image is processed by the image processing chip to obtain the second image, the method further comprises: A first timestamp corresponding to the shooting input is acquired by the image processing chip; N frames of images in a buffer are acquired by the image processing chip, and the buffer stores at least one frame of image collected in a video recording process; The N frames of images are determined as the at least one frame of first image by the image processing chip; The N frames of images comprise a third image corresponding to the first timestamp in the at least one frame of image, or comprise the third image and an image collected before the third image, and N is a positive integer.

3. The method of claim 2, wherein, The N frames of images comprise the third image and the image collected before the third image; The at least one frame of first image is processed by the image processing chip to obtain the second image, comprising: Color correction processing is performed on each frame of image in the N frames of images by the image processing chip to obtain the N frames of corrected images; Multi-frame noise reduction processing is performed on the N frames of corrected images by the image processing chip to obtain one frame of image; Super-resolution processing and pixel format conversion processing are performed on the one frame of image obtained after noise reduction by the image processing chip to obtain the second image in a target format.

4. The method according to claim 2 or 3, characterized in that, The N frames of images comprise N image pairs, and each image pair comprises one frame of long exposure image and one frame of short exposure image; The first image is processed by the image processing chip to obtain the second image, comprising: Color correction processing is performed on the long exposure images in the N image pairs by the image processing chip to obtain N frames of corrected long exposure images, and color correction processing is performed on the short exposure images in the N image pairs by the image processing chip to obtain N frames of corrected short exposure images; Multi-frame noise reduction processing is performed on the N frames of corrected long exposure images by the image processing chip to obtain one frame of long exposure image, and multi-frame noise reduction processing is performed on the N frames of corrected short exposure images by the image processing chip to obtain one frame of short exposure image; Fusion processing is performed on the one frame of long exposure image obtained after noise reduction and the one frame of short exposure image obtained after noise reduction by the image processing chip to obtain a fourth image; Super-resolution processing and pixel format conversion processing are performed on the fourth image by the image processing chip to obtain the second image in a target format.

5. The method of claim 2, wherein, Before the first image is processed by the image processing chip to obtain the second image in the case that a shooting input is received in a video recording process, the method further comprises: The at least one frame of image is collected in a video recording process; acquire at least one timestamp corresponding to the at least one frame of image by the image processing chip, and store the at least one frame of image and the at least one timestamp in a buffer of the image processing chip.

6. The method of claim 1, wherein, The method further comprises: acquiring at least one frame of image during the recording process, the at least one frame of image comprising at least one image group, each image group comprising one frame of long-exposure image and one frame of medium-exposure image; performing fusion processing on each frame of long-exposure image and the corresponding medium-exposure image of each frame of long-exposure image by the image processing chip to obtain at least one frame of fused image; transmitting the at least one frame of fused image to the main control chip by the image processing chip; performing video encoding on the at least one frame of fused image by the main control chip to obtain a recording video.

7. The method of claim 6, wherein, Each image group further comprises one frame of short-exposure image, and the method further comprises: acquiring image parameters during the recording process by the image processing chip, the image parameters comprising at least one of the following: image brightness parameter, image motion parameter, the image motion parameter being a motion parameter of a moving object in the image; adjusting the exposure time length of the short-exposure image based on the image parameters by the image processing chip.

8. An image processing apparatus characterized by comprising: comprise: a main control chip and an image processing chip; the image processing chip is configured to, in a case where a shooting input is received during the recording process, perform processing on at least one frame of first image to obtain a second image, the at least one frame of first image being obtained based on the shooting input; the image processing chip is further configured to transmit the second image to the main control chip; the main control chip is configured to encode the second image and store the encoded second image.

9. The apparatus of claim 8, wherein, the image processing chip is further configured to: acquire a first timestamp corresponding to the shooting input before performing processing on the at least one frame of first image to obtain the second image; acquire N frames of image from a buffer of the image processing chip, the buffer storing at least one frame of image acquired during the recording process; determine the N frames of image as the first image; wherein the N frames of image comprise a third image corresponding to the first timestamp in the at least one frame of image, or comprise the third image and an image acquired before the third image, N being a positive integer.

10. The apparatus of claim 7, wherein, The device further comprises an image sensor. the image sensor is configured to acquire at least one frame of image during the recording process, the at least one frame of image comprising at least one image group, each image group comprising one frame of long-exposure image and one frame of medium-exposure image; the image processing chip is further configured to perform fusion processing on each frame of long-exposure image and the corresponding medium-exposure image of each frame of long-exposure image to obtain at least one frame of fused image; the image processing chip is further configured to transmit the at least one frame of fused image to the main control chip; the main control chip is further configured to perform video encoding on the at least one frame of fused image to obtain a recording video.

11. The apparatus of claim 10, wherein, Each image group further comprises one frame of short-exposure image, and the image processing chip is further configured to During the video recording, an image parameter during the video recording is acquired, the image parameter comprising at least one of: an image brightness parameter, an image motion parameter, the image motion parameter being a motion parameter of a moving object in the image; an exposure time of the short-exposure image is adjusted based on the image parameter.

12. An electronic device, comprising: An image processing device comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the image processing method according to any one of claims 1 to 7.

13. A readable storage medium, characterized by, A readable storage medium storing programs or instructions, the programs or instructions being executed by a processor to implement the steps of the image processing method according to any one of claims 1 to 7.

Images