Image processing method, chip, electronic equipment and storage medium

Abstract

The invention provides an image processing method, a chip, electronic equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of images; performing post-processing based on the multiple frames of images while performing time domain noise reduction based on the multiple frames of images; and obtaining a second image based on target grid data and a first image, the target grid data being grid data obtained after the post-processing, and the first image being a single-frame image obtained after the time domain noise reduction. According to the method provided by the invention, the image processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, chip, electronic device and storage medium. Background Art

[0002] In the camera link of the Image Signal Processor (ISP), after the image is converted from the RGB domain to the YUV domain, a noise reduction operation is usually performed in the time domain. Temporal noise reduction refers to the fusion of multiple frames of images combined with the previous and next frames to obtain a fused single-frame image to achieve the purpose of temporal noise reduction. Next, post-processing operations such as single-frame noise reduction, contrast adjustment, and saturation adjustment in the spatial domain will be performed. Among them, spatial noise reduction refers to the post-processing operations on the fused single-frame image in the horizontal and vertical spatial dimensions to achieve the purpose of spatial noise reduction.

[0003] Although the above-mentioned time domain noise reduction and post-processing operations can achieve the purpose of image noise reduction, the image processing flow of the above-mentioned method is time-consuming and inefficient. Summary of the invention

[0004] The present application provides an image processing method, a chip, an electronic device and a storage medium, which help to improve image processing efficiency.

[0005] In a first aspect, the present application provides an image processing method, the method comprising: acquiring multiple frames of images; performing post-processing based on the multiple frames of images while performing time domain noise reduction based on the multiple frames of images; obtaining a second image based on target grid data and a first image, wherein the target grid data is grid data obtained after the post-processing, and the first image is a single frame image obtained after the time domain noise reduction.

[0006] In the present application, by processing time domain denoising and post-processing operations in parallel, there is no need to perform post-processing operations on a single frame image after time domain denoising, thereby improving the efficiency of image processing.

[0007] In one possible implementation manner, the time domain noise reduction includes processing by a central processing unit CPU, and the post-processing includes processing by a graphics processing unit GPU.

[0008] In one possible implementation manner, the target grid image includes a three-dimensional index, and the three-dimensional index includes a two-dimensional coordinate and a pixel value of a pixel.

[0009] In one possible implementation manner, the multiple frames of images include multiple frames of YUV images, and the grid data includes grid data of a YUV channel.

[0010] In one possible implementation, the post-processing based on the multiple frames of images includes: downsampling the multiple frames of images; constructing first grid data and second grid data based on the downsampled multiple frames of images, wherein a first grid node in the first grid data has a first pixel value, a second grid node in the second grid data has a number of pixels corresponding to the first number of pixels, and the first grid node and the second grid node have the same three-dimensional index; and obtaining target grid data based on the first grid data and the second grid data.

[0011] In one possible implementation, the downsampling includes spatial downsampling and intensity downsampling, wherein the spatial downsampling is used to downsample the image resolution, and the intensity downsampling is used to downsample the pixel value.

[0012] In one possible implementation, obtaining target grid data based on the first grid data and the second grid data includes: filtering the first grid data and the second grid data; and obtaining target grid data based on the filtered first grid data and the filtered second grid data.

[0013] In a second aspect, the present application provides a chip, comprising one or more functional modules, wherein the one or more functional modules are used to execute the image processing method as described in the first aspect.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store a computer program; the processor is used to run the computer program to implement the image processing method as described in the first aspect.

[0015] In a fourth aspect, the present application provides a readable storage medium, in which a program is stored. When the program is executed on an electronic device, the electronic device implements the image processing method as described in the first aspect.

[0016] In a fifth aspect, the present application provides a program, which, when executed on a processor of an electronic device, enables the electronic device to execute the image processing method as described in the first aspect.

[0017] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of an image processing method of the related art;

[0019] Figure 2A schematic diagram of a flow chart of an embodiment of the image processing method provided by the present application;

[0020] Figure 3 A flowchart of another embodiment of the image processing method provided by the present application;

[0021] Figure 4A-4C A schematic diagram of the grid data construction method provided for this application;

[0022] Figure 5 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the embodiments of the present application, unless otherwise specified, the character " / " indicates that the objects before and after the association are in an or relationship. For example, A / B can represent A or B. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0025] It should be pointed out that the words "first", "second", etc. involved in the embodiments of the present application are only used to distinguish the description purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.

[0026] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C may represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C may be an element itself, or a set containing one or more elements.

[0027] In the embodiments of the present application, "exemplary", "in some embodiments", "in another embodiment", etc. are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0028] In the embodiments of the present application, "of", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. In the embodiments of the present application, communication and transmission can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are consistent. For example, transmission can include sending and / or receiving, which can be a noun or a verb.

[0029] The equal to involved in the embodiments of the present application can be used in conjunction with greater than, and is applicable to the technical solution adopted when greater than, and can also be used in conjunction with less than, and is applicable to the technical solution adopted when less than. It should be noted that when equal to is used in conjunction with greater than, it cannot be used in conjunction with less than; when equal to is used in conjunction with less than, it cannot be used in conjunction with greater than.

[0030] In the camera link of the Image Signal Processor (ISP), after the image is converted from the RGB domain to the YUV domain, a noise reduction operation is usually performed in the time domain. Temporal noise reduction refers to the fusion of multiple frames of images combined with the previous and next frames to obtain a single-frame fused image to achieve the purpose of temporal noise reduction. Next, post-processing operations such as single-frame noise reduction, contrast adjustment, and saturation adjustment will be performed in the spatial domain. Among them, spatial noise reduction refers to the post-processing operation of the single-frame fused image in the horizontal and vertical spatial dimensions to achieve the purpose of spatial noise reduction.

[0031] Figure 1 The schematic diagram of the flow chart of the image processing method of the related art is shown as an example, and includes the following steps:

[0032] Step 101, acquiring multiple frames of images.

[0033] Step 102: Perform time domain noise reduction on multiple frame images to obtain a single frame image.

[0034] Step 103, performing post-processing operations on the single frame image to obtain an output image.

[0035] Among them, post-processing operations include single-frame noise reduction in the spatial domain, contrast adjustment, saturation adjustment and other operations.

[0036] Although the above-mentioned time domain noise reduction and post-processing operations can achieve the purpose of image noise reduction, the image processing flow of the above-mentioned method is time-consuming and inefficient.

[0037] Based on the above problems, an embodiment of the present application proposes an image processing method.

[0038] The image processing method provided in the present application is applied to an electronic device having an ISP chip. The electronic device may be a fixed terminal, such as a desktop computer, a surveillance camera, a monitor, etc. Alternatively, the electronic device may be a mobile terminal, such as a mobile phone, a tablet, a vehicle-mounted terminal, etc. The embodiments of the present application do not specifically limit the type of electronic device.

[0039] Now combined Figure 2 The image processing method provided in the embodiment of the present application is exemplified.

[0040] Figure 2 The flowchart of the image processing method provided in the embodiment of the present application specifically includes the following steps:

[0041] Step 201, acquiring multiple frames of images.

[0042] Specifically, the multi-frame images may be images in a YUV format, and the images in the YUV format may be obtained by converting images in an RGB format obtained by shooting.

[0043] Step 202: input multiple frames of images into a central processing unit (CPU), perform time domain noise reduction processing, and obtain a first image.

[0044] Specifically, after a plurality of frames of images are acquired, the plurality of frames of images may be input into a CPU for time domain noise reduction processing to obtain a first image, so as to achieve the purpose of time domain noise reduction.

[0045] Among them, time domain noise reduction can be achieved by fusing multiple frames of images. The specific method of fusing multiple frames of images can be referred to the description in the relevant technology, and will not be repeated here.

[0046] It is understandable that during the fusion process, the details and structure of the image can be maintained. In addition, multiple frames of images can be aligned to avoid image quality degradation due to position deviation during the fusion process.

[0047] The alignment operation may include but is not limited to block alignment, ORB alignment, etc.

[0048] Step 203: input multiple frames of images into a graphics processing unit (GPU) and perform post-processing to obtain grid data.

[0049] Specifically, the grid data may include pixel values.

[0050] The index of the grid data may be three-dimensional data, for example, the index of the grid data may include coordinate values ​​and pixel values ​​of pixels, the coordinate values ​​may include values ​​of two-dimensional coordinates of abscissa (e.g., x direction) and ordinate (e.g., y direction), and the pixel value may be used as a third dimension value. The grid data may be embodied in the form of a data table, or the grid data may be embodied in other data forms, which is not particularly limited in the embodiments of the present application.

[0051] It is understandable that step 202 and step 203 can be executed simultaneously. Since the processing time of time domain noise reduction in step 202 is relatively long, the post-processing operation in the related technology must be performed after the time domain noise reduction is completed, which results in a longer time for the entire image processing. The present application performs post-processing operations while performing time domain noise reduction, thereby reducing the duration of the entire image processing and improving the efficiency of the entire image processing.

[0052] Step 204: obtaining a second image based on the first image and the grid data.

[0053] Specifically, after obtaining the first image and grid data, the grid data can be mapped back to the first image through a slicing operation using the original pixel value index, that is, the post-processing in the grid data can be reflected in the first image, thereby obtaining a second image after post-processing the first image.

[0054] Among them, the specific method of slicing operation can refer to the specific description in the relevant technology, which will not be repeated here.

[0055] Next, through Figure 3 The post-processing operation in step 203 is exemplarily described. Step 203 may include the following steps:

[0056] Step 2031, down-sampling multiple frames of images.

[0057] Specifically, in order to reduce time consumption, each frame of the multiple frames of images may be downsampled.

[0058] The sampling rate may include a spatial sampling rate and an intensity sampling rate. The spatial sampling rate is the sampling rate of the image resolution, and the intensity sampling rate is the sampling rate of the pixel value of each pixel in the image.

[0059] For example, taking the image resolution as x*y, assuming the spatial sampling rate is Ss, the image resolution after downsampling is (x / Ss)*(y / Ss).

[0060] For another example, taking the pixel value of a certain pixel P in an image as K, assuming that the intensity sampling rate is Sr, the pixel value of the pixel P after downsampling is K / Sr.

[0061] It is understandable that grid data can be obtained through post-processing operations, and the grid data can be three-dimensional data. The grid data can include multiple grid nodes, and each grid node of the grid data can include the two-dimensional coordinates and pixel values ​​of the corresponding pixel.

[0062] Among them, the larger the sampling rate, the fewer the number of grid nodes and the less time it takes; the smaller the sampling rate, the more the number of grid nodes and the more time it takes.

[0063] It should be noted that, in the process of downsampling each frame of the image, the Y channel, the U channel and the V channel may be downsampled respectively. By performing post-processing operations on the Y channel, the U channel and the V channel respectively, the grid data of the Y channel, the U channel and the V channel may be obtained respectively, that is, the grid data obtained by the post-processing operation may include the grid data of the Y channel, the U channel and the V channel.

[0064] Step 2032: construct first grid data and second grid data based on the downsampled image.

[0065] Each grid node in the first grid data may include a pixel value, and the index of each grid node in the first grid data may be three-dimensional data, for example, the three-dimensional data may be a pixel coordinate value and a pixel value.

[0066] Each grid node in the second grid data may include the number of pixels, and the index of each grid node in the second grid data may be three-dimensional data, for example, the three-dimensional data may be a pixel coordinate value and a pixel value.

[0067] In some optional embodiments, before constructing the first grid data and the second grid data based on the downsampled image, operations such as contrast adjustment and saturation adjustment may be performed on the downsampled image.

[0068] Among them, the contrast adjustment method may include but is not limited to a histogram equalization method. In some embodiments, it may also be implemented by a contrast enhancement algorithm, which is not particularly limited in the embodiments of the present application.

[0069] After the downsampled image is acquired, the first grid data and the second grid data may be constructed based on the downsampled image, wherein the grid data may include the first grid data and the second grid data of three channels: the Y channel, the U channel and the V channel.

[0070] Next, take the construction of the first grid data and the second grid data of the Y channel as an example, and combine Figure 4A-4C The construction method of the first grid data and the second grid data is exemplified.

[0071] refer to Figure 4A, assuming that the multi-frame image may include three frame images, frame 1, frame 2 and frame 3, image 401 is the image of the Y channel of frame 1, and the image resolution of image 401 is x*y.

[0072] It can be understood that although the above example only uses 3 frames of images as an example for illustrative description, it does not constitute a limitation on the embodiments of the present application. In some embodiments, the above multiple frames of images may also include scenes that are less than 3 frames or greater than 3 frames.

[0073] Next, by traversing each pixel in the image 401, the first grid data 410 and the second grid data 420 can be obtained. Each grid node in the first grid data 410 includes a pixel value, and each grid node has a corresponding three-dimensional index, which can include the two-dimensional coordinates and pixel values ​​of the pixel. Each grid node in the second grid data 420 includes the number of pixels, and each grid node has a corresponding three-dimensional index, which can include the two-dimensional coordinates and pixel values ​​of the pixel.

[0074] Next, assume that the spatial sampling rate is Ss and the intensity sampling rate is Sr.

[0075] Exemplarily, the initial value of the number of pixels of each grid node in the first grid image 410 may be 0, and the initial value of the number of pixels of each grid node in the second grid image 420 may be 0.

[0076] Taking the first pixel P1 in the first row of image 401 as an example, the two-dimensional coordinates of pixel P1 are (0, 0), and the pixel value of pixel P1 is z1, that is, the x coordinate x1 of pixel P1 is 0, and the y coordinate y1 of pixel P1 is 0.

[0077] Next, assume that the spatial sampling rate is Ss=2 and the intensity sampling rate is Sr.

[0078] The two-dimensional coordinates corresponding to the pixel point P1 in the first grid data 410 are the two-dimensional coordinates obtained by downsampling the two-dimensional coordinates of the pixel point P1.

[0079] Among them, the two-dimensional coordinates of the pixel point P1 after downsampling can be (x11, y11), x11=floor(x1 / Ss)=0, y11=floor(y2 / Ss)=0, that is, the two-dimensional coordinates of the pixel point P1 after downsampling are still (0, 0).

[0080] The pixel value of the pixel point P1 after downsampling may be z11=z1 / Sr, thereby obtaining the grid node Q1 corresponding to the pixel point P1 in the first grid data 410 and the grid node Q2 corresponding to the pixel point P1 in the second grid data 420.

[0081] Among them, the three-dimensional index of the grid node Q1 and the grid node Q2 is (x11, y11, z11), that is, the three-dimensional index of the grid node Q1 and the grid node Q2 is (0, 0, z1 / Sr).

[0082] Next, the pixel value z11 after downsampling of the pixel point P1 can be written into the grid node Q1, and the accumulated number of pixels in the grid node Q1 can be written into Q2. On the basis that the initial value of the pixel value of the grid node Q1 is 0, the pixel value of the grid node Q1 can be updated to the pixel value after downsampling of the pixel point P1 (i.e., z11). In addition, on the basis that the initial value of the number of pixels of the grid node Q1 is 0, the three-dimensional index (0, 0, z1 / Sr) corresponds to only one pixel point (i.e., the pixel point P1), and therefore, the number of pixels of the grid node Q2 can be updated to 1.

[0083] Next, take the second pixel P2 in the first row of image 401 as an example, refer to Figure 4B , the coordinates of pixel point P2 are (1, 0), and the pixel value of pixel point P2 is z2, that is, the x coordinate of pixel point P2 is x2=1, and the y coordinate of pixel point P2 is y2=0.

[0084] By downsampling the coordinates of the pixel point P2, the two-dimensional coordinates (x12, y12) of the pixel point P2 after downsampling can be obtained, where x12=floor(x2 / Ss)=0, y12=floor(y2 / Ss)=0, that is, the two-dimensional coordinates of the pixel point P2 after downsampling are still (0, 0).

[0085] The pixel value of the pixel point P2 after downsampling may be z12=z2 / Sr.

[0086] Assuming z12=z11, the three-dimensional index of the grid node of the pixel point P2 in the first grid data 410 is the same as the grid node Q1, and the three-dimensional index of the grid node of the pixel point P2 in the second grid data 420 is the same as the grid node Q2, that is, the grid node corresponding to the pixel point P2 in the first grid data 410 is Q1, and the grid node corresponding to the pixel point P2 in the second grid data 420 is Q2.

[0087] Next, the values ​​of the grid nodes Q1 and Q2 can be updated based on the pixel point P2. For example, the pixel value corresponding to the grid node Q1 can be updated to z11+z12. In addition, the three-dimensional index (0, 0, z12 / Sr) corresponds to two pixels (i.e., pixel point P1 and pixel point P2), so the number of pixels of the grid node Q2 is updated to 2.

[0088] Next, take the third pixel P3 in the first row of image 401 as an example, refer to Figure 4C, the coordinates of the pixel point P3 are (2, 0), and the pixel value of the pixel point P3 is z3, that is, the x coordinate of the pixel point P3 is x3=2, and the y coordinate of the pixel point P3 is y3=0.

[0089] By downsampling the coordinates of the pixel point P3, the two-dimensional coordinates (x13, y13) of the pixel point P3 after downsampling can be obtained, where x13=floor(x3 / Ss)=1, y13=floor(y3 / Ss)=0, that is, the two-dimensional coordinates of the pixel point P3 after downsampling are still (1, 0).

[0090] The pixel value of the pixel point P3 after downsampling may be z13=z3 / Sr, thereby obtaining the grid node Q3 corresponding to the pixel point P3 in the first grid data 410 and the grid node Q4 corresponding to the pixel point P3 in the second grid data 420.

[0091] Among them, the three-dimensional index of the grid node Q3 and the grid node Q4 is (x13, y13, z13), that is, the three-dimensional index of the grid node Q3 and the grid node Q4 is (1, 0, z3 / Sr).

[0092] Next, the pixel value z13 after downsampling of the pixel point P3 can be written into the grid node Q3, and the accumulated number of pixels in the grid node Q3 can be written into Q4. On the basis that the initial value of the pixel value of the grid node Q3 is 0, the pixel value of the grid node Q3 is updated to the pixel value after downsampling of the pixel point P3 (i.e., z13). In addition, on the basis that the initial value of the number of pixels of the grid node Q3 is 0, the three-dimensional index (1, 0, z3 / Sr) corresponds to only one pixel point (i.e., the pixel point P3), and therefore, the number of pixels of the grid node Q4 is updated to 1.

[0093] It can be understood that other pixels in the image 401 can be written into the first grid data 410 and the second grid data 420 in the manner of the above embodiment, and all pixels in frame 2 and frame 3 can be written into the first grid data 410 and the second grid data 420 in the manner of the above embodiment, thereby obtaining the first grid data 410 and the second grid data 420 of the Y channel, which will not be repeated here.

[0094] It can be understood that the construction method of the first grid data 410 and the second grid data 420 of the U channel and the V channel can specifically refer to the construction method of the first grid data 410 and the second grid data 420 of the Y channel mentioned above, which will not be repeated here.

[0095] It can be seen that the present application constructs grid data based on multiple frame images, that is, the grid data can be constructed using the information of multiple frames at the same time, and the information of multiple frame images can be integrated into the grid data, thereby enriching the data information so that the grid data and the first image can jointly restore the desired two-dimensional image.

[0096] Step 2033: Obtain third grid data based on the first grid data and the second grid data.

[0097] Specifically, when the first grid data and the second grid data are obtained, spatial noise reduction processing can be performed based on the first grid data and the second grid data, thereby obtaining the first grid data and the second grid data after noise reduction, and the third grid data can be obtained based on the first grid data and the second grid data after noise reduction.

[0098] It can be understood that the third grid data can be used to fuse with the first image to generate the second image.

[0099] Exemplarily, assuming that the first grid data is T1 and the second grid data is T2, the first grid data after noise reduction may be T1' and the second grid data after noise reduction may be T2'.

[0100] It is understandable that spatial noise reduction can be achieved through Gaussian filtering, or spatial noise reduction can be achieved through other methods, and the embodiments of the present application do not specifically limit this.

[0101] Taking Gaussian filtering to achieve spatial noise reduction as an example, the Gaussian filtering method can be implemented by the following formula:

[0102] T1' = G*T1;

[0103] T2' = G*T2;

[0104] Where G is a three-dimensional Gaussian kernel.

[0105] Next, a division operation is performed based on the first grid data after denoising and the second grid data after denoising to obtain third grid data.

[0106] For example,

[0107] T3 = T1'. / T2'.

[0108] Among them, T3 is the third grid data.

[0109] Compared with the spatial domain denoising based on two-dimensional images in the prior art, the spatial domain denoising based on two-dimensional images usually performs edge-preserving operations in order to blur out isolated noise without losing image details, that is, not only the spatial weight is considered during filtering, but also the value range weight, that is, the similarity weight between pixels, so that denoising can be achieved while retaining the image boundary. However, since the value range and the spatial domain are two dimensions, the calculation takes too long. The grid data in this application promotes the image to three dimensions, unifies the calculation of the value range and the spatial domain, and converts the two-dimensional nonlinear calculation into a three-dimensional linear convolution, which is convenient for accelerating the processing by using data locality, thereby improving the image processing efficiency.

[0110] Figure 5 A schematic diagram of the structure of the chip provided in the embodiment of the present application, such as Figure 5 As shown, the chip 50 may include: an acquisition module 51, a parallel processing module 52 and a fusion module 53; wherein,

[0111] An acquisition module 51 is used to acquire multiple frames of images;

[0112] A parallel processing module 52, configured to perform post-processing based on the multiple frames of images while performing temporal noise reduction based on the multiple frames of images;

[0113] The fusion module 53 is used to obtain a second image based on the target grid data and the first image, wherein the target grid data is the grid data obtained after the post-processing, and the first image is a single-frame image obtained after the time domain denoising.

[0114] In one possible implementation manner, the time domain noise reduction includes processing by a central processing unit CPU, and the post-processing includes processing by a graphics processing unit GPU.

[0115] In one possible implementation manner, the target grid image includes a three-dimensional index, and the three-dimensional index includes a two-dimensional coordinate and a pixel value of a pixel.

[0116] In one possible implementation manner, the multiple frames of images include multiple frames of YUV images, and the grid data includes grid data of a YUV channel.

[0117] In one possible implementation, the parallel processing module 52 is specifically used to downsample the multiple frames of images;

[0118] Constructing first grid data and second grid data based on the downsampled multiple frames of images, wherein a first grid node in the first grid data has a first pixel value, a second grid node in the second grid data has a number of pixels corresponding to the first number of pixels, and the first grid node and the second grid node have the same three-dimensional index;

[0119] Target mesh data is obtained based on the first mesh data and the second mesh data.

[0120] In one possible implementation, the downsampling includes spatial downsampling and intensity downsampling, wherein the spatial downsampling is used to downsample the image resolution, and the intensity downsampling is used to downsample the pixel value.

[0121] In one possible implementation, the parallel processing module 52 is specifically used to filter the first grid data and the second grid data;

[0122] Target mesh data is obtained based on the filtered first mesh data and the filtered second mesh data.

[0123] Figure 5 The chip 50 provided in the illustrated embodiment can be used to execute the technical solution of the method embodiment illustrated in the present application, and its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0124] It should be understood that the division of the various modules of the above chip 50 is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of the terminal device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0125] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0126] Figure 6The present invention provides a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 may include: at least one processor; and at least one memory connected to the processor in communication. The memory stores program instructions executable by the processor. The processor in the electronic device 600 calls the program instructions to perform the actions performed in the storage access method provided in the embodiment of the present invention.

[0127] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: one or more processors 610, a memory 620, a communication bus 640 connecting different system components (including the memory 620 and the processor 610), and a communication interface 630.

[0128] The communication bus 640 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0129] The electronic device 600 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the terminal device, including volatile and non-volatile media, removable and non-removable media.

[0130] The memory 620 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The terminal device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Figure 6Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 640 via one or more data medium interfaces. The memory 620 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0131] A program / utility having a set (at least one) of program modules may be stored in memory 620, such program modules including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.

[0132] The electronic device 600 may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the terminal device, and / or any device that enables the terminal device to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through the communication interface 630. In addition, the electronic device 600 may also communicate with the network adapter ( Figure 6 The network adapter can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the communication bus 640. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Drives; hereinafter referred to as: RAID) systems, tape drives, and data backup storage systems.

[0133] The processor 610 executes various functional applications and data processing by running the programs stored in the memory 620, such as implementing the method provided in the embodiment of the present application.

[0134] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 600. In other embodiments of the present application, the electronic device 600 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0135] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASIC, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0136] An embodiment of the present application also provides a readable storage medium, which stores a program. When the program is run on a terminal device, the terminal device executes the method provided by the embodiment shown in the present application.

[0137] An embodiment of the present application also provides a program product, which includes a program. When the program product is run on a terminal device, the terminal device executes the method provided by the embodiment shown in the present application.

[0138] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0139] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0142] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire multiple frames of images; While performing temporal noise reduction based on the multiple frames of images, post-processing is performed based on the multiple frames of images; A second image is obtained based on target grid data and a first image, wherein the target grid data is grid data obtained after the post-processing, and the first image is a single-frame image obtained after the time domain denoising.

2. The method according to claim 1, characterized in that The time domain noise reduction includes processing by a central processing unit (CPU), and the post-processing includes processing by a graphics processing unit (GPU).

3. The method according to claim 1, characterized in that The target grid image includes a three-dimensional index including two-dimensional coordinates and pixel values ​​of pixels.

4. The method according to claim 1, characterized in that The multiple frames of images include multiple frames of YUV images, and the grid data include grid data of YUV channels.

5. The method according to any one of claims 1 to 4, characterized in that: The post-processing based on the multiple frames of images comprises: Downsampling the multiple frames of images; Constructing first grid data and second grid data based on the downsampled multiple frames of images, wherein a first grid node in the first grid data has a first pixel value, a second grid node in the second grid data has a number of pixels corresponding to the first number of pixels, and the first grid node and the second grid node have the same three-dimensional index; Target mesh data is obtained based on the first mesh data and the second mesh data.

6. The method according to claim 5, characterized in that The downsampling includes spatial downsampling and intensity downsampling, wherein the spatial downsampling is used to downsample the image resolution, and the intensity downsampling is used to downsample the pixel value.

7. The method according to claim 5 or 6, characterized in that: The obtaining target grid data based on the first grid data and the second grid data comprises: filtering the first grid data and the second grid data; Target mesh data is obtained based on the filtered first mesh data and the filtered second mesh data.

8. A chip, characterized in that: The chip comprises: An acquisition module, used for acquiring multiple frames of images; A parallel processing module, used for performing time domain noise reduction based on the multiple frames of images and performing post-processing based on the multiple frames of images; A fusion module is used to obtain a second image based on target grid data and a first image, wherein the target grid data is grid data obtained after the post-processing, and the first image is a single-frame image obtained after the time domain denoising.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the image processing method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program, and when the program is run on an electronic device, the image processing method according to any one of claims 1 to 7 is implemented.

Images