Image signal processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides an image signal processing method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: carrying out the denoising of an image signal collected by a nine-unit sensor, and generating a denoised image signal; performing wavelet decomposition on the denoised image signal to generate a plurality of sub-band signals; calculating an interpolation weight according to the plurality of sub-band signals, and performing channel interpolation operation to generate a target pixel value; the method comprises the following steps: performing linear combination on target pixel values through a reconstruction matrix to generate a reconstructed image signal, and performing denoising preprocessing, wavelet decomposition and multi-sub-band analysis on the image signal, adaptive interpolation based on sub-bands and linear combination of the reconstruction matrix to obtain a reconstructed image signal. The problems of noise interference, color information loss and poor reconstruction quality in the original image of the nine-unit sensor are solved, and information loss or redundancy in the signal decomposition and reconstruction process is avoided in combination with the multi-scale analysis capability of wavelet transform and the adaptive interpolation strategy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image signal processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Image sensors are the core components for achieving high-quality image acquisition. As users' demands for low-light shooting, high-resolution detail capture, and low-power imaging increase, image sensors are gradually evolving towards high pixel density and multi-micro-unit structures.

[0003] In existing technologies, Haar wavelet transform is used to process data acquired by image sensors.

[0004] However, in the existing technology, the Haar wavelet transform method is not applicable to image data acquired by a 9-element sensor, resulting in information loss or redundancy during signal decomposition and reconstruction. Summary of the Invention

[0005] This application provides an image signal processing method, apparatus, electronic device, storage medium, and program product to solve the technical problem of information loss or redundancy in the signal decomposition and reconstruction process in the prior art.

[0006] In a first aspect, embodiments of this application provide an image signal processing method, including:

[0007] The image signals acquired by the nine-unit sensor are denoised to generate denoised image signals.

[0008] The denoised image signal is subjected to wavelet decomposition to generate multiple sub-band signals;

[0009] The interpolation weights are calculated based on the multiple sub-band signals, and channel interpolation is performed to generate the target pixel value.

[0010] A reconstructed image signal is generated by linearly combining the target pixel values ​​using a reconstruction matrix.

[0011] In one possible implementation, the step of denoising the image signal acquired by the nine-unit sensor to generate a denoised image signal includes: performing black level subtraction on the original mode image signal output by the nine-unit sensor to generate an image signal with background noise eliminated; performing boundary pixel mirroring expansion on the image signal with background noise eliminated to generate a boundary-expanded image signal; and determining the boundary-expanded image signal as the denoised image signal.

[0012] In one possible implementation, the step of performing wavelet decomposition on the denoised image signal to generate multiple sub-band signals includes: performing element-wise convolution of multiple pixel blocks in the denoised image signal with a predefined wavelet decomposition matrix to generate multiple sub-band coefficients; arranging the multiple sub-band coefficients according to their spatial positions to obtain multiple sub-band images; and performing channel separation on the multiple sub-band images to generate multiple sub-band signals.

[0013] In one possible implementation, the step of calculating interpolation weights based on the plurality of sub-band signals and performing channel interpolation to generate target pixel values ​​includes: calculating the brightness estimate of a pixel block based on the low-frequency approximate sub-band signal among the plurality of sub-band signals to generate brightness feature parameters; calculating gradient values ​​in multiple directions based on the horizontal high-frequency sub-band and vertical high-frequency sub-band signals among the plurality of sub-band signals to generate directional gradient features; determining the region type of the image based on the brightness feature parameters and directional gradient features to generate an image content classification result; and assigning interpolation weights to the missing color channels based on the image content classification result to generate pixel values.

[0014] In one possible implementation, the step of linearly combining the target pixel values ​​using a reconstruction matrix to generate a reconstructed image signal includes: extracting corresponding sub-band coefficients from multiple sub-band signals based on the color type of the pixel block; performing element-wise convolution and summing of the sub-band coefficients with a predefined reconstruction matrix to generate missing pattern pixel values ​​in the pixel block; merging the reconstructed pixel values ​​with the corresponding pixel values ​​in the original image to generate a full-size pattern image; and performing black level back-addition processing on the full-size pattern image to generate the reconstructed image signal.

[0015] In one possible implementation, before performing element-wise convolution of multiple pixel blocks in the denoised image signal with a predefined wavelet decomposition matrix to generate multiple sub-band coefficients, the method further includes: performing block processing on the denoised image signal to generate multiple pixel blocks; performing local feature detection on the multiple pixel blocks to generate local feature information of the multiple pixel blocks; determining the parameters of the wavelet decomposition matrix based on the local feature information of the multiple pixel blocks; and generating the wavelet decomposition matrix based on the parameters of the wavelet decomposition matrix.

[0016] Secondly, embodiments of this application provide an image signal processing apparatus, comprising:

[0017] The noise reduction module is used to denoise the image signals acquired by the nine-unit sensor and generate denoised image signals.

[0018] The wavelet decomposition module is used to perform wavelet decomposition on the denoised image signal to generate multiple sub-band signals.

[0019] The calculation module is used to calculate the interpolation weights based on the multiple sub-band signals, and perform channel interpolation operations to generate target pixel values;

[0020] The linear combination module is used to linearly combine the target pixel values ​​through a reconstruction matrix to generate a reconstructed image signal.

[0021] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0022] The memory stores computer-executed instructions;

[0023] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0025] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0026] The image signal processing method, apparatus, electronic device, storage medium, and program product provided in this application construct a complete image signal processing link by performing denoising preprocessing, wavelet decomposition and multi-subband analysis, subband-based adaptive interpolation, and linear combination of reconstruction matrices on the image signal. This solves the problems of noise interference, missing color information, and poor reconstruction quality in the original image of the nine-unit sensor. By combining the multi-scale analysis capability of wavelet transform with the adaptive interpolation strategy, the clarity, color reproduction, and detail preservation capability of the reconstructed image are improved, and information loss or redundancy is avoided during signal decomposition and reconstruction. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] Figure 1 This is a schematic diagram of the system structure of a computer device provided in an embodiment of this application;

[0029] Figure 2 Flowchart of the image signal processing method provided in this application Figure 1 ;

[0030] Figure 3 A schematic diagram of the image processing flow acquired by the nine-unit sensor provided in this application;

[0031] Figure 4 A schematic diagram illustrating the brightness estimation of the red pixel centered in this application;

[0032] Figure 5 A schematic diagram illustrating the brightness estimation of the blue pixel centered in this application;

[0033] Figure 6 Schematic diagram of directional gradient calculation provided for this application Figure 1 ;

[0034] Figure 7 Schematic diagram of directional gradient calculation provided for this application Figure 2 ;

[0035] Figure 8 A schematic diagram showing the interpolated pixel values ​​missing in this application;

[0036] Figure 9 Flowchart of the image signal processing method provided in this application Figure 2 ;

[0037] Figure 10 A schematic diagram of the image signal processing apparatus provided in this application;

[0038] Figure 11 A schematic diagram of the structure of the electronic device provided in this application.

[0039] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] Image sensors are the core component for achieving high-quality image acquisition. With increasing user demands for low-light shooting, high-resolution detail capture, and low-power imaging, image sensors are gradually evolving towards higher pixel density and multi-micro-unit structures. Current technologies use Haar wavelet transform to process data acquired by image sensors. However, the Haar wavelet transform method is not suitable for image data acquired by 9-unit sensors, leading to information loss or redundancy during signal decomposition and reconstruction.

[0042] To address the aforementioned technical problems, this application proposes the following technical concept: The inventors considered designing an image signal processing method based on image signals acquired by a nine-unit sensor. By performing denoising preprocessing, wavelet decomposition and multi-subband analysis, subband-based adaptive interpolation, and linear combination of reconstruction matrices on the image signals, a complete image signal processing chain is constructed. Combining the multi-scale analysis capability of wavelet transform with the adaptive interpolation strategy, the clarity, color reproduction, and detail preservation capability of the reconstructed image are improved, avoiding information loss or redundancy during signal decomposition and reconstruction.

[0043] Figure 1 This is a schematic diagram of the system architecture of a computer device provided in an embodiment of this application. Figure 1 As shown, the computer device includes: a receiving device 101, a processing device 102, and a display device 103.

[0044] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the image signal processing method. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0045] In the specific implementation process, the receiving device 101 can be an input / output interface or a communication interface, and can acquire image signals collected by the nine-unit sensor.

[0046] The processing device 102 can perform noise reduction processing on the image signals acquired by the nine-unit sensor to generate reconstructed image signals.

[0047] The display device 103 can be used to display the reconstructed image signal pairs as described above.

[0048] The display device can also be a touch screen, used to receive user commands while displaying the above content, so as to realize the operation interaction with the user.

[0049] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0050] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0051] Figure 2 Flowchart of the image signal processing method provided in this application Figure 1 ,like Figure 2 As shown, the executing entity of this application can be a computer device / base station, or a chip or chip module in a terminal device / base station, and the method includes:

[0052] S201: Denoise the image signal acquired by the nine-unit sensor to generate a denoised image signal.

[0053] Specifically, the original pattern image signal output by the nine-unit sensor is subjected to black level subtraction processing, the image signal with background noise eliminated is subjected to boundary pixel mirroring expansion, and the image signal with boundary expansion is determined as the denoised image signal.

[0054] S202: Perform wavelet decomposition on the denoised image signal to generate multiple sub-band signals.

[0055] Specifically, multiple pixel blocks in the denoised image signal are element-wise convolved with a predefined wavelet decomposition matrix, the coefficients of multiple sub-bands are arranged according to their spatial location, and the channels of multiple sub-band images are separated to generate multiple sub-band signals.

[0056] S203: Calculate interpolation weights based on multiple sub-band signals, perform channel interpolation operations, and generate target pixel values.

[0057] Specifically, the brightness estimate of the pixel block is calculated based on the low-frequency approximate sub-band signal among multiple sub-band signals. The gradient values ​​in multiple directions are calculated based on the horizontal high-frequency sub-band and vertical high-frequency sub-band signals among multiple sub-band signals. Based on the brightness feature parameters and directional gradient features, the region type of the image is determined. The missing color channel is interpolated and weighted to generate pixel values.

[0058] S204: Generate a reconstructed image signal by linearly combining the target pixel values ​​through a reconstruction matrix.

[0059] Specifically, based on the color type of the pixel block, the corresponding sub-band coefficients are extracted from multiple sub-band signals. The sub-band coefficients are then convolved element-wise with a predefined reconstruction matrix and summed to generate the missing pattern pixel values ​​in the pixel block. The reconstructed pixel values ​​are then merged with the corresponding pixel values ​​in the original image. Finally, the full-size pattern image is subjected to black level back-addition processing to generate the reconstructed image signal.

[0060] As can be seen from the above embodiments, by performing denoising preprocessing, wavelet decomposition and multi-subband analysis, subband-based adaptive interpolation, and linear combination of reconstruction matrices on the image signal, a complete image signal processing link is constructed, which solves the problems of noise interference, color information loss, and poor reconstruction quality in the original image of the nine-unit sensor. By combining the multi-scale analysis capability of wavelet transform and the adaptive interpolation strategy, the clarity, color reproduction, and detail preservation capability of the reconstructed image are improved, and information loss or redundancy is avoided during signal decomposition and reconstruction.

[0061] Figure 3 A schematic diagram of the image processing flow acquired by the nine-unit sensor provided in this application.

[0062] like Figure 3 As shown, Figure 3 Image a in the image is the original image acquired by the nine-unit sensor. After black level subtraction, 3×3 wavelet decomposition, green channel interpolation, red / blue channel interpolation, 3×3 wavelet reconstruction, and black level re-addition, the processed image b is obtained.

[0063] In one embodiment of this application, step S201 includes:

[0064] S2011: Perform black level subtraction processing on the raw mode image signal output by the nine-unit sensor to generate an image signal with background noise eliminated.

[0065] In this embodiment, the raw pattern image signal output by the nine-unit sensor carries physical background noise.

[0066] Specifically, the processor reads the raw digital value of each pixel and subtracts a pre-calibrated black level reference value from each pixel value.

[0067] The black level reference value is obtained by averaging multiple frames of images acquired under conditions where the sensor is completely shaded.

[0068] In this embodiment, the image signal acquired by the nine-unit sensor is a 3×3 image signal.

[0069] S2012: Perform boundary pixel mirroring expansion on the image signal with background noise removed to generate the boundary-expanded image signal, and determine the boundary-expanded image signal as the denoised image signal.

[0070] Specifically, using the four boundaries of the image as axes of symmetry, the pixels outside the boundaries are filled with the mirror values ​​of the pixels inside the boundaries to obtain the denoised image signal.

[0071] For example, the value of the extended column outside the left boundary of the image is equal to the mirror symmetric value of the first column inside the left boundary, and the extension width is determined according to the size of the wavelet decomposition matrix.

[0072] As can be seen from the above embodiments, by eliminating the inherent background noise of the sensor and processing the boundary effect, a regular data foundation is provided for subsequent image processing steps. The boundary mirroring expansion operation effectively avoids artifacts or information loss caused by boundary truncation in subsequent wavelet decomposition and other operations, ensuring the consistency of the whole image processing.

[0073] In one embodiment of this application, step S202 includes:

[0074] S2021: Element-wise convolution of multiple pixel blocks in the denoised image signal with a predefined wavelet decomposition matrix to generate multiple subband coefficients.

[0075] In this embodiment, the wavelet decomposition matrix is ​​a 3×3 wavelet decomposition matrix.

[0076] in, This represents the matrix with coordinates (n,n) after one wavelet decomposition.

[0077] The physical meanings represented by each matrix in the wavelet decomposition matrix are as follows:

[0078] , representing the low-frequency approximate component subband of the signal.

[0079] , representing the horizontal high-frequency approximate component subband of the signal.

[0080] , representing the horizontal high-frequency approximate component subband of the signal.

[0081] , representing the vertical high-frequency approximate component subband of the signal.

[0082] , representing the diagonal high-frequency approximate component subband.

[0083] , representing the diagonal high-frequency approximate component subband.

[0084] , representing the vertical high-frequency approximate component subband.

[0085] , representing the diagonal high-frequency approximate component subband.

[0086] , representing the diagonal high-frequency approximate component subband.

[0087] Specifically, for the same 3×3 color pixel signal in the nine-unit sensor image. The transformed subband signal is obtained by element-wise convolution of the wavelet decomposition matrix.

[0088] The transformed subband signal is represented as follows:

[0089]

[0090] S2022: Arrange multiple sub-band coefficients according to their spatial location to obtain multiple sub-band images.

[0091] Specifically, the subband coefficients obtained by decomposing the pixel blocks are recombined according to their spatial positions in the original image to obtain a subband image with a reduced size.

[0092] S2023: Perform channel separation on multiple sub-band images to generate multiple sub-band signals.

[0093] Specifically, each sub-band image is decomposed into multiple single-channel sub-band signals according to the spectral type of each unit within the macro-pixel.

[0094] As can be seen from the above embodiments, by generating subband coefficients through convolution of pixel blocks with a predefined wavelet matrix, and arranging them according to spatial position and separating channels, the image signal is decomposed in multiple scales and directions in the frequency domain. This provides rich and physically meaningful feature inputs for subsequent intelligent interpolation based on different frequency band features, enabling the interpolation process to distinguish between smooth regions, edge regions and texture regions, and to perform differentiated processing.

[0095] In one embodiment of this application, step S203 includes:

[0096] S2031: Calculate the brightness estimate of the pixel block based on the low-frequency approximate sub-band signal among multiple sub-band signals, and generate brightness feature parameters.

[0097] Specifically, according to The brightness of the sub-band is estimated, and the brightness value is calculated.

[0098] Figure 4 A schematic diagram illustrating the brightness estimation of the red pixel at the center, provided in this application.

[0099] like Figure 4 As shown, Figure 4 The formula for calculating brightness in the image is:

[0100]

[0101] In the formula, 'r' represents brightness; 'g' represents red pixels; 'b' represents blue pixels.

[0102] Figure 5 A schematic diagram illustrating the brightness estimation of the blue pixel at the center, as provided in this application.

[0103] like Figure 5 As shown, Figure 5 The formula for calculating brightness in the image is:

[0104]

[0105] S2032: Calculate gradient values ​​in multiple directions based on the horizontal and vertical high-frequency sub-band signals from multiple sub-band signals, and generate directional gradient features.

[0106] Specifically, according to , , and The sub-band performs gradient calculations in the east, west, south, and north directions to generate directional gradients.

[0107] Figure 6 Schematic diagram of directional gradient calculation provided for this application Figure 1 .

[0108] like Figure 6 As shown, the formula for calculating the westward gradient is:

[0109]

[0110] In the formula, Indicates the gradient in the west direction; Indicates row index; This indicates the column index; abs() represents the absolute value operation.

[0111] The formula for calculating the gradient in the east direction is:

[0112]

[0113] In the formula, This represents the gradient in the eastward direction.

[0114] Figure 7 Schematic diagram of directional gradient calculation provided for this application Figure 2 .

[0115] like Figure 7 As shown, the formula for calculating the gradient in the north direction is:

[0116]

[0117] In the formula, This represents the gradient in the north direction.

[0118] The formula for calculating the gradient in the south direction is:

[0119]

[0120] In the formula, This represents the gradient in the north direction.

[0121] S2033: Based on brightness feature parameters and directional gradient features, determine the region type of the image and generate image content classification results.

[0122] In this embodiment, the region types of the image include, but are not limited to, highlighted regions, horizontal regions, vertical regions, and other regions.

[0123] S2034: Based on the image content classification results, interpolation weights are assigned to the missing color channels to generate pixel values.

[0124] Specifically, content detection is performed on the image region, interpolation weights are set, and the interpolation weights are inversely proportional to the calculated directional gradient. The missing green channel pixel values ​​in the nine sub-band signals are calculated based on the image region. Based on the calculated green channel pixel values, the red channel pixel values ​​and blue channel pixel values ​​are calculated.

[0125] As can be seen from the above embodiments, by extracting brightness features from low-frequency sub-bands and calculating directional gradients from high-frequency sub-bands, and then determining the region type and dynamically allocating interpolation weights, intelligent channel interpolation with content awareness is realized. By adaptively selecting the optimal interpolation strategy based on the different characteristics of whether the local area of ​​the image is a flat area, an edge area, or a texture area, the edge sharpness and texture details are preserved to the maximum extent while effectively reconstructing the missing color channels, thus avoiding the blurring and false color problems of traditional interpolation methods.

[0126] In one embodiment of this application, step S204 includes:

[0127] S2041: Extract the corresponding sub-band coefficients from multiple sub-band signals based on the color type of the pixel block.

[0128] Figure 8 This is a schematic diagram of the missing pixel values ​​for interpolation provided in this application.

[0129] like Figure 8 As shown, calculate Figure 8 The pixel values ​​at the positions of the characters are used to reconstruct the image.

[0130] S2042: Perform element-wise convolution of the subband coefficients with the predefined reconstruction matrix and sum them to generate the missing pattern pixel values ​​in the pixel block.

[0131] In this embodiment, the reconstruction matrix is ​​a 3×3 wavelet reconstruction matrix.

[0132] in, This represents the matrix with coordinates (n,n) after wavelet reconstruction.

[0133] The individual matrices in the wavelet reconstruction matrix are represented as follows:

[0134]

[0135] S2043: Merge the reconstructed pixel values ​​with the corresponding pixel values ​​in the original image to generate a full-size mode image.

[0136] Specifically, using , , , and The matrix is ​​convolved with the nine sub-band signals to obtain the reconstructed pixel values.

[0137] Specifically, when 3×3 is Green_red, the calculated pixel value is:

[0138]

[0139] Specifically, when 3×3 is Green_blue, the calculated pixel value is:

[0140]

[0141] Specifically, when 3×3 is Red, the calculated pixel value is:

[0142]

[0143] Specifically, when 3×3 is Blue, the calculated pixel value is:

[0144]

[0145] S2044: Perform black level re-addition processing on the full-size mode image to generate a reconstructed image signal.

[0146] Specifically, the subtracted black level value is added back to each pixel of the full-size mode image to generate the reconstructed image.

[0147] As can be seen from the above embodiments, through...

[0148] Figure 9 Flowchart of the image signal processing method provided in this application Figure 2 ,like Figure 9As shown, before step S2021, the following steps are also included:

[0149] S301: Performs block processing on the denoised image signal to generate multiple pixel blocks.

[0150] Specifically, the denoised image signal is divided into fixed-size, non-overlapping pixel blocks.

[0151] In this embodiment, the size of each pixel block is matched with the macro-pixel structure of the nine-unit sensor and is set to 3×3 pixels.

[0152] S302: Perform local feature detection on multiple pixel blocks to generate local feature information for multiple pixel blocks.

[0153] Specifically, for each pixel block, local statistical features are calculated as local feature information.

[0154] In this embodiment, local feature information includes, but is not limited to, the mean, variance, maximum and minimum values ​​of pixel values ​​within the block, texture energy based on the gray-level co-occurrence matrix, and the calculated gradient direction and intensity.

[0155] S303: Determine the parameters of the wavelet decomposition matrix based on the local feature information of multiple pixel blocks.

[0156] Specifically, based on local feature information, the parameters of the wavelet decomposition matrix of the matching pixel block are determined using predefined decision rules.

[0157] For example, if the pixel block has rich texture and obvious edges, a wavelet basis with short support and linear phase is selected; if the pixel block is smooth, a wavelet basis with high vanishing moment is selected.

[0158] S304: Generate the wavelet decomposition matrix based on the parameters of the wavelet decomposition matrix.

[0159] Specifically, based on the determined parameters, the low-pass and high-pass filter coefficients of the corresponding wavelet basis are extracted from the pre-constructed wavelet filter library and combined into a two-dimensional wavelet decomposition matrix.

[0160] As can be seen from the above embodiments, by dividing the image into blocks and performing local feature detection, the appropriate wavelet decomposition matrix parameters are dynamically determined and generated, which improves the adaptive capability of wavelet decomposition. This allows the decomposition process to select the optimal transformation basis according to the local characteristics of different regions of the image, thereby obtaining a sparser and more efficient sub-band representation, and further improving the processing quality and efficiency of subsequent interpolation and reconstruction stages.

[0161] Figure 10 A schematic diagram of the image signal processing apparatus provided in this application is shown below. Figure 10As shown, the image signal processing device 100 provided in this embodiment includes: a denoising module 1001, a wavelet decomposition module 1002, a calculation module 1003, and a linear combination module 1004.

[0162] The denoising module 1001 is used to denoise the image signals acquired by the nine-unit sensor and generate denoised image signals.

[0163] The wavelet decomposition module 1002 is used to perform wavelet decomposition on the denoised image signal to generate multiple sub-band signals.

[0164] The calculation module 1003 is used to calculate the interpolation weights based on multiple sub-band signals and perform channel interpolation operations to generate target pixel values.

[0165] The linear combination module 1004 is used to linearly combine the target pixel values ​​through the reconstruction matrix to generate a reconstructed image signal.

[0166] In one possible implementation, the noise reduction module 1001 includes:

[0167] The first generation unit is used to perform black level subtraction processing on the raw pattern image signal output by the nine-unit sensor to generate an image signal with background noise eliminated.

[0168] The expansion unit is used to perform boundary pixel mirroring expansion on the image signal with background noise removed, generate the boundary-expanded image signal, and determine the boundary-expanded image signal as the denoised image signal.

[0169] In one possible implementation, the wavelet decomposition module 1002 includes:

[0170] The convolution unit is used to perform element-wise convolution between multiple pixel blocks in the denoised image signal and a predefined wavelet decomposition matrix to generate multiple subband coefficients.

[0171] The arrangement unit is used to arrange multiple sub-band coefficients according to their spatial location to obtain multiple sub-band images.

[0172] The channel separation unit is used to separate multiple sub-band images into multiple sub-band signals.

[0173] In one possible implementation, the computing module 1003 includes:

[0174] The first calculation unit is used to calculate the brightness estimate of the pixel block based on the low-frequency approximate sub-band signal among multiple sub-band signals, and generate brightness feature parameters.

[0175] The second calculation unit is used to calculate gradient values ​​in multiple directions based on the horizontal high-frequency sub-band and vertical high-frequency sub-band signals in multiple sub-band signals, and generate directional gradient features.

[0176] The judgment unit is used to determine the region type of the image based on the brightness feature parameters and directional gradient features, and generate image content classification results.

[0177] The allocation unit is used to perform interpolation weight allocation on the missing color channels based on the image content classification results, and generate pixel values.

[0178] In one possible implementation, the linear combination module 1004 includes:

[0179] The extraction unit is used to extract the corresponding sub-band coefficients from multiple sub-band signals based on the color type of the pixel block.

[0180] The summation unit is used to perform element-wise convolution and summation of the subband coefficients with a predefined reconstruction matrix to generate the missing pattern pixel values ​​in the pixel block.

[0181] The merging unit is used to merge the reconstructed pixel values ​​with the corresponding pixel values ​​in the original image to generate a full-size mode image.

[0182] The second generation unit is used to perform black level re-addition processing on the full-size mode image to generate a reconstructed image signal.

[0183] In one possible implementation, the wavelet decomposition module 1002 further includes:

[0184] The block unit is used to divide the denoised image signal into blocks to generate multiple pixel blocks.

[0185] The detection unit is used to perform local feature detection on multiple pixel blocks and generate local feature information for multiple pixel blocks.

[0186] The determination unit is used to determine the parameters of the wavelet decomposition matrix based on the local feature information of multiple pixel blocks.

[0187] The third generation unit is used to generate the wavelet decomposition matrix based on the parameters of the wavelet decomposition matrix.

[0188] The image signal processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0189] Figure 11 A schematic diagram of the structure of the electronic device provided in this application. Figure 11As shown, the electronic device 110 provided in this embodiment includes at least one processor 1101 and a memory 1102. Optionally, the electronic device 110 further includes a communication component 1103. The processor 1101, the memory 1102, and the communication component 1103 are connected via a bus.

[0190] In the specific implementation process, at least one processor 1101 executes computer execution instructions stored in memory 1102, causing at least one processor 1101 to execute the above-described image signal processing method.

[0191] The specific implementation process of processor 1101 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0192] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0193] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0194] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0195] This application also provides a chip, which includes at least one processor for executing program instructions to perform the image signal processing method described above.

[0196] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image signal processing method.

[0197] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described image signal processing method.

[0198] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0199] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0200] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0201] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0202] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0203] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0204] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0205] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image signal processing method, characterized in that, include: The image signals acquired by the nine-unit sensor are denoised to generate denoised image signals. The denoised image signal is subjected to wavelet decomposition to generate multiple sub-band signals; The interpolation weights are calculated based on the multiple sub-band signals, and channel interpolation is performed to generate the target pixel value. A reconstructed image signal is generated by linearly combining the target pixel values ​​using a reconstruction matrix.

2. The method according to claim 1, characterized in that, The denoising process for the image signals acquired by the nine-unit sensor to generate denoised image signals includes: The raw pattern image signal output by the nine-unit sensor is processed by black level subtraction to generate an image signal with background noise eliminated; The image signal with background noise removed is subjected to boundary pixel mirroring expansion to generate a boundary-expanded image signal, and the boundary-expanded image signal is determined as the denoised image signal.

3. The method according to claim 1, characterized in that, The denoised image signal is subjected to wavelet decomposition to generate multiple sub-band signals, including: Multiple pixel blocks in the denoised image signal are element-wise convolved with a predefined wavelet decomposition matrix to generate multiple sub-band coefficients. The multiple sub-band coefficients are arranged according to their spatial location to obtain multiple sub-band images; Channel separation is performed on the multiple sub-band images to generate multiple sub-band signals.

4. The method according to claim 1, characterized in that, The step of calculating interpolation weights based on the multiple sub-band signals and performing channel interpolation to generate target pixel values ​​includes: The brightness estimate of the pixel block is calculated based on the low-frequency approximate sub-band signal from multiple sub-band signals, and the brightness feature parameters are generated. Based on the horizontal and vertical high-frequency sub-band signals from the multiple sub-band signals, gradient values ​​in multiple directions are calculated respectively to generate directional gradient features; Based on the brightness feature parameters and directional gradient features, the region type of the image is determined, and the image content classification result is generated; Based on the image content classification results, interpolation weights are assigned to the missing color channels to generate pixel values.

5. The method according to claim 1, characterized in that, The step of linearly combining the target pixel values ​​using a reconstruction matrix to generate a reconstructed image signal includes: Extract the corresponding sub-band coefficients from multiple sub-band signals based on the color type of the pixel block; The sub-band coefficients are convolved element-wise with a predefined reconstruction matrix and summed to generate the missing pattern pixel values ​​in the pixel block. The reconstructed pixel values ​​are merged with the corresponding pixel values ​​in the original image to generate a full-size mode image; The full-size mode image is subjected to black level re-addition processing to generate a reconstructed image signal.

6. The method according to claim 3, characterized in that, Before performing element-wise convolution of multiple pixel blocks in the denoised image signal with a predefined wavelet decomposition matrix to generate multiple sub-band coefficients, the method further includes: The denoised image signal is divided into blocks to generate multiple pixel blocks; Local feature detection is performed on the multiple pixel blocks to generate local feature information of the multiple pixel blocks; The parameters of the wavelet decomposition matrix are determined based on the local feature information of the multiple pixel blocks; The wavelet decomposition matrix is ​​generated based on the parameters of the wavelet decomposition matrix.

7. An image signal processing apparatus, characterized in that, include: The noise reduction module is used to denoise the image signals acquired by the nine-unit sensor and generate denoised image signals. The wavelet decomposition module is used to perform wavelet decomposition on the denoised image signal to generate multiple sub-band signals. The calculation module is used to calculate the interpolation weights based on the multiple sub-band signals, and perform channel interpolation operations to generate target pixel values; The linear combination module is used to linearly combine the target pixel values ​​through a reconstruction matrix to generate a reconstructed image signal.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the image signal processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image signal processing method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image signal processing method according to any one of claims 1 to 6.

Images