137 results about "Demosaicing" patented technology

The invention discloses a combined denoising and demosaicing method for a low-exposure original image under low light, and the method is divided into two stages: in the first stage, introducing a high and low frequency fusion module based on wavelet transform, and generating relatively clean pseudo long exposure data to reduce noise; and in the second stage, through a color space attention subnet, image reconstruction is carried out by using the pseudo long exposure data generated in the first stage, the color is recovered, and texture details are enhanced. In addition, the invention also provides a wavelet loss function to balance detail recovery and noise suppression, and discusses the influence of three different training strategies on the performance of the model. Experimental results show that the method provided by the invention is superior to most advanced algorithms in the aspects of quantitative indexes, color accuracy, detail retention and the like.

The invention discloses an ultra-high-speed imaging device based on acousto-optic filtering modulation. The ultra-high-speed imaging device comprises a spectrum modulation system, a spectrum imaging system and a data processing system. The spectrum modulation system utilizes an acousto-optic tunable filter (AOTF) to dynamically modulate broadband continuous spectrum light, narrow-band illumination light which rapidly changes along with time is generated, and time information of a dynamic scene is effectively mapped to a spectrum dimension. And the spectral imaging system images the spectral coded dynamic scene to the hyperspectral camera to realize acquisition of spectral space-time information. The data processing system performs spectral crosstalk correction, channel separation and demosaicing reconstruction on the acquired original mosaic image, and recovers a time sequence image according to a spectrum-time mapping relation, thereby realizing single-shot superspeed imaging in a dynamic process. The device has the advantages of being simple and compact in structure, flexible in time window, high in imaging fidelity and the like, and has wide application prospects in the fields of rapid dynamics such as microfluidics, laser processing and ultrafast physics.

The invention provides an array image demosaicing method based on dynamic convolution and adaptive coding, which relates to the technical field of image processing, and comprises the following steps: acquiring single-channel original image data and a corresponding color filtering array arrangement type identifier; converting the arrangement type identifier into a multi-dimensional physical feature vector, and inputting the multi-dimensional physical feature vector into a feature processor of a neural network model to generate a weighting coefficient vector; carrying out weighted combination on the plurality of special arrangement transformation matrixes through a weighting coefficient vector to obtain a transformation component, adding the transformation component and a basic convolution kernel parameter to obtain a dynamic convolution kernel parameter, and carrying out directional modulation on a specific spatial position of the dynamic convolution kernel parameter based on a direction weight component in a multi-dimensional physical feature vector; and performing convolution operation on the original image data by using the dynamic convolution kernel parameters, extracting multi-scale features, reconstructing image features, and outputting multi-channel color image data, so that the method can be adaptive to different color filter array types, and the demosaicing precision and generalization capability are improved.

Apparatuses, systems, and techniques to process image data. In at least one embodiment, a neural network is trained to perform demosaicing of two-dimensional image data obtained from an image sensor.

The present application discloses an image reconstruction method, an apparatus, a device, a medium, and a product. According to the image reconstruction method in the embodiments of the present application, all Swin Transformer modules of an encoding module are connected by means of a patch merging layer; all Swin Transformer modules of a decoding module are connected by means of a patch expanding layer; the encoding module and the decoding module are symmetrical in structure; and finally, an upsampled feature map is input into a reconstruction module so as to obtain an RGB image by means of conversion. By constructing a network on the basis of a U-Net structure, the present solution is able to restore a corresponding high-resolution RGB image from a feature map, and interpolate missing values and defective pixels in a RAW image captured by an event camera while performing demosaicing, thereby obtaining a high-quality RGB image. Moreover, the use of Swin Transformer architecture as a feature extractor allows for the effective capturing of long-range dependencies, and ensures that consistency of local and global information is maintained.

The invention provides an image processing model training method and device, an image processing method and device, equipment and a medium, and relates to the technical field of image processing, and the training method comprises the steps: obtaining a sample image of a first format and a reference image of a second format; performing position coding on the sample image by adopting a coding network in the image processing model to obtain a coding feature map; adopting a demosaicing network in an image processing model to carry out demosaicing processing on the coded feature map to obtain an output feature map; fusing the coding feature map and the output feature map to obtain a fused feature map; and training an image processing model based on the fused feature map and the reference image. Therefore, position coding is introduced in the modeling stage, the performance and effect of the model can be improved, and the quality of the finally generated image in the second format is improved.

The invention discloses an image subgradient constrained dual-camera snapshot spectral imaging system and an imaging method, and relates to the field of spectral imaging, in particular to an image subgradient constrained dual-camera snapshot spectral imaging method. The method aims at the technical bottleneck that a traditional spectral imaging system cannot be suitable for a dynamic scene and a low-illumination environment due to low time resolution and limited light utilization rate, and the problem that an existing SCSI system is poor in reconstruction accuracy of a decoding algorithm due to the fact that the compression ratio of space-spectral information is too high or information complementarity is insufficient. The method comprises the following steps: 1, obtaining a preprocessed broadband filter array RAW image and an SD-CASSI compressed encoding image Y; 2, obtaining a demosaiced multiband image; 3, obtaining a coarse estimation spectrum image; 4, calculating the subgradient of the three-dimensional total variation function f3DTV (.) at the roughly estimated spectral image, wherein the subgradient represents the spatial characteristics of the spectral image; and 5, iteratively solving a spectral image recovery problem by using an optimization algorithm to reconstruct a spectral image.

Disclosed is a joint denoising and demosaicking method for a color RAW image guided by a monochrome image. The method comprises: constructing a synthetic image dataset of a monochrome-color binocular camera system for training and testing of network modules; constructing an aligned guidance image generation module by utilizing a structural correlation between a monochrome image and a color image, using a clean grayscale image corresponding to the RAW image as supervision, and training the aligned guidance image generation module by using a perceptual loss function to generate a high-quality aligned guidance image; using the generated aligned guidance image to guide the joint denoising and demosaicking process of the color RAW image; and training a guided denoising and demosaicking module ensure the accuracy of the color of the denoising and demosaicking result while accurately transferring the guidance image structure.

An imaging system includes an image sensor, a wobulator employed to perform sub-pixel shifts; and processor(s) configured to: obtain, in a cycle, two or three sub-images from the image sensor; control the wobulator to perform, during said cycle, one or two sub-pixel shifts, wherein step sizes of sub-pixel shifts vary within at least one of: same cycle, different cycles, wherein at least one of step sizes is X pixel, wherein X is a fraction lying between 0 and 1, at least one other of step sizes is Y pixels, wherein Y is an integer lying in a range from 1 to Z, Z being equal to a number of pixels of same colour lying along a direction of sub-pixel shift in a smallest repeating M×N array (204a-b) in the image sensor; and process the two or three sub-images to generate image(s).

The image demosaicing method comprises the following steps: determining a red image pixel value, a green image pixel value and a blue image pixel value for a red pixel of a Bayer unit, such that each of the first weighted sum, the second weighted sum, and the third weighted sum of the red image pixel value, the green image pixel value, and the blue image pixel value is equal to the respective sensor pixel value. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to an average value of green sensor pixel values output by green pixels of the Bayer unit. The third weighted sum is equal to a blue sensor pixel value output by a blue pixel of the Bayer cell. The method further includes outputting a demosaiced image having an RGB triple at a position in the demosaiced image corresponding to a position of the red pixel, the RGB triple including a red image pixel value, a green image pixel value, and a blue image pixel value.

This disclosure provides a demosaicing device and a demosaicing method for image sensor. The method includes capturing a panchromatic image by using a pixel array including switchable pixels and clear pixels, where the panchromatic image is captured when the switchable pixels and the clear pixels are under a clear state. The method includes capturing a chromic image by using the pixel array when the switchable pixels are under a color state and the clear pixels are under the clear state. The method includes performing a weighted average calculation and a signal subtraction on the chromic image and the panchromatic image to generate first resolution chromic images. A resolution of the first resolution chromic images is the same as a resolution of the panchromatic image and the chromic image. The method includes fusing the first resolution chromic images with the panchromatic image to generate demosaiced chromic images.

Disclosed are systems and techniques for image demosaicing with minimal artifacts. The techniques include computing a color value within a first color space for a first pixel of a plurality of pixels based at least on color values within the first color space of a first group of neighboring pixels of the first pixel and computing a first chrominance value within a second color space for the first pixel based at least on the computed color value within the first color space. The techniques include computing a luminance value within the second color space for the first pixel based at least on the first chrominance value within the second color space and converting the luminance value within the second color space and the first chrominance value within the second color space to an output pixel value within a third color space.

A method for generating a digital zoom image by using generative AI may comprise the steps of: acquiring a multi-frame input; performing demosaicing and denoising on the multi-frame input, wherein a denoising intensity is adjusted to be weaker as the number of a plurality of frames included in the multi-frame input increases; acquiring a base image by merging the plurality of frames on which the demosaicing and denoising have been performed; and acquiring a zoomed image by performing super-resolution processing on the base image.

A method of constructing a dictionary of kernels for blending in multi-frame processing (MFP) includes generating, for each ground truth (GT) image in a set of GT images represented in a full color space, a corresponding feature matrix by generating a set of synthetic raw images represented in a color filter mosaic space, performing demosaicing and registration operations on each of the synthetic raw images to generate a corresponding registered image, choosing a set of patches in each registered image, the location of each patch in one registered image coinciding with the location of a corresponding patch in each other registered image, generating, for each patch, a corresponding pixel value matrix containing pixel values of each color channel of each pixel included in the corresponding patch, and extracting features from the pixel value matrices and generating the feature matrix corresponding to the GT image based on the extracted features.

The invention relates to a method (200) for enriching a raw image (IMBP) of a scene on the basis of one or more secondary raw images (IMBS1-IMBSl) of the scene, the method (200) comprising an enrichment phase (210) for enriching the primary raw image (IMBP) and providing an enriched raw image (IMBE), the enrichment phase (210) comprising, for at least one pattern, referred to as the target pattern, of the primary raw image (IMBP), the following steps: - identifying (212), in at least one secondary raw image (IMBS1-IMBSk), a pattern corresponding to the target pattern; and adjusting (216,218) at least one value of the target pattern according to one or more values of identified corresponding patterns. The invention also relates to a method for demosaicing a raw image, to an image processing method, to an image acquisition method, to a computer program, to a device, to a system and to a vehicle implementing such a demosaicing method.

An image demosaicing method includes determining, for a red pixel of a Bayer unit, a red image-pixel value, a green image-pixel value, and a blue image-pixel value such that each of a first, a second, and a third weighted sum of the red, the green, and the blue image-pixel values equals a respective sensor-pixel value. The first weighted sum equals a red sensor-pixel value output by the red pixel. The second weighted sum equals an average of a green sensor-pixel values output by green pixels of the Bayer unit. The third weighted sum equals a blue sensor-pixel value output by a blue pixel of the Bayer unit. The method also includes outputting a demosaiced image having, at a location in the demosaiced image corresponding to a location of the red pixel, an RGB triplet that includes the red image-pixel value, the green image-pixel value, and the blue image-pixel value.

The invention discloses a video processing method and device, and relates to the technical field of video processing. The method comprises the steps of obtaining a to-be-processed video; dividing an image in the video to be processed according to a preset moment node to obtain a frame image at each moment; and inputting the frame image at each moment into the trained residual branch model for image processing to obtain a video processing result. Compared with the prior art, the method has the advantages that the corresponding processing branches are set for decoupling of video processing, noise reduction, fusion and demosaicing, and connection is performed through the residual structure, so that the processing logic relationship among the processing branches is defined, the functions of the trained residual branch model are more comprehensive, and the robustness of the residual branch model is improved. According to the method, noise reduction, demosaicing and fusion tasks in the video can be processed, processing of HDR type and NonHDR type videos is considered at the same time, and the problem that in a related method, processing models are independently set for different tasks, and universality is poor is solved.

The invention discloses an unmanned aerial vehicle detection method and device based on polarization information enhancement and polarization attention characteristics, a medium and equipment, and the method comprises the steps: carrying out the multi-channel splitting and demosaicing of an infrared polarization mosaic image through a polarization decoding student network, and generating a multi-dimensional polarization information graph; carrying out feature extraction on the multi-dimensional polarization information graph based on a YOLOv8 backbone network, and outputting preliminary features; and inputting the initial feature into a polarization attention feature network, generating a target positioning feature, and inputting the target positioning feature into a detection head to obtain an unmanned aerial vehicle detection result. Multi-dimensional polarization information can be obtained from the polarization mosaic image through the polarization decoding student network, and the contrast between the target and the background is improved by using different polarization information; preliminary features of the multi-dimensional polarization information are obtained through a YOLOv8 backbone network, the preliminary features are processed through a polarization attention feature network, and accurate detection of an unmanned aerial vehicle target is achieved on the multi-dimensional polarization information with the contrast ratio remarkably improved.

This disclosure provides systems, methods, and devices for image signal processing that support improved demosaicing of color image signals. In a first aspect, an image processing method includes receiving a first image frame and a second image frame. The method may also include determining a first demosaiced image frame by applying a first demosaicking process to the first image frame, and determining a second demosaiced image frame by applying a second demosaicking process to the first image frame based on the second image frame. A blending weight may be determined based on the first image frame and the second image frame, and a blended image frame may be determined by combining pixel values from corresponding portions of the first demosaiced image frame and the second demosaiced image frame according to the blending weight. Other aspects and features are also claimed and described.

Disclosed are systems and circuits for image demosaicing with minimal artifacts. The circuits are to compute a color value within a first color space for a first pixel of a plurality of pixels based at least on color value within the first color space of a first group of neighboring pixels of the first pixel and compute a first chrominance value within a second color space for the first pixel based at least on the computed color value within the first color space. The circuits are also to compute a luminance value within the second color space for the first pixel based at least on the first chrominance value within the second color space and convert the luminance value within the second color space and the first chrominance value within the second color space to an output pixel value within a third color space.

Methods and systems for providing a real time light weight non-Bayer color filter array (CFA) artificial intelligence (AI) demosaic through an architecture which leverages position dependent interpolation, position aware gradient, and light weight AI demosaic model are provided. The methods include position dependent interpolation and position aware gradient such that maximum information is preserved for efficient training of light weight deep neural network (DNN). The methods include producing high quality demosaic output from non-Bayer CFA image data without suffering from loss of details, texture and resolution power while maintaining low inference time.

The present invention relates to an IR imaging device including: an image capturing unit which includes a pixel array having a spatially varying exposure (SVE) pixel pattern of a size of N x N in which pixels are classified into pixel groups so that pixels having an identical sensitivity belong to an identical group and pixels having multiple sensitivities are arranged so that pixels having an identical sensitivity are not adjacent to each other, wherein the multiple pixels having the multiple sensitivities concurrently detect light with the multiple sensitivities so as to capture an image; and an image restoring unit for acquiring multiple images having different sensitivities by generating an image having a low resolution for each pixel group by using only data of multiple pixels belonging to an identical pixel group among the data of the captured image and interpolating data of pixels belonging to different pixel groups in a demosaicing method to restore the resolution thereof.

The present application relates to the field of endoscope imaging, in particular to a single-channel filtering demosaicing method and system for endoscope images; the horizontal direction local gradient intensity and the vertical direction local gradient intensity of green pixels in a color filter array image are calculated, the green pixels are adaptively interpolated according to the difference between the horizontal direction local gradient intensity and the vertical direction local gradient intensity, and an initial green channel is obtained; the pixels in the initial green channel are subjected to bilateral filtering, and a filtered green channel is obtained; based on the filtered green channel, residual interpolation is used to respectively reconstruct a red channel and a blue channel, and a demosaiced image is obtained; the present application reduces the risk of false color and noise amplification, has the characteristics of strong edge preservation ability, high overall color restoration precision, simple algorithm and easy hardware implementation.

The application discloses a snapshot multispectral camera demosaicing method and device based on a binary tree coating array, according to a multispectral original image, a plurality of single spectral band mosaic images are extracted according to spectral bands, and a pseudo panchromatic estimation image is estimated according to the multispectral original image; a difference value between each spectral band sparse pseudo panchromatic estimation image and an original spectral band sparse image is calculated, and the difference value at all pixel positions is calculated; then, the difference value image after filtering and interpolation is added to the pseudo panchromatic estimation image, to obtain a demosaicing reconstruction result based on the pseudo panchromatic estimation image; and the color and spatial correlation between adjacent spectral bands is utilized to enhance the spatial high-frequency information of a binary tree low node spectral band image based on a binary tree high node spectral band image. The above method can solve the artifact problem caused by the weak spectral correlation of a wide spectral distribution filter, reduce the artifact of a reconstructed image, and improve the spatial resolution of the image after demosaicing.

The method and device for denoising of extremely low light original image can improve the generalization ability of the joint denoising and demosaicing method for extremely weak light image, greatly reduce the number of parameters without losing performance, and have better robustness. The method comprises the following steps: (1) inputting an extremely low light original image into a system; (2) image reconstruction, reconstructing a single-channel original image into an RGBG four-channel image with a resolution of 1 / 4 of the original image; (3) black level normalization; (4) enhancing the signal at a specified magnification ratio; (5) inputting an SUnet++ neural network; (6) outputting an RGB*4 twelve-channel image with the same resolution as the image input into the neural network; (7) reconstructing the output image into an RGB three-channel image with the same resolution as the original system input image; and (8) outputting an RGB image with normal brightness.

First image data including first subsampled image data of at least a first part of a first field of view a first image sensor is obtained. Second image data including second subsampled image data of at least a second part of a second field of view of a second image sensor is obtained. The first part and the second part have at least a part of an overlapping field of view. The first image data and the second image data are processed to generate a first image and a second image, respectively. During processing, interpolation and demosaicking on the first subsampled image data is performed by utilising the second subsampled image data, while interpolation and demosaicking on the second subsampled image data is performed by utilising the first subsampled image data.

Systems and circuits for demosaicing images with minimal artifacts are disclosed. The circuits are designed to calculate a color value within a first color space for the first pixel of a plurality of pixels, based at least on a color value within the first color space of a first group of pixels adjacent to the first pixel, and to calculate a first chrominance value within a second color space for the first pixel, based at least on the calculated color value within the first color space. The circuits are also designed to calculate a luminance value within the second color space for the first pixel, based at least on the first chrominance value within the second color space, and to convert the luminance value within the second color space and the first chrominance value within the second color space into an output pixel value within a third color space.

The embodiment of the invention provides an image processing method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining a to-be-processed image which is an image in a Bayer format; based on the image processing model, performing demosaicing and purple edge correction processing on the to-be-processed image to generate a target image; wherein the image processing model comprises a first network and a second network, and the image processing model carries out preliminary training on the first network through a first mosaic image and a first reference image which have a first corresponding relation, and then uses a second mosaic image and a second reference image which have a second corresponding relation to obtain a second mosaic image; performing joint training on the first network and the second network; wherein during joint training, the first network further transmits the generated multi-level feature map to the second network through long jump connection. By adopting the technical scheme, end-to-end optimization from demosaicing to purple edge correction can be realized, and the quality of the target image is improved.

This application provides an image processing method, apparatus, and electronic device, relating to the field of image processing technology. These methods can more effectively suppress false color and restore more high-frequency details during image depigmentation, denoising, and super-resolution processing, thereby improving image quality. The image processing method includes: acquiring an RGBW image from an image sensor, the RGBW image including a red channel, a green channel, a blue channel, and a white channel; inputting the RGBW image into a convolutional neural network model to obtain an RGB image output by the convolutional neural network model, the convolutional neural network model including a cascaded feature extraction module, a reconstruction module, and an upsampling module, the feature extraction module and the reconstruction module being used for the depigmentation and denoising processes, and the upsampling module being used for the super-resolution processing process. The RGB image includes a red channel, a green channel, and a blue channel, and the resolution of the RGB image is greater than that of the RGBW image.

Image processing systems and image processing techniques leveraging neural networks (e.g., convolutional neural networks (CNNs)) for image restoration tasks (e.g., for demosaicing tasks) are described. In certain aspects, Mixture of Experts (MoE) techniques may be employed, where multiple different expert networks are used to divide a problem space (e.g., image reconstruction tasks) into homogenous regions. For example, each MoE module may reconstruct a certain problem in an image, and a gating component may activate certain MoE modules to provide a reconstructed image. In some aspects, training and optimization techniques are described for each expert of the MoE architecture, to increase individual performance (e.g., a sub-task for each expert of an image processing system may be imposed in a residual manner, a gating function may be trained, etc.). Accordingly, image processing systems may leverage MoE architectures to support a large number of neural network parameters for improved image reconstruction applications.