198 results about "Hybrid image" patented technology

The invention discloses a solar cell defect detection method based on a multi-mode sensing technology. The system comprises the following steps: 1, multi-modal data acquisition: acquiring multi-modal data in real time by using four sensing technologies of near-infrared laser, area array visible light, PL and EL; 2, multi-modal data processing: carrying out denoising, edge detection and feature extraction on the acquired near-infrared, visible light, PL and EL images by adopting a mixed image processing algorithm to obtain key defect features; 3, multi-modal data fusion: combining a multi-modal data fusion method with a decision-making level strategy, synthesizing different technical data, improving the detection precision, and classifying, positioning and labeling the defects of cracks, broken gates, scratches, depressions, poor welding and hot spots by using a convolutional neural network to ensure accurate distinguishing of surface and internal defects; and 4, defect marking and production line adjustment: marking defects and adjusting the production line through a feedback and automatic control module, and feeding back a real-time visual result to a management layer.

The invention discloses a multi-task lung cancer brain metastasis lifetime prediction method based on multi-modal data fusion, and relates to the field of medical image processing, and the method comprises the steps: obtaining non-small cell lung cancer CT image data, gene data and clinical data; performing focus segmentation on the non-small cell lung cancer CT image data; feature extraction is carried out based on the focus segmentation result, and radiomics features and depth image features are obtained; performing feature extraction on the gene data to obtain gene features; missing value deletion and numeralization preprocessing are carried out on the clinical data to obtain clinical features; local cross-modal attention fusion is carried out on the radiomics features and the depth image features; performing global cross-modal attention fusion on the preliminary mixed image features, the gene features and the clinical features to obtain original mixed features; and performing conditional guidance diffusion based on the original mixed features to obtain a lung cancer brain metastasis classification judgment result and a lifetime prediction risk. According to the invention, the lifetime prediction accuracy and stability can be improved.

The invention provides an unmanned aerial vehicle inspection defect sample generation method and system based on a diffusion model, and relates to the technical field of data processing, and the method comprises the steps: collecting an image of equipment, carrying out the feature compression and mapping operation of the image, carrying out texture gradient analysis and region density calculation, recognizing a local region with strong texture change in the image, and carrying out the feature compression and mapping operation; the method comprises the following steps: performing continuous aggregation on a local area, simulating a space range of defect growth, calculating a gradient difference value, determining consistent distribution of an image on a physical structure, determining a diffusion direction and a boundary position of a defect generation process, performing center feature extraction on the image, and determining a defect starting point and a growth speed of the defect generation process. And performing back diffusion to construct defect features, performing feature fusion on the defect features and the potential image data to obtain mixed image data, and performing feature reduction and image reconstruction to obtain defect image sample data. The defect sample can be reversely generated according to the equipment image.

Various structures for a hybrid pixel array are disclosed. The hybrid pixel array includes a combination of square photodiode (PD) pixel structures and split PD pixel structures. Square PD pixel structures include one photodiode per pixel unit while split PD pixel structures include two photodiodes per pixel unit. In certain instances, square PD pixel structures are used for green light pixels and split PD pixel structures are used for blue and red light pixels in the hybrid pixel array. The combination of square PD pixel structures and split PD pixel structures provides autofocus capability with higher signal strengths. Various techniques for row addressing and readout from the hybrid pixel array are also disclosed.

A method includes obtaining image frames from each camera disposed along a vehicle, where each image frame corresponds to a same timestamp. The method further includes constructing a first birds-eye view (BEV) image from each image frame with a first BEV module and constructing a second BEV image from each image frame by Inverse Perspective Mapping (IPM) with a second BEV module. The first BEV module extracts features of an external environment of the vehicle from each image frame, transforms the features to a three-dimensional space, and projects the three-dimensional space onto an overhead two-dimensional plane. Subsequently, a merging module merges the first and second BEV images to produce a hybrid BEV image. Features of an external environment of the vehicle within the hybrid BEV image are detected by a deep learning neural network and the hybrid BEV image is displayed to a user in the vehicle.

This invention provides an image processor, a processing method, a storage medium, and a mixed reality display system. The image processor includes: a first color management module, whose input is connected to a rendering module, for acquiring virtual layer data from the rendering module and performing first color processing on it; a second color management module, whose input is connected to a camera module, for acquiring real-world layer data from the camera module and performing second color processing on it; a layer blending module, whose inputs are respectively connected to the first and second color management modules, for performing layer blending processing on the color-processed virtual layer data and real-world layer data to determine mixed image data; and a third color management module, whose input is connected to the layer blending module, for acquiring mixed image data from the layer blending module, performing third color processing on it, and outputting the third-color-processed mixed image data to a mixed reality display.

Hybrid image sensors with video frame interpolation (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system comprises one or more event vision sensor (EVS) pixels, and a plurality of CMOS image sensor (CIS) pixels. Each EVS pixel can be configured to capture event data corresponding to contrast information of light incident on the EVS pixel. Each CIS pixel can be configured to capture CIS data corresponding to intensity of light incident on the CIS pixel. The imaging system can further comprise a deblur circuit configured to deblur the CIS data captured by the plurality of CIS pixels using a first portion of the event data captured by the one or more EVS pixels, and a system processor configured to interpolate a video frame using the deblurred CIS data and all or a subset of the event data.

The present disclosure provides a network model training method, a data processing method, and an apparatus. The network model training method comprises: acquiring target sample data, wherein the target sample data comprises text sample data and image sample data; inputting the target sample data into a network model to be trained to obtain a sample recognition result; and adjusting a parameter of a text encoder on the basis of a text recognition result and first supervision data corresponding to the text recognition result, adjusting a parameter of an image encoder on the basis of an image recognition result and second supervision data corresponding to the image recognition result, and a hybrid image-text recognition result and third supervision data corresponding to the hybrid image-text recognition result, and adjusting a parameter of a hybrid encoder on the basis of the hybrid image-text recognition result and the third supervision data corresponding to the hybrid image-text recognition result to obtain the trained network model formed by the text encoder, the image encoder, and the hybrid encoder.

Methods for operating hybrid image sensors having different CIS-to-EVS resolutions (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system includes a hybrid image sensor including (a) an event driven sensing array including one or more event vision sensor (EVS) pixels arranged in one or more EVS pixel rows and configured to capture EVS data having an EVS resolution, and (b) a pixel array including a plurality of CMOS image sensor (CIS) pixels arranged in one or more CIS pixel rows and configured to capture CIS data having a CIS resolution. The imaging system can further include control circuitry configured to adjust the CIS resolution of the CIS data and / or the EVS resolution of the EVS data such that a mismatch between the CIS resolution and the EVS resolution is reduced.

The invention discloses a monocular four-dimensional imaging device and method based on a metasurface. The monocular four-dimensional imaging device comprises an image sensor and an imaging optical system arranged in front of the image sensor. Wherein the imaging optical system comprises a metasurface, the metasurface is designed to simultaneously generate a self-acceleration Airy point spread function with depth-dependent lateral displacement for incident left-hand circularly polarized light and right-hand circularly polarized light, and the self-acceleration Airy point spread function can generate a self-acceleration Airy point spread function with depth-dependent lateral displacement under the condition that a mechanical moving part is not used. Carrying out polarization separation and focusing on the left-hand circularly polarized light and the right-hand circularly polarized light; the image sensor is used for recording a mixed image containing the left-hand circularly polarized light information and the right-hand circularly polarized light information at the same time through single exposure so as to obtain three-dimensional depth and polarization information of the metasurface, namely monocular four-dimensional imaging of the metasurface. The scene perception and understanding capability of a machine is greatly enhanced, and the method has unique advantages in the aspects of target identification, material discrimination, severe environment perception and the like.

Hybrid image sensors with multiple operating modes are disclosed herein. In one embodiment, a pixel arrangement includes a first photosensor, a first floating diffusion, a second photosensor, a second floating diffusion, and a mode switch. The mode switch can include (a) a first switch selectively coupling the second floating diffusion to the first floating diffusion, and (b) a second switch configured to selectively couple the second floating diffusion to event vision sensor (EVS) readout circuitry. The mode switch can be used to transition the pixel arrangement between (i) a first mode in which the pixel arrangement is controllable to output intensity information corresponding to first light incident on the first photosensor and / or second light incident on the second photosensor, and (ii) a second mode in which the pixel arrangement is controllable to output contrast information corresponding to the first light and / or the second light.

The present disclosure proposes methods, apparatuses, computer program products and non-transitory computer-readable medium for providing real-time virtual background in a video session. Real-time environment status information of a target user may be obtained, the real-time environment status information at least comprising geographic location information of the target user. A virtual visual representation corresponding to the real-time environment status information may be determined. A real-time virtual background may be formed through adding the virtual visual representation into a predetermined layout template. A mixed image corresponding to the target user may be formed through combining the real-time virtual background and a real-time human image of the target user. The mixed image may be presented in a user display region corresponding to the target user in a user interface of the video session.

The invention discloses a face in-vivo detection method and system based on scene optimization, a medium and a product, and relates to the field of computer systems based on a specific calculation model, and the method comprises the steps: extracting a mixed image sample, and applying physical drive transformation to obtain an enhanced image sample; scene comparison loss is calculated; based on living body classification loss and scene comparison loss, training the basic model to obtain a teacher model; inputting the scene sample into a scene branch of the teacher model to obtain a scene target vector; inputting the mixed image sample into a scene branch to obtain a scene vector set; calculating the vector similarity between the scene target vector and the vector in the scene vector set, screening out a corresponding image sample, and obtaining a scene matching data subset; taking the teacher model as supervision, executing knowledge distillation, and generating a living body detection model; and inputting a to-be-detected face image to the living body detection model to obtain a classification result. By implementing the method, the living body detection performance of the model in a specific deployment scene can be improved.

Hybrid image sensors with on-chip image deblur (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an image sensor includes (a) a plurality of CMOS image sensor (CIS) pixels configured to capture CIS data corresponding intensity of light incident on CIS pixels of the plurality of CIS pixels, (b) an event vision sensor (EVS) pixel configured to capture EVS data corresponding to events detected in light incident on the EVS pixel, and (c) a deblur circuit configured to generate deblurred image data based on the CIS data and an accumulation of events in the EVS data. The deblur circuit can be configured to compute the accumulation of events, such as using an event-based double integral model.

Systems and methods for intraoperative tracking and visualization are disclosed. A current minimally invasive surgical (MIS) instrument pose may be determined based on a live intraoperative input video stream comprising a current image frame captured by a MIS camera. In addition, an instrument activation state and at least one parameter value associated with the instrument may also be determined. Intraoperative graphic visualization enhancements may be determined based on the activation state of the instrument, and / or a comparison of parameter values with corresponding parametric thresholds. The visualization enhancements may be applied to a current graphics frame. The current graphics frame may also include visualization enhancements related to proximate anatomical structures with proximity determined from the instrument pose and an anatomical model. The current graphics frame may be blended with the current input image frame to obtain an output blended image frame, which may form part of an output video stream.

Hybrid image sensors with adjustable contrast thresholds (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system comprises one or more event vision sensor (EVS) pixels, and a plurality of CMOS image sensor (CIS) pixels. Each EVS pixel can be configured to, based on a contrast threshold, capture event data corresponding to contrast information of light incident on the EVS pixel. Each CIS pixel can be configured to capture CIS data corresponding to intensity of light incident on the CIS pixel. The imaging system can further comprise (i) a contrast threshold calibration circuit configured to adjust a value of the contrast threshold over time, and (ii) a deblur circuit configured to generate deblurred image data by deblurring the CIS data captured by the plurality of CIS pixels using at least a portion of the event data captured by the one or more EVS pixels.

The embodiment of the invention provides a medical image segmentation method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring original sample medical images of different modalities; determining a dominant mode and a secondary mode; performing image weighted fusion on the original sample medical image and the dominant weight corresponding to the dominant mode and the original sample medical image and the secondary weight corresponding to the secondary mode to obtain a mixed mode image; performing complementary mask processing on the mixed modal image to obtain a mixed image mask; performing feature extraction on each mixed image mask through a pre-constructed initial image feature extractor to obtain initial image features; and performing model training on the initial image feature extractor according to the initial image features and the original sample medical image corresponding to the dominant mode to obtain a target image feature extractor so as to obtain an image segmentation model. According to the embodiment of the invention, the image segmentation precision of the multi-modal medical image can be improved.

The present disclosure provides a method and system for adapting a machine learning model, such as an object detection model, to account for domain shift. The method includes receiving a labeled data elements and target image samples and performing a plurality of model adaptation epochs. Each adaptation epoch includes: predicting for each of the target image samples, using the machine learning model configured by a current set of configuration parameters, a corresponding target class label for the respective target data object included in the target image sample; generating a plurality of labeled mixed data elements that each include: (i) a mixed image sample including a source data object from one of the source image samples and a target data object from one of the target image samples, and (ii) the corresponding source class label for the source data object and the corresponding target class label for the target data object. The method also includes adjusting the current set of configuration parameters to minimize a loss function for the machine learning model for the plurality of mixed data elements. The method results in adapted machine learning model that accounts for domain shift and that has improved performance at inference on new target image samples.

The invention discloses an adversarial sample generation method based on uniform scaling and mixed masks, and the method specifically comprises the steps: 1, inputting an original image with a real label into an image classification model, and setting hyper-parameters; step 2, uniformly scaling the input original image to obtain a scaled copy; step 3, obtaining mixed images after the interference images are sampled, and obtaining a mixed mask of each mixed image; step 4, multiplying the scaled copy and each mixed mask element by element to obtain a transformation image; 5, obtaining the gradient of a loss function through the transformed image, and superposing the gradient; and step 6, updating the adversarial sample by using a gradient, and carrying out next iteration or output. According to the method, uniform scaling is used in a specified interval, nonlinear image mixing is carried out by using a mixed mask to replace a traditional image mixing strategy based on linear addition, the attack success rate and mobility of an adversarial sample are improved, and the problem that the migration effect of adversarial attacks among different models is poor is solved.

The application discloses an open set image classification field self-adaption method based on self-step learning, first, the original image is preprocessed to obtain an image set, then a feature extraction module and a double multi-class classifier module are constructed and trained to align shared class features of source domain images and target domain images and separate target domain private class features, a multi-criteria cross-domain hybrid module is further constructed and trained, cross-domain hybrid images are generated by using the source domain images and the target domain images, and the shared class features are self-learned, and finally, a classification result of the target domain image is output. Compared with the existing open set image classification field self-adaption method, the application covers smooth and non-smooth class distribution, and does not need to empirically adjust the threshold for distinguishing common class images and private class images in the inference stage, so that the model has good robustness under different hyperparameters and experimental settings.

The invention discloses a quantum classical mixed image classification method based on group isovariant and delayed aggregation. The method comprises the following steps: lifting a convolution module, inputting a target image, initializing a convolution kernel and rotating the convolution kernel, and carrying out convolution and stacking on the target image to obtain an output image; the group convolution module is used for converting the output image of the convolution module into an image sample and carrying out convolution and reverse remodeling operation to obtain an output image; the group pooling and flattening module is used for carrying out adaptive average pooling on an output image of the group convolution module, compressing the output image into a feature vector and obtaining an output feature vector; the group equivariant quantum feature processing module encodes the feature vectors to quantum bits according to the feature vectors output by the group pooling and flattening module; and the delay aggregation and output module measures the quantum bits, aggregates and classifies measurement results, and completes classification of target images. The method provided by the invention has strong nonlinear feature fusion capability, and can realize more accurate image classification.

A device for unmixing images of samples with fluorescent dyes includes one or more hardware processors. The device is configured to obtain a mixed image of a sample; determine an unmixed image based on the mixed image; determine a noise-map image based on the mixed image; determine a signal-to-noise image based on the unmixed image and the noise map-image; determine a denoised signal-to-noise image based on the signal-to-noise image; and determine a noise-reduced unmixed image of the sample based on the denoised signal-to-noise image and on the noise-map image.

The invention relates to a motor vehicle (1) with a display (8) and with an image generator (41) for generating a mixed image which comprises an edge region and a functional region for displaying and / or operating a function of the motor vehicle (1), wherein the mixed image and / or the edge region and / or the functional region can be displayed by means of the display, wherein the edge region at least partially surrounds the functional region, and wherein the motor vehicle (1) comprises a horizon determination module (42) for determining a horizon or the horizon, wherein the image generator (41) serves for such a fixed coupling of the mixed image and / or the functional region to the horizon, so that the position of the functional region follows the horizon and / or is dependent on the horizon.

To determine a composition ratio suitable for generation of a composite image having an extended dynamic range.SOLUTION: An image processing device includes: acquisition means that, for each of a plurality of HDR (High Dynamic Range) images photographed with different amounts of exposure, acquires exposure information of photographing and an upper limit value of a dynamic range expressed in the image; determination means that, for a composition process for composing the plurality of HDR images to generate a composite image in which the dynamic range is further extended, determines a composition ratio of each HDR image for each signal region of the composite image, based on the exposure information and the upper limit value of the dynamic range acquired by the acquisition means.SELECTED DRAWING: Figure 4

This disclosure describes systems, methods, and devices related to video post-processing using a single local directional pattern (LDP) for multiple post-processing steps. A method may include identifying video received by a first device from a second device and decoded by the first device; generating a LDP of a video frame of the decoded video; detecting, based on the LDP and the video frame input to a blurred region detection algorithm, a blurred region and a non-blurred region of the video frame; applying, based on the LDP and the video frame input to a super resolution algorithm, super resolution on the non-blurred region of the video frame without applying the super resolution to the blurred region; and generating, based on the LDP and the video frame input to a blended image algorithm, a blended image of a low-resolution image of the video frame and a high-resolution image of the video frame.

This application provides a multi-scale hybrid DR image enhancement method, apparatus, electronic device, and storage medium. The method includes: acquiring a target image to be identified; dividing the target image into N blocks according to N different block scales to obtain a first block image corresponding to each block scale; for each block scale, performing histogram equalization processing on each first block image corresponding to that block scale according to its corresponding contrast constraint to obtain a corresponding second block image; for each block scale, performing edge smoothing processing on each second block image corresponding to that block scale to obtain a corresponding third block image; and fusing all third block images corresponding to the N different block scales to obtain an enhanced image. This application solves the problem in related technologies where single local histogram equalization is difficult to simultaneously capture image details at different scales.

The invention discloses a single building image reflection artifact elimination method and system based on a multi-scale space attention mechanism, and the method mainly comprises the steps: constructing a single building image reflection artifact elimination network, taking a mixed image as a network input end, and carrying out the reflection artifact elimination of a single building image through the multi-characteristic module coupled single building image reflection artifact elimination network. The output end is a building entity and reflection artifact image; and designing a combined graph perception loss function, constraining the training process of the two-dimensional reflection artifact elimination network of the single building image by utilizing combined convergence of multiple graph perception losses, and guiding the decoupling of the mixed graph layer. According to the method, the reflection artifact elimination problem of the complex building image is solved, and the method is of great significance to image-based intelligent perception and urban ecological inversion.

The invention belongs to the technical field of calculation, and relates to a mixed image data correction system and method, and the method comprises the steps: enabling a first industrial robot to run according to a preset path, continuously shooting the images of a lower road surface in the running process, obtaining the position information and illumination intensity of a marked region in each image, and obtaining the position information and illumination intensity of the marked region; generating standard image information corresponding to the preset path; the first industrial robot sends the standard image information to an upper computer, and the standard image information is synchronized to all the industrial robots through the upper computer; after the industrial robot starts to work, the industrial robot runs according to a preset path, continuously shoots images of a road surface below the industrial robot in the running process, obtains position information and illumination intensity of a marked area in each image, and compares the position information and the illumination intensity with corresponding standard image information; whether driving deviates or not and whether the environment illumination intensity changes or not are judged, so that the calculation power needed by the industrial robot during judgment and comparison is reduced, and the comparison time is shortened.

This disclosure provides systems, methods, and devices for image signal processing that support improved demosaicing of color image signals. In a first aspect, an image processing method includes receiving a first image frame and a second image frame. The method may also include determining a first demosaiced image frame by applying a first demosaicking process to the first image frame, and determining a second demosaiced image frame by applying a second demosaicking process to the first image frame based on the second image frame. A blending weight may be determined based on the first image frame and the second image frame, and a blended image frame may be determined by combining pixel values from corresponding portions of the first demosaiced image frame and the second demosaiced image frame according to the blending weight. Other aspects and features are also claimed and described.

A system for producing an image to design a product can include a processor and a memory. The memory can store a regularizing module, a blending module, a denoising module, and a communications module. The regularizing module can produce a regularized image of a denoised image of an interpolation of a first diffused image and a second diffused image. The regularized image can be regularized with respect to a visual pattern. The blending module can: (1) determine a blending weight and (2) produce, based on the blending weight, a blended image of the denoised image and the regularized image. The denoising module can denoise the blended image to produce the image to design the product. The communications module can cause the image to be sent to a computer-aided design system to design the product.