245 results about "Hybrid image" patented technology

The invention discloses a solar cell defect detection method based on a multi-mode sensing technology. The system comprises the following steps: 1, multi-modal data acquisition: acquiring multi-modal data in real time by using four sensing technologies of near-infrared laser, area array visible light, PL and EL; 2, multi-modal data processing: carrying out denoising, edge detection and feature extraction on the acquired near-infrared, visible light, PL and EL images by adopting a mixed image processing algorithm to obtain key defect features; 3, multi-modal data fusion: combining a multi-modal data fusion method with a decision-making level strategy, synthesizing different technical data, improving the detection precision, and classifying, positioning and labeling the defects of cracks, broken gates, scratches, depressions, poor welding and hot spots by using a convolutional neural network to ensure accurate distinguishing of surface and internal defects; and 4, defect marking and production line adjustment: marking defects and adjusting the production line through a feedback and automatic control module, and feeding back a real-time visual result to a management layer.

The invention discloses a multi-task lung cancer brain metastasis lifetime prediction method based on multi-modal data fusion, and relates to the field of medical image processing, and the method comprises the steps: obtaining non-small cell lung cancer CT image data, gene data and clinical data; performing focus segmentation on the non-small cell lung cancer CT image data; feature extraction is carried out based on the focus segmentation result, and radiomics features and depth image features are obtained; performing feature extraction on the gene data to obtain gene features; missing value deletion and numeralization preprocessing are carried out on the clinical data to obtain clinical features; local cross-modal attention fusion is carried out on the radiomics features and the depth image features; performing global cross-modal attention fusion on the preliminary mixed image features, the gene features and the clinical features to obtain original mixed features; and performing conditional guidance diffusion based on the original mixed features to obtain a lung cancer brain metastasis classification judgment result and a lifetime prediction risk. According to the invention, the lifetime prediction accuracy and stability can be improved.

The invention provides a deep counterfeit detection method based on self-mixing, and compared with the prior art, the method generates a mixed image containing counterfeit artifacts by using a self-mixing technology, and carries out the precise detection and recognition of an artifact region of an input image through combining a backbone network OfficientNet-b3 and a counterfeit artifact detection module. The method comprises the following steps of: firstly, performing framing processing on a real image, generating a source image and a target image by utilizing a self-mixing technology, and generating a mixed image with counterfeit artifacts by combining a mask generator with an alpha blowing model; and then, the backbone network extracts global features from the mixed image, and the fake artifact detection module reinforces local feature expression of a mixed region through an artifact space cooperation attention module, so that an artifact region is accurately positioned. And finally, fusing global and local features by using residual connection, and transmitting the fused features to a full connection layer to complete classification. According to the method, various forged image types can be effectively detected, and excellent generalization ability is shown in cross-dataset and unknown scenes.

Disclosed in the present invention is a multi-source domain adaptive semantic segmentation method based on multi-level domain correlation, comprising: (1) pre-training a cross-domain semantic segmentation model F on all source domains and target domains; on the basis of the pre-trained F, calculating a multi-level domain correlation, comprising domain-level source-target correlation di and pixel-level source-target correlation wi (h,w); on the basis of the wi (h,w), constructing a source-target domain mixed image xh i and a corresponding pseudo label yh i; on the basis of a random sampling method, constructing a source-target domain mixed image xm i and a corresponding pseudo label ym i; on the basis of the di, the wi (h,w), the yh i, the xm i and the ym i, training the F in all the source domains and target domains; and using the trained F to perform multi-source domain adaptive semantic segmentation on an image to be tested, and extracting a target object in said image. The method can increase weights of source domains and source domain pixel points having high correlation with target domains while reducing weights of source domains and source domain pixel points having low correlation with the target domains, thereby avoiding the interference of unrelated information in the multi-source domains for training.

The invention provides an image restoration method and system based on dynamic decomposition and fusion, and the method comprises the steps: decomposing a to-be-restored image with hybrid features according to a degradation mode, and extracting degradation features of which the number is the same as the type number of the degradation mode; respectively processing each degradation feature to obtain a recovery feature corresponding to each degradation mode; and the recovery features are dynamically fused to obtain a clear recovery image with the same size as the to-be-recovered image, so that the function of performing high-quality recovery on the complex hybrid image is realized. According to the method, the recovery feature map is decomposed, processed and fused under multiple scales, the recovery feature map fusing the recovery features of the multiple scales is obtained, and the visual effect of hybrid image recovery is improved. According to the method, the symmetric U-shaped network is constructed based on the corresponding model formed by the method, and the processing sub-module corresponding to the degradation mode is constructed, so that complex and diversified degraded images in a real environment are well dealt with, and an efficient and reliable solution is provided for the field of image restoration.

The invention provides an unmanned aerial vehicle inspection defect sample generation method and system based on a diffusion model, and relates to the technical field of data processing, and the method comprises the steps: collecting an image of equipment, carrying out the feature compression and mapping operation of the image, carrying out texture gradient analysis and region density calculation, recognizing a local region with strong texture change in the image, and carrying out the feature compression and mapping operation; the method comprises the following steps: performing continuous aggregation on a local area, simulating a space range of defect growth, calculating a gradient difference value, determining consistent distribution of an image on a physical structure, determining a diffusion direction and a boundary position of a defect generation process, performing center feature extraction on the image, and determining a defect starting point and a growth speed of the defect generation process. And performing back diffusion to construct defect features, performing feature fusion on the defect features and the potential image data to obtain mixed image data, and performing feature reduction and image reconstruction to obtain defect image sample data. The defect sample can be reversely generated according to the equipment image.

The invention relates to the field of image processing, in particular to a multi-modal guided progressive image generation method, which comprises the following steps of: generating multi-modal features for received data; generating a final layer layout for each main object by using a layout denoising network through a noise sampling and multi-modal feature fusion method; extracting potential codes of a reference image through an inversion algorithm, combining layer text codes, performing iteration denoising through a stable diffusion model, and extracting a reference key value matrix; the initial noise, the final layer layout, the multi-modal features and the reference key value matrix are input into a layered diffusion model, and after multi-step denoising, a main body transparent image layer is generated through decoding; and after all the main body transparent layers are superposed to form a mixed image, driving the image recovery network to generate a target image based on global text coding. According to the method, the defects of an existing method in processing a complex scene are overcome; and the generated image is ensured to be consistent with the text description globally and locally.

Various structures for a hybrid pixel array are disclosed. The hybrid pixel array includes a combination of square photodiode (PD) pixel structures and split PD pixel structures. Square PD pixel structures include one photodiode per pixel unit while split PD pixel structures include two photodiodes per pixel unit. In certain instances, square PD pixel structures are used for green light pixels and split PD pixel structures are used for blue and red light pixels in the hybrid pixel array. The combination of square PD pixel structures and split PD pixel structures provides autofocus capability with higher signal strengths. Various techniques for row addressing and readout from the hybrid pixel array are also disclosed.

The invention provides an image classification method and device based on a student model, equipment and a medium, and relates to the technical field of knowledge distillation, and the method comprises the steps: obtaining a to-be-classified image; inputting the to-be-classified image into a target student model to obtain a category corresponding to the to-be-classified image; inputting the initial image set into a trained teacher model and a preset initial student model, and obtaining a teacher model imbalance category probability and a student model imbalance category probability; inputting the mixed image set into a trained teacher model and a preset initial student model, and obtaining a teacher model imbalance category probability and a student model imbalance category probability; determining a corresponding loss; according to the loss, performing back propagation to update student model parameters so as to obtain a target student model; when the image is classified, the resource computing power required by computer hardware can be reduced, the response speed of a computer is improved, and the phenomenon of computer jamming is avoided.

A method includes obtaining image frames from each camera disposed along a vehicle, where each image frame corresponds to a same timestamp. The method further includes constructing a first birds-eye view (BEV) image from each image frame with a first BEV module and constructing a second BEV image from each image frame by Inverse Perspective Mapping (IPM) with a second BEV module. The first BEV module extracts features of an external environment of the vehicle from each image frame, transforms the features to a three-dimensional space, and projects the three-dimensional space onto an overhead two-dimensional plane. Subsequently, a merging module merges the first and second BEV images to produce a hybrid BEV image. Features of an external environment of the vehicle within the hybrid BEV image are detected by a deep learning neural network and the hybrid BEV image is displayed to a user in the vehicle.

This invention provides an image processor, a processing method, a storage medium, and a mixed reality display system. The image processor includes: a first color management module, whose input is connected to a rendering module, for acquiring virtual layer data from the rendering module and performing first color processing on it; a second color management module, whose input is connected to a camera module, for acquiring real-world layer data from the camera module and performing second color processing on it; a layer blending module, whose inputs are respectively connected to the first and second color management modules, for performing layer blending processing on the color-processed virtual layer data and real-world layer data to determine mixed image data; and a third color management module, whose input is connected to the layer blending module, for acquiring mixed image data from the layer blending module, performing third color processing on it, and outputting the third-color-processed mixed image data to a mixed reality display.

Hybrid image sensors with video frame interpolation (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system comprises one or more event vision sensor (EVS) pixels, and a plurality of CMOS image sensor (CIS) pixels. Each EVS pixel can be configured to capture event data corresponding to contrast information of light incident on the EVS pixel. Each CIS pixel can be configured to capture CIS data corresponding to intensity of light incident on the CIS pixel. The imaging system can further comprise a deblur circuit configured to deblur the CIS data captured by the plurality of CIS pixels using a first portion of the event data captured by the one or more EVS pixels, and a system processor configured to interpolate a video frame using the deblurred CIS data and all or a subset of the event data.

The present disclosure provides a network model training method, a data processing method, and an apparatus. The network model training method comprises: acquiring target sample data, wherein the target sample data comprises text sample data and image sample data; inputting the target sample data into a network model to be trained to obtain a sample recognition result; and adjusting a parameter of a text encoder on the basis of a text recognition result and first supervision data corresponding to the text recognition result, adjusting a parameter of an image encoder on the basis of an image recognition result and second supervision data corresponding to the image recognition result, and a hybrid image-text recognition result and third supervision data corresponding to the hybrid image-text recognition result, and adjusting a parameter of a hybrid encoder on the basis of the hybrid image-text recognition result and the third supervision data corresponding to the hybrid image-text recognition result to obtain the trained network model formed by the text encoder, the image encoder, and the hybrid encoder.

The invention is suitable for the technical field of medical treatment, and provides a calcification image generation method, electronic equipment and a computer program product. The calcification image generation method comprises the following steps: acquiring an energy spectrum CT image; performing first base material decomposition operation on the energy spectrum CT image to obtain a material density map; performing second base material decomposition operation on the energy spectrum CT image to obtain a virtual plain scanning image; extracting a calcium area mask according to the material density map; performing weighting operation on the energy spectrum CT image to obtain a mixed image; and generating a calcification image according to the calcium region mask, the virtual plain-scan image and the mixed image. The accuracy of calcification evaluation can be improved.

The present disclosure provides a domain generalization method, device, server, and client. The method includes acquiring image discrimination information uploaded from multiple clients, wherein the image discrimination information is obtained by discriminators in the clients, the discriminators evaluating enhanced image generated by a generator in the server based on initial image of the respective clients, the initial image including amplitude information; and updating the generator based on the image discrimination information to obtain an updated generator; sending a multi-domain mixed image generated by the updated generator to the clients; and determining domain generalization parameters for classifiers in each client, based on model update parameters obtained after the clients update their classifiers using the initial image and the multi-domain mixed image.

Methods for operating hybrid image sensors having different CIS-to-EVS resolutions (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system includes a hybrid image sensor including (a) an event driven sensing array including one or more event vision sensor (EVS) pixels arranged in one or more EVS pixel rows and configured to capture EVS data having an EVS resolution, and (b) a pixel array including a plurality of CMOS image sensor (CIS) pixels arranged in one or more CIS pixel rows and configured to capture CIS data having a CIS resolution. The imaging system can further include control circuitry configured to adjust the CIS resolution of the CIS data and / or the EVS resolution of the EVS data such that a mismatch between the CIS resolution and the EVS resolution is reduced.

A method includes obtaining, using at least one processing device of an electronic device, raw image frames of a scene. The raw image frames include different sets of raw image frames captured at different viewpoints and different viewing angles relative to the scene. The method also includes performing, using the at least one processing device, blending of each set of raw image frames in order to generate blended image frames of the scene. The method further includes training, using the at least one processing device, a machine learning model using the blended image frames. The machine learning model is trained to generate three-dimensional (3D) information about the scene from viewpoints and viewing angles not captured in the sets of raw image frames.

The invention discloses a monocular four-dimensional imaging device and method based on a metasurface. The monocular four-dimensional imaging device comprises an image sensor and an imaging optical system arranged in front of the image sensor. Wherein the imaging optical system comprises a metasurface, the metasurface is designed to simultaneously generate a self-acceleration Airy point spread function with depth-dependent lateral displacement for incident left-hand circularly polarized light and right-hand circularly polarized light, and the self-acceleration Airy point spread function can generate a self-acceleration Airy point spread function with depth-dependent lateral displacement under the condition that a mechanical moving part is not used. Carrying out polarization separation and focusing on the left-hand circularly polarized light and the right-hand circularly polarized light; the image sensor is used for recording a mixed image containing the left-hand circularly polarized light information and the right-hand circularly polarized light information at the same time through single exposure so as to obtain three-dimensional depth and polarization information of the metasurface, namely monocular four-dimensional imaging of the metasurface. The scene perception and understanding capability of a machine is greatly enhanced, and the method has unique advantages in the aspects of target identification, material discrimination, severe environment perception and the like.

Hybrid image sensors with multiple operating modes are disclosed herein. In one embodiment, a pixel arrangement includes a first photosensor, a first floating diffusion, a second photosensor, a second floating diffusion, and a mode switch. The mode switch can include (a) a first switch selectively coupling the second floating diffusion to the first floating diffusion, and (b) a second switch configured to selectively couple the second floating diffusion to event vision sensor (EVS) readout circuitry. The mode switch can be used to transition the pixel arrangement between (i) a first mode in which the pixel arrangement is controllable to output intensity information corresponding to first light incident on the first photosensor and / or second light incident on the second photosensor, and (ii) a second mode in which the pixel arrangement is controllable to output contrast information corresponding to the first light and / or the second light.

The present disclosure proposes methods, apparatuses, computer program products and non-transitory computer-readable medium for providing real-time virtual background in a video session. Real-time environment status information of a target user may be obtained, the real-time environment status information at least comprising geographic location information of the target user. A virtual visual representation corresponding to the real-time environment status information may be determined. A real-time virtual background may be formed through adding the virtual visual representation into a predetermined layout template. A mixed image corresponding to the target user may be formed through combining the real-time virtual background and a real-time human image of the target user. The mixed image may be presented in a user display region corresponding to the target user in a user interface of the video session.

The invention discloses a face in-vivo detection method and system based on scene optimization, a medium and a product, and relates to the field of computer systems based on a specific calculation model, and the method comprises the steps: extracting a mixed image sample, and applying physical drive transformation to obtain an enhanced image sample; scene comparison loss is calculated; based on living body classification loss and scene comparison loss, training the basic model to obtain a teacher model; inputting the scene sample into a scene branch of the teacher model to obtain a scene target vector; inputting the mixed image sample into a scene branch to obtain a scene vector set; calculating the vector similarity between the scene target vector and the vector in the scene vector set, screening out a corresponding image sample, and obtaining a scene matching data subset; taking the teacher model as supervision, executing knowledge distillation, and generating a living body detection model; and inputting a to-be-detected face image to the living body detection model to obtain a classification result. By implementing the method, the living body detection performance of the model in a specific deployment scene can be improved.

The invention discloses a cross-domain remote sensing image semantic segmentation method based on image mixing and prototype comparison, and the method comprises the steps: inputting a target domain image into a cross-domain remote sensing image semantic segmentation model, and outputting a confidence coefficient; the step of training the cross-domain remote sensing image semantic segmentation model comprises the steps of preprocessing a source domain image with a label and a target domain image without a label; mixing the preprocessed source domain image and the target domain image through an adaptive mixing strategy to obtain a mixed image; semantic features of the source domain image and the target domain image are extracted, and the semantic features of the source domain and the target domain are aligned through prototype contrast learning; and training a cross-domain remote sensing image semantic segmentation model based on a semantic segmentation framework Segform by using the mixed image, the source domain image and the confidence coefficient. According to the method, information interaction between the source domain and the target domain is effectively utilized through the hybrid strategy, and the model is better helped to learn semantic features of the same category between different domains.

Hybrid image sensors with on-chip image deblur (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an image sensor includes (a) a plurality of CMOS image sensor (CIS) pixels configured to capture CIS data corresponding intensity of light incident on CIS pixels of the plurality of CIS pixels, (b) an event vision sensor (EVS) pixel configured to capture EVS data corresponding to events detected in light incident on the EVS pixel, and (c) a deblur circuit configured to generate deblurred image data based on the CIS data and an accumulation of events in the EVS data. The deblur circuit can be configured to compute the accumulation of events, such as using an event-based double integral model.

Systems and methods for intraoperative tracking and visualization are disclosed. A current minimally invasive surgical (MIS) instrument pose may be determined based on a live intraoperative input video stream comprising a current image frame captured by a MIS camera. In addition, an instrument activation state and at least one parameter value associated with the instrument may also be determined. Intraoperative graphic visualization enhancements may be determined based on the activation state of the instrument, and / or a comparison of parameter values with corresponding parametric thresholds. The visualization enhancements may be applied to a current graphics frame. The current graphics frame may also include visualization enhancements related to proximate anatomical structures with proximity determined from the instrument pose and an anatomical model. The current graphics frame may be blended with the current input image frame to obtain an output blended image frame, which may form part of an output video stream.

Hybrid image sensors with adjustable contrast thresholds (and associated systems, devices, and methods) are disclosed herein. In one embodiment, an imaging system comprises one or more event vision sensor (EVS) pixels, and a plurality of CMOS image sensor (CIS) pixels. Each EVS pixel can be configured to, based on a contrast threshold, capture event data corresponding to contrast information of light incident on the EVS pixel. Each CIS pixel can be configured to capture CIS data corresponding to intensity of light incident on the CIS pixel. The imaging system can further comprise (i) a contrast threshold calibration circuit configured to adjust a value of the contrast threshold over time, and (ii) a deblur circuit configured to generate deblurred image data by deblurring the CIS data captured by the plurality of CIS pixels using at least a portion of the event data captured by the one or more EVS pixels.

The embodiment of the invention provides a medical image segmentation method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring original sample medical images of different modalities; determining a dominant mode and a secondary mode; performing image weighted fusion on the original sample medical image and the dominant weight corresponding to the dominant mode and the original sample medical image and the secondary weight corresponding to the secondary mode to obtain a mixed mode image; performing complementary mask processing on the mixed modal image to obtain a mixed image mask; performing feature extraction on each mixed image mask through a pre-constructed initial image feature extractor to obtain initial image features; and performing model training on the initial image feature extractor according to the initial image features and the original sample medical image corresponding to the dominant mode to obtain a target image feature extractor so as to obtain an image segmentation model. According to the embodiment of the invention, the image segmentation precision of the multi-modal medical image can be improved.

The present invention relates to a remote sensing image cloud detection method based on multi-attention and multi-level feature mixing, which belongs to the field of satellite remote sensing image processing technology, including: constructing a cloud and fog detection data set through an existing data set or collecting real remote sensing satellite image data, and marking the cloud and fog areas therein; constructing a cloud and fog detection network model; training and verifying the cloud and fog detection network model; inputting the remote sensing image to be detected into the cloud and fog detection network model to obtain the cloud and fog mask area, and realizing the cloud detection task. The method of the present invention is simple and reasonably designed. The encoder module adopts an efficient self-attention mechanism and a channel attention mechanism to enhance the ability of the model network to extract image features, and mixes image features from different directions and angles respectively. The decoder module introduces multi-level MLP and upsampling, which fully improves the utilization ability of features at all levels, improves the detection ability of clouds and fogs of different scales, and improves the cloud detection accuracy.

The present disclosure provides a method and system for adapting a machine learning model, such as an object detection model, to account for domain shift. The method includes receiving a labeled data elements and target image samples and performing a plurality of model adaptation epochs. Each adaptation epoch includes: predicting for each of the target image samples, using the machine learning model configured by a current set of configuration parameters, a corresponding target class label for the respective target data object included in the target image sample; generating a plurality of labeled mixed data elements that each include: (i) a mixed image sample including a source data object from one of the source image samples and a target data object from one of the target image samples, and (ii) the corresponding source class label for the source data object and the corresponding target class label for the target data object. The method also includes adjusting the current set of configuration parameters to minimize a loss function for the machine learning model for the plurality of mixed data elements. The method results in adapted machine learning model that accounts for domain shift and that has improved performance at inference on new target image samples.

A goggle system is provided. The goggle system includes a computing device, a goggle device configured to be worn by a user and including a detector configured to simultaneously acquire image data of a subject in a first image mode and a second image mode, at least one eye assembly configured to display at least one of an image in the first image mode, an image in the second image mode, and a hybrid image including pixels of image data from the first image mode and pixels of image data from the second image mode, and a communications module configured to transmit acquired image data from the goggle device to the computing device.

The invention discloses an adversarial sample generation method based on uniform scaling and mixed masks, and the method specifically comprises the steps: 1, inputting an original image with a real label into an image classification model, and setting hyper-parameters; step 2, uniformly scaling the input original image to obtain a scaled copy; step 3, obtaining mixed images after the interference images are sampled, and obtaining a mixed mask of each mixed image; step 4, multiplying the scaled copy and each mixed mask element by element to obtain a transformation image; 5, obtaining the gradient of a loss function through the transformed image, and superposing the gradient; and step 6, updating the adversarial sample by using a gradient, and carrying out next iteration or output. According to the method, uniform scaling is used in a specified interval, nonlinear image mixing is carried out by using a mixed mask to replace a traditional image mixing strategy based on linear addition, the attack success rate and mobility of an adversarial sample are improved, and the problem that the migration effect of adversarial attacks among different models is poor is solved.