Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

681 results about "Image alignment" patented technology

Image alignment is the process of matching one image called template (let's denote it as T) with another image, I (see the above figure). There are many applications for image alignment, such as tracking objects on video, motion analysis, and many other tasks of computer vision.

Sample handling apparatus and image registration methods

A method for aligning a sample to an array is provided. An image of sample image of a sample can be received by a data processor. The sample image having a first resolution. An array image including an overlay of an array with the sample and an array fiducial can be received by the data processor. The array image having a second resolution lower than the first resolution of the sample image. The sample image can be registered to the array image by aligning the sample image and the array image. An aligned image can be generated based on the registering. The aligned image can include an overlay of the sample image with the array. The aligned image can be provided by the data processor. A method for detecting fiducials associated with an array is provided. Systems and non-transitory computer readable mediums performing the method are also provided.
Owner:10X GENOMICS INC

Fruit tree pest detection method based on machine vision

The invention relates to the technical field of agricultural intelligent detection, in particular to a fruit tree disease and pest detection method based on machine vision, which comprises the following steps: acquiring visible light, infrared light and ultraviolet light images of a fruit tree at multiple angles through a multispectral camera to obtain complete fruit tree surface information, and preprocessing the acquired images to obtain a fruit tree disease and pest detection result; comprising the steps of denoising, illumination correction and image alignment, extracting feature vectors, including the color, shape and size of scabs, pest distribution and other features, of pest and disease damage from preprocessed images through a convolutional neural network, fusing the pest and disease damage features with environmental data by using a multi-modal data fusion algorithm, and enhancing the accuracy of pest and disease damage classification. Classifying and identifying diseases and pests based on a support vector machine classification algorithm; according to the dynamically adjusted threshold value and suggestions provided by the expert system, a disease and pest diagnosis report is generated, and prevention and control measure suggestions are provided; according to the invention, efficient and accurate detection and management of fruit tree diseases and insect pests can be realized, and the intelligent level of agricultural management is improved.
Owner:ZHAOQING UNIV

Efficient panoramic image splicing method and system based on multi-view fusion

The invention relates to an image processing technology, and discloses an efficient panoramic image splicing method and system based on multi-view fusion, and the method comprises the steps: collecting a plurality of images from different views; performing multi-scale feature extraction on each image, generating a feature descriptor for each extracted feature point, and matching a corresponding feature point pair; performing multi-view geometric constraint screening on the feature point pairs; according to the screened feature point pairs, estimating a homography matrix between adjacent images, and carrying out global optimization on the homography matrix; aligning all the images into the same coordinate system; determining an overlapping region between adjacent images; and according to the pixel information in the overlapping areas, fusing the overlapping areas by adopting a self-adaptive weighted fusion algorithm so as to splice the plurality of images into a panoramic image. The invention further discloses a control device and a computer readable storage medium. The invention aims to improve the efficiency and accuracy of generating the multi-view fused panoramic image.
Owner:SHENZHEN QINUO TECH CO LTD

Multi-modal medical image fusion diagnosis system based on artificial intelligence

The invention discloses a multi-modal medical image fusion diagnosis system based on artificial intelligence, and the system comprises the following steps: extracting shared features of CT and MRI images through a convolutional neural network, and mapping the shared features to the same feature space; a two-way step-by-step alignment strategy is adopted, a three-dimensional deformation field matrix is generated, and cross-modal image anatomical structure alignment is achieved; calculating modal feature weights and eliminating distribution differences through an attention mechanism and an adversarial domain adaptation layer; constructing a CT-MRI image block contrast learning task, and optimizing a shared feature encoder; a conditional generative adversarial network is used for generating a false image of a missing mode according to the semantic segmentation map, and data distribution is constrained through a Wasserstein distance; uniform feature extraction of multi-modal medical images is realized through a shared feature encoder, the cross-modal image alignment accuracy is improved in combination with a bidirectional deformation field prediction module, and the comprehensiveness and accuracy of fusion features are enhanced by using a multi-modal feature fusion module.
Owner:SHANXI MEDICAL UNIV

Three-dimensional model reconstruction method based on point cloud data processing

The invention relates to the technical field of three-dimensional model reconstruction, and discloses a three-dimensional model reconstruction method based on point cloud data processing, which comprises the following steps: step 1, point cloud data acquisition and multi-modal fusion: acquiring point cloud data through a laser radar, an RGB camera and multi-view three-dimensional reconstruction to obtain a point cloud set, aligning the point cloud data with an RGB image to obtain a point cloud image set; establishing a mapping relation between point cloud coordinates and image pixel coordinates through an internal parameter matrix and an external parameter matrix of the camera to obtain aligned multi-modal point cloud data; 2, preprocessing the point cloud data: performing cleaning and downsampling processing on the multi-modal point cloud data obtained in the step 1; by combining point cloud data obtained by a laser radar, an RGB camera and multi-view three-dimensional reconstruction, accurate alignment of different sensor data is realized, high-precision and complete multi-modal point cloud data is obtained, blind areas of all sensors are effectively supplemented, and the precision and reliability of a three-dimensional reconstruction result are improved.
Owner:CHINA TELEVISION DIGITAL (BEIJING) CULTURE MEDIA CO LTD

Embedded reconstructed text-image alignment style migration method

The invention relates to an innovative embedded reconstructed text-image alignment style migration method, which comprises the following steps of: constructing a basic framework based on a diffusion model, and adding noise through a forward process and denoising to learn image style characteristics. And extracting style embedding from the artwork data set by utilizing perceptron attention and a feedforward network. Text embedding is generated through a text encoder, and guidance is provided for style migration. A cross attention mechanism is adopted to fuse style embedding and text embedding, multi-modal embedding is generated, and interaction of text and image information is optimized. And fusing multi-modal embedding and image embedding through linear interpolation, splicing with text embedding to form complete prompt embedding, and integrating the prompt embedding into a diffusion model to generate a style image matched with text description. According to the method, the quality and diversity of artistic image style migration are remarkably improved, the method has wide application potential in the field of artistic creation and design, rich and diversified stylized visual achievements can be provided, and the requirements of multiple creation scenes are met.
Owner:TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL

Utilizing individual-concept text-image alignment to enhance compositional capacity of text-to-image models

The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize a text-image alignment loss to train a diffusion model to generate digital images from input text. In particular, in some embodiments, the disclosed systems generate a prompt noise representation form a text prompt with a first text concept and a second text concept using a denoising step of a diffusion neural network. Further, in some embodiments, the disclosed systems generate a first concept noise representation from the first text concept and a second concept noise representation from the second text concept. Moreover, the disclosed systems combine the first and second concept noise representation to generate a combined concept noise representation. Accordingly, in some embodiments, by comparing the combined concept noise representation and the prompt noise representation, the disclosed systems modify parameters of the diffusion neural network.
Owner:ADOBE INC

Cross-modal adaptive matching laser radar and camera on-line external parameter calibration method and system

The invention provides a cross-modal adaptive matching online external parameter calibration method and system for a laser radar and a camera, and belongs to the technical field of online external parameter calibration of the laser radar and the camera, initial external parameters are obtained by using a static calibration result, a laser radar point cloud is projected to a visual angle of a virtual camera, an LIP image aligned with a camera image is generated, and a calibration result is obtained. Performing cross-modal segmentation on the LIP image and the RGB image, extracting a semantic mask, generating a cross-modal feature point pair through a strategy, and performing preliminary external parameter calculation in combination with a PnP algorithm; global optimization is carried out through a multi-modal loss function, and an inter-frame smoothing and increment optimization strategy is introduced to ensure accurate estimation of external parameters. Finally, the optimized external parameter matrix is applied to projection and matching of subsequent frames, and can adapt to environment change and data dynamic update in real time. The method has high precision and real-time performance, overcomes the defects that off-line calibration depends on a specific calibration target and cannot cope with dynamic environment changes, and adapts to application requirements in high-speed motion and complex environments.
Owner:BEIJING JIAOTONG UNIV

Liver focus three-dimensional modeling method

The invention provides a liver focus three-dimensional modeling method, and belongs to the technical field of image processing based on computer vision. Firstly, a multi-view spatial registration method based on optical flow optimization is designed, pixel-level displacement information of different view images is estimated by calculating an optical flow field, accurate image alignment is achieved, and spatial consistency of three-dimensional reconstruction is improved. And secondly, a three-dimensional reconstruction strategy based on two-dimensional focus segmentation is proposed, the two-dimensional focus segmentation is completed by adopting a lightweight U-Net variant, and a segmentation result is mapped to a three-dimensional space through a voxel probability projection method, so that 3D focus reconstruction is realized, and the calculation cost is reduced. And finally, extracting high-frequency features of the three-dimensional model by adopting a local edge enhancement method based on a Laplacian operator, and strengthening a focus boundary and a key anatomical structure through interpolation optimization, so that the three-dimensional model is more accurate and clearer. Compared with a traditional method, the method has the advantages that the mode of purely depending on image superposition is avoided, and the accuracy of three-dimensional modeling is improved.
Owner:QINGDAO MUHUA DATA TECHNOLOGY CO LTD

Building defect detection method and intelligent imaging device

The invention relates to the technical field of building structure health monitoring, in particular to a building defect detection method based on infrared thermal imaging and visible light bimodal fusion and a matched intelligent imaging device. According to the method, infrared thermal imaging and visible light bimodal image fusion are combined, and an improved YOLOv8m-seg model is combined, so that accurate detection of the defects of the outer wall of the building is realized. Firstly, a bimodal data set containing multi-material temperature difference data is constructed, ORB feature matching is combined with a feature alignment module (FAM) to achieve accurate image alignment, the defect detection robustness is improved through an improved bimodal feature fusion network, and finally automatic recognition and quantitative analysis of cracks, fractures and other defects are achieved. The problems that single-mode detection is interfered by dirt and is poor in multi-material adaptability are solved, the detection accuracy rate reaches 96% or above, the recall rate exceeds 93%, the single image detection time is shorter than or equal to 0.3 s, and the method is suitable for efficient detection of various building outer wall defects.
Owner:CHANGSHA XINTAI INSTR CO LTD

Method of using imaging devices in surgery

A method for generating and updating a three-dimensional representation of a surgical site based on imaging data from an imaging system is disclosed. The method comprises the steps of generating a first image of the surgical site based on structured electromagnetic radiation emitted from the imaging system, receiving a second image of the surgical site, aligning the first image and the second image, generating a three-dimensional representation of the surgical site based on the first image and the second image as aligned, displaying the three-dimensional representation on a display screen, receiving a user selection to manipulate the three-dimensional representation, and updating the three-dimensional representation as displayed on the display screen from a first state to a second state according to the received user selection.
Owner:CILAG GMBH INTERNATIONAL

Printing quality online detection method based on artificial intelligence

The invention belongs to the technical field of quality detection, and discloses a printing quality online detection method based on artificial intelligence. The method comprises: collecting a printed matter image; preprocessing the printed matter image to obtain a preprocessed image; acquiring a reference image corresponding to the printed matter image; aligning the preprocessed image with the reference image, calculating an overall offset, and judging whether an overall offset instruction is generated or not; if the overall offset instruction is not generated, performing image segmentation on the preprocessed image to obtain a segmented image; key point matching is carried out on the segmented image, local offset is calculated, and whether a local offset instruction is generated or not is judged; pattern deviation in the printing process can be automatically recognized, manual intervention and errors are reduced, the rejection rate is reduced, and therefore the production efficiency and the product quality of printed matter are improved.
Owner:ZHONGSHAN HAIMEI PAPER CO LTD

Immersive video enhancement method and device based on frequency domain boundary collaborative optimization

The invention discloses an immersive video enhancement method and device based on frequency domain boundary collaborative optimization, and relates to the field of video processing, and the method comprises the steps: obtaining a to-be-reconstructed compressed multi-view texture and depth video sequence, and inputting the sequence into a trained immersive video enhancement model; a current to-be-enhanced video frame passes through a feature extraction module, and high-frequency features and low-frequency features are extracted respectively; the high-frequency features and the low-frequency features pass through a frequency domain enhancement module to obtain a frequency domain enhanced image; inputting the frequency domain enhancement image and the current video frame to be enhanced into a boundary enhancement module to obtain a fused image; and inputting the fused image and an adjacent video frame of the current to-be-enhanced video frame into a space-time deformable convolution module to obtain an aligned fused image, and performing prediction on the aligned fused image through a quality enhancement module to obtain an enhanced residual error and generate a corresponding reconstructed video. According to the method, the problems of compression artifacts, boundary artifacts, low quality of immersive videos and the like are solved.
Owner:HUAQIAO UNIVERSITY +2

Reloading pedestrian re-identification method and system based on visual language pre-training model

The invention relates to the technical field of computer vision, in particular to a reloading pedestrian re-identification method based on a visual language pre-training model. The method comprises the following steps: a training stage: inputting an image to obtain a clothing mask image, and generating clothing irrelevant / relevant prompts; text encoder parameters are fixed, prompt weights are optimized, text prompts are input into an encoder to obtain features, and a classifier is constructed to achieve image-text alignment through cross entropy loss; using a visual encoder to extract mask pattern features, and constraining a class center Euclidean distance to realize image-image alignment; stripping clothes characteristics: extracting clothes area characteristics and corresponding text characteristics, optimizing by a classifier, and introducing orthogonal loss to decouple clothes correlation; in the reasoning stage, a query image is input into a trained image encoder to extract features, and cosine similarity ranking and result returning are calculated according to the features of the image library. According to the technical scheme, the recognition accuracy of the pedestrian re-recognition method under the condition of pedestrian clothing change can be improved.
Owner:重庆脑与智能科学中心

Forest land landslide collapse area detection method based on unmanned aerial vehicle visual remote sensing

A forest land landslide collapse area detection method based on unmanned aerial vehicle visual remote sensing is characterized by comprising the following steps: step 1, constructing a forest land landslide collapse area detection system based on unmanned aerial vehicle visual remote sensing; 2, the unmanned aerial vehicle shoots remote sensing image data; 3, an image preprocessing module performs preprocessing operation on the historical satellite image data and the remote sensing image data to obtain a remote sensing image and a historical image; step 4, matching the feature points of the remote sensing image and the historical image by an image alignment module, calculating a transformation matrix, and completing alignment of the images before and after the landslide through the transformation matrix; step 5, identifying a landslide area by a landslide area identification module; step 6, a landslide area identification module extracts a landslide area contour; and 7, calculating the average gradient of the landslide area by a landslide area calculation module, and calculating the actual area of the slope of the landslide in combination with the plane projection area and the average gradient. The method has the advantage that the forest land landslide collapse area can be rapidly and accurately detected.
Owner:CHONGQING UNIV +1

Infrared image enhancement method and system based on local phase correlation

The invention relates to the technical field of image processing, and discloses an infrared image enhancement method and system based on local phase correlation, and the method comprises the steps: obtaining a plurality of continuous frames of infrared images, carrying out the intelligent partitioning of a reference frame, and calculating the variance feature, the method comprises the following steps: selecting regions of interest with rich information, independently executing phase correlation operation in each region to extract a local translation vector, obtaining global displacement estimation through weighted fusion, adopting an abnormal value detection algorithm to improve robustness, and finally realizing sub-pixel-level image alignment and intelligent weighted fusion. The method is suitable for real-time enhancement processing of satellite-borne infrared remote sensing images, the resource constraint requirement of an embedded platform is met while the processing quality is guaranteed, and an efficient and reliable technical scheme is provided for space remote sensing image processing.
Owner:SHANGHAI WEIXING DATA TECH CO LTD

Social network false message detection method based on large model and multi-modal fusion

The invention discloses a social network false message detection method based on large model and multi-modal fusion. According to the method, based on a given multi-modal message, for an image and a text, an embedding vector and aligned embedding are obtained respectively; utilizing the large model to obtain a background knowledge embedding vector; splicing and inputting the two aligned embedding pieces into a full connection layer to obtain a multi-modal hybrid aligned embedding vector; inputting image embedding, text embedding, hybrid alignment embedding and background knowledge embedding into an N-layer stacked progressive Transform model, and outputting an embedded vector after multi-modal fusion; and inputting the fused embedded vector into a classifier, and setting a cross entropy loss function at the same time. According to the detection method, texts, images, text-image alignment characteristics and background knowledge assisted by a large model are fused, false message detection is assisted by the background knowledge generated by the large language model, and different modal information is effectively fused by a designed stacking progressive model; and the accuracy of false message detection is improved from the perspective of multi-modal fusion.
Owner:HANGZHOU NORMAL UNIVERSITY

Biodiversity monitoring method and device and terminal equipment

The invention provides a biodiversity monitoring method and device and terminal equipment, and is suitable for the technical field of data processing, and the method comprises the steps: obtaining biological image information and biological sound information; performing feature extraction on the biological sound information to obtain multiple pieces of biological audio feature information; performing feature extraction on the biological image information to obtain multiple pieces of biological image feature information; performing alignment processing on the biological audio feature information and the biological image feature information to obtain a biological audio alignment feature and a biological image alignment feature; enhancing the biological audio alignment feature and the biological image alignment feature to obtain a biological audio monitoring feature and a biological image monitoring feature; and monitoring organisms according to the biological audio monitoring features and the biological image monitoring features. According to the invention, the audio information and the image information are combined, so that the accuracy and comprehensiveness of real-time monitoring of biodiversity are improved.
Owner:BIRDS DATA

Task-oriented grabbing method and system for cross-level constraint reasoning

The invention discloses a task-oriented grabbing method and system based on cross-level constraint reasoning, and belongs to the technical field of robot grabbing control. The method comprises the following steps: aligning a text instruction with an input image through a mask alignment module, generating a target area mask by utilizing SAM-Clip, and generating a target point cloud by taking the target area mask as spatial prior; and further analyzing an internal physical structure of the target by using VLM, guiding to generate a 6-DOF grabbing attitude, performing collision detection and quality scoring by combining high-level function and bottom-level geometric prior, and outputting an optimal grabbing attitude. According to the method, the problems that task understanding and scene perception are disjointed, and the grabbing posture lacks constraint are solved, and the grabbing success rate and the intelligent level of the robot in the open environment are remarkably improved.
Owner:XI'AN UNIVERSITY OF ARCHITECTURE AND TECHNOLOGY

Tumor space-occupying brain network neural image alignment method based on multi-modal fusion

The invention discloses a tumor space-occupying brain network neural image alignment method based on multi-modal fusion, and belongs to the technical field of medical image processing and artificial intelligence crossing. The method comprises the following core steps of multi-modal image heterogeneous feature decoupling, tumor occupation deformation field modeling, functional network topological structure maintenance, cross-modal feature adversarial alignment, dynamic deformation constraint optimization and clinical interpretability verification, and construction of a three-dimensional non-rigid registration network based on a double attention mechanism. And differential homeomorphic mapping of a tumor focus area and normal brain tissue is realized through the cascaded spatial transformation module. Aiming at the problems of insufficient multi-modal feature alignment and brain network topology distortion in the prior art, the invention provides a function connection constrained cross-modal fusion strategy, a graph convolution network is adopted to encode resting state function connection features, and network node displacement caused by tumor occupation is dynamically corrected in combination with deformable convolution and a bidirectional feature competition mechanism; a space consistency loss function based on white matter fiber bundle tracing is designed, and through diffusion tensor imaging feature guide structure-function bimodal joint optimization, the problems of insufficient registration precision in a focus area and whole brain network connection distortion of a traditional method are solved. Experiments show that the registration precision of the method in glioma cases reaches 0.82 mm and is improved by 37% compared with that of a traditional method, and dissection-function consistency of functional network reconstruction around tumors is remarkably improved.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

House full life cycle intelligent monitoring method based on machine vision

The invention discloses a house full life cycle intelligent monitoring method based on machine vision, particularly relates to the field of machine vision, and is used for solving the problems of insufficient image alignment precision, aging change recognition lag and incapability of accurately quantifying life cycle evolution trend in existing house structure aging detection. The method comprises the following steps of: acquiring an image sequence by setting a shooting point position of a house facade component, performing spatial composite alignment by utilizing affine transformation and feature edge fitting, extracting component key points in an aligned image, dividing the component key points into segmented regions, and constructing a structure life cycle image stacking group; then calculating color distribution dispersion and texture roughness entropy of each section of region, establishing an aging evolution feature sequence, executing trend mutation point detection, extracting a material state jump section, and constructing an aging trend jump map; and finally, based on the map, identifying a jump risk area and outputting house component aging information, thereby realizing accurate monitoring and trend study and judgment of the house outer wall structure state.
Owner:宁夏国科综合检验监测有限公司

Intelligent robot path planning method based on visual identification

The invention relates to the technical field of intelligent robot path planning based on visual identification, and discloses an intelligent robot path planning method based on visual identification. According to the scheme, the problems of inconsistency of environment sensing data and lack of accurate geometric information are solved by utilizing data acquisition and calibration processing of the camera and the laser radar. The camera collects images, and parameters are determined by hardware; the internal reference matrix of the camera ensures clear correspondence between pixels and actual coordinates. The laser radar collects point cloud data, records three-dimensional coordinates of the environment, and is high in precision and real-time. Through data fusion and calibration, point cloud and image alignment by the system, camera internal reference matrix establishment and external reference calibration, accurate mapping and back projection are realized, and grassland area and obstacle information is extracted. According to the method, the sensing precision is improved, support is provided for path planning and navigation, and the robot has accurate positioning and efficient path planning capabilities.
Owner:FUYANG ZEXI INFORMATION TECHNOLOGY CO LTD

Multi-frame photoacoustic image reconstruction method based on optical flow alignment and depth feature fusion

The invention discloses a multi-frame photoacoustic image reconstruction method based on optical flow alignment and depth feature fusion. The method comprises the following steps: S1, obtaining photoacoustic signal data; s2, reconstructing a photoacoustic cross-sectional image; s3, performing optical flow calculation and image alignment; and S4, training the deep feature fusion network. According to the method, through optical flow motion correction and a potential space learning mechanism, space-time information and complementary features in the aligned multiple frames of images are dynamically integrated, complementary information in the multiple frames of images is adaptively fused, noise is suppressed, and finally reconstruction of high signal-to-noise ratio and high spatial resolution images of biological tissues is achieved. Experimental results show that the method can significantly improve image quality, recover image distortion and detail loss caused by motion and noise, and provide a new effective scheme for promoting robust clinical application of a photoacoustic imaging technology.
Owner:CHANGCHUN NORMAL UNIV

Training-free text-image generation method based on diffusion model

The invention provides a training-free text-image generation method based on a diffusion model, and relates to the technical field of computer graphic processing and artificial intelligence. The method comprises the following steps: extracting semantic phrases and layout information in an input text by utilizing a natural language model, inputting the input text, the semantic phrases and the layout information as additional conditions into a diffusion model, and extracting cross attention maps of different time steps; a positive and negative sample concept and a foreground and background concept based on an object are constructed, a new loss function is calculated on a cross attention map for semantic information and layout information, the loss function combines semantic loss, regional loss and original loss of a diffusion model and is used for updating a potential space image, and the image is generated through iterative denoising and a decoder. According to the method, additional training is not needed, image generation output based on the diffusion model better meets text requirements, and a better text and image alignment effect is achieved.
Owner:SHENYANG JIANZHU UNIVERSITY

Brain multi-modal multi-sequence data registration method and device based on deep learning

The invention discloses a brain multi-modal multi-sequence data registration method and device based on deep learning. The method comprises the steps of performing first iteration processing on a target image and a first moving image to obtain a first deformation field, wherein the first iteration processing comprises image alignment constraint processing of the target image, smooth constraint processing of the first deformation field, and area alignment constraint processing of a tumor area in the image; registering the first moving image to the target image based on the first deformation field to obtain a second moving image; performing second iteration processing on the target image and the second moving image to obtain a second deformation field; and registering the second moving image to the target image based on the second deformation field to obtain a registered moving image. In the process of performing unsupervised training on the deep learning model, after multiple times of iterative processing, model parameters are updated based on image alignment constraint loss with a target image, deformation field smooth constraint loss and region alignment constraint loss of a tumor region in the image.
Owner:WEST CHINA HOSPITAL SICHUAN UNIV +1

Multi-fluorescence channel microsphere image alignment method, device and equipment

The invention discloses a multi-fluorescent-channel microsphere image alignment method, device and equipment. The method comprises the following steps: respectively acquiring fluorescent microsphere images at the same position through multiple fluorescent channels; preprocessing each fluorescent microsphere image to obtain a fluorescent correction image; performing microsphere region segmentation on each fluorescence correction image, and calculating a region center point corresponding to each microsphere region; intercepting a pixel matrix from the fluorescence correction image according to the central point of each region, and extracting a corresponding feature vector; any one fluorescence correction image is selected as a reference image, the remaining fluorescence correction images are subjected to pairwise registration according to the feature vectors, and adjustment offset is determined respectively; and correcting each fluorescence correction image according to the adjustment offset, and superposing the corrected fluorescence correction images to the reference image to generate a fluorescence channel alignment image. Therefore, high-precision image alignment is realized without depending on deep learning in a mode of performing offset registration on the fluorescent microsphere image in combination with the gradient direction.
Owner:LEAD HEALTHCARE TECHNOLOGY (GUANGZHOU) CO LTD

Multi-modal large model training system and method applied to traffic field

The invention provides a multi-modal large model training system and method applied to the traffic field. The system comprises a large language model module, a feature alignment module, a multi-modal module and a user interaction module. Wherein the large language model module adopts a decoder architecture improved in Transform, and traffic text data and non-traffic text data which are collected in advance are used for training to obtain a language model; the feature alignment module extracts high-level semantic features of an image through an image encoder, and performs image and text alignment training in combination with a language model to obtain a text image alignment model; the multi-modal module adopts multi-modal instruction data containing a text image question and answer pair and combines the text image alignment model to carry out fusion training to obtain a multi-modal large model; and the user interaction module is used for receiving a user request, performing reasoning prediction through a multi-modal large model, and providing prediction and interaction results. The method solves various complex and tedious problems in the traffic field, and has excellent expansibility and robustness.
Owner:ZHONGDIAN DATA IND CO LTD +1

Prostate cancer diagnosis method based on multimodal large model prompt learning mechanism

ActiveCN120954689AMedical automated diagnosisBiological modelsProstate ultrasoundRadiology
The invention belongs to the field of characterization learning, and particularly relates to a prostate cancer diagnosis method based on a multimodal large model prompt learning mechanism. Comprising the following steps: step 1, data preprocessing; 2, key frames and similarity are calculated, and irrelevant editing is compressed; step 3, text and image alignment training; step 4, a test stage; according to the method, a similarity-based screening mechanism is provided, ultrasonic videos under coarse-grained labels are preliminarily screened under segmentation of a large model, and focus areas are focused on in time sequence; meanwhile, a pre-processing mechanism based on a large model is provided, and the influence of a large amount of irrelevant information existing in prostate ultrasonic image scanning on the model is compressed at a data end, so that the diagnosis effect of the model is improved.
Owner:SHANGHAI TENTH PEOPLES HOSPITAL +1

Model training method, electronic equipment and computer readable storage medium

The invention discloses a model training method, electronic equipment and a computer readable storage medium, and relates to the technical field of data processing and large models. The method comprises the steps of obtaining a training data set, wherein the training data set comprises image annotation data of various types of concerned objects and description text annotation data associated with the image annotation data; training the initial multi-modal retrieval model by adopting the training data set to generate a learnable prompt; carrying out image coding training on the initial multi-modal retrieval model by adopting the training data set and the learnable prompt to generate an intermediate multi-modal retrieval model; the training data set is adopted to conduct text and image alignment training on the middle multi-modal retrieval model, a target multi-modal retrieval model is generated, and the target multi-modal retrieval model is used for conducting multi-modal retrieval on the target retrieval request to obtain a target retrieval result. According to the method and the device, the technical problems that a multi-modal retrieval model in related technologies cannot adapt to variability and large-scale data requirements, and the model performance is relatively poor are solved.
Owner:ALIBABA CLOUD FEITIAN (HANGZHOU) CLOUD COMPUTING TECH CO LTD

HDR video reconstruction method based on standardized stream

The invention discloses an HDR video reconstruction method based on a standardized stream, and belongs to the technical field of high dynamic range image processing. The method comprises the following steps of: firstly, constructing a convolution optical flow estimation module with a self-adaptive normalized structure, wherein the convolution optical flow estimation module is used for accurately acquiring optical flow information between adjacent frames in an alternative exposure LDR video image sequence; then, carrying out multi-level feature alignment on the image sequence through an image alignment module so as to reduce alignment errors caused by illumination difference and movement; and finally, inputting the aligned and fused multi-level LDR image features into a standardized flow reconstruction network to realize high-quality HDR video image reconstruction. Aiming at the video reconstruction problem under the alternate exposure condition, the invention designs a standardized flow modeling structure considering the optical flow estimation precision and the feature alignment effect, and effectively improves the HDR video reconstruction quality in a complex dynamic scene.
Owner:BEIHANG UNIV