18182 results about "Image pair" patented technology

Disclosed in the present invention is a salient contour matching-based method for target measurement in a severe imaging environment. The method specifically comprises: (1) acquiring a binocular image of a target; (2) establishing a global-local joint constraint-based background light estimation model, and removing a scattering effect of a medium in an imaging environment to obtain a restored left eye image and a restored right eye image; (3) learning an original image, and on the basis of a residual between a network reconstructed image and the original image, obtaining target localization prediction maps of the left eye image and the right eye image; and (4) respectively extracting contour lines of the target in the left eye image and the right eye image, constructing feature matching descriptors of contour points, performing stereo matching on the two sets of contour lines by minimizing matching cost, and performing three-dimensional reconstruction on the contour lines in light of calibrated intrinsic and extrinsic parameters to complete the measurement of a key size. According to the present invention, the key sizes of different targets in a severe environment can be accurately measured, thereby providing an effective solution for the problem of measuring the sizes of targets in a severe environment.

The invention discloses a stainless steel surface defect detection method based on machine vision, and relates to the field of material defect detection.The stainless steel surface defect detection method comprises the steps that stainless steel surface images are collected, the collected multispectral surface images are subjected to image fusion, and a multispectral composite image is generated; performing detail enhancement on the low-resolution image through a super-resolution reconstruction algorithm to obtain a high-resolution image set; positioning a defect area position from the high-resolution image set through a defect detection algorithm, and obtaining a defect position data set; extracting a historical image defect feature set and a real-time image defect feature set from the defect position data set, and constructing a defect classification model according to the historical image defect feature set; inputting the real-time image defect feature set into a defect classification model, and performing defect classification and evaluation to obtain a defect classification data set; and generating and distributing a defect detection report. By combining the multispectral imaging algorithm, the super-resolution reconstruction algorithm and the defect classification model, the automation, precision and efficiency of stainless steel surface defect detection are improved.

The invention relates to the technical field of image processing, and discloses a backlight effect image edge enhancement method based on intelligent identification, which comprises the following steps of: judging a field environment type; performing global optimization on the original image based on a set environment perception type enhancement mechanism according to the judged field environment type, and outputting a global pre-processing image; constructing a backlight area segmentation model for the globally preprocessed image; outputting a local enhanced image; designing a structure perception type local adaptive threshold algorithm for the local enhanced image, and outputting a binary image keeping structural continuity; extracting an edge image of the binarized image through an edge detection algorithm, and optimizing a topological structure of a contour in the binarized image; and comparing with a wood template processing standard feature library, outputting an edge quality evaluation result and feeding back to a processing control system. Defect detection and machining control depth linkage is achieved, passive detection is changed into active optimization, and the production efficiency and the yield are improved.

The invention relates to the technical field of wire disaster prevention analysis, and discloses an overhead transmission line icing monitoring and disaster prevention method and system. The method comprises the following steps: acquiring meteorological parameters, stress data and icing images through distributed sensing nodes; performing time-space synchronization processing on the data; calculating icing state information; evaluating an icing risk level; selectively starting the deicing device; and storing the data and extracting a law to optimize a disaster prevention plan. According to the invention, accurate monitoring of the icing state, scientific assessment of risks, intelligent selection of deicing measures and continuous optimization of disaster prevention strategies are realized, so that the ability of the power grid to deal with icing disasters is significantly improved, and economic losses and potential safety hazards are reduced.

The invention relates to the technical field of chip testing, in particular to a chip testing method and a chip testing system. The method comprises the following steps: acquiring a thermal imaging image of a working state of a chip; global color temperature offset optimization is carried out on the thermal imaging image of the working state of the chip, and spatial heat distribution mining is carried out, so that a chip heat distribution diagram is constructed; performing multi-moment heat distribution tracking processing on the chip heat distribution diagram, performing dynamic heat distribution evolution fitting, and constructing a dynamic heat distribution trend evolution diagram; acquiring a high-definition image of a to-be-tested chip; performing chip assembly visual monitoring on the high-definition image of the to-be-tested chip, performing heat distribution trend rendering based on the dynamic heat distribution trend evolution graph, and constructing a chip assembly heat distribution trend rendering model. According to the invention, through limit working condition test analysis, the evaluation efficiency and accuracy of the chip performance are improved.

The invention provides a toll vehicle type identification method and system based on expressway passing images, and the method comprises the steps: obtaining an image set of passing vehicles, carrying out the multi-scale brightness compensation processing of a vehicle side contour image, generating a side feature enhancement image, carrying out the contour sharpening processing of a vehicle top contour image, and obtaining a vehicle top contour image; and generating a top feature enhanced image. And performing dual-channel feature fusion on the side feature enhanced image and the top feature enhanced image, generating a vehicle multi-dimensional fusion feature map, inputting the vehicle multi-dimensional fusion feature map into a pre-trained vehicle type matching model, and obtaining a vehicle type feature matching result output by the vehicle type matching model. And performing similarity comparison according to a vehicle model feature matching result and a pre-stored vehicle model feature library, determining a charging vehicle model category corresponding to the passing vehicle, and generating a vehicle model identification mark. According to the method, the robustness of vehicle type recognition in a complex illumination environment can be improved, and meanwhile, the accuracy of vehicle type classification and the charging and passing efficiency are improved by fusing dual-view feature enhancement and a multi-dimensional feature matching mechanism.

The invention relates to the technical field of remote sensing monitoring, in particular to a remote sensing image ocean green tide monitoring method and system fusing physical prior and spatio-temporal evolution. The method comprises the following steps: acquiring a multi-modal remote sensing monitoring image; performing multi-modal feature extraction on the acquired image, wherein the multi-modal feature extraction comprises spectral reflectivity feature extraction, ocean dynamics feature extraction and feature alignment and unified representation; establishing a physical prior of a green tide characteristic wave band by using an ocean optical radiation transmission model; constructing a dynamic space-time diagram based on the extracted multi-modal features to obtain a node global feature vector and a dynamic adjacency matrix; carrying out adaptive graph convolution feature coding based on physical prior and a dynamic space-time diagram; through fusion of multi-spectral images of multiple platforms such as satellites and unmanned aerial vehicles and ocean dynamic data and combination of atmospheric correction and wave band resampling, consistency processing and high-precision extraction of multi-source features are realized, and comprehensiveness and reliability of green tide feature recognition are remarkably improved.

The invention relates to an image processing technology, and discloses an efficient panoramic image splicing method and system based on multi-view fusion, and the method comprises the steps: collecting a plurality of images from different views; performing multi-scale feature extraction on each image, generating a feature descriptor for each extracted feature point, and matching a corresponding feature point pair; performing multi-view geometric constraint screening on the feature point pairs; according to the screened feature point pairs, estimating a homography matrix between adjacent images, and carrying out global optimization on the homography matrix; aligning all the images into the same coordinate system; determining an overlapping region between adjacent images; and according to the pixel information in the overlapping areas, fusing the overlapping areas by adopting a self-adaptive weighted fusion algorithm so as to splice the plurality of images into a panoramic image. The invention further discloses a control device and a computer readable storage medium. The invention aims to improve the efficiency and accuracy of generating the multi-view fused panoramic image.

The invention relates to the technical field of intelligent vending, solves the problem that in the prior art, commodity identification cannot be accurately carried out in a multi-target scene, and provides a multi-target commodity identification method, device and system based on multi-modal data processing. The method comprises the following steps: acquiring multiple frames of real-time images in a commodity transaction scene; performing preprocessing and label information extraction on the real-time image, and determining character information corresponding to the target image and the commodity label; performing instance segmentation on the target image, and determining commodity position information; performing feature extraction on the target image, and determining commodity image feature information; according to pre-collected multi-source privatized data in an intelligent vending scene, performing fine adjustment and optimization processing on the open-source multi-modal visual language model to obtain a multi-modal large model; and inputting the commodity image feature information and the text information into the multi-modal large model for information fusion, and determining a commodity target identification result. According to the invention, commodity identification can be accurately carried out in a multi-target scene.

A computer implemented method, system and non-transitory computer-readable device for a remote deposit environment. Upon receiving a user request, based on interactions with the UI, the method implements an electronic deposit of a financial instrument by activating a camera on the client device to generate a live stream of image data of a field of view of at least one camera, wherein the live stream includes imagery of at least a portion of the financial instrument. The method continues by extracting in real-time, based on the formation of byte array objects from the live stream of image data, data fields from a ranked sequence of imagery to be processed by an optical character recognition (OCR) program resident on the client device. The OCR process extracts one or more data fields from of the financial instrument that are communicated to a remote deposit server to complete the electronic deposit.

The invention discloses a quartzite surface defect detection method, system and device, and relates to the technical field of image processing, and the method comprises the following steps: fixing a to-be-detected quartzite on a detection platform, and carrying out preprocessing; constructing a double-path polarization imaging light path; polarization parameter optimization is carried out based on the optical anisotropy characteristic of quartz stone, so that a polarization state difference is generated between a defect area and a normal area; under the condition of polarization parameter optimization, synchronously acquiring a first polarization image and a second polarization image through a double-path polarization imaging light path; performing polarization difference calculation on the first polarization image and the second polarization image to generate a polarization difference image; and carrying out contrast enhancement processing on the polarization difference image, identifying a defect area, carrying out defect classification, and outputting a quartz stone surface defect detection result. By optimizing polarization imaging parameter configuration and combining image processing, high-precision detection and intelligent classification of quartz stone surface defects are achieved, and the automation level and reliability of detection are improved.

The invention provides an engineering construction defect automatic detection and classification method based on deep learning, and the method comprises the steps: obtaining a welding seam surface image through the shooting of an unmanned plane, and carrying out the denoising and illumination normalization processing of the welding seam surface image, and obtaining a standardized image; welding seam surface texture features are extracted from the standardized image, a convolutional neural network is adopted to analyze the spatial distribution characteristics of textures, and vectorization processing is carried out to obtain texture feature vectors; segmenting a weld surface corresponding to abnormal region distribution by adopting a region growing algorithm, and analyzing pore and weld discontinuity in combination with the texture feature vector to obtain a defect candidate region; performing threshold division on the sizes and the numbers of the defects according to the defect types and the feature vectors of the candidate regions to obtain a severity grading result of each type of defects; and severity features are extracted from a grading result, and a Bayesian network is adopted to fuse texture feature vectors and defect type labels to obtain a welding quality evaluation score.

The invention discloses an engineering drawing label identification method and system based on multi-modal information extraction, and the method comprises the steps: obtaining an engineering drawing image, carrying out the preprocessing of the image, and setting a data structure Schema for controlling an extraction target and a relation mode; detecting a drawing label area in the drawing through the trained drawing label detection model to obtain a bounding box coordinate of each drawing label, mapping the detected bounding box coordinates back to a coordinate system of the original image, and cutting out a corresponding drawing label image from the original image; performing text recognition on the cut-out signature image, and extracting text content and corresponding textbox coordinate information; and inputting the clipped signature image and the text recognition result into a trained multi-modal information extraction model, performing information extraction according to a set Schema, and outputting a structured extraction result. According to the method, the extraction target can be flexibly defined, unified extraction of the two-tuples and the three-tuples is supported, and the extraction accuracy is high.

The invention discloses an MURA defect detection method based on background reconstruction and related equipment, and relates to the field of image processing. The method comprises the following steps: acquiring an initial image displayed by a display panel; performing discrete cosine transform processing on the initial image to extract low-frequency background components; denoising the low-frequency background components based on a singular value decomposition algorithm to obtain main background features; performing fitting reconstruction on the main background features through a polynomial surface fitting algorithm to obtain a background reconstruction image; performing contrast enhancement processing on the background reconstruction image to obtain a background enhancement image; subtracting the background enhancement image from the initial image to obtain a difference image; determining an MURA defect area to be detected according to the difference image; and performing identification processing on the MURA defect to-be-detected area through a preset MURA defect detection model to obtain an MURA defect detection result of the display panel. According to the technical scheme, the MURA defect detection precision can be improved.

The invention discloses a multi-modal entity and relation extraction method and system based on cross-modal alignment and fusion, and the method comprises the steps: carrying out the processing and coding of an input text and an image, and obtaining a plurality of types of image and text features; performing feature alignment on the fine-grained and coarse-grained text features and the pixel-level image representation by taking the semantic representation of the image as an anchor point, and mapping the image and the text features to the same semantic space; performing multi-granularity feature fusion through text-guided dynamic gating aggregation, visual prefix cross-modal fusion and cross-modal image-text matching, modeling association between noun phrases and image objects in a text while increasing feature complementarity, and obtaining multi-granularity multi-modal feature representation; fusing multi-granularity multi-modal features through entity guidance attention gating, and gathering visual information related to a text entity to obtain final multi-modal fusion representation; according to the multi-modal fusion representation, task prediction of multi-modal named entity recognition and multi-modal relation extraction is carried out.

The invention discloses an unmanned aerial vehicle multi-modal feature fusion target tracking method and system based on natural language description, belongs to the technical field of computer vision and image processing, and solves the problem that in the prior art, when the quality of an image collected by an unmanned aerial vehicle is poor or image features are not obvious, the target tracking capability and the long-time tracking capability are poor. Natural language description is carried out on a traffic accident scene in an image of an unmanned aerial vehicle visual angle, and a language prompt is obtained; constructing a scene-context feature pyramid network to perform context information enhancement processing on the image of the view angle of the unmanned aerial vehicle to obtain a feature-enhanced image; respectively carrying out visual coding and language coding on the enhanced image and language prompt to obtain visual features and language feature vectors, and carrying out visual-language bimodal feature local alignment; and fully fusing the obtained aligned new language features with the visual features to obtain multi-modal features for target tracking. The method is used for multi-modal feature fusion target tracking of the unmanned aerial vehicle.

The invention discloses a bridge health state detection early warning method and system based on image recognition, and relates to the field of bridge health monitoring, and the method comprises the steps: collecting a bridge crack image, carrying out the preprocessing of the bridge crack image, and forming a bridge crack image data set; carrying out manual amplification on the bridge crack image data set through a sliding window algorithm, and carrying out feature extraction to generate a feature data set; performing model training by adopting a convolutional neural network to obtain a crack recognition model; acquiring bridge health state data in real time through a sensor, and performing crack detection by using a crack recognition model; modeling the health state of the bridge by adopting a time sequence analysis method, and generating a dynamic damage prediction result; carrying out bridge health state evaluation, and carrying out early warning on bridge damage; according to the invention, amplification and feature extraction of the bridge crack image are carried out through the adaptive sliding window algorithm, and the precision and robustness of crack identification are improved in combination with the convolutional neural network and the dynamic attention mechanism.

The invention discloses an unmanned aerial vehicle laser and vision fusion inspection method and system for bridge bottom disease detection, and the method comprises the steps: carrying out the synchronous data collection through employing a calibrated laser radar, a camera and an IMU, and obtaining a three-dimensional laser point cloud and a two-dimensional visual image of the appearance of a bridge; sharpening the image containing the motion blur and completing brightness self-adaption of the image; stable feature points are extracted, multi-frame matching is carried out, the corresponding poses of the images are estimated, and bridge dense point cloud reconstruction is completed; performing geometric component segmentation on the point cloud to generate a geometric prior region; component segmentation is carried out on a support area in the image, and a continuous and accurate component segmentation result is obtained in combination with a geometric prior area; screening the image, calling a targeted disease detection model in a corresponding component area, and generating a segmentation mask for the disease; obtaining a real disease three-dimensional point cloud, and carrying out quantitative calculation on the physical size of the disease; and displaying the real disease three-dimensional point cloud data and the physical size of the disease. The method is high in efficiency and precision.

The invention discloses an underwater binocular positioning method and device based on target assistance and a storage medium, and belongs to the technical field of underwater environment perception, and the method comprises the following steps: obtaining internal and external parameters of a binocular camera; solving to obtain refraction parameters of the binocular camera; the binocular camera is controlled to shoot and collect the target image, an initial underwater cooperative positioning target image is obtained, the initial underwater cooperative positioning target image is corrected, and a corrected underwater cooperative positioning target image is obtained; a YOLOv8 model is adopted to detect a target, and an area where the target is located in the underwater cooperative positioning target image is selected; feature points are extracted and matched through an ORB algorithm, and matched feature points are obtained; and calculating to obtain the three-dimensional coordinate of the target in the camera coordinate system. According to the invention, accurate positioning of the underwater target can be realized through a binocular vision method, and working requirements in a complex underwater environment are met.

The invention relates to a bill voucher information extraction method based on multi-mode and OCR model fusion. The method comprises the following steps: S1, obtaining an image of a bill voucher; s2, preprocessing the image; s3, identifying the preprocessed image by using an OCR engine to obtain the text content and the corresponding two-dimensional coordinates of each text block; s4, taking the recognized text segments and the original image as input, performing joint coding by using a pre-trained multi-modal model, evaluating and outputting the matching degree of each text segment and a predefined field category by the model, and determining candidate texts of each field and confidence of the candidate texts; s5, accurately positioning and extracting the key field, and verifying the consistency of the OCR output and the semantic result; s6, if the verification result conflicts or the identification reliability of a certain field is lower than a threshold value, error correction operation is carried out; and S7, outputting the structured bill voucher information. Through multi-modal fusion and iterative correction, the error rate of non-standard voucher information extraction is effectively reduced, and the method is suitable for various voucher formats and complex scenes.

The invention provides an aluminum alloy surface oxidation spot defect identification method and device based on machine vision, and relates to the field of intelligent manufacturing and industrial automation, and the method comprises the steps: obtaining an aluminum alloy surface color image, and carrying out the preprocessing of the image, so as to extract a brightness component image; self-adaptive threshold segmentation of local contrast enhancement is carried out on the brightness component image, a defect area binary mask is generated, morphological connected domains are extracted according to the mask, and three basic feature indexes of the area pixel value, the contour Fourier descriptor complexity and the area gray scale standard deviation contrast of each connected domain are calculated; and extracting and marking a connected domain boundary, verifying a boundary closed topological structure, and dynamically generating a curvature-driven self-adaptive sampling point through multi-scale B-spline curvature extreme value detection. Through optical-algorithm-process three-level collaborative innovation, the curved surface reflection false alarm rate is reduced, the pinhole detection rate is increased, and the boundary precision is + / -0.2 pixel.

The invention discloses a distribution network tree barrier real-time analysis method and system based on dynamic vision and SLAM. The method comprises the following steps: generating a dynamic visual baseline by cooperatively controlling the translation and flight displacement of an unmanned aerial vehicle holder, and constructing a bionic binocular model to simulate a time sequence image into a binocular image pair; generating a depth point cloud through epipolar correction and stereo matching; key targets are recognized and extracted through a semantic segmentation network, and semantic point clouds are generated; establishing a dimensionality reduction motion model by utilizing pan-tilt stability augmentation, and fusing a visual inertial odometer and RTK data by adopting a filtering or optimization algorithm to realize centimeter-level pose estimation; and finally, performing optimization processing on the semantic point cloud, completing three-dimensional reconstruction based on multi-modal fusion, and outputting a risk assessment result through tree line spacing calculation and safety margin analysis. According to the invention, accurate, efficient and automatic routing inspection and risk early warning of the distribution network tree obstacles are realized.

Provided are an advertisement generation method and system, a medium and a device. The method comprises: acquiring standard-product information and environment information of at least one standard product (S201); generating a standard-product image corresponding to the standard-product information on the basis of artificial intelligence generated content (AIGC), measuring each dimensionality of the standard-product image to obtain label information of each dimensionality of the standard-product image, and performing semantic combination on the label information of each dimensionality and the environment information to generate an advertisement copywriting (S202); and matching an advertisement template from an advertisement template library according to the label information of each dimensionality, and combining the standard-product image and the advertisement copywriting into the advertisement template to generate a standard-product advertisement (S203). The solution has the technical advantages of simplicity and high efficiency.

The invention provides a cross-domain target detection method and system based on semi-supervised learning in extreme weather, and the method comprises the steps: enabling a student model to learn bounding box regression knowledge of a trained teacher model through distillation, so as to achieve the regression capability in extreme weather; performing instance-level feature adversarial training on a pre-training detection model which is obtained in a distillation stage and has a regression capability in extreme weather by using an intermediate-state image which is generated by a diffusion model and is closer to a target domain and has smaller noise, and determining a new teacher model and a new student model based on the detection model after the adversarial training; in the fine tuning stage, a dynamic category pseudo label screening strategy is utilized to screen pseudo labels generated by a new teacher model, and a screened target domain pseudo label is utilized to train a new student model to obtain a target detection model used for detecting a to-be-detected image in extreme weather. The phenomena of leak detection and error detection in extreme weather scenes in the prior art can be reduced, and the original reasoning speed is kept.

The invention belongs to the technical field of digital restoration of computer vision and cultural heritage, and particularly relates to a traditional picture restoration method fusing low-resolution prior and efficient visual selection, which comprises the following steps: constructing a multi-source image data set, taking images in the multi-source image data set as high-resolution images, preprocessing the high-resolution images to obtain low-resolution images, and carrying out high-resolution priori and high-efficiency visual selection on the low-resolution images. The high-resolution image and the low-resolution image are respectively masked to generate simulated damage mask images, and the simulated damage mask images comprise a regular damage mask image and an irregular damage mask image; taking the multi-source image data set and the preprocessed multi-source image data set as training data, and training a multi-source image model; the dual-stage repair network comprises a coarse repair network and a fine repair network; according to the method, the problems of structural semantic loss, high priori information dependency and insufficient global and local coordination when an existing image restoration method is used for processing a complex scene and a large-range missing region are solved.

The invention discloses a two-way visual saliency detection method and device combining difference guidance and texture enhancement, and the method comprises the following steps: 1, obtaining an image to be subjected to saliency detection, and carrying out the marking and preprocessing of a data set; 2, constructing a visual saliency detection model which comprises a dual-path encoder (a saliency detection path and an image reconstruction path), an adaptive interaction network, a decoder network and an output network; a significance detection path in the dual-path encoder network uses a pre-trained ConvNeXt encoder, and an image reconstruction path uses VQ-VAE as a backbone network; the adaptive interactive network comprises a multi-scale convolution module and a gating fusion module; the decoder network comprises a mutual conversion attention module and a double-gating fusion module; the output network comprises a multi-level feature fusion module; 3, training the saliency detection model to obtain a trained saliency detection model; and 4, carrying out saliency detection on the image data by adopting the trained saliency detection model.

The invention discloses an image restoration method, an image restoration device and a storage medium, which are used for improving the structure restoration precision and the detail restoration capability of image restoration. The method comprises the following steps: acquiring multi-dimensional inertial data in real time; performing frequency domain analysis on the multi-dimensional inertial data by adopting sliding window short-time Fourier transform to obtain vibration intensity; if the vibration intensity does not exceed the preset threshold value, acquiring an image; calculating a definition index of the image; determining whether the image is a blurred image according to a preset definition standard and the definition index of the image; if the image is judged to be a blurred image, dividing the blurred image into a motion blurred image and a focusing blurred image; respectively constructing point spread function models of the motion blurred image and the focusing blurred image; performing deconvolution processing or depth reconstruction on the blurred image through a point spread function model to obtain a clear image; and recalculating the definition index of the clear image, and if the definition index does not exceed the definition threshold, triggering reacquisition or switching the repair model to execute secondary repair.

The invention discloses a model training method, a garment version image generation method, equipment and a readable medium. The method comprises the following steps: acquiring training data of which the type comprises text data and / or image data, and encoding the training data to obtain a training condition encoding vector; determining an original token sequence and a token position code according to the initial data of the clothing version and a preset clothing version coding rule; according to each original token sequence and a preset time step, obtaining a noise-containing token sequence; inputting each training condition coding vector, the token position code, a preset time step and each noisy token sequence into an initial clothing pattern prediction model to obtain a noise reduction token sequence output by the initial clothing pattern prediction model for each training condition coding vector; the noise reduction token sequence can be used for generating a garment pattern image corresponding to the training data; and adjusting parameters of the initial clothing model prediction model until a convergent and fitted clothing model prediction model is obtained.

The invention discloses a shaving board surface defect detection method based on deep learning, and the method comprises the steps: collecting shaving board surface images, and screening out shaving board images with surface defects; manually marking the surface defects of the shaving board in the screened shaving board image, constructing a shaving board surface defect data set, and dividing the shaving board surface defect data set into a training set, a verification set and a test set in proportion; an improved YOLOv11 target detection model is constructed; inputting the training set and the verification set into an improved YOLOv11 target detection model for training; performing comprehensive evaluation and analysis on a training result by using an evaluation index; and testing the trained improved YOLOv11 target detection model by using the data of the test set. According to the method, various types of shaving board surface defects can be accurately predicted, the detection precision of tiny defects and complex defects on the surface of the shaving board is also improved, meanwhile, the detection precision and efficiency of the shaving board surface defects are greatly improved, and the labor cost is greatly reduced.

The invention provides a multi-modal image fusion method based on modal self-adaption and modal interaction compensation, and the method comprises the following steps: S1, obtaining a multi-modal image fusion data set, and obtaining a training data set through preprocessing; S2, analyzing the modal difference characteristics of infrared and visible light images, and evaluating the correlation characteristics of image pairs in different scenes; s3, capturing a cross-modal feature dependency relationship through a self-attention mechanism; s4, a differential feature extraction strategy is adopted, model parameters are optimized through iterative training, and multi-modal image fusion is completed; s5, a modal interaction compensation module is additionally arranged, unit dynamic balance common features and modal exclusive features are fused, feature complementation is achieved in channel and space dimensions, parameters of the modal interaction compensation module are optimized, the model is made to learn the optimal fusion weight of the multi-modal features in a self-adaptive mode, and multi-modal fusion image generation optimization is achieved through the model; according to the invention, multi-modal image fusion can be accurately and effectively carried out.