94 results about "Multispectral image fusion" patented technology

The invention relates to a high-voltage equipment defect detection method based on deep learning and multispectral image fusion, and relates to the technical field of electric power high-voltage equipment state detection. The method comprises the following steps: acquiring an ultraviolet image, an infrared image and a visible light image of the surface of the high-voltage equipment; carrying out image pixel feature-based fusion processing on the ultraviolet image, the infrared image and the visible light image through an image fusion method; establishing a high-voltage equipment defect detection model, and training the high-voltage equipment defect detection model by using the fused image data to obtain a high-voltage equipment defect identification model based on the YOLO-STrans multispectral fusion network; and inputting the ultraviolet image, the infrared image and the visible light image of the outer surface of the power high-voltage equipment into a high-voltage equipment defect identification model to obtain a fault identification result of the to-be-detected power high-voltage equipment. The method can improve the recognition precision of the extremely early insulation degradation and temperature anomaly defects of the surface of the high-voltage power equipment.

The invention discloses a multispectral image fusion model based on a double-branch self-attention-generative adversarial network and a fusion method thereof, and the model sequentially comprises an input preprocessing module which is used for carrying out the same-amplitude mapping, normalization and overlapping block embedding of visible light and infrared original images, and generating a to-be-fused feature block; the double-branch encoder module captures a cross-modal long-distance dependence and overall brightness structure through multi-head deep convolution transpose attention (MDTA) and a gated deep convolution feedforward network (GDFN), and extracts high-frequency texture and edge information by using a reversible residual gating layer and detail DetailNode iteration; the fusion decoder module is used for carrying out multi-level self-attention-convolution reconstruction on the two paths of features after channel dimension splicing, and outputting a single-frame high-resolution fusion image; and the double-domain discriminator module comprises a visible light domain discriminator and an infrared domain discriminator which are respectively used for carrying out adversarial evaluation on the fused image and the corresponding modal truth value image so as to improve the detail authenticity and the thermal target contrast ratio of the fusion result. The technical problems that an existing infrared-visible light image fusion method is insufficient in detail reservation, unbalanced in brightness and contrast, poor in unsupervised training stability and the like are solved, the method can be deployed on embedded platforms needing real-time and multi-modal information enhancement such as night monitoring, unmanned driving and edge security and protection, and high-contrast and high-information-amount fusion imaging is achieved.

The invention discloses a high-resolution hyperspectral image reconstruction method based on wavelet domain feature fusion and contrast learning, and belongs to the technical field of image fusion and super-resolution reconstruction. The method comprises the following steps: constructing a fusion network model comprising a wavelet transformation module, a cross-modal feature fusion module, a high-frequency contrast learning module and an image reconstruction module; performing end-to-end supervised training by using a training data set containing the low-resolution hyperspectral image and the high-resolution multispectral image; and after training is completed, inputting a test image pair to realize image reconstruction. According to the method, the detail retention capability is improved by combining wavelet decomposition and a directional fusion mechanism, the cross-modal high-frequency feature alignment capability is enhanced through comparative learning, a fusion image with high spatial resolution and high spectral consistency is finally generated, and the method is suitable for multi-modal image reconstruction tasks such as remote sensing, medical and natural images.

The invention discloses a hyperspectral and multispectral image fusion algorithm. The algorithm comprises the following steps: step 1, constructing a fusion model based on regular term constraint; step 2, constructing a multi-view clustering model of hypergraph manifold and low-rank tensor constraint; 3, performing spatial and spectral manifold learning based on a multi-view clustering model; 4, introducing a regular hyperspectral and multispectral image fusion model by utilizing the space and spectral manifold constraint obtained by learning; and 5, solving a fusion problem by adopting an ADMM algorithm. On the basis of inspiration of a low-dimensional manifold structure in a hyperspectral image, by considering space and spectrum height correlation and utilizing manifold constraints and high-dimensional characteristics to cluster an internal low-dimensional space of the hyperspectral image, a low-dimensional structure of the hyperspectral image in space and spectrum dimensions is described, and multi-view constraints are introduced to ensure the accuracy of an incidence matrix. A solution space of a to-be-restored hyperspectral image is constrained, self-expression constraints of a space and a spectral low-dimensional space are introduced, and a proposed algorithm is optimized by using an alternating direction multiplier method.

The invention discloses a metal processing detection system and method based on computer vision, and relates to the technical field of defect detection.Rotatable polarization filters are installed in front of a lens of a vision sensor, the polarization filters are arranged at four angles respectively, and four sets of polarization images are collected; calculating the polarization degree of each pixel point on the surface during metal processing by using the gray values of the four groups of polarization images, setting a polarization region interval, and dividing a metal surface reflection region into a specular reflection region and a diffuse reflection region; acquiring images of different wavebands by using a multispectral vision sensor, and constructing a multispectral image set; for the same wave band, the images of different exposure times are synthesized to obtain an HDR high dynamic range image; setting a weight for the HDR image of each wave band, and combining the weighting of the HDR image and the suppression of the polarization degree to carry out multispectral image fusion so as to obtain a final image; and a gray threshold value and an area threshold value are respectively calculated through the average value and the standard deviation, and the metal processing defects are judged by using the two threshold values.

The invention relates to the technical field of intelligent maintenance of tunnel electromechanical facilities, in particular to a tunnel lamp recognition and spatial positioning method based on binocular vision, which comprises the following steps: synchronously acquiring tunnel environment images through left and right cameras subjected to polar horizontal correction, inputting the images into a lightweight tunnel lamp detection model, and detecting an area image containing a lamp bounding box; extracting feature points in the bounding box by adopting a multi-scale adaptive FAST feature extraction algorithm, describing the feature points by rotating an invariant binary descriptor, and generating a disparity map in combination with an adaptive Hamming distance matching algorithm; calculating three-dimensional space coordinates of the lamp according to the disparity map by using a binocular camera triangulation principle and combining camera parameters; when the ambient illumination is lower than a threshold value, a multispectral image fusion module is started to enhance the feature matching robustness; and when continuous matching fails, visible light communication assisted positioning is started. According to the invention, the problem of low positioning precision of the lamp in a complex tunnel environment is solved, and the reliability and efficiency of identification and positioning are improved.

The invention discloses a multi-spectral image fusion building surface biological attachment area detection method and device and a medium, relates to the technical field of image processing, and discloses a multi-spectral image fusion building surface biological attachment area detection method comprising the following steps: based on a to-be-detected building, obtaining a visible light image collected by a visible light camera of an unmanned aerial vehicle, the multispectral camera acquires a multispectral image sequence based on at least two different frequency bands; registering the visible light image and the multispectral image based on a calibration and preprocessing module to obtain a pixel-level aligned visible light image and multispectral image sequence; and according to a deep learning feature identification module, identifying the visible light image and the multispectral image sequence after pixel-level alignment, and obtaining a pixel-level segmentation result of the organism attachment area of the to-be-detected building. Therefore, detection is carried out based on the unmanned aerial vehicle, deep learning and an image feature fusion mode are combined, and the building surface bioattachment recognition accuracy is improved.

The invention discloses a hyperspectral and multispectral image fusion method and system based on Reform residual iteration. The method comprises the following steps: acquiring an image data set of a hyperspectral remote sensing image and a multispectral remote sensing image in the same region and preprocessing the image data set; performing spatial down-sampling on the preprocessed data set to generate a low-resolution hyperspectral image and a low-resolution multispectral image; an image priori network model is constructed, the image priori network model comprises a space priori module and a spectrum priori module, the space priori module captures a long-range spatial dependency relationship through a Reformer layer, and the spectrum priori module extracts spectrum features through a depth separable convolutional layer; establishing a multi-level iteration model, setting a residual connection reconstruction function, and training the model by adopting an optimization algorithm and a space-spectrum joint loss function to obtain a trained model; and finally, generating a hyperspectral image with high spatial resolution. According to the method, the parameter quantity required by fusion can be effectively reduced.

The invention discloses a hyperspectral and multispectral image fusion method based on heterogeneous double-branch collaboration, and the method fully utilizes the local feature extraction capability of convolution operation and the long-distance dependency relationship of the feature extraction operation of a visual converter through a feature extraction normal form of the collaboration of the convolution operation and the feature extraction operation of the visual converter. In the feature extraction process, a cross-modal cross attention module is used for carrying out interactive fusion on local spatial features and global semantic features, and correlation among different modal features is captured through a channel local attention enhancement mechanism, so that the feature expression ability is enhanced. According to the method, in the fusion process of the low-resolution hyperspectral image and the high-resolution multispectral image, the calculation cost is reduced, the local feature extraction capability is improved, and the problems of spatial detail loss, insufficient cross-modal information interaction capability and the like are avoided.

The invention relates to the technical field of multispectral image fusion, and provides a multispectral image fusion method and system based on spatial spectral difference prior guidance, and the method comprises the steps: carrying out the up-sampling of an LRMS image, carrying out the subtraction and splicing of the LRMS image and a panchromatic image in a waveband-by-waveband manner, and obtaining a spatial difference input feature map; extracting a multi-scale spatial feature map through spatial difference prior guide branches; subtracting the up-sampling LRMS image from the degraded panchromatic image band by band, and splicing with the up-sampling LRMS image to obtain a spectral difference input feature map; extracting a multi-scale spectral feature map through a spectral difference prior guide branch; and fusing the multi-scale spatial feature map and the multi-scale spectral feature map through the spatial spectral feature fusion branch to obtain an HRMS image. According to the method, the spectral and spatial difference information of the panchromatic image and the LRMS image at the pixel level is used as priori, the extraction and fusion of complementary information are guided, and the accurate reconstruction of the high-resolution multispectral image is realized.

The invention relates to a multispectral-based printed circuit board detection method, which comprises the following steps of: firstly, acquiring and marking images and structural attributes of a printed circuit board under a multispectral channel, constructing a structured data set, and sequentially performing spatial correction, resolution unification and spectral normalization to form a consistent standard image; according to the method, space coordinate embedding and consistency modeling are utilized to realize accurate alignment of each channel image, and space and spectrum fusion parameters are optimized based on a joint adversarial learning network to obtain a self-adaptive enhanced image. On the basis, the micro-feature residual error is subjected to regional enhancement through a structural sensitive attention mechanism, and the detectability of the micro-defect is improved. And finally, the intelligent identification network realizes automatic screening and precise positioning of the micro-defects, and guarantees the detection precision and model adaptability based on a quality evaluation model and a self-correction feedback mechanism. According to the method, the PCB multispectral image fusion quality and the micro-defect detection capability can be remarkably improved.

The invention relates to the technical field of image processing, and discloses a method and system for generating a hyperspectral image based on hyperspectral and multispectral image fusion. The method comprises the following steps: respectively acquiring a hyperspectral image and a multispectral image with different resolutions and atmospheric state parameters at an imaging moment, and carrying out geometric registration on the images to obtain a registered image; analyzing according to the atmospheric state parameters, and constructing a differentiable atmospheric radiation transmission model; constructing an illumination invariance fusion network, defining a joint loss function of the illumination invariance fusion network, and optimizing the illumination invariance fusion network based on the joint loss function to obtain an optimized illumination invariance fusion network; in combination with a differentiable atmospheric radiation transmission model, taking the obtained registered image and atmospheric state parameters as input of an optimized illumination invariance fusion network, and obtaining an illumination-invariant high-resolution hyperspectral image; the method has stable ground object identification and analysis capability in different illumination scenes such as cloud shadow, time sequence and the like.

The invention relates to the technical field of water body pollution monitoring, in particular to a multispectral image fused water body pollutant accurate recognition system, which comprises a data acquisition module, a cloud processing module, a boundary processing module, a pollution recognition module, a diffusion prediction module and a visualization module, the system constructs sub-pixel representation of a water pollutant boundary by using a differential geometry manifold theory, and realizes high-precision pollutant boundary description through a multi-scale analysis and curvature flow optimization technology; enhancing pollutant characteristic expression by adopting multispectral image fusion and an optimal wave band selection technology; using a support vector machine model to accurately identify various pollutant types such as oil films, oil spots, algae blooms and the like; based on a boundary fine description result and a pollutant type identification result, the diffusion trend of pollutants is accurately predicted in combination with historical flow, wind direction and wind speed data, and the system improves the water pollutant boundary identification precision and enhances the identification capability of complex boundary forms and low-contrast regions.

The embodiment of the invention provides a hyperspectral and multispectral image fusion reconstruction method guided by spatial-spectral diffusion priori, and the method achieves the extraction and generation of spatial-spectral priori through a two-stage design, and comprises the steps: extracting spatial and spectral priori information from a real high-resolution hyperspectral image through a spatial-spectral priori encoder in a first stage; and the spatial features of the multispectral image and the spectral features of the low-resolution hyperspectral image are fused through a bidirectional multiscale prior fusion module to complete multiscale image reconstruction. And in the second stage, through a lightweight diffusion priori generation module, the spatial-spectral priori information is generated based on the up-sampled low-resolution hyperspectral image, and the real priori information is estimated, so that the problem of priori information loss caused by lack of a real image in the reasoning stage of the existing method is solved. Through two-stage joint training and a prior injection mechanism, not only are spatial texture and spectrum details reserved, but also video memory occupation and calculation cost of a diffusion model are reduced, and efficient high-resolution hyperspectral image reconstruction is realized.

The invention discloses an image processing system and method for salt mist particle size detection, and relates to the technical field of salt mist tests, the system captures salt mist particle images and acquires multi-dimensional information through a multi-spectral imaging technology by using light sources with different wavelengths and high-resolution imaging equipment, and establishes an image information set in combination with environment temperature and humidity data. Through image preprocessing and multispectral image fusion, salt mist particle features are extracted, and a comprehensive data set is constructed. And calculating a salt mist particle size distribution coefficient and a particle concentration coefficient according to the comprehensive data set to ensure that experimental conditions meet preset requirements and avoid experimental errors. Further analyzing the motion change and the form change of the particles, and calculating a dynamic change coefficient to ensure the stable behavior of the particles. And finally, training and screening the data by using a convolutional neural network, automatically identifying the salt mist particles conforming to the standard, and optimizing the accuracy and efficiency of the salt mist experiment.

The invention relates to the technical field of multispectral image fusion, and discloses a real-time multispectral image fusion system based on deep learning. The system comprises a multispectral image spectrum module, a process spectrum synthesis module, a personalized process extraction module, a process vertex labeling module and a fusion process verification module. The multispectral image map module stores various image category templates of a map structure, and each template corresponds to a fusion flow chart containing a processing vertex and a connecting edge; the process map synthesis module fuses a plurality of templates to generate a unified hypergraph through vertex clustering and edge redirection; the personalized process extraction module analyzes a user demand text and converts the user demand text into a demand graph, and a unified hypergraph is matched with a sub-graph to serve as a basic fusion process; the flow vertex labeling module identifies adjustable vertexes and adds marks; and the fusion process verification module verifies logic through a deep learning model, detects abnormal vertexes and generates a report. The system adapts to multiple scenes, and the flexibility and reliability of the fusion process are improved.

The present invention discloses a hyperspectral and multispectral image fusion method based on adaptive multi-scale features. First, a hyperspectral image with low spatial resolution and high spectral resolution and a multispectral image with high spatial resolution and low spectral resolution are used as the input of the network, and efficient feature expansion is achieved through upsampling by a channel interaction upsampling module. Then, they are fused into a super-multispectral image with the same size as the reference image. Next, diverse features are captured and spatial information is reconstructed via a multi-scale adaptive spatial reconstruction module. The optimized feature map enters a spectral self-attention enhancement module for spectral attention enhancement, calculates the correlation between feature maps to generate an attention map, and reconstructs spectral information. This avoids problems such as loss of spatial details, insufficient cross-domain generalization ability, and spectral distortion caused by low spectral resolution during the image fusion process.

The invention provides a multispectral image fusion method and system for security monitoring, and the method comprises the steps: carrying out the Retinex decomposition based on an obtained visible light image and an infrared image, and obtaining a visible light reflection component, a visible light illumination component, an infrared reflection component and an infrared illumination component; initial division of semantic regions is carried out based on the visible light reflection component and the infrared reflection component, and a joint semantic graph containing a plurality of semantic regions is obtained; based on the joint semantic graph, respectively carrying out regional adaptive smoothing constraint on the visible light illuminance component and the infrared illuminance component to obtain a visible light target smoothing component and an infrared target smoothing component; and performing illumination field alignment fusion on the visible light target smooth component and the infrared target smooth component based on the joint semantic map to obtain a fused illumination component. According to the method, the problems of edge blurring, artifact interference and key target structure distortion caused by uniform processing and ignoring of physical characteristic differences of semantic regions in an existing method are accurately solved.

The application provides a multispectral image fusion method based on an interpretable neural network, first, multispectral images are obtained through a multispectral sensor, a panchromatic image of the same area is obtained through the multispectral sensor, and then a data fidelity term is constructed; a residual relationship between a fusion image and the panchromatic image is established through a deep network, and used as a prior constraint term; a fusion model is constructed through the data fidelity term and the deep learning prior term; based on the fusion model, an optimization model is iteratively solved through a proximal gradient descent method; the iterative solving step is mapped into a deep network architecture, a required multispectral panchromatic image deep network model is generated through training of a training data set; and based on the generated multispectral panchromatic image deep network model, multispectral and panchromatic images of a given input are fused to generate a final fusion image. The application generates optimal parameter settings through data driving, thereby improving fusion performance.

This application provides a method and system for smoke and fire recognition based on multispectral image fusion, relating to the field of data processing. The method acquires visible light imaging sequences, infrared thermal imaging sequences, and gimbal pose sequences of the target monitoring area and constructs scene commitment identifiers; it generates joint observation units based on unified spatiotemporal calculations and extracts multimodal increments to form cross-modal increment ledgers; it filters candidate anomalies based on structural destructiveness, thermal behavior persistence, spatial expansion, and cross-modal synergy; it performs cross-modal playback comparisons using a counterfactual surrogate fragment library and generates causal preemption chains; it combines asynchronous evidence ledgers to determine self-sustaining signs and establishes cross-regional inheritance relationship graphs; and it constructs a risk takeover ledger based on modal dominance switching trajectories and outputs graded smoke and fire alarm results. Implementing this technical solution facilitates the effective differentiation between smoke and fire anomalies and non-fire disturbances under multispectral observation conditions, thereby improving the accuracy of smoke and fire recognition based on multispectral image fusion.

The invention provides a hyperspectral and multispectral image fusion method and system. The method comprises the following steps: acquiring hyperspectral and multispectral data of the same spatial region; performing up-sampling on the low-resolution hyperspectral image by adopting bilinear interpolation; one-dimensional wavelet transform is utilized to extract reflection peak and spectrum change characteristics in a spectrum dimension, two-dimensional wavelet transform is utilized to extract texture and edge information in a space dimension, and pixel-by-pixel wavelet basis adaptive selection is realized through Softmax gating; multi-granularity feature modeling is realized for the spectral branches by adopting grouped multi-scale convolution, and a self-adaptive weighting mechanism is formed for the spatial branches through global average pooling, a convolution layer and Sigmoid activation so as to enhance spatial details and compensate spatial feature loss by utilizing MSI; a cross-scale two-way attention mechanism is introduced, and long-range dependence modeling and information interaction of spectrum and spatial characteristics are achieved; and a high-resolution hyperspectral image is generated through a feature aggregation and reconstruction module.

The application relates to the technical field of image processing, and discloses a method and system for generating a hyperspectral image based on hyperspectral and multispectral image fusion; the method comprises the following steps: collecting a hyperspectral image and a multispectral image with different resolutions and atmospheric state parameters at an imaging moment, performing geometric registration on the images to obtain registered images; analyzing the atmospheric state parameters to construct a differentiable atmospheric radiation transmission model; constructing an illumination invariance fusion network, defining a joint loss function of the illumination invariance fusion network, optimizing the illumination invariance fusion network based on the joint loss function, and obtaining an optimized illumination invariance fusion network; combining the differentiable atmospheric radiation transmission model, taking the registered images and the atmospheric state parameters as inputs of the optimized illumination invariance fusion network, and obtaining an illumination-invariant high-resolution hyperspectral image; and the application has stable ground object recognition and analysis capability under different illumination scenes such as cloud shadows and time sequences.

The application relates to the technical field of image processing, and discloses a multispectral fusion imaging camera module and an image processing method, which comprises the following steps: performing spatial mapping on infrared imaging images and near-infrared imaging images based on the spatial arrangement order of corner points and the direction of continuous edge broken lines; determining a spectral corresponding structure according to the light and dark trend changes of the same scene in three imaging channels; performing regional division on a geometric alignment result based on the spectral corresponding structure; performing direction consistency correction on geometric edges in a visible light imaging image based on a dominant contour line; performing extensibility superposition on texture lines in the spatially mapped near-infrared imaging image based on the texture direction of a texture detail area; performing superposition on the brightness levels of the visible light imaging image based on the brightness extension direction of a background smooth area; and performing connectivity checking on the boundary direction, the texture direction and the brightness extension direction. The application ensures the reliability of multispectral image fusion.

The present invention discloses a method for establishing a fusion network for multispectral images based on a self-attention mechanism, comprising: establishing a training set comprising multiple groups of image pairs, each group of image pairs comprising an infrared image and a visible light image; inputting the image pairs in the training set into a generator employing a self-attention mechanism to generate a preliminary fused image, and calculating the generator's loss function value; inputting the image pairs, combined with the preliminary fused image, into two discriminators for discrimination processing, and calculating the combined loss function value of the two discriminators; and adjusting the weights of the generator weights to minimize the generator's loss function value corresponding to the image pairs in the training set and the combined loss function value of the discriminator. The present invention also provides a corresponding fusion method. Implementing the present invention can improve the fusion effect of visible light images and infrared images, and can retain more high-frequency and low-frequency details.

The invention discloses a building curtain wall automatic detection method and system, and relates to the technical field of building curtain wall detection based on visual inspection, and the method comprises the steps: obtaining a high-resolution image covering visible light to a near-infrared band, and classifying chemical components through a support vector machine algorithm; a chemical composition classification map containing distribution regions of corrosion products and fouling types is generated. And three-dimensional point cloud data is further obtained through a laser range finder, fused with the multispectral image and input into a convolutional neural network to predict the thickness of the deposit, and a thickness distribution diagram is generated. And generating an aging degree grading distribution diagram through the grading classification model, and determining a preferential maintenance area and a processing mode in combination with aging correlation analysis. According to the method, the types of the deposits on the surface of the curtain wall are accurately identified through an automatic technical means, the thickness and the distribution range of the deposits are accurately measured, accurate evaluation and intelligent maintenance decision making of the aging state of the surface of the curtain wall are achieved, and the maintenance efficiency and the curtain wall performance are remarkably improved.

The present invention discloses a visible light and infrared image fusion method based on feature extraction. First, the infrared image is preprocessed by a hybrid filtering algorithm. Secondly, the gray histogram of the image is extended to fill the entire gray range, highlighting the infrared target and weakening the influence of the background. Then, Gaussian image pyramids are processed for different spectral images respectively, and the significant regions of the images at different scales are obtained through Laplacian pyramids. Next, the infrared region of interest is calculated, and through the dominant information comparison strategy, the significant regions of different spectra are compared to obtain the infrared image regions of interest at each scale. Finally, the infrared regions of interest at different scales are fused into the visible light image to obtain the fusion results of different spectra. The multi-spectral image fusion method of the present invention retains complete detail information and has strong robustness, solving the problem of limited spectral detection range of a single visible light sensor.

The present application relates to the technical field of multispectral image fusion, and discloses a real-time multispectral image fusion system based on deep learning. The system comprises a multispectral image atlas module, a flow atlas synthesis module, a personalized flow extraction module, a flow vertex labeling module and a fusion flow verification module. The multispectral image atlas module stores a plurality of image category templates of graph structures, each template corresponding to a fusion flow graph containing processing vertices and connecting edges; the flow atlas synthesis module fuses a plurality of templates into a unified hypergraph through vertex clustering and edge redirection; the personalized flow extraction module analyzes user demand text and converts it into a demand atlas, and matches a subgraph from the unified hypergraph as a basic fusion flow; the flow vertex labeling module identifies adjustable vertices and adds labels; and the fusion flow verification module verifies logic through a deep learning model, detects abnormal vertices and generates a report. The system is suitable for multiple scenarios and improves the flexibility and reliability of the fusion flow.

The present application relates to the technical field of multispectral image fusion, and particularly relates to a multi-modal biological signal fusion method and system, multi-modal biological signal acquisition: at least two different types of biological signals are collected; time synchronization processing: a unified time axis T_ref=[0, 10ms, 20ms, …, T_end] with a resolution of 10ms is constructed, and alignment of each modal signal with the unified time axis is realized through a three-level synchronization strategy; cross-device standardization processing: the biological signals after time synchronization are subjected to standardization processing, and characteristic differences among different devices are eliminated; signal quality calculation: the signal quality Q(t) of each modal signal at time t is calculated in real time; dynamic weight updating: based on the real-time quality Q(t) of each modal signal, the fusion weight of each modal is dynamically updated; the present application has the advantages that: through a three-level mixed time synchronization algorithm, a unified time axis is constructed, and precise alignment of multi-modal signals with different sampling rates and different timing characteristics is realized.

The invention discloses contact appearance automatic detection equipment and a detection method thereof, relates to the technical field of contact appearance detection, and aims to solve the problem that tiny defects are difficult to effectively recognize in the prior art. Comprising a multispectral imaging module and a deep learning processing module. The multispectral imaging module comprises a multi-light source assembly and a multi-camera synchronous acquisition assembly and is used for acquiring a spectral image of the contact; and the deep learning processing module comprises a multispectral image fusion sub-module and a defect identification sub-module, and is used for analyzing and detecting the quality of the coating on the surface of the contact according to the spectral image and transmitting a detection result to the interaction module. The method has the advantages of improving the fine defect identification accuracy and reducing the false detection rate.

The application discloses a multispectral image fusion method and device based on multi-target segmentation, and equipment, and relates to the technical field of image processing. Different fusion methods are adopted for different targets in the feature domain according to the category of multi-target segmentation, so that the image conforming to the human eye visual perception is generated, and the significant target in the infrared image is effectively highlighted. The method comprises the following steps: collecting a visible light image and an infrared image, performing image registration processing on the visible light image and the infrared image to obtain a registered target visible light image and a target infrared image; performing multi-target semantic segmentation on the target visible light image and the target infrared image by using a multi-target segmentation network to generate a multi-target segmentation image; extracting a first deep feature corresponding to the target visible light image and a second deep feature corresponding to the target infrared image based on a multispectral image fusion network; and generating a target fusion image by fusing the first deep feature, the second deep feature and the multi-target segmentation image.