Method, device and medium for detecting biofouling areas on building surfaces using multispectral image fusion

Abstract

The application discloses a kind of multispectral image fusion building surface biological attachment area detection method, equipment and medium, it is related to image processing technical field, disclose multispectral image fusion building surface biological attachment area detection method, comprising: based on the building to be detected, obtain the visible light image that unmanned aerial vehicle's visible light camera collects, and multispectral camera based on at least two different frequency bands collected multispectral image sequence;Based on calibration and preprocessing module registration the visible light image and the multispectral image, obtain the pixel-level alignment the visible light image and the multispectral image sequence;According to deep learning feature recognition module, identify the pixel-level alignment the visible light image and the multispectral image sequence, obtain the pixel-level segmentation result of the biological attachment area of the building to be detected.This is based on unmanned aerial vehicle detection, combined with deep learning and image feature fusion mode, improve building surface biological attachment recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device and medium for detecting bio-attached areas on building surfaces using multispectral image fusion. Background Technology

[0002] Throughout their entire life cycle, buildings are inevitably exposed to the natural environment for extended periods. In humid, rainy, and densely vegetated areas, commonly used building materials such as concrete, masonry, and wood are particularly susceptible to biofouling. The hazards of biofouling are mainly manifested in three aspects: first, it damages the aesthetics of the building surface, causing pollution and discoloration; second, it accelerates material degradation and reduces durability; and third, the long-term secretion of organic acids and other chemicals may lead to a decline in structural performance, threatening the safety of the building's external envelope.

[0003] The detection of biofouling on building surfaces typically relies on manual inspections and personal experience. Even with current methods that incorporate spectral index detection using spectral imaging, or methods that combine visible light images with deep learning models, misjudgments can occur under conditions with significant interference, such as on real building surfaces, resulting in detection accuracy that fails to meet practical detection needs.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device and medium for detecting bio-attachment areas on building surfaces using multispectral image fusion, aiming to solve the technical problem of low accuracy in detecting bio-attachment on building surfaces in real building inspection scenarios.

[0006] To achieve the above objectives, this application proposes a method for detecting biodegradation areas on building surfaces using multispectral image fusion, the method comprising:

[0007] Based on the building to be detected, acquire visible light images collected by the visible light camera of the UAV, and multispectral image sequences collected by the multispectral camera based on at least two different bands;

[0008] The visible light image and the multispectral image are registered based on the calibration and preprocessing module to obtain a pixel-aligned sequence of the visible light image and the multispectral image;

[0009] Based on the deep learning feature recognition module, the visible light image and the multispectral image sequence after pixel-level alignment are identified to obtain the pixel-level segmentation result of the bio-attachment area of ​​the building to be detected.

[0010] In one embodiment, the step of registering the visible light image and the multispectral image based on the calibration and preprocessing module to obtain a pixel-aligned sequence of the visible light image and the multispectral image includes:

[0011] Obtain the intrinsic parameter matrices and distortion parameters of the visible light camera and the multispectral camera;

[0012] The visible light image and the multispectral image are corrected according to the intrinsic parameter matrix and the distortion parameters to obtain the corrected visible light image and the corrected multispectral image;

[0013] Determine a specific channel image in the corrected multispectral image as the target image, and calculate the affine transformation matrix from all other images to the target image;

[0014] Based on the affine transformation matrix, perform image transformation on all other images to register all other images pixel-level to the target image coordinate system.

[0015] In one embodiment, the step of calculating the affine transformation matrix from all other images to the target image includes:

[0016] In the multispectral image sequence, a certain channel spectral image is selected as a reference image, and the target registration region is determined in the reference image;

[0017] The first feature points of the target registration region and the second feature points of all other images are extracted based on the scale-invariant feature transform algorithm.

[0018] The random sampling consensus algorithm determines high-confidence point pairs between the first feature point and the second feature point that have a higher confidence level than a preset confidence level, and calculates the affine transformation matrix of all other images to the target image based on the high-confidence point pairs.

[0019] In one embodiment, the step of obtaining the intrinsic parameter matrix and distortion parameters of the visible light camera and the multispectral camera includes:

[0020] Acquire standard checkerboard images captured from multiple angles by the visible light camera and the multispectral camera;

[0021] The intrinsic parameter matrix and distortion parameters of the visible light camera, as well as the intrinsic parameter matrix and distortion coefficients of the multispectral camera, are obtained by analyzing multiple standard checkerboard images according to a preset calibration algorithm.

[0022] The step of correcting the visible light image and the multispectral image based on the intrinsic parameter matrix and the distortion parameters to obtain the corrected visible light image and the corrected multispectral image includes:

[0023] Based on the intrinsic parameter matrix and distortion parameters of the visible light camera, the visible light image is calibrated using a preset correction function to obtain a corrected visible light image;

[0024] Based on the intrinsic parameter matrix and distortion parameters of the multispectral camera, the multispectral image is calibrated using a preset correction function to obtain a corrected multispectral image.

[0025] In one embodiment, the step of identifying the pixel-aligned visible light image and the multispectral image sequence based on the deep learning feature recognition module to obtain the pixel-level segmentation result of the bio-attachment region of the building to be detected includes:

[0026] Based on the spectral feature extraction unit of the deep learning feature recognition module, the visible light global features of the pixel-level aligned visible light image and the multispectral global features of the multispectral image are extracted.

[0027] Extract the fused feature obtained by splicing the visible light global feature and the multispectral global feature, and input the fused feature into the deep learning feature recognition module;

[0028] Obtain the pixel-level segmentation results of the biological attachment area of ​​the building to be detected, output by the deep learning recognition module.

[0029] In one embodiment, the step of extracting the fused feature obtained by splicing the visible light global feature and the multispectral global feature, and inputting the fused feature into the deep learning feature recognition module includes:

[0030] Extract a single-band raster feature map calculated based on multi-band pixels using a predetermined operator, wherein the single-band raster feature map is extracted based on the multispectral image sequence;

[0031] The visible light global features are subjected to convolution and ReLU activation processing to generate weighted channels;

[0032] Convolution and Sigmoid activation are applied to the multispectral global features and the single-band raster feature map to obtain feature vectors;

[0033] The result of the channel-by-channel element-wise multiplication of the weight channels and the feature vectors is calculated to obtain the attention weight. New tag The determined weights;

[0034] The attention focus New tag The feature obtained by multiplying the determined weights and the visible light global features is then added to the visible light global features to obtain the enhanced visible light global features;

[0035] The enhanced visible light global features and the multispectral global features are spliced ​​together to obtain the fused features;

[0036] The fused features are input into the deep learning feature recognition module.

[0037] In one embodiment, the spectral feature extraction unit includes a spectral channel attention module, and the step of identifying the pixel-level aligned visible light image and the multispectral image sequence based on the deep learning feature recognition module to obtain the pixel-level segmentation result of the bio-attachment region of the building to be detected includes:

[0038] Pooling is performed on the visible light image sequence of the visible light channel of the spectral channel attention module to obtain the visible light global features;

[0039] The multispectral image sequence of the multispectral channels of the spectral channel attention module is subjected to pooling processing to obtain the multispectral global features.

[0040] In one embodiment, prior to the steps of acquiring visible light images from a visible light camera of a drone and multispectral images from a multispectral camera based on at least two different frequency bands, the method for detecting bio-attachment areas on building surfaces through multispectral image fusion further includes:

[0041] The multispectral payload for performing image acquisition actions is determined based on a prior library of bioattachment spectra.

[0042] Furthermore, to achieve the above objectives, this application also proposes a multispectral image fusion-based biodegradation detection device for building surfaces. The multispectral image fusion-based biodegradation detection device for building surfaces includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the multispectral image fusion-based biodegradation area detection method for building surfaces as described above.

[0043] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the multispectral image fusion method for detecting biological attachment areas on building surfaces as described above.

[0044] One or more technical solutions proposed in this application have at least the following technical effects:

[0045] By acquiring visible light images from a UAV's visible light camera and multispectral image sequences acquired from a multispectral camera based on at least two different frequency bands, pixel-level aligned visible light and multispectral image sequences are obtained through calibration and preprocessing modules. Then, a deep learning feature recognition module is used to identify the aligned images to obtain pixel-level segmentation results of bio-attached areas. Thus, using airborne multispectral and visible light imaging equipment on a UAV platform, non-contact, wide-area detection of large building facades and infrastructure can be achieved while maintaining high resolution. Simultaneously, preprocessing and feature recognition of the detected UAV image data effectively overcomes the adverse effects of shadows, uneven lighting, and building surface humidity interference on detection in multispectral backgrounds. It also compensates for the insufficient information representation of single image types in complex interference environments. Pixel-level alignment ensures the accuracy of multi-source image feature fusion, and the combination of deep learning feature recognition capabilities enables more accurate and reliable identification of bio-attached areas on building surfaces, improving the automation level, accuracy, and comprehensiveness of bio-attached detection on building facades and infrastructure surfaces. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the functional modules of the detection equipment for the multispectral image fusion method for detecting biodegradation areas on building surfaces, as described in this application.

[0049] Figure 2 This is a flowchart illustrating the first embodiment of the method for detecting biodegradation areas on building surfaces using multispectral image fusion according to this application.

[0050] Figure 3 This is a flowchart illustrating the second embodiment of the method for detecting biodegradation areas on building surfaces using multispectral image fusion according to this application.

[0051] Figure 4 A simplified flowchart illustrating the image calibration and data preprocessing process provided in the second embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the multispectral image fusion method for detecting bio-attachment areas on building surfaces in this application embodiment.

[0053] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0055] The main solution of this application embodiment is: based on the building to be detected, obtain visible light images captured by the visible light camera of the UAV, and multispectral image sequences captured by the multispectral camera based on at least two different frequency bands;

[0056] The visible light image and the multispectral image are registered based on the calibration and preprocessing module to obtain a pixel-aligned sequence of the visible light image and the multispectral image;

[0057] Based on the deep learning feature recognition module, the visible light image and the multispectral image sequence after pixel-level alignment are identified to obtain the pixel-level segmentation result of the bio-attachment area of ​​the building to be detected.

[0058] Currently, the detection of biofouling on building surfaces mainly relies on manual inspections and experience-based judgment. This method is time-consuming, labor-intensive, and highly dependent on the subjective experience of the inspectors, making it difficult to achieve effective identification over a large scale, with high precision, and at early stages. In recent years, the academic community has attempted to introduce spectral index detection methods based on multispectral imaging or to employ deep learning methods based on visible light images. However, the former usually only relies on vegetation indices (such as the Normalized Difference Vegetation Index, NDVI) for rough judgments and lacks the ability to distinguish between complex backgrounds and non-green organisms; while the latter has made progress in pattern recognition, it often suffers from misjudgments and missed detections under shadows, uneven lighting, or surface humidity interference. More importantly, most of these studies are based on static, localized sampled images and lack large-scale coverage capabilities.

[0059] Based on this, this application provides a solution that utilizes an airborne multispectral and visible light imaging device mounted on a drone platform to achieve non-contact, wide-area detection of large-area building facades and infrastructure while maintaining high resolution. Simultaneously, it preprocesses and performs feature recognition on the detected drone image data, effectively overcoming the adverse effects of shadows, uneven lighting, and building surface humidity interference on detection in multispectral backgrounds. It also compensates for the insufficient information representation of a single image type in complex and interfering environments. Pixel-level alignment ensures the accuracy of multi-source image feature fusion, and combined with deep learning feature recognition capabilities, it achieves more accurate and reliable identification of bio-attached areas on building surfaces, improving the automation level, identification accuracy, and application comprehensiveness of bio-attached detection on building facades and infrastructure surfaces.

[0060] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a multispectral image fusion device for detecting bio-attachments on building surfaces. The multispectral image fusion device for detecting bio-attachments on building surfaces can be a drone, or a network device accessed by the drone, such as a cloud platform or computer. The following description uses a multispectral image fusion device for detecting bio-attachments on building surfaces as an example to illustrate this embodiment and the subsequent embodiments.

[0061] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0062] Taking the detection system connected to drones as an example, please refer to... Figure 1 The detection system comprises a data acquisition module, an image calibration and data preprocessing module, and an intelligent recognition module. The data acquisition module utilizes a visible light camera and a multispectral camera mounted on a drone gimbal to simultaneously acquire visible light images (RGB images) and multispectral images. The multispectral images include at least sensitive bands such as the near-infrared (NIR) band, the red (R) band, and the RE band. Therefore, during drone flight, the imaging equipment can simultaneously capture images of building surfaces to obtain multimodal image data. The image calibration and preprocessing module performs spatial registration and data preprocessing on the images acquired by the drone. This module includes a geometric correction unit, a feature matching and transformation unit, and a spatial resampling unit. These units perform visible light camera geometric correction, ROI-guided feature matching, spatial resampling, and feedback training using registered RGB and multispectral image pairs. This achieves camera distortion correction, cross-modal image alignment, and the generation of a highly consistent multimodal image dataset.

[0063] The intelligent recognition module is used for the detection and recognition of bioattachment based on preprocessed multimodal images. This module includes a spectral feature extraction unit and a deep learning recognition unit. The spectral feature extraction unit extracts spectral information from the multispectral images, while the deep learning recognition unit combines the spatial texture features of visible light with the spectral features of the multispectral images to output pixel-level segmentation results of bioattachment areas on building surfaces. Specifically, the spectral feature extraction unit processes RGB visible light images and multispectral images using a spectral channel attention mechanism to obtain spectral information, and then uses a deep learning semantic segmentation network such as Unet to perform bioattachment recognition, resulting in a pixel-level segmentation image of the bioattachment.

[0064] Through the coordinated operation of the three modules, high-precision, wide-area detection of biological attachments on building surfaces can be achieved.

[0065] Based on this, embodiments of this application provide a method for detecting bio-attachment areas on building surfaces using multispectral image fusion, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the method for detecting bio-attachment areas on building surfaces using multispectral image fusion, as described in this application.

[0066] In this embodiment, the method for detecting bio-attachment areas on building surfaces using multispectral image fusion includes steps S10 to S30:

[0067] Step S10: Based on the building to be detected, acquire visible light images collected by the visible light camera of the UAV, and multispectral image sequences collected by the multispectral camera based on at least two different frequency bands.

[0068] In this embodiment, the frequency bands acquired by the multispectral camera include at least sensitive bands such as the NIP band, R band, and RE band. Therefore, the multispectral image sequence is a collection of multiple frames of images continuously acquired by the multispectral camera on the same building area under at least two preset different frequency bands, containing spectral feature data of the building surface under each frequency band. It should be noted that the multispectral camera includes at least spectral channels sensitive to biofouling, such as the red band and near-infrared band, which can be used to characterize chlorophyll-rich biofouling organisms such as mosses and algae. Using the above spectral information, indices such as the Normalized Difference Vegetation Index (NDVI) and the Green Normalized Difference Vegetation Index (GNDVI) can be calculated, thereby enhancing the difference in spectral response between biofouled areas and abiotic areas.

[0069] As an optional implementation, the detection system first plans the drone's flight path using the building's 3D model and GPS positioning data, ensuring the path fully covers all surfaces of the building to be inspected, including walls and roofs. Then, the system controls the drone equipped with a visible light camera to acquire images at a preset frame rate. Simultaneously, a multispectral camera is triggered, acquiring images synchronously at the same frame rate across both near-infrared and red bands. Thus, during the inspection process, the visible light camera acquires high-resolution RGB images, while the multispectral camera acquires multispectral images covering the bio-attachment sensitive bands, providing source data for subsequent registration. After acquiring the image data, the drone transmits the acquired visible light and multispectral image sequences to the detection system in real time via its communication module, avoiding storage capacity limitations and ensuring the continuous execution of subsequent steps.

[0070] As an alternative implementation, the detection system can first control a drone to scan the building to be detected and generate 3D point cloud data based on the scan results. Then, it can plan an adaptive flight path using the point cloud data. Next, it controls the drone to acquire images using visible light cameras and multispectral cameras, and finally feeds the acquired images back to the detection system.

[0071] For example, after setting the drone's flight path, the drone's visible light camera acquires visible light images at a resolution of 2560×1440 at a frame rate of 20fps, while the multispectral camera selects two frequency bands, 900nm (near-infrared) and 1200nm (short-wave infrared), and acquires images corresponding to these two frequency bands according to the set acquisition frequency. After acquiring the images, the drone wirelessly transmits the acquired image data to the ground processing terminal.

[0072] Understandably, during image acquisition, image data from various surface types can be collected to enhance the model's applicability. These surface types include, but are not limited to, concrete walls, wooden structures, masonry components, stone surfaces, and ancillary structures such as municipal infrastructure pipelines. Comprehensive acquisition across multiple scenarios covers common bio-attachment types found in buildings and municipal facilities, thus providing more stable and sufficient data support for subsequent intelligent recognition.

[0073] Furthermore, the differences in spectral reflectance / absorption characteristics between biological attachments such as moss, mold, and lichen and building substrates such as concrete, glass, metal, and masonry are the core basis for multispectral detection. However, the spectral characteristics of different types of biological attachments vary significantly. For example, moss has a significantly higher reflectance in the near-infrared 850nm band than concrete, and mold has a clearly distinguishable absorption peak in the short-wave infrared 1200nm band compared to glass. Moreover, shadows, uneven lighting, and surface humidity can cause spectral feature distortion or overlap. Blindly selecting the detection frequency band for the multispectral payload may result in the multispectral image failing to effectively distinguish the target from the background because the frequency band does not cover the key difference range between biological attachments and the substrate, or the frequency band is sensitive to environmental interference, thus reducing the accuracy of subsequent feature recognition and segmentation. Therefore, before acquiring images using a drone, it is necessary to determine the multispectral payload for image acquisition based on a priori library of biological attachment spectra. Among them, the bio-attachment spectral prior library stores standard spectral data of different types of bio-attachment and different building substrates in the full spectrum range, as well as the spectral variation patterns under different environmental disturbances such as shadow, humidity, and light changes. It can provide a scientific basis for the frequency band selection and parameter setting of multispectral loads, ensuring that the acquired multispectral images can accurately capture the characteristic information of bio-attachment, and solve the problem of low detection accuracy in multispectral background from the source.

[0074] Step S20: Register the visible light image and the multispectral image based on the calibration and preprocessing module to obtain a pixel-aligned sequence of the visible light image and the multispectral image.

[0075] Because multispectral cameras typically employ a multi-lens configuration, with each band imaged by an independent lens, spatial misalignment between bands can easily occur. Furthermore, visible light cameras and multispectral cameras differ in field of view, resolution, and distortion levels, and the optical centers of different cameras are difficult to keep strictly consistent. The combined effect of these factors can lead to significant deviations during image fusion, impacting the accuracy of subsequent feature extraction and deep learning recognition.

[0076] Therefore, this embodiment uses a calibration and preprocessing module to perform spatial registration and data preprocessing on the images acquired by the UAV, thereby solving the spatial inconsistency problem caused by the imaging process between the multispectral camera and the visible light camera.

[0077] Specifically, as an optional implementation, the calibration process includes camera geometric calibration, ROI-guided feature matching and affine transformation estimation, and spatial resampling and image alignment. First, the camera is geometrically calibrated to correct nonlinear distortion introduced by the lens. Then, the region of interest is manually selected using a region-constrained feature matching algorithm. Based on the feature extraction algorithm, stable matching point pairs are selected using the Randomized Multivariate Sampling (RANSAC) algorithm to estimate the affine transformation matrix. Finally, the source image is resampled onto a grid consistent with the reference image using inverse mapping and bilinear interpolation, ensuring all images are aligned to a standard network of uniform resolution, providing standardized input for subsequent multimodal fusion and deep learning recognition.

[0078] As another optional implementation, the calibration and preprocessing module of the detection system first performs radiometric calibration on the received visible light images and multispectral image sequences, correcting the image grayscale values ​​through a preset camera response function to eliminate radiometric distortion caused by changes in illumination intensity. Then, a quadratic polynomial distortion correction algorithm is used to correct the geometric deformation of the image caused by the drone's flight attitude tilt. Next, SIFT feature points (scale-invariant feature transform) are extracted from each frame of the visible light and multispectral image sequences. The FLANN matching algorithm is used to find homologous feature point pairs, followed by the RANSAC algorithm to remove mismatched feature point pairs. Finally, the homography matrix is ​​solved using the least squares method to achieve pixel-level alignment.

[0079] Step S30: Based on the deep learning feature recognition module, the visible light image and the multispectral image sequence after pixel-level alignment are identified to obtain the pixel-level segmentation result of the biological attachment area of ​​the building to be detected.

[0080] In this embodiment, the pixel-level segmentation result is a binarized image in which each pixel in the image is marked as either biologically attached or non-biologically attached, which can accurately locate the spatial range and boundary of the biologically attached region.

[0081] The deep learning feature recognition module, as a multimodal image segmentation network incorporating an attention mechanism, is used to achieve automatic detection of biological attachments on building surfaces.

[0082] Specifically, the deep learning feature recognition module includes a spectral feature extraction unit and a deep learning feature recognition module. The spectral feature extraction unit extracts the visible light global features of the visible light image and the multispectral global features of the multispectral image, respectively. Then, the two features are spliced ​​and fused. Finally, the fused features are input into the deep learning feature recognition module, and finally, the pixel-level segmentation result of the biological attachment area of ​​the building to be detected is obtained from the output of the deep learning recognition module.

[0083] This embodiment provides a method for detecting bio-attachment areas on building surfaces using multispectral image fusion. It utilizes a drone to collect visible light and multi-band spectral information from building surfaces, ensuring high image resolution while enabling non-contact, wide-area detection of large building facades and infrastructure. This overcomes the limitations of single-image-type information representation. Pixel-level registration eliminates image distortion and spatial bias, laying the foundation for feature fusion. Finally, deep learning feature recognition and segmentation, integrating the advantages of multi-source features, effectively resists interference from shadows, uneven lighting, and building surface humidity. This achieves accurate identification and localization of bio-attachment areas on building surfaces, accurately reflecting the actual distribution and boundary conditions of bio-attachment, and providing a reliable technical basis for building maintenance.

[0084] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S20 includes steps S21 to S24:

[0085] Step S21: Obtain the intrinsic parameter matrix and distortion parameters of the visible light camera and the multispectral camera;

[0086] When performing geometric calibration on a camera, it is necessary to obtain the intrinsic parameter matrix and distortion parameters. In this embodiment, based on the checkerboard calibration method, the camera's intrinsic parameter matrix and distortion parameters are obtained by capturing standard checkerboard images from multiple angles and using the Zhang Zhengyou calibration method.

[0087] Specifically, first, standard checkerboard images captured by a visible light camera and a multispectral camera from multiple angles are acquired. Then, multiple standard checkerboard images are analyzed according to a preset calibration algorithm to obtain the intrinsic parameter matrix and distortion parameters of the visible light images and the multispectral camera.

[0088] For example, in an indoor non-reflective calibration environment, a standard checkerboard calibration board with a size of 12×9 grids, each grid having a side length of 30mm and a high contrast between black and white is prepared. The checkerboard material uses a diffuse reflection coating to ensure stable reflectivity under visible light and multispectral frequency bands, without spectral absorption interference. It is then fixed on a vertical calibration frame, and a drone carrying the visible light camera and multispectral camera to be calibrated is controlled to take pictures around the calibration board from 15 different angles: including 3 vertical shots from the front, 2 shots each at a horizontal 45° tilt, 2 shots each at a pitch angle of ±15°, and 4 shots at a combined pitch and tilt angle. Each image ensures that the checkerboard completely occupies more than 60% of the camera's field of view, and that the corners are clearly identifiable. Finally, 15 sets of simultaneously acquired visible light standard checkerboard images and multispectral standard checkerboard images are obtained. Next, Zhang Zhengyou's calibration algorithm was used as the preset calibration algorithm to analyze all the acquired standard checkerboard images. First, the corner coordinates of the checkerboard in each image were extracted using a corner detection algorithm. Then, a sub-pixel optimization algorithm was used to improve the corner positioning accuracy to the 0.1 pixel level. Finally, based on a perspective projection model, the correspondence between the corner pixel coordinates and world coordinates was established. The intrinsic parameter matrix and distortion parameters of the visible light camera, as well as the intrinsic parameter matrix and distortion parameters of the multispectral camera for multiple frequency bands, were solved using the least squares method. This provides accurate camera calibration parameters to support the subsequent pixel-level registration of visible light and multispectral images, effectively eliminating the influence of camera lens distortion on registration accuracy.

[0089] Step S22: Correct the visible light image and the multispectral image according to the intrinsic parameter matrix and the distortion parameters to obtain the corrected visible light image and the corrected multispectral image.

[0090] When correcting visible light and multispectral images, respective correction parameters are required. Specifically, the visible light image needs to be calibrated using a preset correction function based on the intrinsic parameter matrix and distortion parameters of the visible light camera to obtain the corrected visible light image. Similarly, the multispectral image needs to be calibrated using a preset correction function based on the intrinsic parameter matrix and distortion parameters of the multispectral camera to obtain the corrected multispectral image. The correction functions provided by the OpenCV library in the Python programming language can be used to correct the image, resulting in a high-resolution image with distortion removal. The specific correction process is not limited in this application.

[0091] Step S23: Determine a certain channel image in the corrected multispectral image as the target image, and calculate the affine transformation matrix from all other images to the target image.

[0092] In this embodiment, the affine transformation matrix can be calculated using the region-constrained SIFT-RANSAC feature matching method. During the calculation, taking the R or RE bands in the multispectral image as a reference image, the region of interest (ROI) needs to be manually selected, and the feature point extraction range is limited to reduce the impact of background interference on the matching. Subsequently, feature points are extracted using the SIFT algorithm, and stable matching point pairs are selected using the RANSAC method to estimate the affine transformation matrix. This matrix can describe the scaling, rotation, shearing, and translation relationships between images, realizing spatial mapping between multi-band and visible light images.

[0093] Therefore, when determining the affine transformation matrix, it is necessary to determine a certain channel image in the corrected image as the target image. The target image can be determined according to the user's input instructions, or the detection system can randomly select a channel image as the target image, which can be the image that other images need to be aligned with.

[0094] After obtaining the target image, a specific channel of the multispectral image sequence needs to be selected as the reference image, and the target registration region needs to be determined within the reference image. This target registration region can be selected in response to an operation command or randomly. Next, the first feature point of the target registration region and the second feature points of all other images are extracted using a scale-invariant feature transform algorithm. Subsequently, a random sampling consensus algorithm is used to determine high-confidence point pairs between the first and second feature points that have a higher than a preset confidence level. Based on these high-confidence point pairs, the affine transformation matrix from all other images to the target image is calculated.

[0095] For example, the core of calculating the transformation matrix lies in finding the spatial correspondence between a multispectral image and a visible light image by limiting the range, extracting key features, and filtering effective matches. This process can be broken down as follows: First, a reference image is determined by selecting a band from the multispectral image as the reference, and all subsequent images must be aligned to the spatial position of this image. Next, a Region of Interest (ROI) is manually defined by selecting one or more ROIs in the chosen reference image, and feature points are extracted only within this region. Then, the SIFT algorithm is used to extract feature points separately. Subsequently, based on the description of the feature points, pairs of points that appear similar in the two images are initially identified, while the RANSAC algorithm is used to filter out erroneous point pairs, retaining only those that truly correspond to the same physical location. Finally, the affine transformation matrix is ​​calculated using the filtered reliable point pairs. With this matrix, each pixel of the visible light image (or other multispectral bands) can be mapped to the corresponding position in the reference image through mathematical calculations, achieving spatial alignment.

[0096] Step S24: Perform image transformation on all other images based on the affine transformation matrix to register all other images pixel-level to the target image coordinate system.

[0097] In this embodiment, after obtaining the affine transformation matrix, image transformation processing can be performed directly through matrix transformation, so that all other image pixels are registered to the target image coordinate system, i.e., the position that needs to be calibrated, thereby achieving pixel-level alignment of the image.

[0098] Specifically, after obtaining the affine transformation matrix, spatial registration is performed on each non-reference image. This can be achieved by resampling the source images onto a uniform grid consistent with the reference images using inverse mapping and bilinear interpolation. Bilinear interpolation not only effectively avoids aliasing but also maintains spatial smoothness. Finally, all images are aligned to a uniform resolution, such as 2592×1944 pixels, providing standardized input for subsequent multimodal fusion and deep learning recognition.

[0099] For example, to help understand the implementation process of the multispectral image fusion method for detecting bio-attachment areas on building surfaces obtained by combining this embodiment with the first embodiment described above, please refer to... Figure 4 , Figure 4 A simplified flowchart of image calibration and data preprocessing is provided. Specifically, after the images are acquired, camera geometry calibration is performed on the visible light image and the multispectral image. Then, feature point extraction and feature point matching are performed with ROI constraints. Next, affine transformation matrix estimation is performed based on the feature points. Then, other images are spatially resampled using the affine transformation matrix. Finally, pixel-level aligned RGB visible light image and multispectral image are obtained.

[0100] This embodiment provides a method for detecting bio-attachment regions on building surfaces through multispectral image fusion. The calibration and preprocessing modules provide a precise geometric reference for image alignment through camera geometric calibration, ensuring the consistency of multi-source images in the physical space dimension. ROI-guided feature matching can focus on the effective detection area of ​​the building surface, avoiding interference from background noise and non-target areas on feature extraction. At the same time, combined with affine transformation estimation, it accurately models the spatial mapping relationship between different images, significantly improving the robustness of feature matching and the estimation accuracy of the transformation matrix, and reducing mismatch problems caused by environmental factors such as shadows and uneven lighting. Through spatial resampling technology, non-target images are mapped to the target image coordinate system according to the optimal affine transformation matrix, achieving pixel-level precise alignment. This ensures the effective correlation and complementary fusion of the texture and color features of the visible light image and the spectral features of the multispectral image in the same spatial location, providing high-quality and highly consistent input data for the subsequent deep learning feature recognition module to fully mine the bio-attachment features in multimodal data, thereby improving the boundary accuracy of bio-attachment region segmentation and the reliability of region recognition.

[0101] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, step S30 may include steps S31 to S33:

[0102] Step S31: Based on the spectral feature extraction unit of the deep learning feature recognition module, extract the visible light global features of the pixel-level aligned visible light image and the multispectral global features of the multispectral image.

[0103] In this embodiment, the global features can be the semantic features of the image. The spectral feature extraction unit includes a spectral channel attention module.

[0104] When extracting global features, pooling can be performed on the visible light image of the visible light channel in the spectral channel attention module to obtain visible light global features. Similarly, pooling can be performed on the multispectral image of the multispectral channel in the spectral channel attention module to obtain multispectral global features. Specifically, global average pooling can be performed on the RGB channel and the multispectral channel separately to extract their respective global semantic vectors.

[0105] Step S32: Extract the fused feature obtained by splicing the visible light global feature and the multispectral global feature, and input the fused feature into the deep learning feature recognition module.

[0106] In this embodiment, before fusing RGB and multispectral images, a lightweight spectral channel attention module is introduced to perform contextual modeling of different bands, enabling the network to better understand and model the dependencies between different spectral bands, strengthen the coupling between RGB features and multispectral features, and thus improve feature representation capabilities.

[0107] Therefore, after obtaining the global features, single-band raster feature maps calculated based on multi-band pixels according to a predetermined operator can be extracted from the multispectral image sequence, such as by performing Normalized Difference Vegetation Index (NDVI) processing.

[0108] .

[0109] in, Represents the reflectivity in the near-infrared band. This index represents the reflectance in the red light band. By enhancing the strong absorption of chlorophyll and red light and the high reflectance to the near-infrared in biological tissues, this index can significantly improve the spectral contrast between biological attachment areas and the non-biological background, making it suitable as an auxiliary discriminative feature input to subsequent networks.

[0110] Next, convolution and ReLU activation are applied to the visible light global features to generate weighted channels. Then, convolution and Sigmoid activation are applied to the multispectral global features and single-band raster feature maps to obtain feature vectors. Finally, the element-wise multiplication of the weighted channels and feature vectors is calculated to obtain the attention weights. New tag After determining the weights, attention will then be focused on New tag The weighted features are multiplied by the visible light global features, and then added to the visible light global features to obtain enhanced visible light global features. Finally, the enhanced visible light global features and the multispectral global features are concatenated to obtain fused features, which are then input into the deep learning feature recognition module. The single-band raster feature map can be a vegetation index map.

[0111] For example, in this process, global average pooling is performed on the RGB and multispectral channels respectively to extract their respective global semantic vectors. The RGB global features are then activated by a 1×1 convolution followed by ReLU to generate weighted channels representing the importance of RGB. Simultaneously, for the multispectral image, single-band raster feature maps calculated based on multi-band pixels using predetermined operators are extracted. The multispectral global features and single-band raster feature maps are then activated by a 1×1 convolution followed by Sigmoid to generate modulation vectors, which characterize the modulating effect of multispectral light on RGB. These two results are then multiplied channel-by-channel to form attention weights for recalibration. The weighted RGB features are then residually concatenated with the original input visible light global features to ensure no loss of underlying feature information, resulting in enhanced RGB global features. Finally, the enhanced RGB global features are concatenated with the multispectral global features to obtain the input parameters for subsequent networks.

[0112] Step S33: Obtain the pixel-level segmentation result of the biological attachment area of ​​the building to be detected, output by the deep learning recognition module.

[0113] For the fused features after input enhancement, this embodiment can perform feature extraction and progressive restoration of spatial resolution through an improved encoder-decoder structure such as U-Net or its lightweight variant. Specifically, the encoder part can be based on other feature extraction networks such as MobileNetv3 for efficient extraction of multimodal features; the decoder part upsamples layer by layer and fuses shallow information to generate high-resolution segmentation results.

[0114] Therefore, after obtaining the fused features of the two images, a recognition module is used to accurately locate the spatial range and boundaries of the bio-attached area. For example, the Softmax activation function is used to classify the output feature map, with a probability threshold of 0.5. Pixels with a probability ≥ 0.5 are marked as "bio-attached areas," while others are not marked. At the same time, the classification result is transformed into a probability distribution using the Softmax function. Finally, small areas of mold in corners are segmented, and the segmentation result highly matches the actual mold area. Alternatively, pixels in the moss area can be marked, and the segmentation result clearly shows the distribution range and edge contour of the moss.

[0115] This embodiment provides a method for detecting biological attachment regions on building surfaces through multispectral image fusion. By using a lightweight spectral channel attention module to assign weights to different bands, the network can better understand and model the dependencies between different spectral bands, thereby explicitly characterizing the cross-band relationship between RGB and multispectral, while keeping the number of parameters extremely small and the computational cost low, which greatly improves the discrimination ability of RGB features in a multispectral background.

[0116] This application provides a multispectral image fusion-based biodegradation detection device for building surfaces. The multispectral image fusion-based biodegradation detection device for building surfaces includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the multispectral image fusion-based biodegradation area detection method for building surfaces described in the first embodiment above.

[0117] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a building surface biodegradation detection device suitable for implementing multispectral image fusion in the embodiments of this application. The multispectral image fusion building surface biodegradation detection device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The multispectral image fusion-based bioadhesion detection device for building surfaces shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0118] like Figure 5As shown, the multispectral image fusion building surface biodegradation detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the multispectral image fusion building surface biodegradation detection device. The processing unit 1001, the read-only memory 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the multispectral image fusion building surface bio-attachment detection device to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a multispectral image fusion building surface bio-attachment detection device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0119] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0120] The multispectral image fusion-based biodegradation detection device for building surfaces provided in this application, employing the multispectral image fusion-based biodegradation area detection method described in the above embodiments, can solve the technical problem of low accuracy in detecting biodegradation on building surfaces in real building inspection scenarios. Compared with the prior art, the beneficial effects of the multispectral image fusion-based biodegradation detection device for building surfaces provided in this application are the same as those of the multispectral image fusion-based biodegradation area detection method for building surfaces provided in the above embodiments, and other technical features of this multispectral image fusion-based biodegradation detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0121] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0123] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the multispectral image fusion method for detecting bio-attachment areas on building surfaces in the above embodiments.

[0124] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM, or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0125] The aforementioned computer-readable storage medium may be included in a multispectral image fusion building surface bio-attachment detection device; or it may exist independently and not be assembled into a multispectral image fusion building surface bio-attachment detection device.

[0126] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the multispectral image fusion building surface biodegradation detection device, cause the multispectral image fusion building surface biodegradation detection device to:

[0127] Based on the building to be detected, acquire visible light images collected by the visible light camera of the UAV, and multispectral image sequences collected by the multispectral camera based on at least two different frequency bands;

[0128] The visible light image and the multispectral image are registered based on the calibration and preprocessing module to obtain a pixel-aligned sequence of the visible light image and the multispectral image;

[0129] Based on the deep learning feature recognition module, the visible light image and the multispectral image sequence after pixel-level alignment are identified to obtain the pixel-level segmentation result of the bio-attachment area of ​​the building to be detected.

[0130] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Python, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0133] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for detecting bio-attachment areas on building surfaces using multispectral image fusion. This method can solve the technical problem of low accuracy in detecting bio-attachment on building surfaces during real-world building exterior wall inspections. Compared with existing technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the multispectral image fusion method for detecting bio-attachment areas on building surfaces provided in the above embodiments, and will not be elaborated upon here.

[0134] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting bio-attached areas on building surfaces using multispectral image fusion, characterized in that, The method for detecting bio-attached areas on building surfaces through multispectral image fusion includes: Based on the building to be detected, a visible light image is acquired by a visible light camera of a drone, and a multispectral image sequence is acquired by a multispectral camera based on at least two different frequency bands. The images of the visible light camera and the multispectral camera are acquired at the same time. The multispectral images are used to calculate vegetation spectral indices related to the characteristics of the corresponding detected object. The visible light image and the multispectral image are registered using a calibration and preprocessing module to obtain a pixel-aligned sequence of visible light images and multispectral images. This includes acquiring the intrinsic parameter matrices and distortion parameters of the visible light camera and the multispectral camera; correcting the visible light image and the multispectral image based on the intrinsic parameter matrices and the distortion parameters to obtain corrected visible light images and corrected multispectral images; determining a specific channel image in the corrected multispectral image as the target image; calculating the affine transformation matrix from all other images to the target image; and performing image transformation on all other images based on the affine transformation matrix to pixel-level register all other images to the target image coordinate system. Based on the deep learning feature recognition module, the pixel-aligned visible light image and the multispectral image sequence are identified to obtain the pixel-level segmentation result of the bio-attachment region of the building to be detected, including: Based on the spectral feature extraction unit of the deep learning feature recognition module, the visible light global features of the pixel-level aligned visible light image and the multispectral global features of the multispectral image are extracted. Extract a single-band raster feature map calculated based on multi-band pixels using a predetermined operator, wherein the single-band raster feature map is extracted based on the multispectral image sequence; The visible light global features are subjected to convolution and ReLU activation processing to generate weighted channels; Convolution and Sigmoid activation are applied to the multispectral global features and the single-band raster feature map to obtain feature vectors; The weights after attention recalibration are obtained by calculating the result of the element-wise multiplication of the weight channels and the feature vectors. The enhanced visible light global feature is obtained by multiplying the weights after the attention recalibration with the visible light global feature and then adding the enhanced visible light global feature to the visible light global feature. By splicing the enhanced visible light global features and the multispectral global features, a fused feature is obtained; The fused features are input into the deep learning unit of the deep learning feature recognition module; The pixel-level segmentation result of the biological attachment region of the building to be detected is obtained from the output of the deep learning unit. The pixel-level segmentation result is a binary image in which each pixel in the image is marked as biologically attached or non-biologically attached, thereby accurately locating the spatial range and boundary of the biological attachment region.

2. The method for detecting bio-attachment areas on building surfaces using multispectral image fusion as described in claim 1, characterized in that, The steps for calculating the affine transformation matrix from all other images to the target image include: In the multispectral image sequence, a certain channel spectral image is selected as a reference image, and the target registration region is determined in the reference image; The first feature points of the target registration region and the second feature points of all other images are extracted based on the scale-invariant feature transform algorithm. The random sampling consensus algorithm determines high-confidence point pairs between the first feature point and the second feature point that have a higher confidence level than a preset confidence level, and calculates the affine transformation matrix of all other images to the target image based on the high-confidence point pairs.

3. The method for detecting bio-attachment areas on building surfaces using multispectral image fusion as described in claim 1, characterized in that, The steps for obtaining the intrinsic parameter matrices and distortion parameters of the visible light camera and the multispectral camera include: Acquire standard checkerboard images captured from multiple angles by the visible light camera and the multispectral camera; The intrinsic parameter matrix and distortion parameters of the visible light camera, as well as the intrinsic parameter matrix and distortion coefficients of the multispectral camera, are obtained by analyzing multiple standard checkerboard images according to a preset calibration algorithm. The step of correcting the visible light image and the multispectral image based on the intrinsic parameter matrix and the distortion parameters to obtain the corrected visible light image and the corrected multispectral image includes: Based on the intrinsic parameter matrix and distortion parameters of the visible light camera, the visible light image is calibrated using a preset correction function to obtain a corrected visible light image; Based on the intrinsic parameter matrix and distortion parameters of the multispectral camera, the multispectral image is calibrated using a preset correction function to obtain a corrected multispectral image.

4. The method for detecting bio-attachment areas on building surfaces using multispectral image fusion as described in claim 1, characterized in that, The spectral feature extraction unit includes a spectral channel attention module. The step of identifying the pixel-level aligned visible light image and the multispectral image sequence based on the deep learning feature recognition module to obtain the pixel-level segmentation result of the bio-attachment region of the building to be detected includes: Pooling is performed on the visible light image sequence of the visible light channel of the spectral channel attention module to obtain the visible light global features; The multispectral image sequence of the multispectral channels of the spectral channel attention module is subjected to pooling processing to obtain the multispectral global features.

5. The method for detecting bio-attachment areas on building surfaces using multispectral image fusion as described in claim 1, characterized in that, Before the steps of acquiring visible light images from a visible light camera of a drone and multispectral images from a multispectral camera based on at least two different frequency bands, the method for detecting bio-attached areas on building surfaces through multispectral image fusion further includes: The multispectral payload for performing image acquisition actions is determined based on a prior library of bioattachment spectra.

6. A multispectral image fusion device for detecting bio-attachment on building surfaces, characterized in that, The multispectral image fusion building surface bio-attachment detection device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multispectral image fusion building surface bio-attachment area detection method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the method for detecting bio-attached areas on building surfaces using multispectral image fusion as described in any one of claims 1 to 5.

Images