A microwave reflection detection method for external wall hollowing based on multi-modal fusion

Abstract

The application belongs to the technical field of microwave measurement, and discloses a kind of microwave reflection detection method for outer wall hollowing based on multi-modal fusion, comprising multi-modal scanning to the building outer wall to be measured, obtaining visible light image data and infrared temperature distribution data of outer wall surface, and carrying out spatial registration and coordinate mapping to form unified multi-modal fusion dataset;Abnormal screening is carried out on the multi-modal fusion dataset, and the thermal anomaly area is identified by analyzing the infrared temperature distribution data;Combining the texture and topographic anomaly features of visible light image data, detect the bulge or crack area;Information fusion is carried out on the thermal anomaly area and the bulge or crack area, and the candidate detection area of suspected hollowing is extracted;High-precision identification and quantitative evaluation of outer wall hollowing are realized, and the precision and stability of outer wall hollowing detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of microwave measurement, more particularly, the present application relates to a kind of external wall hollow microwave reflection detection method based on multimodal fusion. BACKGROUND

[0002] The existing external wall hollow microwave reflection detection method has the following defects:

[0003] External wall hollow is one of the common structural hidden troubles of building facade, which is usually caused by material aging, construction defects or environmental stress. If it cannot be identified and treated in time, it may cause safety accidents such as peeling and falling. Therefore, the non-contact detection technology of external wall hollow has become an important research direction in the field of building safety detection.

[0004] At present, external wall hollow detection mainly uses visible light detection, infrared thermal imaging or single-frequency microwave reflection detection method. Visible light detection relies on surface deformation or color anomaly characteristics, and it is difficult to find shallow hollow that has not been exposed. Although infrared thermal imaging can reflect the difference in heat diffusion, it is greatly affected by environmental temperature, wind speed, sunlight conditions and other factors, and is easy to misjudge. Microwave detection has strong penetration ability and dielectric sensitivity, and becomes the main development direction of hollow detection.

[0005] The existing microwave detection method mainly uses single-frequency or narrow-band frequency signal, only records the echo amplitude information of a single frequency point, and ignores the continuous response characteristics of phase and amplitude caused by frequency change. This single frequency point detection method cannot accurately distinguish the small dielectric constant difference between wall material and air layer, so it is difficult to identify shallow or small hollow.

[0006] Traditional microwave reflection detection often relies on experience judgment or statistical model to evaluate whether there is hollow, and lacks inversion model based on physical parameters. Since the phase information and frequency spectrum distribution characteristics of echo signal are not fully utilized, the existing technology cannot invert the depth, thickness and dielectric constant change of hollow through continuous frequency response. Especially in the case of multiple medium interfaces in the wall, the weak reflection signal between adjacent layers is easily ignored, resulting in missing thickness distribution information and poor detection result accuracy and repeatability.

[0007] In view of this, the present application provides a kind of external wall hollow microwave reflection detection method based on multimodal fusion to solve the above problems. SUMMARY

[0008] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0009] A kind of external wall hollow microwave reflection detection method based on multimodal fusion, comprising:

[0010] S1, multi-modal scanning is performed on the building outer wall to be tested to obtain visible light image data and infrared temperature distribution data of the outer wall surface, and spatial registration and coordinate mapping are performed to form a unified multi-modal fusion data set;

[0011] S2, the multi-modal fusion data set is subjected to abnormal screening, the thermal abnormal area is identified by analyzing the infrared temperature distribution data, the texture and topography abnormal features of the visible light image data are combined to detect the bulge or crack area, and the thermal abnormal area and the bulge or crack area are information fused to extract the candidate detection area of suspected hollowing;

[0012] S3, through the adjustable frequency multi-frequency point scanning mechanism, the candidate detection area is subjected to cluster beam scanning, and the multi-frequency reflection signals at different frequencies are recorded; the phase change and group time delay difference of the multi-frequency reflection signals are calculated, the propagation path and equivalent dielectric constant of the microwave in the wall are calculated, and the depth and thickness distribution of the hollowing are obtained;

[0013] S4, the depth and thickness distribution of the hollowing are subjected to multi-dimensional feature clustering and geometric boundary fitting to form a continuous hollowing area and hollowing area characteristic parameter;

[0014] S5, according to the hollowing area characteristic parameter, the outer wall hollowing is quantitatively evaluated, the hollowing area is divided into different disease grades; and visual output is performed to generate a detection report containing position, depth, area and grade.

[0015] Specifically, the method for obtaining the visible light image data and the infrared temperature distribution data of the outer wall surface comprises:

[0016] A UAV platform carrying a visible light imaging sensor and an infrared thermal imager is used to synchronously scan the building outer wall at different angles and different flight lines. In the scanning process, the visible light imaging sensor collects the visible light image data of the outer wall surface, and the infrared thermal imager synchronously records the thermal radiation intensity of the corresponding area and automatically converts it into infrared temperature distribution data;

[0017] The visible light image data includes outer wall surface texture data, outer wall surface geometric shape data and outer wall color brightness data; the infrared temperature distribution data includes outer wall surface temperature data, corresponding area thermal radiation intensity data and temperature gradient distribution data.

[0018] Specifically, the method for forming a unified multi-modal fusion data set comprises:

[0019] The obtained visible light image data and infrared temperature distribution data are time-synchronized, the wall corner points and texture features in the visible light image data are extracted as visible light image feature points based on Harris corner point detection, and the pre-calibrated feature points in the infrared temperature distribution data are extracted as infrared temperature distribution feature points;

[0020] The spatial correspondence between the feature points of the visible light image and the feature points of the infrared temperature distribution is established by using a feature matching algorithm, and the infrared temperature distribution data is mapped into the spatial coordinate system of the visible light image data according to the internal parameters of the visible light imaging sensor and the infrared thermal imager and the position information of the unmanned aerial vehicle;

[0021] The mapped infrared temperature distribution data and the visible light image data are aligned at the pixel level by using a bilinear or bicubic interpolation method; the aligned visible light image data and infrared temperature distribution data are fused in the same spatial coordinate system to form a unified multi-modal fusion data set.

[0022] Specifically, the method for identifying the thermal anomaly region comprises:

[0023] The infrared temperature distribution data in the multi-modal fusion data set is subjected to noise suppression by using a Gaussian filter, and the filtered infrared temperature distribution data is subjected to normalization processing, so as to map the infrared temperature distribution data into a preset standard range;

[0024] Temperature anomaly points and hot spot features are extracted from the normalized infrared temperature distribution data by using a local extremum detection method, as abnormal temperature feature points; the abnormal temperature feature points are merged into continuous thermal anomaly regions by using a spatial clustering algorithm.

[0025] Specifically, the method for detecting the bulge or crack region comprises:

[0026] The texture features of the visible light image data are extracted by using an LBP algorithm, a neighborhood of pixels is selected with each pixel in the visible light image as the center, the gray values of the neighborhood pixels are compared with the gray value of the center pixel, and the neighborhood pixels are marked as 1 if the gray value is greater than or equal to the gray value of the center pixel, otherwise marked as 0;

[0027] The comparison results are converted into a binary string in order and then into a decimal number to generate the LBP value of each pixel; the visible light image is divided into fixed-size grid regions, the LBP value distribution of each region is counted to form a texture feature vector, and the texture feature vector of each region is subjected to Euclidean distance calculation with a normal wall texture feature vector established in advance, and when the distance is greater than a preset distance threshold, the region is marked as a texture anomaly region;

[0028] The geometric topographic features of the wall, including the wall contour, the crack line, and the local protrusion or depression boundary, are extracted by using an edge detection algorithm; the local protrusion height, the depression depth, and the crack width are calculated according to the edge detection results;

[0029] Regions with local bulge height greater than a preset local bulge height threshold, depression depth greater than a preset depression depth threshold, or crack width greater than a preset crack width threshold are marked as abnormal morphology regions; texture abnormality regions are combined with abnormal morphology regions as bulge or crack regions.

[0030] Specifically, the method for extracting candidate detection regions for suspected hollow areas includes:

[0031] Place the bulge or crack area and the thermal anomaly area in a unified spatial coordinate system. For areas where the boundaries do not completely overlap, use an interpolation method to adjust and match the thermal anomaly area and the bulge or crack area.

[0032] The matched thermal anomaly regions and bulge or crack regions are fused by logical AND operation. Only regions that simultaneously possess thermal anomaly characteristics and bulge or crack characteristics are marked as candidate detection regions for suspected hollow areas.

[0033] Specifically, the method for recording multi-frequency reflection signals at different frequencies includes:

[0034] Within each candidate detection area, a frequency-tunable directional transmitter is used to perform multi-frequency scanning within a preset frequency band. The transmission frequency is controlled to scan point by point at a fixed step interval between the preset minimum and maximum frequencies to form a multi-frequency scanning sequence. Echo signals are collected at each frequency point, and the amplitude, phase, and propagation delay information of the echo signals are recorded respectively. The phases of the echo signals of all scanning frequencies are arranged sequentially to form a phase sequence.

[0035] Due to the difference in dielectric constant between the wall material and the internal air gap, microwaves are reflected at the interface. The amplitude and phase characteristics of the reflected signal are related to the equivalent dielectric constant of the material. The material difference characteristics are characterized by calculating the reflection coefficient of the wall interface. The multi-frequency scanning sequence and phase sequence obtained by multi-frequency point scanning together constitute the multi-frequency reflection signal set.

[0036] Specifically, the method for obtaining the depth and thickness distribution of the void includes:

[0037] Based on the phase sequence and corresponding frequency information recorded by the multi-frequency echo signal, a phase-frequency relationship function is established. By utilizing the propagation characteristics of microwaves at the interface between wall materials and air gaps, the change of echo phase with frequency reflects the propagation path of microwaves within the wall.

[0038] According to the phase frequency relationship function, the group delay formed by the rate of change of phase with frequency is calculated, and the depth of the hollow is obtained by using the relationship between the group delay and the dielectric constant; the equivalent dielectric constant of the wall material is inversely deduced by combining the amplitude of the echo signal and the wall interface reflection coefficient, so as to obtain the dielectric distribution of the hollow layer; the thickness distribution of the hollow is estimated by combining the depth of the hollow and the amplitude attenuation characteristics and the change of the wall interface reflection intensity of the multi-frequency signal.

[0039] Specifically, the method for forming the continuous hollow area and the hollow area feature parameter comprises:

[0040] The depth and thickness distribution of the hollow are normalized and weighted to construct a fusion feature set; a density clustering algorithm with spatial constraint conditions is performed based on the fusion feature set, and candidate detection areas of suspected hollows that are adjacent in spatial position and have continuous depth and thickness changes are aggregated to form a continuous hollow candidate area;

[0041] All hollow candidate areas are mapped to a spatial coordinate system to generate a region mask map, and after isolated noise points and small holes are removed by morphological closing operation, the hollow area boundary is extracted, and curve fitting is used to smooth the hollow area boundary, and then the continuous hollow boundary contour is obtained; the continuous hollow area is delineated based on the continuous hollow boundary contour, the feature parameter of the hollow area is calculated, and then the hollow area feature parameter is obtained.

[0042] Specifically, the method for generating a detection report containing position, depth, area and level comprises:

[0043] According to the hollow area feature parameter, the hollow of the outer wall is quantitatively evaluated, the hollow area feature parameter is linearly normalized and weighted fused to obtain a comprehensive score representing the severity of the hollow area; different comprehensive score threshold intervals are preset to divide the hollow area into different disease levels; the hollow areas of different disease levels are identified by different colors, visual output is performed, and a detection report containing position, depth, area and level is automatically generated.

[0044] The technical effects and advantages of the outer wall hollow microwave reflection detection method based on multi-modal fusion of the present application are:

[0045] The multi-frequency scanning is performed in the candidate detection area by the adjustable frequency directional transmitting device, amplitude, phase and propagation time delay information is obtained, a complete multi-frequency reflection signal set is formed, and a reliable physical data basis is provided for subsequent depth and dielectric characteristic inversion. The relationship between the echo amplitude and phase and the equivalent dielectric constant of the wall material is used, the reflection coefficient is calculated, the difference of the wall interface material is quantitatively described, and the physical quantitative hollow identification is realized, instead of relying on visual or experience judgment. The echo phases of each frequency point are sequentially arranged to form a phase sequence, which provides continuous signal data for the multi-frequency inversion algorithm, and the depth and thickness distribution of the hollow can be accurately calculated to realize non-destructive and quantitative detection. The multi-frequency, multi-point and continuous collection mode can effectively overcome the defects of single frequency or single mode signal, such as being easily affected by noise, material non-uniformity and surface environment, so as to improve the accuracy and repeatability of the detection results.

[0046] The phase frequency function is formed by using the relationship between the echo phase of each frequency point and the frequency, and the propagation characteristics of the microwave in the wall material and the air sandwich interface, which can quantize the microwave propagation path and realize accurate inversion of the hollow depth. The rate of change of the phase with the frequency is calculated as the group time delay, and the one-way depth of the hollow can be quantitatively calculated by combining the dielectric constant relationship, which is more accurate and reliable than the traditional experience judgment. The equivalent dielectric constant of the wall material is obtained by backstepping the wall interface reflection coefficient through the echo amplitude and the dielectric distribution of the hollow layer. The thickness of the hollow layer is accurately estimated by using the phase mutation point and the amplitude attenuation characteristics in combination with the depth information, and the thickness is represented as the difference between the upper and lower interface depths, so that the quantitative measurement of the thickness of the hollow layer is realized. The depth, thickness and dielectric characteristics of the hollow can be obtained simultaneously without damaging the wall structure, and the comprehensive, continuous and quantitative detection of the hollow layer is realized, and the detection results have higher accuracy, repeatability and physical basis. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a process schematic diagram of a multi-modal fusion-based external wall hollow microwave reflection detection method.

[0048] Figure 2 It is a structure schematic diagram of a multi-modal fusion-based external wall hollow microwave reflection detection system. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0050] Embodiment 1: Please refer to Figure 1As shown, the embodiment provides a microwave reflection detection method for external wall hollowing based on multi-modal fusion, which specifically includes the following steps:

[0051] S1, multi-modal scanning is performed on the external wall of the building to be measured to obtain visible light image data and infrared temperature distribution data of the external wall surface, and spatial registration and coordinate mapping are performed to form a unified multi-modal fusion data set;

[0052] S2, abnormal screening is performed on the multi-modal fusion data set, thermal abnormal areas are identified by analyzing the infrared temperature distribution data, and bulge or crack areas are detected by combining the texture and topography abnormal features of the visible light image data; information fusion is performed on the thermal abnormal areas and the bulge or crack areas to extract candidate detection areas of suspected hollowing;

[0053] S3, through an adjustable frequency multi-frequency point scanning mechanism, each candidate detection area is subjected to a cluster beam scanning, and multi-frequency reflection signals at different frequencies are recorded; the phase change and group time delay difference of the multi-frequency reflection signals are calculated to calculate the propagation path and equivalent dielectric constant of the microwave in the wall, and the depth and thickness distribution of the hollowing are obtained;

[0054] S4, multi-dimensional feature clustering and geometric boundary fitting are performed on the depth and thickness distribution of the hollowing to form continuous hollowing areas and hollowing area characteristic parameters;

[0055] S5, according to the hollowing area characteristic parameters, quantitative evaluation of the external wall hollowing is performed, the hollowing area is divided into different disease grades; and visual output is performed to generate a detection report containing position, depth, area and grade.

[0056] The method for obtaining the visible light image data and the infrared temperature distribution data of the external wall surface includes:

[0057] An unmanned aerial vehicle platform carrying a visible light imaging sensor and an infrared thermal imager is used to synchronously scan the building external wall at different angles and different flight lines. In the scanning process, the visible light imaging sensor collects visible light image data of the external wall surface, and the infrared thermal imager synchronously records the thermal radiation intensity of the corresponding area and automatically converts it into infrared temperature distribution data;

[0058] The visible light image data includes external wall surface texture data, external wall surface geometric shape data and external wall color brightness data; the infrared temperature distribution data includes external wall surface temperature data, corresponding area thermal radiation intensity data and temperature gradient distribution data.

[0059] The visible light imaging sensor is used to collect the texture, cracks and color change information of the outer wall surface, and to perform denoising, distortion correction and brightness equalization processing to improve the definition of the visible light image; the visible light image data is a high-resolution texture image of the outer wall surface; the infrared thermal imager is used to collect the temperature field distribution of the building outer wall in real time to obtain thermal distribution information.

[0060] The specific implementation is: using a UAV platform equipped with a visible light imaging sensor and an infrared thermal imager to partition and infrared scan the building outer wall; according to the area and shape of the building outer wall, a plurality of scanning areas are divided, and the size of each scanning area is reasonably set according to the field of view angle and measurement distance of the infrared thermal imager to ensure complete coverage of the scanning area.

[0061] During the infrared scanning process, the scanning frequency is set to

[10] times per minute to ensure that the temperature information of the building outer wall surface can be quickly obtained; the scanning angle range is adjusted according to the actual situation of the building outer wall, and for vertical walls, the scanning angle is kept within ± [5]° in the vertical direction to ensure that the temperature distribution of the wall surface can be comprehensively detected; for parts of the building outer wall with inclination or special structure, the scanning angle is appropriately adjusted to cover all areas that need to be detected.

[0062] During the scanning process of adjacent areas, the image overlap rate is set and maintained above

[20] %; the emissivity of the infrared thermal imager is adjusted according to the characteristics of the building outer wall material; different outer wall materials, such as concrete, ceramic tiles, and metal, have different emissivities, which can be obtained by consulting relevant materials or using an emissivity measuring device, and the corresponding settings are made in the infrared thermal imager to ensure that the measured infrared temperature distribution data can accurately reflect the true temperature of the building outer wall surface.

[0063] The method for forming a unified multi-modal fusion data set includes:

[0064] The acquired visible light image data and infrared temperature distribution data are time-synchronized, the wall corner points and texture features in the visible light image data are extracted as visible light image feature points based on Harris corner point detection, and the pre-calibrated feature points in the infrared temperature distribution data are extracted as infrared temperature distribution feature points; specifically, the known position feature points formed by the pre-arranged calibration board or thermal markers, and the stable geometric structure points in the infrared image, such as wall corners or window frame edges, are used as infrared temperature distribution feature points for spatial matching with the visible light image feature points.

[0065] The spatial correspondence between the feature points of the visible light image and the feature points of the infrared temperature distribution is established by using a feature matching algorithm (such as SIFT, SURF, or ORB), and the infrared temperature distribution data is mapped into the spatial coordinate system of the visible light image data according to the internal parameters of the visible light imaging sensor and the infrared thermal imager and the position information of the unmanned aerial vehicle.

[0066] The mapped infrared temperature distribution data and the visible light image data are aligned at the pixel level by using a bilinear or bicubic interpolation method, and the aligned visible light image data and infrared temperature distribution data are fused in the same spatial coordinate system to form a unified multi-modal fusion data set.

[0067] The method for identifying thermal anomaly regions includes:

[0068] The infrared temperature distribution data in the multi-modal fusion data set is subjected to noise suppression by using a Gaussian filter, and the filtered infrared temperature distribution data is subjected to normalization processing to map the infrared temperature distribution data into a pre-set standard range;

[0069] Temperature anomaly points and hot spot features are extracted from the normalized infrared temperature distribution data by using a local extremum detection method (such as a maximum / minimum value search based on a 3x3 or 5x5 pixel neighborhood) as abnormal temperature feature points, and the abnormal temperature feature points are merged into continuous thermal anomaly regions by using a spatial clustering algorithm (such as a density clustering, k-means clustering, or region growing algorithm).

[0070] The method for detecting bulge or crack regions includes:

[0071] The texture features of the visible light image data are extracted by using an LBP algorithm, the neighborhood pixels are selected with each pixel in the visible light image as the center, the gray values of the neighborhood pixels are compared with the gray value of the center pixel, and the neighborhood pixels with a gray value greater than or equal to the gray value of the center pixel are marked as 1, otherwise as 0;

[0072] The comparison results are converted into a binary string in order and then into a decimal number to generate the LBP value of each pixel, the visible light image is divided into fixed-size grid regions, the LBP value distribution of each region is counted to form a texture feature vector, and the texture feature vector of each region is subjected to Euclidean distance calculation with a pre-established normal wall texture feature vector, and when the distance is greater than a pre-set distance threshold, the region is marked as a texture anomaly region;

[0073] The wall geometry features, including the wall contour, crack lines, and local protrusion or depression boundaries, are extracted by using an edge detection algorithm, and the local protrusion height, depression depth, and crack width are calculated according to the edge detection results.

[0074] The region with a local protrusion height greater than a preset local protrusion height threshold, a recess depth greater than a preset recess depth threshold, or a crack width greater than a preset crack width threshold is marked as a topographic anomaly region; the texture anomaly region and the topographic anomaly region are combined as a blister or crack region.

[0075] The method for extracting the candidate detection region of suspected hollowing includes:

[0076] The blister or crack region and the thermal anomaly region are placed in a unified spatial coordinate system, and for regions with incomplete boundary overlap, an interpolation method is used to adjust and match the thermal anomaly region and the blister or crack region.

[0077] The matched thermal anomaly region and the blister or crack region are subjected to logical AND operation fusion, and only the region with both thermal anomaly characteristics and blister or crack characteristics is marked as a candidate detection region of suspected hollowing.

[0078] The method for recording multi-frequency reflection signals at different frequencies includes:

[0079] In each candidate detection region, a tunable frequency directional emission device (a tunable frequency microwave signal source such as a vector signal source, a frequency synthesizer, or a sweep signal source) is used to perform multi-frequency point scanning within a preset frequency band range, and the emission frequency is scanned point by point at a fixed step interval between a preset minimum frequency and a maximum frequency to form a multi-frequency scanning sequence;

[0080] Echo signals are collected at each frequency point, and the amplitude, phase, and propagation time delay information of the echo signals are recorded respectively; the phases of the echo signals at all scanning frequencies are arranged in sequence to form a phase sequence;

[0081] The phase of the echo signal is: ; wherein, φn represents the echo phase at the nth frequency point; fn represents the emission frequency at the nth frequency point; tn represents the round-trip propagation time delay of the microwave from emission to reception; n represents the index of the frequency point;

[0082] Due to the difference in the dielectric constant of the wall material and the internal air layer, the microwave is reflected at the interface, and the amplitude and phase characteristics of the reflected signal are related to the equivalent dielectric constant of the material. By calculating the reflection coefficient of the wall interface, the material difference characteristics are represented; the reflection coefficient is: ; wherein, Γ represents the reflection coefficient of the wall interface; ε represents the equivalent dielectric constant of the wall material;

[0083] ​​The multi-frequency scanning sequence obtained by the multi-frequency point scanning and the phase sequence jointly constitute a multi-frequency reflection signal set.

[0084] The existing technology has the following technical problems: the existing technology only uses visible light, infrared or single frequency microwave for detection, and cannot accurately distinguish the small dielectric difference between wall materials and air sandwich, resulting in difficulty in identifying shallow hollow. The existing method often records the echo information of a single frequency point, ignores the continuous change characteristics of frequency on phase and amplitude, and cannot invert the hollow depth and dielectric characteristics through physical quantization. The traditional method relies on experience or statistical model, and cannot quantitatively process the difference of material dielectric constant, hollow depth and thickness distribution, resulting in inaccurate detection results or lack of repeatability.

[0085] The beneficial effects of the existing technology are: through the adjustable frequency directional emission device, the multi-frequency scanning is carried out in the candidate detection area, the amplitude, phase and propagation time delay information are obtained, and the complete multi-frequency reflection signal set is formed, which provides reliable physical data basis for subsequent depth and dielectric characteristic inversion. The relationship between echo amplitude and phase and the equivalent dielectric constant of wall material is used, the reflection coefficient is calculated, the difference of wall interface material is quantitatively described, and the physical quantitative hollow identification is realized, instead of relying on vision or experience. The echo phases of each frequency point are arranged in sequence to form a phase sequence, which provides continuous signal data for multi-frequency inversion algorithm, can accurately calculate the hollow depth and thickness distribution, and realizes non-destructive and quantitative detection. The multi-frequency, multi-point and continuous collection mode can effectively overcome the defects of single frequency or single mode signal, such as being easily affected by noise, material non-uniformity and surface environment, so as to improve the accuracy and repeatability of the detection results.

[0086] The method for obtaining the depth and thickness distribution of the hollow includes:

[0087] According to the phase sequence recorded by the multi-frequency echo signal and the corresponding frequency information, a phase frequency relationship function is established, the propagation characteristics of microwave at the interface between wall material and air sandwich are used, and the change of echo phase with frequency reflects the propagation path of microwave in the wall; the phase frequency relationship function is: ; wherein, c represents the speed of light;

[0088] According to the phase frequency relationship function, the group time delay formed by the change rate of phase with frequency is calculated, the group time delay is: ; wherein, cgt represents the group time delay, and represents the delay response of phase to frequency caused by the round trip propagation of microwave in the medium; and the relationship between the group time delay and the dielectric constant is used to obtain the depth of the hollow; the depth of the hollow is: ; wherein, c represents the depth of the hollow, that is, the one-way distance of microwave propagation;

[0089] The equivalent dielectric constant of the wall material is deduced from the amplitude of the echo signal and the wall interface reflection coefficient, so as to obtain the dielectric distribution of the hollow layer; the thickness distribution of the hollow layer is estimated by the amplitude attenuation characteristics of the multi-frequency signal and the change of the wall interface reflection intensity, combined with the depth of the hollow.

[0090] Specifically, according to the amplitude attenuation characteristics of the multi-frequency reflection signal and the change of the interface reflection intensity, the reflection interface of the adjacent two layers of medium in the wall corresponds to the phase mutation point of the echo signal. Assuming that the depths corresponding to the upper and lower interfaces are and , the thickness of the hollow layer can be expressed as: .

[0091] The following technical problems existing in the prior art are solved: the conventional method usually relies on single-frequency microwave or infrared thermal imaging for detection, and cannot accurately invert the depth of the hollow through the phase information of continuous frequency. The existing technology can only determine the existence of the hollow, and lacks a quantitative measurement means for the thickness of the hollow layer. The small echo signal change corresponding to the adjacent medium interface in the wall is difficult to identify, resulting in the loss of thickness distribution information. The dielectric constant difference between the wall material and the air layer is the physical basis for hollow detection, but most of the existing methods rely on experience or statistical models, and lack a means for directly inverting the dielectric constant through the echo amplitude and phase change, which reduces the detection accuracy and reliability.

[0092] The beneficial effects of the prior art are: by using the relationship between the echo phase and the frequency of each frequency point, forming a phase frequency function, and combining the propagation characteristics of the microwave at the interface between the wall material and the air layer, the microwave propagation path can be quantified, and the depth of the hollow can be accurately inverted. The rate of change of the phase with the frequency is calculated as the group delay, and the one-way depth of the hollow can be quantitatively calculated by combining the dielectric constant relationship, which is more accurate and reliable than the traditional empirical judgment. The equivalent dielectric constant of the wall material is deduced from the echo amplitude and the wall interface reflection coefficient, and the dielectric distribution of the hollow layer is obtained. By using the phase mutation point and the amplitude attenuation characteristics, combined with the depth information, the thickness of the hollow layer can be accurately estimated, and the thickness of the hollow layer is expressed as the difference between the depths of the upper and lower interfaces, realizing the quantitative measurement of the thickness of the hollow. Without damaging the wall structure, the depth, thickness and dielectric characteristics of the hollow can be obtained at the same time, realizing the comprehensive, continuous and quantitative detection of the hollow layer, and the detection result has higher accuracy, repeatability and physical basis.

[0093] The method for forming a continuous hollow area and a characteristic parameter of the hollow area comprises:

[0094] To avoid the dimensional difference between different feature dimensions from affecting the subsequent clustering, the depth and thickness distribution of the hollow are normalized and weighted, and a fusion feature set is constructed; based on the fusion feature set, a density clustering algorithm with spatial constraints is executed, and candidate detection regions of suspected hollows that are adjacent in space and have continuous depth and thickness changes are aggregated to form continuous hollow candidate regions;

[0095] It should be noted that the density clustering algorithm with spatial constraints (DBSCAN) means that on the basis of the traditional density clustering algorithm, spatial continuity constraints and physical similarity constraints are introduced, so that the clustering results not only satisfy the feature similarity (such as depth and thickness similarity), but also satisfy the spatial continuity of the sampling points in the physical space coordinate system;

[0096] In the detection of hollows in external walls, the physical boundaries of hollow regions often exhibit spatial connectivity and continuous changes in thickness and depth, so simply using the traditional density clustering algorithm (only according to feature similarity) can easily aggregate physically dispersed but signal feature similar sampling points to form "false hollow regions"; therefore, a spatial constraint term is added to the traditional density clustering algorithm to ensure that only spatially adjacent points with continuous depth and thickness can be aggregated into a class, thereby effectively excluding false hollows;

[0097] All hollow candidate regions are mapped to the spatial coordinate system to generate a region mask image, after removing isolated noise points and small holes through morphological closing operation, the hollow region boundary is extracted, and curve fitting is used to smooth the hollow region boundary, and then the continuous hollow boundary contour is obtained;

[0098] It should be noted that the region mask image refers to an image or data structure representing the spatial distribution of the region in the two-dimensional or three-dimensional space coordinate system in the form of a binary (or multi-value) matrix;

[0099] Taking a binary region mask image as an example: in two-dimensional form, the binary region mask image is a gray image (or two-dimensional matrix) corresponding to the size of the external wall surface, each pixel corresponds to a sampling point on the wall surface, and each hollow candidate region appears as a connected white region in the region mask image; wherein, the pixel value of 1 (or white) indicates that the position is identified as a hollow; the pixel value of 0 (or black) indicates that the position is identified as a normal wall;

[0100] Based on the continuous hollow boundary contour, a continuous hollow region is delineated, the feature parameters of the hollow region are calculated, and the hollow region feature parameters including spatial position, morphological geometry and physical properties are obtained.

[0101] It should be noted that: the spatial position includes the center coordinates of the hollow area, the boundary contour coordinate set of the hollow area, the spatial range of the hollow area, etc., which are used to locate the specific position and size of the hollow on the wall surface; the shape geometry includes the area, perimeter, length-width ratio, boundary irregularity of the hollow area, etc., which are used to describe the geometric contour characteristics and morphological complexity of the hollow area; the physical properties include the average depth, maximum depth, thickness distribution range, average reflection intensity, infrared temperature difference of the hollow area, etc., which are used to describe the internal physical characteristics of the hollow area.

[0102] The method for generating a detection report containing position, depth, area and level includes:

[0103] According to the hollow area feature parameters, the external wall hollow quantitative evaluation is carried out, the hollow area feature parameters are linearly normalized and weighted fused to obtain a comprehensive score representing the severity of the hollow area; different comprehensive score threshold intervals are preset (different comprehensive score thresholds are preset to form different comprehensive score threshold intervals, and the different comprehensive score thresholds can be set according to historical experience data), and the hollow area is divided into different disease levels; the hollow area of different disease levels is marked with different colors, visual output is carried out, and a detection report containing position, depth, area and level is automatically generated.

[0104] The preset distance threshold is set by the staff, the average value of multiple distances is taken as the preset distance threshold by collecting different distances, and the preset local protrusion height threshold, the preset recess depth threshold and the preset crack width threshold are set in the same way.

[0105] The embodiment carries out multi-frequency scanning in the candidate detection area through the adjustable frequency directional emission device, obtains amplitude, phase and propagation time delay information, forms a complete multi-frequency reflection signal set, and provides reliable physical data basis for subsequent depth and dielectric characteristic inversion. By using the relationship between echo amplitude and phase and the equivalent dielectric constant of wall material, the reflection coefficient can be calculated to quantitatively describe the difference of wall interface material, so as to realize physical quantitative hollow identification instead of relying on visual or experience judgment. The echo phases of each frequency point are arranged in sequence to form a phase sequence, which provides continuous signal data for the multi-frequency inversion algorithm, can accurately calculate the hollow depth and thickness distribution, and realizes non-destructive and quantitative detection. The multi-frequency, multi-point and continuous collection mode can effectively overcome the defects that single frequency or single mode signal is easily affected by noise, material non-uniformity and surface environment, so as to improve the accuracy and repeatability of the detection result.

[0106] The phase-frequency function is formed by using the echo phase and frequency relationship of each frequency point, and the propagation characteristics of microwaves in the wall material and the air interlayer interface are combined, so that the microwave propagation path can be quantified, and the accurate inversion of the hollow depth is realized. The rate of change of the phase with the frequency is calculated as the group delay, and the single-pass depth of the hollow can be quantitatively calculated in combination with the dielectric constant relationship, which is more accurate and reliable than the traditional empirical judgment. The equivalent dielectric constant of the wall material is obtained by backstepping the echo amplitude and the wall interface reflection coefficient, and the dielectric distribution of the hollow layer is obtained. The thickness of the hollow layer is accurately estimated by using the phase mutation point and the amplitude attenuation characteristics in combination with the depth information, and the thickness is represented as the depth difference between the upper and lower interfaces, so that the quantitative measurement of the hollow thickness is realized. Without damaging the wall structure, the depth, thickness and dielectric characteristics of the hollow can be obtained at the same time, realizing the comprehensive, continuous and quantitative detection of the hollow layer, and the detection results have higher precision, repeatability and physical basis.

[0107] Embodiment 2: see Figure 2 As shown in the figure, some parts of this embodiment are not described in detail and are described in embodiment 1. A microwave reflection detection system for external wall hollow based on multi-modal fusion is provided, comprising:

[0108] A data acquisition registration module performs multi-modal scanning on the external wall of the building to be measured, obtains visible light image data and infrared temperature distribution data of the surface of the external wall, and performs spatial registration and coordinate mapping to form a unified multi-modal fusion data set;

[0109] An abnormality screening module screens the multi-modal fusion data set for abnormalities, identifies thermal abnormal areas by analyzing the infrared temperature distribution data, detects bulge or crack areas in combination with the texture and topography abnormal features of the visible light image data, and fuses the thermal abnormal areas and the bulge or crack areas to extract candidate detection areas of suspected hollows;

[0110] A microwave directional scanning module performs beam scanning on each candidate detection area through an adjustable frequency multi-frequency point scanning mechanism, records multi-frequency reflection signals at different frequencies, calculates the phase change and group delay difference of the multi-frequency reflection signals, calculates the propagation path and equivalent dielectric constant of microwaves in the wall, and obtains the depth and thickness distribution of the hollow;

[0111] A region boundary reconstruction module performs multi-dimensional feature clustering and geometric boundary fitting on the depth and thickness distribution of the hollow to form a continuous hollow region and hollow region characteristic parameters;

[0112] A disease grading and evaluation module performs quantitative evaluation of the external wall hollow according to the hollow region characteristic parameters, divides the hollow region into different disease grades, and visually outputs a detection report containing the position, depth, area and grade.

[0113] Since the electronic device introduced in the embodiment is the electronic device used in the implementation of the method for detecting external wall hollowing by microwave reflection based on multi-modal fusion in the embodiment, the specific implementation of the electronic device and various changes thereof can be understood by those skilled in the art based on the method for detecting external wall hollowing by microwave reflection based on multi-modal fusion in the embodiment, and therefore, how the electronic device implements the method in the embodiment will not be described in detail. As long as the electronic device used in the implementation of the method for detecting external wall hollowing by microwave reflection based on multi-modal fusion in the embodiment is implemented by those skilled in the art, it belongs to the scope of protection of the present application.

[0114] The above formulas are dimensionless values calculated, the formulas are obtained by collecting a large amount of data to simulate the most recent real situation, and the preset parameters and threshold values in the formulas are set by those skilled in the art according to actual conditions.

[0115] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application is within the protection scope of the present application. It should be noted that, for ordinary technical users in the technical field, some improvements and refinements without departing from the principles of the present application are also considered to be within the protection scope of the present application.

Claims

1. A method for detecting microwave reflection of hollow areas in exterior walls based on multimodal fusion, characterized in that, include: S1. Perform multimodal scanning on the exterior wall of the building to be tested to acquire visible light image data and infrared temperature distribution data of the exterior wall surface, and perform spatial registration and coordinate mapping to form a unified multimodal fusion dataset; S2. Perform anomaly screening on the multimodal fusion dataset and identify thermally abnormal areas by analyzing infrared temperature distribution data; By combining the texture and morphological anomaly features of visible light image data, bulge or crack areas are detected; information is fused between thermal anomaly areas and bulge or crack areas to extract candidate detection areas suspected of being hollow. S3. Through the adjustable frequency multi-frequency scanning mechanism, the candidate detection area is scanned by a cluster beam to record the multi-frequency reflection signals at different frequencies; the phase change and group delay difference of the multi-frequency reflection signals are calculated to estimate the propagation path and equivalent dielectric constant of microwaves in the wall, and the depth and thickness distribution of the void are obtained. Methods for obtaining the depth and thickness distribution of voids include: Based on the phase sequence and corresponding frequency information recorded from the multi-frequency echo signals, a phase-frequency relationship function is established. Utilizing the propagation characteristics of microwaves at the interface between the wall material and the air gap, the change in echo phase with frequency reflects the propagation path of the microwaves within the wall. The phase-frequency relationship function is as follows: ;in, Represents the speed of light; Based on the phase-frequency relationship function, the group delay formed by the rate of change of phase with frequency is calculated. The group delay is: ;in, Let group delay represent the phase delay response to frequency caused by microwaves propagating back and forth in the medium; and using the relationship between group delay and dielectric constant, the depth of the cavity is obtained; the depth of the cavity is: ;in, This indicates the depth of the void, which is the one-way distance of microwave propagation. By combining the amplitude of the echo signal and the reflection coefficient of the wall interface, the equivalent dielectric constant of the wall material is inferred, thereby obtaining the dielectric distribution of the hollow layer; by combining the amplitude attenuation characteristics of the multi-frequency signal and the change in the reflection intensity of the wall interface with the depth of the hollow, the thickness distribution of the hollow is estimated. S4. Perform multidimensional feature clustering and geometric boundary fitting on the depth and thickness distribution of the hollow area to form continuous hollow areas and hollow area feature parameters. S5. Quantitatively assess the hollow areas of the exterior wall based on the characteristic parameters of the hollow areas, classify the hollow areas into different disease levels, and generate a visual output, including a test report containing location, depth, area and level.

2. The method for detecting external wall voids based on multimodal fusion according to claim 1, characterized in that, The method for acquiring visible light image data and infrared temperature distribution data of the exterior wall surface includes: A drone platform equipped with a visible light imaging sensor and an infrared thermal imager was used to simultaneously scan the building's exterior wall from different angles and along different flight paths. During the scanning process, the visible light imaging sensor collected visible light image data of the exterior wall surface, and the infrared thermal imager simultaneously recorded the thermal radiation intensity of the corresponding area and automatically converted it into infrared temperature distribution data. Visible light image data includes exterior wall surface texture data, exterior wall surface geometry data, and exterior wall color and brightness data; infrared temperature distribution data includes exterior wall surface temperature data, corresponding area thermal radiation intensity data, and temperature gradient distribution data.

3. The method for detecting external wall voids based on multimodal fusion according to claim 2, characterized in that, The method for forming a unified multimodal fusion dataset includes: The acquired visible light image data and infrared temperature distribution data are synchronized in time. Based on Harris corner detection, wall corners and texture features are extracted from the visible light image data as visible light image feature points; pre-calibrated feature points are extracted from the infrared temperature distribution data as infrared temperature distribution feature points. The spatial correspondence between visible light image feature points and infrared temperature distribution feature points is established using a feature matching algorithm. Based on the intrinsic parameters of the visible light imaging sensor and the infrared thermal imager and the UAV position information, the infrared temperature distribution data is mapped to the spatial coordinate system of the visible light image data. The mapped infrared temperature distribution data and visible light image data are aligned at the pixel level using bilinear or bicubic interpolation methods. The aligned visible light image data and infrared temperature distribution data are then fused in the same spatial coordinate system to form a unified multimodal fusion dataset.

4. The method for detecting external wall voids based on multimodal fusion according to claim 3, characterized in that, The method for identifying thermal anomaly regions includes: Gaussian filtering is used to suppress noise in the infrared temperature distribution data of the multimodal fusion dataset, and the filtered infrared temperature distribution data is normalized to map the infrared temperature distribution data to a preset standard range. Temperature anomalies and hot spot features are extracted from normalized infrared temperature distribution data using the local extremum detection method, and these are used as anomalous temperature feature points. Spatial clustering algorithm is then used to merge these anomalous temperature feature points into continuous thermal anomaly regions.

5. The method for detecting external wall voids based on multimodal fusion according to claim 4, characterized in that, The method for detecting bulges or cracks includes: The LBP algorithm is used to extract texture features from visible light image data. Neighboring pixels are selected with each pixel in the visible light image as the center. The gray values ​​of the neighboring pixels are compared with the gray values ​​of the center pixel. If the gray value of the neighboring pixel is greater than or equal to the gray value of the center pixel, it is recorded as 1; otherwise, it is recorded as 0. The comparison results are sequentially formed into a binary string and converted into a decimal number to generate the LBP value for each pixel; the visible light image is divided into fixed-size grid regions, and the LBP value distribution of each region is statistically analyzed to form a texture feature vector; the texture feature vector of each region is then compared with the pre-established normal wall texture feature vector using Euclidean distance calculation, and when the distance is greater than a preset distance threshold, the region is marked as a texture anomalous region; The geometric features of the wall are extracted using an edge detection algorithm, including the wall outline, crack lines, and boundaries of local protrusions or depressions; the height of local protrusions, the depth of depressions, and the width of cracks are calculated based on the edge detection results. Regions with local bulge height greater than a preset local bulge height threshold, depression depth greater than a preset depression depth threshold, or crack width greater than a preset crack width threshold are marked as abnormal morphology regions; texture abnormality regions are combined with abnormal morphology regions as bulge or crack regions.

6. The method for detecting external wall voids based on multimodal fusion according to claim 5, characterized in that, The method for extracting candidate detection regions for suspected hollow areas includes: Place the bulge or crack area and the thermal anomaly area in a unified spatial coordinate system. For areas where the boundaries do not completely overlap, use an interpolation method to adjust and match the thermal anomaly area and the bulge or crack area. The matched thermal anomaly regions and bulge or crack regions are fused by logical AND operation. Only regions that simultaneously possess thermal anomaly characteristics and bulge or crack characteristics are marked as candidate detection regions for suspected hollow areas.

7. The method for detecting external wall voids based on multimodal fusion according to claim 6, characterized in that, The method for recording multi-frequency reflection signals at different frequencies includes: Within each candidate detection area, a frequency-tunable directional transmitter is used to perform multi-frequency scanning within a preset frequency band. The transmission frequency is controlled to scan point by point at a fixed step interval between the preset minimum and maximum frequencies to form a multi-frequency scanning sequence. Echo signals are collected at each frequency point, and the amplitude, phase, and propagation delay information of the echo signals are recorded respectively. The phases of the echo signals of all scanning frequencies are arranged sequentially to form a phase sequence. Due to the difference in dielectric constant between the wall material and the internal air gap, microwaves are reflected at the interface. The amplitude and phase characteristics of the reflected signal are related to the equivalent dielectric constant of the material. The material difference characteristics are characterized by calculating the reflection coefficient of the wall interface. The multi-frequency scanning sequence and phase sequence obtained by multi-frequency point scanning together constitute the multi-frequency reflection signal set.

8. The method for detecting external wall voids based on multimodal fusion according to claim 7, characterized in that, The method for forming continuous hollow regions and hollow region characteristic parameters includes: The depth and thickness distribution of the hollow area are normalized and weighted to construct a fusion feature set. Based on the fusion feature set, a density clustering algorithm with spatial constraints is executed to aggregate the candidate detection regions of suspected hollow areas that are spatially adjacent and have continuous changes in depth and thickness, forming continuous hollow area candidate regions. All candidate hollow regions are mapped to a spatial coordinate system to generate a region mask image. After removing isolated noise and pinholes through morphological closing operations, the boundaries of the hollow regions are extracted, and curve fitting is used to smooth the boundaries of the hollow regions to obtain continuous hollow boundary contours. Based on the continuous hollow boundary contours, continuous hollow regions are delineated, and feature parameters of the hollow regions are calculated to obtain the feature parameters of the hollow regions.

9. The method for detecting external wall voids based on multimodal fusion according to claim 8, characterized in that, The method for generating an inspection report that includes location, depth, area, and grade includes: A quantitative assessment of external wall hollowness is conducted based on the characteristic parameters of the hollow area. The characteristic parameters of the hollow area are linearly normalized and weighted to obtain a comprehensive score that characterizes the severity of the hollow area. Different comprehensive score threshold ranges are preset to divide the hollow area into different disease levels. Hollow areas of different disease levels are marked with different colors for visualization output, and an inspection report containing location, depth, area and level is automatically generated.

Images