Structural surface disease diagnosis method and system based on multi-modal edge computing

Abstract

The present application relates to the technical field of surface defect detection, in particular to a structural surface disease diagnosis method and system based on multi-modal edge computing, comprising the following steps: acquiring an infrared thermal image frame and constructing a temperature difference graph set, enhancing visible light texture features to generate an enhanced graph set, performing image registration to extract a combined feature vector and classification identification, mapping a defect area boundary to generate a coordinate set, and counting disease information to generate a visual display layer. The present application, through temperature mapping and spatial division operations of the infrared thermal imaging image, enables visual expression of thermal anomaly features in different regions, realizes accurate discrimination of defect types through a trained neural network model, extracts boundary coordinate point sets of structural surface defects through image mapping means, and combines connectivity operation and rectangular frame construction methods to label and position the defect position, effectively improving the accuracy of building surface disease identification, the reliability of boundary extraction, and the integrity of the partition risk presentation.

Description

Technical Field

[0001] This invention relates to the field of surface defect detection technology, and in particular to a method and system for diagnosing structural surface defects based on multimodal edge computing. Background Technology

[0002] The field of surface defect detection technology mainly involves a technical system for the automated identification, location, and quantitative analysis of defects or abnormal states on the surfaces of materials, components, or structures. This field integrates multiple disciplines such as image processing, machine vision, deep learning, pattern recognition, and non-destructive testing. By acquiring target surface image data and employing processing algorithms such as image enhancement, edge extraction, and target segmentation, it models and analyzes various surface defects, including cracks, peeling, bulging, and discoloration, to assist in subsequent diagnosis, maintenance, and decision-making. Surface defect detection is widely used in the assessment of component safety and integrity in industries such as construction, machinery manufacturing, transportation, and energy facilities, and features non-contact operation, high efficiency, and high accuracy.

[0003] The structural surface defect diagnosis method is used to identify and diagnose defects appearing on the surface of building structures, including the detection and classification of various defect types such as cracks, spalling, weathering, and corrosion. This method acquires and processes images of the building structure surface, combines them with a pre-set defect feature database for discriminative analysis, and achieves accurate identification of defect areas, boundary location, and severity assessment, providing decision support for structural maintenance, reinforcement design, and lifespan assessment.

[0004] Traditional diagnostic methods do not perform joint analysis of thermal imaging and visible light information during image acquisition and processing. This results in a failure to integrate and express the temperature change characteristics and surface texture characteristics of the affected area. When faced with hidden distributions and blurred boundaries of defects such as hollowing or peeling, relying solely on single-source images for identification is prone to errors. Traditional methods only identify the type of defect through discriminant analysis without labeling the location of the defect area with reverse coordinates. This makes the identification results lack traceability and difficult to directly support subsequent positioning and structural maintenance operations. For example, in wall areas with dense crack distribution, it is difficult to conduct risk assessment and maintenance planning based solely on the identification results of the defect type, affecting the practicality of the diagnostic results and the efficiency of subsequent work deployment. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and to propose a method and system for diagnosing structural surface defects based on multimodal edge computing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for diagnosing structural surface defects based on multimodal edge computing, comprising the following steps:

[0007] S1: Obtain the frame sequence captured by the infrared thermal imager on the surface of the building structure's exterior wall, map the pixel gray values ​​in the image to the blackbody calibration reference value, and construct a partitioned thermal anomaly matrix based on the division of the building surface area to generate a partitioned temperature difference mapping map group.

[0008] S2: Based on the visible light image frame of the building wall, and through image enhancement, perform local gray-level histogram statistics on the filtered image, transform and reconstruct the gray-level values ​​of each region, reconstruct the visual features of the wall surface texture, and generate a set of enhanced structural surface texture images.

[0009] S3: Based on the partitioned temperature difference mapping group and the structural surface texture enhancement group, feature fusion is performed on the same spatial coordinate regions in the image to extract the temperature gradient, gray-level co-occurrence matrix contrast, edge closure degree and shape factor of the region, and the trained convolutional neural network is used to classify defects to generate a structural partitioned disease identification layer.

[0010] S4: Call the boundary labels of the defective areas identified in the structural partition defect identification layer, perform position mapping based on the pixel coordinate relationship in the original image, and generate the building defect boundary coordinate set by extracting the closed boundary contour point set.

[0011] The present invention is improved in that the partitioned temperature difference mapping map group includes a regional temperature anomaly threshold band, a pixel heat value spatial distribution grid, a wall surface thermal radiation non-uniformity area, and a residual area for thermal imaging noise suppression; the structural surface texture enhancement map group includes a texture image high-frequency detail distribution layer, an image grayscale equalization layer, a texture edge coherence feature map, and an illumination interference correction area map; the structural partitioned defect identification layer includes a defect type label area, a building component partition index encoding, a feature vector prediction distribution map, and a convolution feature response activation map; and the building defect boundary coordinate set includes a boundary bounding box coordinate index, a contour closed structural point matrix, structural patch geometric reference points, and a boundary topological connectivity mapping.

[0012] The present invention is improved in that the specific steps for obtaining the partitioned temperature difference mapping group are as follows:

[0013] S111: Obtain the frame sequence of the building structure's exterior wall surface captured by an infrared thermal imager, call the spatial resolution parameters and frame rate parameters configured in the image acquisition control terminal, filter according to the frame number order and acquisition time interval of each frame, compare the image sharpness index and brightness histogram peak position in consecutive frames, and remove image frames with sharpness index below the set limit to generate a thermal image screening group of the structure surface.

[0014] S112: Based on the pixel grayscale values ​​in each image frame of the thermal image screening group of the structure surface, call the grayscale conversion table to map the grayscale values ​​to the calibration points in the standard blackbody temperature calibration reference table, compensate and correct the non-calibrated pixels through linear interpolation, construct the image pixel temperature matrix, and record the inter-frame temperature floating segments according to the image index order to generate the regional pixel temperature mapping set.

[0015] S113: Based on the spatial coordinate index of the image frames in the region pixel temperature mapping set, combined with the boundary division of the building wall structure, perform range analysis on the temperature pixel set in each region, delineate the boundary of the thermal anomaly matrix according to the temperature change response threshold, and perform image number index association on the two-dimensional thermal field of the delineated region to establish a partitioned temperature difference mapping map group.

[0016] The present invention is improved in that the steps for obtaining the enhanced surface texture map group are as follows:

[0017] S211: Obtain visible light image frames of building walls, perform median filtering on each frame, call the gray value sequence of each pixel in the filtering window, sort the sequence in ascending order and extract the median value to replace the gray value of the center pixel, record the number of replacements according to the total number of pixels in each frame, and output the processed image frames in the original index order to generate a structure surface filtered image sequence.

[0018] S212: Based on the filtered image sequence of the structure surface, extract the set of local pixel gray values ​​of each region in each frame image, construct a gray-level histogram according to the region boundary, normalize the pixel frequency of each gray level in the histogram, calculate the cumulative distribution value corresponding to each gray level, and establish a gray-level transformation function based on the cumulative distribution function to generate a set of partitioned gray-level mapping functions.

[0019] S213: According to the grayscale transformation function of each region in the partitioned grayscale mapping function set, call the corresponding pixel coordinate index to replace the grayscale values ​​in the structure surface filtered image sequence, merge the remapped image frames of each region according to the original image sequence, and mark the layer index of the texture enhancement result according to the region number to generate a structure surface texture enhancement map group.

[0020] The present invention is improved in that the specific steps for obtaining the structural partition disease identification layer are as follows:

[0021] S311: Based on the partitioned temperature difference mapping group and the structural surface texture enhancement group, call the image alignment parameters, perform spatial coordinate transformation and geometric constraint matching on the pixels of infrared image frames and visible light image frames in the same building area, align the corresponding frames of the infrared channel and the visual channel, and generate a multimodal image registration reference set.

[0022] S312: Based on each pair of image frames in the multimodal image registration reference set, extract the infrared channel temperature gradient distribution value, visual channel gray-level co-occurrence matrix contrast, contour boundary closure degree and image block shape factor in the same region of the image at the edge processing node, and construct the four types of numerical features into a structured array according to the detection region number to generate a set of region fusion feature vectors.

[0023] S313: Call the set of regional fusion feature vectors, input each feature vector into the trained convolutional neural network structure, load the labeled sample label set of hollow, crack and leakage, perform parallel feature prediction at the edge nodes, and summarize the defect type classification results of each detection area to generate a structural partition disease identification layer.

[0024] The present invention is improved in that the steps for obtaining the coordinate set of the building defect boundary are as follows:

[0025] S411: Call the area labels of hollow, cracked or leaking areas in the structural partition disease identification layer, perform coordinate mapping on the pixel positions in the original image coordinate system according to the boundary attribute index of each type of disease in the layer, retrieve the set of pixel points corresponding to each disease area, construct the area boundary contour point column, and generate the image disease contour point set.

[0026] S412: Based on the image disease contour point set, extract the two-dimensional coordinate pairs of contour boundary points within the closed area. Based on the region boundary connectivity, boundary point density, and enclosing area, calculate the compactness of the boundary distribution of each disease region and determine whether the boundary closure condition is met. Calculate the minimum bounding rectangle of the region that meets the condition, obtain the boundary enclosing coordinate segment, and generate the minimum boundary set of the disease region.

[0027] S413: Based on the rectangular coordinate segments corresponding to each defect in the minimum boundary set of the defect area, construct a data table with rectangular vertex coordinate index and defect category label, aggregate it into a unified layer data structure according to the regional index order, and establish a building defect boundary coordinate set.

[0028] The present invention has an improvement, wherein the method further includes the following steps:

[0029] S5: Call the coordinate and label data in the building defect boundary coordinate set and the structural partition disease identification layer, overlay the category and location of the disease in each area, evaluate the defect risk level of each partition, construct a defect statistics layer in combination with the numbering information, call the display interface to output the layer to the user terminal visual component, and generate structural disease distribution display information.

[0030] The structural defect distribution display information includes the corresponding layer for the partition number, defect category icon labels, statistical distribution charts, and a visual interactive interface for the layer.

[0031] The present invention is improved in that the steps for obtaining the structural defect distribution display information are as follows:

[0032] S511: Based on the building defect boundary coordinate set and the coordinate index and category label data in the structural partition disease identification layer, call the layer visualization rendering interface to perform coordinate overlay processing on each structural area, and bind the disease type label with the corresponding boundary rectangle coordinate to the layer, implement visible mark rendering, and generate a disease label mapping layer dataset.

[0033] S512: Based on the index information of each structural region in the disease label mapping layer dataset, extract the number of diseases, the occupied area and the layer projection area of ​​each region, calculate the disease frequency, disease density and area ratio by region, calculate the risk level index of each structural partition, combine the original partition number to perform risk level labeling mapping, and generate structural risk level assessment results.

[0034] S513: Based on the structural risk level assessment results and the area layers that have been classified in the disease label mapping layer dataset, construct a multi-dimensional field structure including disease location, category, density and risk level, combine and generate the layer output data structure and transmit it to the user-end visual component to establish structural disease distribution display information.

[0035] A structural surface defect diagnosis system based on multimodal edge computing, wherein the structural surface defect diagnosis system based on multimodal edge computing is used to implement the above-mentioned structural surface defect diagnosis method based on multimodal edge computing, the system comprising:

[0036] The zoned temperature difference mapping module acquires a sequence of frame images captured by an infrared thermal imager on the surface of the building's exterior wall, maps the pixel grayscale values ​​in the images to the blackbody calibration reference values, and constructs a zoned thermal anomaly matrix based on the division of the building surface area to generate a zoned temperature difference mapping map group.

[0037] The surface enhancement processing module uses visible light image frames of the building wall surface and performs local gray-level histogram statistics on the filtered image through image enhancement. It transforms and reconstructs the gray-level values ​​of each region to rebuild the visual features of the wall surface texture and generate a set of structural surface texture enhancement images.

[0038] The structural defect classification module, based on the partitioned temperature difference mapping map group and the structural surface texture enhancement map group, performs feature fusion on the same spatial coordinate regions in the image, extracts the temperature gradient, gray-level co-occurrence matrix contrast, edge closure degree and shape factor of the region, and performs defect classification by the trained convolutional neural network to generate a structural partitioned defect recognition layer.

[0039] The defect boundary extraction module calls the boundary labels of the defective areas identified in the structural partition defect identification layer, performs position mapping based on the pixel coordinate relationship in the original image, and generates a set of building defect boundary coordinates by extracting the closed boundary contour point set;

[0040] The defect distribution display module calls the coordinate and label data in the building defect boundary coordinate set and the structural partition defect identification layer, overlays the defect category and location in each area, assesses the defect risk level of each partition, constructs a defect statistics layer with the number information, calls the display interface to output the layer to the user-end visual component, and generates structural defect distribution display information.

[0041] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0042] In this invention, temperature mapping and spatial segmentation operations of infrared thermal imaging images enable the visual representation of thermal anomaly features in different regions. Combined with texture enhancement processing of visible light images, multiple detailed textures of the building surface are restored and reconstructed. Based on the fusion of image features, multi-dimensional quantitative indicators are extracted. A trained neural network model is used to accurately identify defect types. The boundary coordinate point set of structural surface defects is extracted through image mapping. Combined with connectivity operations and rectangular box construction, the defect locations are marked and located. A risk level assessment layer for each zone is constructed through frequency statistics and area analysis. The distribution information of structural defects is output in a visual form, effectively improving the accuracy of building surface defect identification, the reliability of boundary extraction, and the completeness of zone risk presentation. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention;

[0044] Figure 2 This is a flowchart of the process for obtaining a group of zoned temperature difference mapping maps according to the present invention;

[0045] Figure 3 This is a flowchart of the process for obtaining a set of enhanced surface texture maps for the present invention;

[0046] Figure 4 This is a flowchart illustrating the process of obtaining a structural partition disease identification layer according to the present invention;

[0047] Figure 5 This is a flowchart illustrating the process of obtaining the coordinate set of building defect boundaries according to the present invention;

[0048] Figure 6 This is a flowchart illustrating the process of obtaining information on the distribution of structural defects according to the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0051] Please see Figure 1 This invention provides a technical solution: a method for diagnosing structural surface defects based on multimodal edge computing, comprising the following steps:

[0052] S1: Acquire a sequence of frames captured by an infrared thermal imager on the surface of the building's exterior wall. Filter the images frame by frame based on the spatial resolution and frame rate set by the image acquisition control terminal. Perform grayscale and temperature conversion on the filtered images. Call the image temperature mapping component to map the pixel grayscale values ​​in the image to the blackbody calibration reference values. Construct a partitioned thermal anomaly matrix based on the division of the building surface area to generate a partitioned temperature difference mapping map group.

[0053] The blackbody calibration reference value refers to the gray-temperature mapping table established in infrared imaging equipment by setting the target temperature of a standard blackbody radiation source;

[0054] S2: Based on the visible light image frames of the building wall, perform median filtering on each frame to remove random interference, and perform local gray-level histogram statistics on the filtered image through image enhancement. Construct a pixel mapping function according to the regional gray-level equalization coefficient, transform and reconstruct the gray-level values ​​of each region, and reconstruct the visual features of the wall surface texture in the entire image range to generate a set of enhanced structural surface texture images.

[0055] S3: Based on the partitioned temperature difference mapping group and the structural surface texture enhancement group, image registration is performed. The corresponding image pairs are loaded to the infrared channel and the visual channel through the edge processing node. Feature fusion is performed on the same spatial coordinate regions in the image. The temperature gradient, gray-level co-occurrence matrix contrast, edge closure degree and shape factor of the region are extracted to construct a four-dimensional combined feature vector. The trained convolutional neural network loads the sample labels of hollow, crack and leakage samples in the sample library for vector prediction and classification. After local parallel inference is completed at the edge node, the defect type judgment results of each detection area are summarized to generate a structural partition disease identification layer.

[0056] Edge closure refers to the closure ratio of the connected regions of the boundary pixels; the shape factor is commonly defined as the ratio of its area to the area of ​​its bounding rectangle.

[0057] S4: Call the boundary labels of the defective areas identified in the structural partition disease identification layer, perform position mapping based on the pixel coordinate relationship in the original image, extract the closed boundary contour point set, calculate the connectivity of the point set and perform the minimum bounding rectangle operation on the contour to construct the bounding box of the area with coordinate labels, and generate the building defect boundary coordinate set.

[0058] S5: Call the coordinate and label data of the building defect boundary coordinate set and the structural partition defect identification layer, overlay the defect category and location in each area, calculate the defect frequency, distribution density and surrounding area area for each structural partition, assess the defect risk level of each partition, construct a defect statistics layer with the number information, call the display interface to output the layer to the user-end visual component, and generate structural defect distribution display information.

[0059] The zoned temperature difference mapping set includes regional temperature anomaly threshold bands, pixel thermal value spatial distribution grids, wall surface thermal radiation non-uniformity areas, and residual areas for thermal imaging noise suppression. The structural surface texture enhancement set includes a texture image high-frequency detail distribution layer, an image grayscale equalization layer, a texture edge coherence feature map, and an illumination interference correction area map. The structural zoned defect identification layer includes a defect type label area, building component zone index encoding, feature vector prediction distribution map, and convolution feature response activation map. The building defect boundary coordinate set includes a boundary bounding box coordinate index, contour closed structural point matrix, structural patch geometric reference points, and boundary topological connectivity mapping. The structural defect distribution display information includes a zone number corresponding layer, defect category icon labels, statistical distribution charts, and a layer visual interactive interface.

[0060] Please see Figure 2 The specific steps for obtaining the zoned temperature difference mapping group are as follows:

[0061] S111: Obtain the frame sequence of the building structure's exterior wall surface captured by an infrared thermal imager, call the spatial resolution parameters and frame rate parameters configured in the image acquisition control terminal, filter according to the frame number order and acquisition time interval of each frame, compare the image sharpness index and brightness histogram peak position in consecutive frames, and remove image frames with sharpness index below the set limit to generate a thermal image screening group of the structure surface.

[0062] 30. Acquire a sequence of frames captured by an infrared thermal imager on the surface of the building's exterior wall. Then, call the spatial resolution and frame rate parameters configured in the image acquisition control terminal, where the spatial resolution is set to... The resolution is set to 30 frames per second (fps). Frames are filtered based on their frame number and acquisition time interval. Specifically, from 100 consecutive frames (FRM0001 to FRM0100), with an acquisition time interval of 33.3 milliseconds, the sharpness index and luminance histogram peak position are extracted for each frame. The sharpness index is obtained by calculating the Tenenbaum gradient value of the image, i.e., convolving the image with Sobel operators in the horizontal and vertical directions, and then calculating the square root of the sum of the squares of the gradient magnitudes in both directions for all pixels. The luminance histogram peak position is determined by statistically analyzing the gray level with the most pixels in the 0-255 grayscale range. For example, for the image with frame number FRM0005, the calculated sharpness index is 485.6, and the luminance histogram peak position is at gray level 128. The sharpness index and luminance histogram peak positions of consecutive frames are compared, and frames are eliminated by setting a sharpness index limit. The sharpness index was determined based on statistical analysis of 1000 sample images collected under normal lighting and focusing conditions. The 15th percentile of its statistical distribution was taken as the threshold. The specific calculation process is as follows: the 1000 sharpness index values ​​are sorted in ascending order, and the 150th value is selected as the limit value, which is set to 500.0. At the same time, in order to avoid artifacts caused by sudden changes in ambient light, image frames whose peak position of the brightness histogram moves within a range of more than 50 gray levels within 10 consecutive frames are also removed. As shown in Table 1, the sharpness index of frame FRM0003 (475.2) is lower than the set limit value of 500.0, and the sharpness index of frame FRM0008 (491.5) is lower than the set limit value of 500.0. Therefore, these two frames were removed.

[0063] Table 1 Image Frame Filtering Data:

[0064] ;

[0065] As shown in Table 1, after the screening is completed, the remaining eight frames of images, namely FRM0001, FRM0002, FRM0004, FRM0005, FRM0006, FRM0007, FRM0009, and FRM0010, are combined in the order of the original frame numbers to generate a thermal image screening group of the structural surface.

[0066] S112: Based on the pixel grayscale values ​​in each image frame of the thermal image screening group of the structural surface, call the grayscale conversion table to map the grayscale values ​​to the calibration points in the standard blackbody temperature calibration reference table, compensate and correct the non-calibrated pixels through linear interpolation, construct the image pixel temperature matrix, and record the inter-frame temperature fluctuation segments according to the image index order to generate the regional pixel temperature mapping set.

[0067] Based on the pixel grayscale values ​​within each image frame of the structural surface thermal image screening group, specifically taking image frame FRM0005 as an example, a grayscale conversion table is used to map the grayscale values ​​to calibration points in a standard blackbody temperature calibration reference table. This reference table establishes the correspondence between temperature and grayscale values ​​by photographing a standard blackbody of known temperature using an infrared thermal imager of the same model as the acquisition equipment under controlled conditions, recording its output grayscale values. For example, the calibration point data is: grayscale value... Corresponding temperature grayscale value Corresponding temperature grayscale value Corresponding temperature For non-calibrated pixels in the image, compensation and correction are performed through linear interpolation. For example, the pixel with coordinates (100, 150) in the FRM0005 image has a gray value of Its grayscale value is between and Between, the corresponding temperature Calculated in the following way:

[0068] ;

[0069] Substitute the values ​​into the calculation:

[0070] ;

[0071] All within the FRM0005 image frame Perform the same mapping and compensation correction operations on each pixel to construct a... The image pixel temperature matrix is ​​used, where each element represents the temperature of the corresponding pixel. The inter-frame temperature fluctuation range is recorded in image index order. Specifically, the temperature values ​​of pixels at the same coordinate (100, 150) in adjacent retained frames (e.g., FRM0004 and FRM0005) are compared. If the temperature of that point in FRM0004 is 17.8℃, then the temperature fluctuation of that point during the time interval between FRM0004 and FRM0005 is recorded as 17.8℃. The temperature matrix of all pixels and the temperature fluctuation records between all adjacent frames are collected to generate a regional pixel temperature mapping set.

[0072] S113: Based on the spatial coordinate index of the image frames in the regional pixel temperature mapping set, combined with the boundary division of the building wall structure, perform range analysis on the temperature pixel set in each region, delineate the boundary of the thermal anomaly matrix according to the temperature change response threshold, and perform image number index association on the two-dimensional thermal field of the delineated region to establish a partitioned temperature difference mapping map group.

[0073] Based on the spatial coordinate index of image frames in the regional pixel temperature mapping set, and combined with the boundary delineation of building wall structure, for example, a single image... The image of pixels is divided into four The rectangular regions are labeled as regions A, B, C, and D. Range analysis is performed on the temperature pixel set within each region. Taking region A as an example, all values ​​within that region are extracted. Find the highest temperature among the temperature values ​​of each pixel. and lowest temperature Calculate the temperature range of this region as follows: The boundary of the thermal anomaly matrix is ​​defined based on the temperature anomaly response threshold. This threshold is set based on long-term monitoring data of the normal thermal performance of similar building materials under similar environmental conditions. The range of their surface temperature distribution is statistically analyzed, and the 98th percentile of the statistical distribution is taken as the threshold. Since the temperature range of 12.3℃ in area A exceeds the threshold of 5.0℃, a thermal anomaly is determined to exist in area A. Subsequently, within area A, all areas with temperatures higher than the average temperature of that area are... Add half of the threshold ( The set of pixels, and all pixels whose temperature is below the average temperature of the region minus half the threshold. The system identifies a set of pixels and records the coordinates of these pixels. Combine them into a thermal anomaly matrix, and perform image number index association on the two-dimensional thermal field of the defined area, that is, associate the thermal anomaly matrix with the frame number FRM0005 and record it as {FRM0005, area A, anomaly matrix coordinate set}. Repeat this process for all areas and all filtered image frames to establish a partitioned temperature difference mapping group.

[0074] Please see Figure 3 The specific steps for obtaining the surface texture enhancement map set are as follows:

[0075] S211: Obtain visible light image frames of building walls, perform median filtering on each frame, call the gray value sequence of each pixel in the filtering window, sort the sequence in ascending order and extract the median value to replace the gray value of the center pixel, record the number of replacements according to the total number of pixels in each frame, and output the processed image frames in the original index order to generate a structure surface filtered image sequence.

[0076] Acquire a visible light image frame of the building wall. This frame was acquired at the same time as FRM0005 in the infrared thermal image sequence. Perform median filtering on this frame and call a... The filtering window moves pixel by pixel across the image. During each movement, it extracts the grayscale value sequence of the nine pixels covered by the filtering window. For example, taking the pixel at coordinates (200, 250) as the center, its... The gray value sequence of the neighborhood is Sort the sequence in ascending order to obtain The median value of the sequence, i.e., the 5th value 118, is extracted and used to replace the original grayscale value 180 of the center pixel (200, 250). This operation effectively removes isolated highlights or shadows caused by noise or minor defects. The number of replacements is recorded based on the total number of pixels in each frame. If the image has a total of 307,200 pixels, then the replacement operation needs to be performed 307,200 times (edge ​​pixels are processed by boundary mirroring). The processed image frames are output while retaining their original index order (e.g., associated with FRM0005). The same median filtering process is performed on all visible light image frames corresponding to the filtered thermal image group to generate a structure surface filtered image sequence.

[0077] S212: Based on the structured surface filtered image sequence, extract the local pixel gray value set of each region in each frame image, construct a gray level histogram according to the region boundary, normalize the pixel frequency of each gray level in the histogram, calculate the cumulative distribution value corresponding to each gray level, and establish a gray level transformation function based on the cumulative distribution function to generate a set of partitioned gray level mapping functions.

[0078] Based on the structured surface filtered image sequence, a set of local pixel grayscale values ​​is extracted for each region corresponding to the thermal image partitions (regions A, B, C, and D). Taking region A as an example, a grayscale histogram of 76,800 pixels within this region is constructed. The horizontal axis of this histogram represents the grayscale levels from 0 to 255, and the vertical axis represents the number of pixels at each grayscale level. The pixel frequency of each grayscale level in the histogram is normalized by dividing the number of pixels at each grayscale level by the total number of pixels in region A, 76,800. For example, if there are 1,536 pixels with grayscale level 120, then its normalized frequency is... Calculate the cumulative distribution value corresponding to each gray level. That is, starting from gray level 0, accumulate all normalized frequencies less than or equal to the current gray level. For example, the cumulative distribution value for gray level 120 is... It equals the sum of all normalized frequencies from gray level 0 to 120, and a gray-level transformation function is established based on this cumulative distribution function. The form is ,in The original grayscale value. This is a rounding function, for example, if The new grayscale value after transformation is Repeat this calculation for all gray levels in area A to obtain the gray-level transformation function specific to area A. Similarly, establish gray-level transformation functions for areas B, C, and D respectively to generate a set of partitioned gray-level mapping functions.

[0079] S213: Based on the grayscale transformation function of each region in the partitioned grayscale mapping function set, call the corresponding pixel coordinate index to replace the grayscale values ​​in the structure surface filtered image sequence, merge the remapped image frames of each region according to the original image sequence, and mark the layer index of the texture enhancement result according to the region number to generate a structure surface texture enhancement map group.

[0080] Based on the grayscale transformation function for each region in the partitioned grayscale mapping function set, the corresponding pixel coordinate index is called to replace the grayscale values ​​in the structure surface filtered image sequence. Continuing with region A as an example, the grayscale transformation function generated for region A in the previous step is called. Traverse region A (coordinate range) From 0 to 319, For each pixel within the range of 0 to 239, read its grayscale value in the filtered image. And calculate the new grayscale value. ,use Replace the original grayscale value of this pixel. For example, the pixel at coordinates (50, 80) has an original grayscale value of 120. According to the previous example, its new grayscale value is replaced with 102. For pixels in areas B, C, and D, their respective grayscale transformation functions are called. , , The grayscale value replacement is performed, and the remapped image frames of each region are merged and output according to the original image sequence. That is, the pixel data of the four regions A, B, C, and D that have undergone grayscale transformation are reassembled into a complete image. The image is processed, and the layer index of the texture enhancement result is labeled according to the region number. For example, metadata is generated to record the layer index 1 corresponding to region A, the layer index 2 corresponding to region B, etc. Finally, a complete image frame after partition histogram equalization is obtained. This operation is performed on each frame in the structure surface filtered image sequence to generate a structure surface texture enhancement map group.

[0081] Please see Figure 4 The specific steps for obtaining the structural partition disease identification layer are as follows:

[0082] S311: Based on the partitioned temperature difference mapping group and the structural surface texture enhancement group, the image alignment parameters are called to perform spatial coordinate transformation and geometric constraint matching on the pixels of infrared image frames and visible light image frames in the same building area, and the corresponding frames of the infrared channel and the visual channel are aligned to generate a multimodal image registration reference set.

[0083] Based on the partitioned temperature difference mapping image set and the structural surface texture enhancement image set, specifically selecting the thermal anomaly image and texture enhancement image corresponding to FRM0005, and calling the image alignment parameters, which are obtained by calibrating the intrinsic and extrinsic parameters and relative pose relationship between the infrared and visible light cameras, including a... rotation matrix and Translation vector Spatial coordinate transformation and geometric constraint matching are performed on pixels in infrared and visible light image frames within the same building area. Specifically, for a thermal anomaly point in region A of the infrared image, its pixel coordinates are... First, the intrinsic parameter matrix of the infrared camera is used. Convert it to a 3D point in the camera coordinate system Then through rotation and translation transformations Transform it to a three-dimensional point in the visible light camera coordinate system. Finally, the intrinsic parameter matrix of the visible light camera is used. Projecting it onto a visible light image yields the matching pixel coordinates. For example, the thermal anomaly point (55, 90) in region A of the infrared image, after the above coordinate transformation calculation, corresponds to the coordinates (68, 102) on the visible light image. The positions of the corresponding frames of the infrared channel and the visual channel are aligned, that is, the pixel positions of one image (e.g., the infrared image) are adjusted so that its feature points coincide with the corresponding feature points of the other image in space. Finally, image pairs with consistent size and content alignment are generated to form a multimodal image registration reference set.

[0084] S312: Based on each pair of image frames in the multimodal image registration reference set, extract the infrared channel temperature gradient distribution value, visual channel gray-level co-occurrence matrix contrast, contour boundary closure degree and image block shape factor in the same region of the image at the edge processing node, and construct the four types of numerical features into a structured array according to the detection region number to generate a set of region fusion feature vectors.

[0085] Based on each pair of image frames in the multimodal image registration reference set, four types of numerical features within the same region of the image are extracted at the edge processing node. Taking the aligned region A as an example, firstly, the infrared channel temperature gradient distribution value is extracted by calculating the average temperature difference between each pixel and its neighboring pixels in the horizontal and vertical directions within the thermal anomaly region. For example, the average temperature gradient within an anomaly region is 2.5℃ / pixel. Secondly, the visual channel gray-level co-occurrence matrix contrast is extracted by calculating the weighted sum of the squares of the gray-level differences between pixel pairs at a specified direction and distance. This value reflects the clarity and depth of the image texture. For example, the calculated contrast value is 1.8. Next, the contour boundary closure is extracted by performing edge detection on the defective areas in the texture-enhanced image to obtain the boundary contour. The Euclidean distance between the first and last points of the contour is calculated. If the distance is less than 3 pixels, it is considered closed, and the closure is recorded as 1.0; otherwise, it is recorded as 0.0. Finally, the image block shape factor is extracted and calculated as follows: For example, if a region has an area of ​​500 pixels and a perimeter of 100 pixels, then the shape factor is... According to the detection area number A, these four types of numerical features are... The feature extraction process is performed on all regions with thermal anomalies, constructing a structured array and generating a set of region fusion feature vectors.

[0086] S313: Call the set of regional fusion feature vectors, input each feature vector into the trained convolutional neural network structure, load the labeled sample label set of hollow, crack and leakage, perform parallel feature prediction at the edge nodes, and summarize the defect type classification results of each detection area to generate a structural partition disease identification layer.

[0087] Call the region fusion feature vector set to combine the feature vectors of region A. The input is fed into a pre-trained convolutional neural network structure. This network is pre-trained on a dataset containing 10,000 labeled samples and loaded with a set of labeled samples for hollow areas, cracks, and leaks. Hollow areas are labeled 0, cracks are labeled 1, and leaks are labeled 2. Parallel feature prediction is performed at edge nodes. The network output layer is a softmax layer, outputting a vector containing the probabilities of the three types of defects. For example, for the input vector of region A, the network output is... This indicates that the probability of the area being hollow is 92%, the probability of it being a crack is 5%, and the probability of it being a leak is 3%. The defect type classification results of each detection area are summarized, and the category with the highest probability is selected as the defect type of the area. Therefore, area A is classified as hollow. The classification result is bound to the boundary information of area A, and the prediction results of all areas are summarized to generate a structural partition defect identification layer.

[0088] Please see Figure 5 The specific steps for obtaining the boundary coordinate set of building defects are as follows:

[0089] S411: Call the area labels identified as hollow, cracked or leaking in the structural partition disease identification layer, perform coordinate mapping on the pixel position in the original image coordinate system according to the boundary attribute index of each type of disease in the layer, retrieve the set of pixel points corresponding to each disease area, construct the area boundary contour point column, and generate the image disease contour point set.

[0090] The system retrieves the area labels identified as hollow, cracked, or leaking from the structural zoning defect identification layer. Taking area A, previously identified as "hollow," as an example, it uses the boundary attribute index of this type of defect in the layer. This index records the boundary pixel coordinates of the thermal anomaly matrix defined in the zoning temperature difference mapping group (S113). The pixel positions are then mapped in the original image coordinate system, and the system retrieves the set of all boundary pixels corresponding to the hollow defect area in area A. For example, the set might be... , which includes Each boundary point is used to construct a regional boundary contour point set. This means connecting these discrete boundary points into an ordered point sequence according to spatial adjacency. For example, starting from the top left corner point, the nearest next boundary point is connected in a clockwise or counterclockwise direction until the starting point is returned, thus generating an image lesion contour point set.

[0091] S412: Based on the image defect contour point set, extract the two-dimensional coordinate pairs of contour boundary points within the closed region. Then, based on the region boundary connectivity, boundary point density, and enclosing area, use the following formula:

[0092] ;

[0093] The calculation obtains the compactness of the boundary distribution of each diseased area, and determines whether it meets the boundary closure condition. The minimum bounding rectangle of the area that meets the condition is calculated, the coordinate segment enclosed by the boundary is obtained, and the minimum boundary set of the diseased area is generated.

[0094] in, This indicates the pixel position of the boundary point on the horizontal coordinate axis. This indicates the pixel position of the boundary point on the vertical coordinate axis. , These represent the arithmetic mean of the horizontal and vertical pixel positions of all boundary points in the diseased area, respectively, derived from the sum of all... , Sum and then divide by the total number of points Calculations show that This indicates the total number of boundary points in the affected area. This represents the normalized value of the enclosed area of ​​the current region, defined as the ratio of the enclosed area of ​​the current region to the sum of the areas of all diseased regions in the same image. This represents the normalized value of the average Euclidean distance between boundary points, calculated as the ratio of the average distance between each pair of adjacent boundary points to the length of the image diagonal. The boundary compactness index is used to determine whether a boundary constitutes a valid closed structure. The boundary closure condition is set based on the calculated boundary compactness index. Less than the set threshold If the spatial distribution of boundary points in the region is too sparse or the outline is discrete, it is considered that the closure requirement is not met, and this threshold is set. It can be obtained by analyzing the distribution pattern of the boundary structure of conventional disease areas in similar structures and performing statistical fitting. It belongs to the statistical empirical threshold of image structure and has engineering adaptability.

[0095] Based on the image defect contour point set, extract the contour boundary points within the hollow closed area of ​​region A. The two-dimensional coordinate pairs are shown in Table 2;

[0096] Table 2. Example table of coordinates of boundary points in diseased areas:

[0097] ;

[0098] As shown in Table 2, based on the region boundary connectivity, boundary point density, and enclosed area, the following formula is used:

[0099] ;

[0100] The calculation obtains the compactness of the boundary distribution of each diseased area. First, in the formula... and Representing the first The boundary points are located at pixel positions in the horizontal and vertical directions of the image. and It is the arithmetic mean of the coordinates of all boundary points in the diseased area, representing the geometric center of the contour, with the summation sign... This indicates the range from the 1st to the 2nd. Accumulate the values ​​at all boundary points and perform square root calculation on the numerator. Essentially, it calculates the root mean square of the distances from all boundary points to their geometric center, reflecting the overall dispersion of the boundary points. The denominator contains... It is the normalized value of the current enclosed area, while This is the normalized value of the average Euclidean distance between boundary points. The entire formula quantifies the compactness of the boundary by calculating the ratio of the dispersion of the boundary points to the area they enclose and the distance between the points. A compact, regular closed boundary has a smaller numerator and a larger denominator, thus yielding a smaller value. The advantage of this formula lies in its comprehensive consideration of the spatial distribution of boundary points (numerator) and the macroscopic size of the region (through area). (embodied) and microscopic continuity (through point spacing) (This demonstrates) the ability to effectively distinguish between well-formed closed defects and non-defect interference with divergent or discontinuous shapes. Next, the boundary points of region A are calculated, starting with the centroid coordinates:

[0101] ;

[0102] ;

[0103] Calculate the dispersion in the molecule:

[0104] ;

[0105] Calculate the normalized area Assuming there is a crack in area B with an area of ​​300 pixels, and the area enclosed by the hollow area in area A is calculated to be 500 pixels, then the total area of ​​all defects is... Pixel, normalized area of ​​region A Calculate the normalized average spacing First, calculate the average distance between adjacent boundary points, which is approximately 1.2 pixels. The image diagonal length is... pixels, then Substitute the parameter values ​​into the formula:

[0106] ;

[0107] It then determines whether the boundary closure condition is met, and this condition is set based on the boundary compactness index. Less than the set threshold threshold The settings were obtained through statistical analysis of image samples containing 500 typical hollow defects, calculating the value of each sample. The value was found to be 95% of the samples. The value is below 70000, therefore the threshold is set. The value is set to 70000, based on the calculated value. If the value 67050.7 is less than the threshold 70000, then the boundary of the region is considered to meet the closure condition. Calculate the minimum bounding rectangle of the region that meets the condition, and the coordinates of its upper left corner are... The coordinates of the lower right corner are The minimum bounding rectangle coordinate segment of region A is calculated to be... The minimum boundary set of the diseased area is generated. The result shows that the calculated boundary compactness index is within the effective range and constitutes an effective closed structure. Therefore, its minimum bounding rectangle can be extracted as the basis for locating the disease.

[0108] S413: Based on the rectangular coordinate segments corresponding to each defect in the minimum boundary set of the defect area, construct a data table with rectangular vertex coordinate index and defect category label, aggregate it into a unified layer data structure according to the area index order, and establish a building defect boundary coordinate set;

[0109] Based on the rectangular coordinate segments corresponding to each defect in the minimum boundary set of the defect area, taking the hollow area A as an example, the coordinates of the vertices of its minimum bounding rectangle are (55, 90) at the top left and (75, 110) at the bottom right, and the defect category label is "hollow area". A data table record containing this information is constructed, specifically {Area Index: "A", Defect Category: "Hollow Area", Top Left Coordinates: (55, 90), Bottom Right Coordinates: (75, 110)}. The same operation is performed on all other defect areas that are identified and verified as valid boundaries, such as the cracks in area B, to obtain their data records {Area Index: "B", Defect Category: "Cracks", Top Left Coordinates: (200, 150), Bottom Right Coordinates: (205, 300)}. All records are aggregated into a unified layer data structure according to the area index order A, B, C, etc. This data structure is an array or list, and each element is a complete information record of a defect, thus establishing a set of building defect boundary coordinates.

[0110] Please see Figure 6 The specific steps for obtaining information on the distribution of structural defects are as follows:

[0111] S511: Based on the coordinate index and category label data in the building defect boundary coordinate set and the structural partition disease identification layer, the layer visualization rendering interface is called to perform coordinate overlay processing on each structural area, and the disease type label is bound to the corresponding boundary rectangle coordinate to perform layer binding, implement visible mark rendering, and generate a disease label mapping layer dataset.

[0112] Based on the coordinate index and category label data in the building defect boundary coordinate set and the structural zoning defect identification layer, the layer visualization rendering interface is called to perform coordinate overlay processing on each structural region. Specifically, on the original visible light image or infrared thermal image, the coordinates of the rectangle in area A are used to... Draw a semi-transparent red rectangle and bind the disease type label "hollow" to the coordinates of the boundary rectangle using a layer. That is, label the text "hollow" next to or inside the rectangle. For the cracks in area B, use a yellow rectangle and the text "crack" for the same visualization and rendering. Overlay the labeling information of all disease areas onto the base image to generate a disease label mapping layer dataset.

[0113] S512: Based on the index information of each structural region in the disease label mapping layer dataset, extract the number of diseases, the occupied area, and the layer projection area of ​​each region. Calculate the disease frequency, disease density, and area proportion by region using the following formula:

[0114] ;

[0115] The risk level index for each structural partition is obtained through calculation, and risk level labeling and mapping are performed in combination with the original partition number to generate structural risk level assessment results.

[0116] in, Indicators representing the disease risk level of structural zones. This indicates the frequency of diseases identified within the current structural partition, obtained by counting the number of diseased areas in the label mapping layer. This represents the maximum disease frequency value among all structural partitions within the layer, obtained by selecting the maximum value after counting the number of diseases in all partitions. This represents the spatial density value of the disease in the current structural partition, which is equal to the total area of ​​the diseased area divided by the projected area of ​​the partition layer. This represents the highest disease density value in the entire layer, derived from all partitions. Take the maximum value from the list. This represents the total area of ​​all areas surrounded by defects within the current structural partition. This represents the area value corresponding to the partition with the largest total area of ​​disease in the entire layer. This represents the average temperature difference of the diseased areas within the current partition, derived from the temperature mapping matrix of the infrared image. This represents the average temperature difference of the diseased areas across all partitions of this layer.

[0117] Based on the index information of each structural region in the disease label mapping layer dataset, the number of diseases, the occupied area, and the layer projection area of ​​each region are extracted. For example, the entire inspection facade is divided into four regions: A, B, C, and D, and the projection area of ​​each region is [missing information]. Pixel, one hollow blemish was identified in area A, area Pixel, there is one crack defect in area B, area Pixel, there are 2 leakage defects in area C, with areas of respectively Pixels Pixels, total area For pixels, area D is free of disease. Disease frequency, density, and area percentage are calculated by region using the following formula:

[0118] ;

[0119] To calculate and obtain the risk level index for each structural partition, firstly, in the formula... For the final calculated risk level index, This refers to the frequency of diseases within the current partition. It is the highest disease frequency among all zones, the first item. By using square root calculations, the absolute impact of the number of diseases is reduced, and more attention is paid to their relative severity. This is the spatial density of diseases in the current partition. It is the highest disease density in all zones, the second item. Using a logarithmic function to process disease density makes the numerical representation of density differences smoother and avoids the influence of extreme values. This represents the total area of ​​disease within the current zone. This is the average temperature difference in the current diseased areas. and These are the maximum total disease area in all zones and the average temperature difference in all diseased areas, respectively. (The third item...) By combining the area of ​​the disease and the degree of thermal performance anomalies, an energy-related relative severity index was constructed. Finally, the average of these three normalized sub-indices (multiplied by 1 / 3) was obtained to yield a comprehensive risk assessment value. The advantage of this formula lies in its ability to more evenly reflect the frequency, spatial distribution, and physical severity of the disease through nonlinear transformations such as square root and logarithmic transformations, as well as multi-dimensional normalization. The assessment results are more comprehensive and robust. Next, the risk level of area C was calculated. First, the parameters, including disease frequency, were determined. The frequency of all partitions is Then the maximum frequency Disease density The density of each zone is Then the maximum density Total area of ​​disease Pixels, total area of ​​disease in each zone is Then the maximum area The average temperature difference between the two leakage defects in Zone C, as determined by infrared data (S112), is as follows: The average temperature difference for diseases in other areas is , Average temperature difference across all affected areas: ;

[0120] Substitute into the formula to calculate the risk level index of area C. :

[0121] ;

[0122] Combine the original partition number to perform risk level labeling and mapping, for example, to set risk intervals: Low risk Medium risk. If the risk level is high, then the risk level of area C is high. The structural risk level assessment result is generated. The result shows that area C has the highest comprehensive risk. Its number of diseases, area and degree of thermal anomaly are the most serious among all areas. Therefore, it is rated as high risk and needs to be given priority attention.

[0123] S513: Based on the structural risk level assessment results and the area layers that have been classified in the disease label mapping layer dataset, construct a multi-dimensional field structure including disease location, category, density and risk level, combine and generate the layer output data structure and transmit it to the user-end visual component to establish structural disease distribution display information.

[0124] Based on the structural risk level assessment results and the regions in the disease label mapping layer dataset that have been classified, construct a multi-dimensional field structure including disease location, category, density and risk level. Combine this data structure of all regions (A, B, C, D) to generate a complete layer output data structure, package it into JSON or XML format, and pass it to the user-end visual component through the preset API interface to establish structural disease distribution display information.

[0125] A structural surface defect diagnosis system based on multimodal edge computing is used to implement the aforementioned structural surface defect diagnosis method based on multimodal edge computing. The system includes:

[0126] The zoned temperature difference mapping module acquires a sequence of frame images captured by an infrared thermal imager on the surface of the building's exterior wall, maps the pixel grayscale values ​​in the images to the blackbody calibration reference values, and constructs a zoned thermal anomaly matrix based on the division of the building surface area to generate a zoned temperature difference mapping map group.

[0127] The surface enhancement processing module uses visible light image frames of the building wall surface and performs local gray-level histogram statistics on the filtered image through image enhancement. It transforms and reconstructs the gray-level values ​​of each region to rebuild the visual features of the wall surface texture and generate a set of structural surface texture enhancement images.

[0128] The structural defect classification module, based on the partitioned temperature difference mapping map group and the structural surface texture enhancement map group, performs feature fusion on the same spatial coordinate regions in the image, extracts the temperature gradient, gray-level co-occurrence matrix contrast, edge closure degree and shape factor of the region, and performs defect classification by the trained convolutional neural network to generate a structural partitioned defect recognition layer.

[0129] The defect boundary extraction module calls the boundary labels of the defective areas identified in the structural partition disease identification layer, performs position mapping based on the pixel coordinate relationship in the original image, and generates the building defect boundary coordinate set by extracting the closed boundary contour point set;

[0130] The defect distribution display module calls the coordinate and label data from the building defect boundary coordinate set and the structural zone defect identification layer to overlay the defect category and location in each area, assess the defect risk level of each zone, construct a defect statistics layer by combining the numbering information, and call the display interface to output the layer to the user-end visual component to generate structural defect distribution display information.

[0131] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for structural surface disease diagnosis based on multi-modal edge computing, characterized in that, The method comprises the following steps: S1: acquiring a frame image sequence collected by an infrared thermal imager on the surface of an external wall of a building structure, mapping and calculating the pixel gray value in the image with a blackbody calibration reference value, and constructing a partition thermal anomaly matrix graph based on the division of the building surface area to generate a partition temperature difference mapping graph set; S2: according to the visible light image frame of the building wall, performing local gray level histogram statistics on the filtered image through image enhancement, transforming and reconstructing the gray value of each region, reconstructing the visual features of the wall surface texture, and generating a structure surface texture enhancement image set; S3: based on the partition temperature difference mapping graph set and the structure surface texture enhancement image set, performing feature fusion on the same spatial coordinate region in the image, extracting the temperature gradient, gray level co-occurrence matrix contrast, edge closure degree and shape factor of the region, and performing defect classification by a trained convolutional neural network to generate a structure partition disease identification layer; S4: calling the boundary label of the defect region determined in the structure partition disease identification layer, performing position mapping based on the pixel coordinate relationship in the original image, generating a building defect boundary coordinate set by extracting the closed boundary contour point set. The partition temperature difference mapping graph set comprises a region temperature anomaly threshold band, a pixel thermal value spatial distribution grid, a wall surface thermal radiation non-uniformity surface domain and a thermal imaging noise suppression residual region, the structure surface texture enhancement image set comprises a texture image high-frequency detail distribution layer, an image gray balance layer, a texture edge coherence feature graph and a light interference correction region graph, the structure partition disease identification layer comprises a disease type label surface domain, a building component partition index code, a feature vector prediction distribution graph and a convolution feature response activation graph, and the building defect boundary coordinate set comprises a boundary enclosing frame coordinate index, a contour closed structure point array, a structure surface piece geometric reference point and a boundary topology connected mapping.

2. The multi-modal edge computing based structural surface distress diagnosis method according to claim 1, wherein, The acquisition step of the partition temperature difference mapping graph set is specifically: S111: acquiring a frame image sequence collected by an infrared thermal imager on the surface of an external wall of a building structure, calling the spatial resolution parameter and frame rate parameter configured in the image acquisition control end, filtering according to the frame number order and acquisition time interval of each image frame, comparing the image definition index and brightness histogram peak position in the continuous frames, and removing the image frames with definition index lower than the set limit value to generate a structure surface thermal image screening image set; S112: based on the pixel gray value in each image frame in the structure surface thermal image screening image set, calling a gray scale conversion table to map the gray value with the calibration points in the standard blackbody temperature calibration reference table, compensating and correcting the non-calibration pixels through linear interpolation, constructing an image pixel temperature matrix, and recording the interframe temperature floating section according to the image index order to generate a region pixel temperature mapping set; S113: according to the spatial coordinate index of the image frame in the region pixel temperature mapping set, combining the building wall structure division boundary, performing range analysis on the temperature pixel set in each region, determining the thermal anomaly matrix boundary according to the temperature anomaly response threshold, and performing image number index association on the two-dimensional thermal field of the divided region to establish a partition temperature difference mapping graph set. 3.The structural surface disease diagnosis method based on multi-modal edge computing according to claim 2, wherein, The structure surface texture enhancement image group acquisition step specifically comprises: S211: Obtain a visible light image frame of a building wall surface, perform image median filtering processing on each image frame, call a gray value sequence of each pixel point in a filtering window, sort the sequence in ascending order and extract a median value to replace a central pixel gray value, record a replacement number according to a total number of pixels of each image frame, output the processed image frame in the original index order, and generate a structure surface filtered image sequence; S212: Based on the structure surface filtered image sequence, extract a local pixel gray value set of each region in each image frame, construct a gray histogram according to a region boundary, perform normalization processing on a pixel frequency of each gray level in the histogram, calculate a corresponding cumulative distribution value of each gray level, and establish a gray transformation function according to a cumulative distribution function, to generate a partition gray mapping function set; S213: According to the gray transformation function of each region in the partition gray mapping function set, call a corresponding pixel coordinate index, replace a gray value in the structure surface filtered image sequence, merge and output a remapping image frame of each region according to the original image sequence, and label a layer index of a texture enhancement result according to a region number, to generate a structure surface texture enhancement image group. 4.The structural surface disease diagnosis method based on multi-modal edge computing according to claim 3, wherein, The structure partition disease identification layer acquisition step specifically comprises: S311: Based on the partition temperature difference mapping image group and the structure surface texture enhancement image group, call image alignment parameters, perform spatial coordinate transformation and geometric constraint matching on pixel points in infrared image frames and visible light image frames in the same building region, positionally align corresponding frames in the infrared channel and the visual channel, and generate a multi-modal image registration set; S312: According to each pair of image frames in the multi-modal image registration set, extract an infrared channel temperature gradient distribution value, a visual channel gray co-occurrence matrix contrast, a contour boundary closure degree and an image block shape factor in the same region of the image in an edge processing node, construct the four types of numerical features into a structured array according to a detection region number, and generate a region fusion feature vector set; S313: Call the region fusion feature vector set, input each feature vector into a trained convolutional neural network structure, load an annotated hollow, crack and leakage sample label set, perform parallel feature prediction in an edge node, and summarize a defect type classification result of each detection region, to generate a structure partition disease identification layer. 5.The structural surface disease diagnosis method based on multi-modal edge computing according to claim 4, wherein, The building defect boundary coordinate set acquisition step specifically comprises: S411: Call a region label determined as a hollow, a crack or a leakage in the structure partition disease identification layer, perform coordinate mapping on a pixel position in an original image coordinate system according to a boundary attribute index of each type of disease in the layer, retrieve a pixel point set corresponding to each disease region, and construct a region boundary contour point column, to generate an image disease contour point set; S412: According to the image disease contour point set, the two-dimensional coordinate pairs of the closed region inner contour boundary points are extracted, the boundary distribution compactness of each disease region is calculated according to the region boundary connectivity, boundary point density and surrounding area, and whether the boundary closed condition is satisfied is judged, the minimum circumscribed rectangle of the region meeting the condition is calculated, the boundary surrounding coordinate section is obtained, and the minimum boundary set of the disease region is generated; S413: According to the rectangular coordinate section corresponding to each disease in the minimum boundary set of the disease region, a data table with rectangular vertex coordinate index and disease category label is constructed, which is aggregated into a unified layer data structure in order of region index, and a building defect boundary coordinate set is established. 6.The structural surface disease diagnosis method based on multi-modal edge computing according to claim 5, wherein, The method further comprises the following steps: S5: Calling the building defect boundary coordinate set and the coordinate and label data in the structure partition disease identification layer, superimposing and displaying the category and position of the determined disease in each region, evaluating the defect risk level of each partition, constructing a defect statistical layer combined with the number information, calling a display interface to output the layer to a user-side visual component, and generating structure disease distribution display information; The structure disease distribution display information comprises a partition number corresponding layer, a defect category icon identifier, a statistical distribution chart and a layer visual interaction interface.

7. The multi-modal edge computing based structural surface distress diagnosis method according to claim 6, wherein, The acquisition step of the structure disease distribution display information is specifically: S511: Based on the coordinate index and category label data in the building defect boundary coordinate set and the structure partition disease identification layer, calling a layer visualization rendering interface to perform coordinate layering processing on each structure region, and performing layer binding of the disease type label and the corresponding boundary rectangular coordinate to implement visualization marking rendering, and generating a disease label mapping layer data set; S512: According to the index information of each structure region in the disease label mapping layer data set, extracting the number of diseases, the occupied area and the layer projection area in each region, calculating the disease frequency, disease density and area ratio according to the region, calculating the risk level index of each structure partition, and performing risk level label mapping combined with the original partition number to generate a structure risk level evaluation result; S513: According to the structure risk level evaluation result and the region layer in the disease label mapping layer data set which has completed classification, a multi-dimensional field structure including disease position, category, density and risk level is constructed, a layer output data structure is generated and transmitted to a user-side visual component, and structure disease distribution display information is established.

8. A structural surface disease diagnosis system based on multi-modal edge computing, characterized by, The system is used to realize the structure surface disease diagnosis method based on multi-modal edge computing according to any one of claims 1-7, and the system comprises: A partition temperature difference mapping module is configured to obtain a frame image sequence collected by an infrared thermal imager on the surface of a building structure outer wall, map and calculate pixel gray values in the image with blackbody calibration reference values, and construct a partition thermal anomaly matrix based on building surface region division to generate a partition temperature difference mapping group. The surface enhancement processing module is configured to perform local gray histogram statistics on a filtered image through image enhancement according to a visible light image frame of a building wall, transform and reconstruct a gray value of each region, reconstruct visual features of a wall surface texture, and generate a structure surface texture enhancement image group; The structure defect division module is configured to perform feature fusion on regions with same spatial coordinates in the image based on the partition temperature difference mapping image group and the structure surface texture enhancement image group, extract a temperature gradient, a gray co-occurrence matrix contrast, an edge closure degree, and a shape factor of the regions, and perform defect classification on the regions by using a trained convolutional neural network to generate a structure partition defect identification layer; The defect boundary extraction module is configured to call boundary labels of defect regions in the structure partition defect identification layer, perform position mapping based on pixel coordinate relationships in an original image, extract a closed boundary contour point set, and generate a building defect boundary coordinate set; The defect distribution display module is configured to call the building defect boundary coordinate set and coordinate and label data in the structure partition defect identification layer, superimpose and display a type and a position of a defect in each region, evaluate a defect risk level of each partition, construct a defect statistical layer in combination with numbering information, call a display interface to output the layer to a user-side visual component, and generate structure defect distribution display information.

Images