A method and system for intelligently locating roof leakage points based on machine vision

Abstract

The present invention provides a method and system for intelligently locating roof leaks based on machine vision, relating to the field of construction engineering detection technology. The method comprises: Step 1, using a dual-channel sensor device for visible light and infrared thermal imaging to collect roof image data, surface texture, temperature field, and ambient lighting parameters, construct a multidimensional dataset, and generate visual data based on the multidimensional dataset through dynamic range correction; Step 2, based on the visual data, divide the collected image into detection areas, extract local temperature gradients, surface texture mutation rates, and liquid flow trajectory parameters, and establish a multi-index feature matrix. By extracting features from roof images using machine vision, the present invention achieves intelligent positioning of roof leaks, risk classification, and dynamic evolution tracking, thereby improving the accuracy of leak location.

Description

Technical Field

[0001] The present invention relates to the technical field of construction engineering detection, and in particular to a method and system for intelligently locating roof leakage points based on machine vision. Background Art

[0002] When it comes to leak detection on large-span roofs of large storage facilities, traditional positioning technology has some limitations. Traditional manual inspection methods rely on the experience and judgment of inspectors, and there are obvious subjective biases. For example, in the inspection of steel structure factory roofs, when rainwater penetrates through roof gaps, it may flow along hidden paths such as purlins and supporting components, causing the location of water stains indoors to be several meters away from the actual leakage point. If inspectors only judge based on the distribution of indoor water stains, they can easily misjudge the source of the leakage, resulting in insufficient targeting of the repair work, and repeated repairs still cannot completely solve the problem.

[0003] In addition, detection technology based on simple instruments is limited by environmental interference and detection range, and accuracy is difficult to guarantee. In a high-temperature and strong sunlight environment, the temperature distribution on the roof surface is affected by direct sunlight, and the temperature difference between the leakage area and the normal area is masked. It is necessary to move the equipment multiple times to detect each area one by one. This is not only inefficient, but may also lead to missed detection of leakage points in the edge areas due to errors during the movement of the equipment. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for intelligent positioning of roof leakage points based on machine vision, so as to realize intelligent identification and positioning of roof leakage points, improve the efficiency of leakage detection, and reduce manual subjective errors and detection costs.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] In a first aspect, a method for intelligently locating roof leakage points based on machine vision is provided, the method comprising:

[0007] Step 1: Use a dual-channel sensor device for visible light and infrared thermal imaging to collect roof image data, surface texture, temperature field, and ambient lighting parameters, build a multidimensional dataset, and generate visual data based on the multidimensional dataset through dynamic range correction;

[0008] Step 2: Based on the visual data, the collected image is divided into detection areas, local temperature gradient, surface texture mutation rate and liquid flow trajectory parameters are extracted, and a multi-index feature matrix is ​​established;

[0009] Step 3: Based on the multi-index feature matrix, a feature parameter weight system is established. This is combined with threshold determination to analyze the leakage risk of the roof area, classifying suspected leakage areas into different levels and generating risk assessment results. For suspected leakage areas, three fixed benchmark detection points are identified to construct a spatial benchmark framework. The coverage area of ​​the spatial benchmark framework is segmented into multiple sub-regions. Dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions.

[0010] Step 4: Based on the dynamic analysis adjustment value, analyze the pixel-level changes in the visual data to obtain dynamic information about the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area;

[0011] Step 5: Based on the dynamic information of the suspected leakage area, three-dimensional laser scanning is used to construct the three-dimensional spatial structure of the roof surface and generate the spatial location data of the leakage point;

[0012] Step 6: Integrate the leakage risk assessment results, dynamic information of suspected leakage areas, and three-dimensional spatial positioning data to generate a comprehensive report.

[0013] Furthermore, step 2 includes:

[0014] Based on visual data, the collected roof fusion image is divided into multiple rectangular grid detection areas, and the infrared image sub-block and visible light image sub-block corresponding to each detection area are obtained;

[0015] For each infrared image sub-block in the detection area, the temperature values ​​of the pixels in the detection area are extracted, and the temperature difference between adjacent pixels is calculated based on the pixel temperature values ​​to obtain the local temperature gradient parameter that characterizes the severity of the temperature change in the detection area;

[0016] For each visible light image sub-block of the detection area, the grayscale information of the pixels in the detection area is extracted. Based on the grayscale information, the texture feature difference value between the pixel point and the neighboring pixels is calculated to obtain the surface texture mutation rate parameter that characterizes the surface structural abnormality of the detection area.

[0017] Based on the visible light image sub-block of each detection area, a multi-frame image sequence including the detection area is obtained, the displacement vectors of the pixel areas of suspected liquid in the image sequence between adjacent frames are identified and tracked, the liquid flow characteristics are calculated based on the displacement vector sequence, and the liquid flow trajectory parameters that characterize the flow direction and diffusion dynamic characteristics of the liquid in the detection area are obtained;

[0018] For each detection area, the local temperature gradient parameters, surface texture mutation rate parameters and liquid flow trajectory parameters are combined in a predetermined order to obtain a multi-dimensional feature vector representing the comprehensive state of the detection area. The feature vectors are arranged in the order of the detection areas to construct a two-dimensional matrix, and a multi-index feature matrix is ​​established to characterize the state of the entire roof detection area.

[0019] Furthermore, step 3 includes:

[0020] Based on the multi-index characteristic matrix, the correlation between each characteristic parameter and leakage risk is analyzed, and a characteristic parameter weight system is established. Based on the characteristic parameter weight system, each characteristic parameter of each detection area in the multi-index characteristic matrix is ​​calculated to obtain the comprehensive risk assessment value of each detection area;

[0021] Compare the comprehensive risk assessment value with the preset risk level threshold, divide the roof inspection area into suspected leakage areas of different risk levels according to the judgment result, and generate risk assessment results based on the risk level;

[0022] For each suspected leakage area, three fixed and non-collinear reference detection points are determined within the area to construct a spatial reference frame covering the suspected leakage area. The spatial range of the spatial reference frame covering the suspected leakage area is then segmented into multiple sub-regions to obtain a set of second-level rectangular sub-regions.

[0023] For each second-level rectangular sub-region, the geometric distribution characteristics are analyzed, and the area, the average distance from the three vertices to the centroid of the suspected leakage area, and the shape factor are calculated to obtain the geometric distribution characteristic parameters of each second-level rectangular sub-region. For each second-level rectangular sub-region, based on the characteristic parameters of the detection area corresponding to the sub-region position in the multi-index feature matrix, the statistical distribution characteristic quantity of the characteristic parameters within the sub-region is calculated.

[0024] Based on the geometric distribution characteristic parameters of each second-level rectangular sub-region and the internal characteristic difference representation of the calculation sub-region, a dynamic analysis adjustment value is generated.

[0025] Furthermore, the reference detection points include a first detection point located in an inner area of ​​a boundary of a suspected leakage area; a second detection point located in a non-leakage area; and a third detection point located in a relatively uniform spatial distribution within the suspected leakage area.

[0026] Furthermore, based on the dynamic analysis adjustment value, the pixel-level changes in the visual data are analyzed to obtain dynamic information about the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area, including:

[0027] Based on the dynamic analysis adjustment value, a frame-by-frame differential operation of pixel grayscale values ​​and texture features is performed on the temporal visual data to generate a pixel-level change feature matrix that represents the changes in the leakage area;

[0028] Perform gradient direction statistics on the pixel-level change feature matrix, determine the main expansion direction of the suspected leakage area based on the spatial distribution density of the gradient vector, and obtain the expansion direction parameter;

[0029] Based on the expansion direction parameter, the displacement trajectory of the leakage boundary point is tracked in the temporal visual sequence, and the diffusion rate parameter is calculated by the ratio of the boundary point displacement to the time interval between adjacent frames.

[0030] The expansion direction parameter is combined with the diffusion rate parameter, and the curvature sequence comparison is performed on the temporal change of the leakage profile to generate the morphological evolution law parameters that describe the profile deformation law.

[0031] Furthermore, based on the dynamic information of the suspected leakage area, 3D laser scanning is used to construct the 3D spatial structure of the roof surface and generate the spatial location data of the leakage point, including:

[0032] Based on the expansion direction parameters and morphological evolution law parameters in the dynamic information of the suspected leakage area, the target roof area range of the 3D laser scanning is determined, and the 3D laser scanning is performed on the target roof area to obtain the original dense point cloud data of the area;

[0033] Preprocess the original dense point cloud data to generate a processed point cloud dataset, and then generate the three-dimensional surface structure of the target roof area through a surface reconstruction algorithm based on the point cloud dataset;

[0034] The dynamic information of the suspected leakage area is mapped onto the three-dimensional spatial surface structure to locate the three-dimensional spatial distribution position of the leakage activity on the structure. Based on the three-dimensional spatial distribution position and combined with the geometric coordinate information of the three-dimensional spatial surface structure, the precise three-dimensional spatial coordinates of the leakage point are calculated to generate the spatial position data of the leakage point.

[0035] Furthermore, the leakage risk assessment results, dynamic information of suspected leakage areas and three-dimensional spatial positioning data are integrated to generate a comprehensive report, including:

[0036] Obtain leakage risk assessment results, dynamic information of suspected leakage areas, and spatial location data of leakage points;

[0037] Perform coordinate analysis on the spatial location data of leakage points to extract the three-dimensional spatial coordinate set of leakage points; perform grade classification mapping on the leakage risk assessment results to generate risk grade distribution data; perform time series feature extraction on the dynamic information of suspected leakage areas to generate dynamic evolution trajectory data;

[0038] The three-dimensional spatial coordinate set, risk level distribution data and dynamic evolution trajectory data are spatially aligned and attribute-associated to construct a structured leakage dataset. Based on the structured leakage dataset, a comprehensive report is automatically generated, including the three-dimensional spatial distribution index of the leakage point, the spatial distribution index of the risk level and the time series record of the dynamic evolution process of the leakage area.

[0039] The second aspect is a machine vision-based intelligent roof leakage location system, comprising:

[0040] The data acquisition module is used to collect roof image data and surface texture, temperature field and ambient light parameters through a dual-channel sensor device of visible light and infrared thermal imaging, construct a multi-dimensional data set, and generate visual data based on the multi-dimensional data set through dynamic range correction;

[0041] The feature extraction module is used to divide the collected images into detection areas based on visual data, extract local temperature gradients, surface texture mutation rates, and liquid flow trajectory parameters, and establish a multi-index feature matrix;

[0042] The assessment and division module is used to establish a characteristic parameter weight system based on a multi-index characteristic matrix, combine threshold judgment to conduct leakage risk analysis on the roof area, divide the suspected leakage areas into different levels, and generate risk assessment results. For the suspected leakage areas, three fixed reference detection points are determined to construct a spatial reference framework. The coverage area of ​​the spatial reference framework is divided into multiple sub-regions, and dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions.

[0043] A dynamic analysis module is used to analyze pixel-level changes in visual data based on dynamic analysis adjustment values ​​to obtain dynamic information on the expansion direction, diffusion rate, and morphological evolution of suspected leakage areas;

[0044] The spatial positioning module is used to construct the three-dimensional spatial structure of the roof surface using three-dimensional laser scanning based on the dynamic information of the suspected leakage area, and generate the spatial location data of the leakage point;

[0045] The comprehensive information module is used to integrate leakage risk assessment results, dynamic information of suspected leakage areas and three-dimensional spatial positioning data to generate a comprehensive report.

[0046] According to a third aspect, a computing device includes:

[0047] one or more processors;

[0048] The storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0049] In a fourth aspect, a computer-readable storage medium stores a program, which implements the method when executed by a processor.

[0050] The above solution of the present invention includes at least the following beneficial effects:

[0051] The combination of machine vision technology and intelligent algorithms provides an efficient and accurate solution for locating roof leakage points, which has many advantages. In terms of detection accuracy, with the help of high-definition image acquisition and deep learning algorithms to intelligently identify leakage characteristics, it can break through the limitations of traditional manual experience judgment and accurately capture the correlation characteristics between rainwater infiltration paths and leakage points. Even if there is a hidden conduction path between the leakage point and the indoor water stain location, it can effectively reduce positioning deviation and improve the accuracy of leakage point positioning; in terms of detection efficiency, this method can realize rapid scanning and comprehensive detection of large-span roofs. Compared with traditional manual point-by-point inspections or small-scale instrument segmented detection methods, it shortens the detection time and is especially suitable for the detection needs of large-area roofs such as industrial plants and large warehouses, reducing the safety risks brought by long-term high-altitude operations.

[0052] From the perspective of cost control, intelligent positioning reduces excessive reliance on the experience of senior inspectors and reduces the cost of repeated repairs caused by human subjective errors; it avoids the manpower and material resources consumed by multiple round-trip inspections of traditional instruments, and long-term use can reduce the overall cost of roof maintenance; in addition, this method has strong environmental adaptability, and can stably identify roof leakage characteristics regardless of high temperature sunshine or rainy weather, effectively expanding the application scenarios of roof leakage detection and providing reliable technical support for the safe maintenance of various roof structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention provides a flow chart of a method for intelligently locating roof leakage points based on machine vision.

[0054] Figure 2 Schematic diagram of a machine vision-based intelligent roof leakage location system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] like Figure 1 As shown, an embodiment of the present invention provides a method for intelligently locating roof leakage points based on machine vision, the method comprising the following steps:

[0057] Step 1: Use a dual-channel sensor device for visible light and infrared thermal imaging to collect roof image data, surface texture, temperature field, and ambient lighting parameters, build a multidimensional dataset, and generate visual data based on the multidimensional dataset through dynamic range correction;

[0058] Step 2: Based on the visual data, the collected image is divided into detection areas, local temperature gradient, surface texture mutation rate and liquid flow trajectory parameters are extracted, and a multi-index feature matrix is ​​established;

[0059] Step 3: Based on the multi-index feature matrix, a feature parameter weight system is established. This is combined with threshold determination to analyze the leakage risk of the roof area, classifying suspected leakage areas into different levels and generating risk assessment results. For suspected leakage areas, three fixed benchmark detection points are identified to construct a spatial benchmark framework. The coverage area of ​​the spatial benchmark framework is segmented into multiple sub-regions. Dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions.

[0060] Step 4: Based on the dynamic analysis adjustment value, analyze the pixel-level changes in the visual data to obtain dynamic information about the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area;

[0061] Step 5: Based on the dynamic information of the suspected leakage area, three-dimensional laser scanning is used to construct the three-dimensional spatial structure of the roof surface and generate the spatial location data of the leakage point;

[0062] Step 6: Integrate the leakage risk assessment results, dynamic information of suspected leakage areas, and three-dimensional spatial positioning data to generate a comprehensive report.

[0063] In an embodiment of the present invention, during the data acquisition stage, a dual-channel sensing device of visible light and infrared thermal imaging is used to synchronously acquire roof images, textures, temperature fields and environmental parameters, and the visual data generated by the dynamic range correction is combined, which effectively breaks through the limitations of a single sensing method and reduces the interference of environmental factors such as light and temperature on the detection results. By dividing the detection area and constructing a multi-index feature matrix, key parameters such as local temperature gradient, texture mutation rate and liquid flow trajectory are systematically integrated. Compared with the traditional single-index analysis method, it can more accurately capture the characteristic differences of the leakage area, make the leakage feature identification more targeted, and reduce the risk of missed detection due to feature omission; establish a feature parameter weight system and combine it with threshold judgment to perform leakage risk analysis, cooperate with the construction of the spatial reference framework and the multi-level sub-region segmentation technology, not only can it scientifically divide the suspected leakage areas of different levels , and also realizes the refined processing of the detection area through dynamic analysis adjustment value, improves the accuracy of leakage risk assessment and the accuracy of regional positioning, and avoids the problem of vague judgment of suspected areas in traditional detection; based on dynamic information, three-dimensional laser scanning is used to construct the three-dimensional structure of the roof surface, and the dynamic evolution law of the leakage point is accurately associated with the spatial position, realizing the leap from plane feature analysis to three-dimensional spatial positioning, making the spatial coordinates of the leakage point more intuitive and clear, and solving the problem of large deviation between the leakage point and the actual position in the traditional method; the comprehensive report generated by multi-dimensional data integration integrates risk assessment, dynamic evolution and three-dimensional positioning information, provides a comprehensive basis for maintenance decision-making, reduces the cost waste caused by blind maintenance, and at the same time improves detection efficiency, shortens the cycle from detection to maintenance, and reduces the structural safety hazards caused by long-term roof leakage.

[0064] In a preferred embodiment of the present invention, step 1 above, using a dual-channel sensor device of visible light and infrared thermal imaging to collect roof image data and surface texture, temperature field, and ambient lighting parameters, construct a multidimensional dataset, and generate visual data based on the multidimensional dataset through dynamic range correction, may include:

[0065] In the embodiment of the present invention, the visible light and infrared thermal imaging dual-channel sensing device is first started to complete the initialization calibration of the equipment. The operator will first perform spatial position calibration on the two sensing devices to ensure that the detection fields of the visible light camera and the infrared thermal imager completely overlap to avoid spatial deviation in data matching; at the same time, time synchronization calibration is performed to ensure that the data collected by the two sensors at the same time point can accurately correspond through the built-in synchronous clock module of the device, ensuring that the texture information and temperature data at the same moment can be referenced to each other; after entering the data acquisition stage, the dual-channel sensing device performs a comprehensive scan of the roof according to the preset path, and the visible light camera continuously captures roof images at a fixed frame rate, one by one. The frames record the physical characteristics of the roof surface, including the integrity of tiles, the distribution of cracks, the diffusion range of stains and other surface texture details. Each frame of the image contains clear pixel coordinates and color information; the infrared thermal imager synchronously collects the temperature values ​​of various points on the roof, generates real-time temperature field distribution data, and converts the infrared radiation signal into a specific temperature value through the built-in temperature sensor, covering the temperature changes of each detection area on the roof; at the same time, the environmental sensing module captures the lighting parameters in real time, including light intensity, light angle, direct sunlight area and other environmental information. Each set of environmental data corresponds to a specific collection time and spatial location, forming a complete record of environmental parameters.

[0066] After the collection is completed, the data preprocessing stage begins. First, the original data is screened for quality, and blurred images or abnormal temperature values ​​caused by equipment jitter and sudden changes in light are eliminated to ensure the validity of the data set; then dynamic range correction is carried out. For visible light images, the pixel brightness of overly bright areas is adjusted through a brightness equalization algorithm, and the detail display of dark areas is enhanced, so that the roof texture remains clearly visible under different lighting conditions; for infrared temperature field data, temperature correction is performed in combination with the synchronously collected ambient light parameters, and the additional temperature increment caused by direct sunlight is deducted to eliminate the interference of light on temperature detection, so that the temperature field data can more realistically reflect the actual temperature distribution of the roof; finally, the corrected visible light texture data is spatially fused with the infrared temperature data, and the texture features of the same spatial position are associated with the temperature information through coordinate matching technology to form multi-dimensional visual data including texture details, temperature distribution, and environmental parameters.

[0067] In a preferred embodiment of the present invention, the above step 2, based on the visual data, divides the collected image into detection areas, extracts local temperature gradients, surface texture mutation rates, and liquid flow trajectory parameters, and establishes a multi-index feature matrix, which may include:

[0068] Step 220: Based on the visual data, the collected roof fusion image is divided into a plurality of rectangular grid detection areas, and infrared image sub-blocks and visible light image sub-blocks corresponding to each detection area are obtained;

[0069] Step 221 , for each infrared image sub-block of the detection area, extract the temperature values ​​of the pixels within the detection area, calculate the temperature difference between adjacent pixels based on the pixel temperature values, and obtain a local temperature gradient parameter that characterizes the severity of the temperature change in the detection area;

[0070] Step 222 , for each visible light image sub-block within the detection area, extract the grayscale information of the pixels within the detection area, calculate the texture feature difference value between the pixel point and the neighboring pixels based on the grayscale information, and obtain a surface texture mutation rate parameter that characterizes the surface structural abnormality of the detection area;

[0071] Step 223 , based on the visible light image sub-block of each detection area, a multi-frame image sequence including the detection area is obtained, displacement vectors of pixel areas suspected of liquid in the image sequence between adjacent frames are identified and tracked, and liquid flow characteristics are calculated based on the displacement vector sequence to obtain liquid flow trajectory parameters that characterize the flow direction and diffusion dynamic characteristics of the liquid in the detection area;

[0072] In step 224, for each detection area, the local temperature gradient parameters, surface texture mutation rate parameters, and liquid flow trajectory parameters are combined in a predetermined order to obtain a multi-dimensional feature vector representing the comprehensive state of the detection area. The feature vectors are arranged in the order of the detection areas to construct a two-dimensional matrix, and a multi-index feature matrix is ​​established to characterize the state of the entire roof detection area.

[0073] In an embodiment of the present invention, before dividing the detection area, the fused image of the roof is preprocessed to remove blurry areas at the image edges caused by the shooting angle, ensuring that the image portion involved in the division is complete and clear. When determining the grid size for division, the actual size of the roof and the image resolution are taken into consideration. For example, if the roof is actually 10 meters wide and 20 meters long, and the image resolution is 1000×2000 pixels, each rectangular grid detection area can be set to 20×20 pixels (corresponding to an actual area of ​​0.2 meters by 0.2 meters). Starting from the first pixel in the upper left corner of the fused image of the roof, marking lines are drawn at intervals of 20 pixels horizontally and 20 pixels vertically. The rectangular area formed by the intersection of these marking lines is the detection area. After each detection area is determined, the pixel range that completely corresponds to the location of the area is cut out from the infrared image as an infrared image sub-block. Simultaneously, the pixel range at the same location is cut out from the visible light image as a visible light image sub-block. Each sub-block is labeled with the corresponding detection area number and coordinate position.

[0074] After obtaining the infrared image sub-block of each detection area, the image sub-block is first subjected to noise reduction processing to remove isolated abnormal temperature pixels caused by equipment interference. Then, each pixel in the sub-block is scanned row by row and column by column, and the specific temperature value of each pixel is recorded. For example, the temperature value of a certain pixel is 32.5°C. Taking the current pixel as the center, check the four adjacent pixels directly above, below, to the left, and to the right. If the pixel is at the edge of the sub-block and lacks adjacent pixels in a certain direction, only the temperature difference in the existing direction is calculated. For example, if the current pixel temperature is 32.5°C and the pixel to the right is 29.8°C, the temperature difference between the two is 2.7°C. The temperature differences of all adjacent pixels in the sub-block are collected, and the maximum and minimum values ​​of these differences, as well as the proportion of differences in different intervals (such as 0-1°C, 1-3°C, and above 3°C) are counted. The maximum temperature difference value and the interval value with the highest proportion of temperature difference intervals are combined to form the local temperature gradient parameter of the detection area. If the maximum temperature difference is 5°C and the interval above 3°C accounts for 30%, it means that the temperature change in this area is relatively drastic.

[0075] When processing visible light image sub-blocks, grayscale normalization is performed first to ensure the consistency of grayscale values ​​under different lighting conditions. Then, grayscale information is extracted pixel by pixel. The grayscale value of each pixel is represented by an integer between 0 and 255. For each pixel, a 3×3 neighborhood range around it is selected, that is, a square area formed by extending one pixel up, down, left, and right with the pixel as the center. The grayscale difference between the center pixel and the 8 pixels in the neighborhood is calculated. For example, the grayscale value of the center pixel is 150 and the grayscale value of the upper pixel is 80, then the texture feature difference value between the two is 70. The preset threshold is set to 25. When a difference value exceeds 25, it is recorded as a texture mutation. The number of texture mutations in each pixel neighborhood is counted. For example, if 3 difference values ​​of 8 neighboring pixels exceed the threshold, the texture mutation ratio of the pixel is , sum up the texture mutation ratios of all pixels in the entire image sub-block, and then divide it by the total number of pixels in the sub-block. The average value obtained is the surface texture mutation rate parameter of the detection area.

[0076] When acquiring a multi-frame image sequence containing the detection area, ensure that the shooting time interval between two adjacent frames is the same, for example, one frame is shot every 1 second, and a total of 30 frames of images are obtained to form a sequence. First, perform inter-frame alignment on the image sequence to eliminate image offset caused by slight shaking of the shooting equipment. In the first frame of the image, by identifying the area with a pixel grayscale value significantly lower than the surrounding area and with a continuous boundary, mark it as a suspected liquid pixel area. In the second frame of the image, find the position of the suspected liquid area and calculate the number of pixels moved in the horizontal and vertical directions between the centers of the two areas, for example, 5 pixels moved horizontally and 3 pixels moved vertically. This is the displacement vector. Repeat this operation for each two adjacent frames to obtain a series of displacement vectors. Observe the direction of these displacement vectors. If most vectors point to the southeast, it is determined that the main flow direction of the liquid is southeast; calculate the average length of the displacement vector per unit time, for example, an average movement of 4 pixels per second, to determine the liquid flow speed, and obtain the liquid flow trajectory parameters by combining the flow direction and speed.

[0077] For each detection area, the specific values ​​of the three parameters are first clarified. For example, the local temperature gradient parameter is 4.2, the surface texture mutation rate parameter is 0.25, and the liquid flow trajectory parameter is comprehensively described by direction and speed as "southeast, 3 pixels per second". According to the fixed order of local temperature gradient, surface texture mutation rate, and liquid flow trajectory, these three parameters are arranged in sequence to form the multi-dimensional feature vector of the detection area. After completing the construction of the feature vectors of all detection areas, according to the arrangement order of the detection areas on the roof, that is, starting from the first area in the upper left corner, the first row is arranged horizontally, and then the second row, the third row, and so on are arranged in sequence. Each feature vector is written in sequence as a row of the matrix. The final two-dimensional matrix has the same number of rows as the total number of detection areas and 3 columns. Each cell corresponds to a specific parameter value, which fully characterizes the comprehensive state of the entire roof detection area.

[0078] When dividing the inspection area, machine vision can ensure that each area is of consistent size and has precise boundaries through preset fixed sizes and strict segmentation rules, avoiding problems of inconsistent area sizes and blurred boundaries caused by subjective judgment during manual division. During the parameter extraction process, the temperature and grayscale values ​​of each pixel are collected and calculated point by point, without missing any subtle changes, so that every link of the inspection process from area division to parameter extraction is traceable and consistent, thereby improving the precision of inspection. When extracting local temperature gradient parameters, machine vision calculates adjacent temperature differences pixel by pixel and statistically distributes the characteristics, avoiding inaccurate temperature change judgments caused by deviations in measurement point selection during manual measurement. When extracting the surface texture mutation rate, the calculation is based on a fixed neighborhood range and threshold standard, eliminating subjective deviations in texture mutation judgments caused by differences in experience when manually observing textures. The extraction of liquid flow trajectory parameters is achieved through multi-frame tracking and displacement calculation. It is not affected by factors such as lack of concentration during manual tracking, making the extraction results of various parameters more objective and accurate. It can simultaneously monitor three types of roof characteristics: temperature changes, surface texture, and liquid flow, covering multiple dimensions of possible roof problems. For abnormal conditions that are difficult for humans to detect, such as slight temperature fluctuations, subtle texture mutations, and slow liquid flow, machine vision can capture them in time through precise parameter calculation. The multi-indicator feature matrix presents the status of each area of ​​the roof in a unified format. The parameters of each area are recorded according to the same calculation standards and arrangement order, making the status comparison between different areas more intuitive and convenient. Whether it is a horizontal comparison of different areas of the same roof or a vertical comparison of the same area at different detection times, it can be based on standardized matrix data, avoiding the comparison difficulties caused by different description methods and parameter standards during manual recording, and providing reliable data support for long-term monitoring and trend analysis of roof status.

[0079] In a preferred embodiment of the present invention, step 3 above establishes a feature parameter weight system based on a multi-index feature matrix, performs leakage risk analysis on the roof area in combination with threshold determination, divides suspected leakage areas into different levels, and generates risk assessment results. For the suspected leakage area, three fixed reference detection points are determined to construct a spatial reference framework, and the coverage area of ​​the spatial reference framework is divided into multiple sub-regions. Based on the geometric distribution characteristics and internal feature differences of the sub-regions, dynamic analysis adjustment values ​​are generated, which may include:

[0080] Step 330: Based on the multi-index feature matrix, the correlation between each characteristic parameter and the leakage risk is analyzed, a characteristic parameter weight system is established, and based on the characteristic parameter weight system, each characteristic parameter of each detection area in the multi-index feature matrix is ​​calculated to obtain a comprehensive risk assessment value for each detection area;

[0081] Step 331: Compare the comprehensive risk assessment value with a preset risk level threshold, divide the roof inspection area into suspected leakage areas of different risk levels according to the determination result, and generate a risk assessment result based on the risk level;

[0082] Step 332: For each suspected leakage area, three fixed and non-collinear reference detection points are determined within the area to construct a spatial reference frame covering the suspected leakage area. The spatial range of the spatial reference frame covering the suspected leakage area is segmented into multiple sub-regions to obtain a set of second-level rectangular sub-regions. Specifically, the reference detection points include: a first detection point located within the boundary of the suspected leakage area; a second detection point located in a non-leakage area; and a third detection point located in a relatively evenly distributed area within the suspected leakage area.

[0083] Step 333: For each second-level rectangular sub-region, the geometric distribution characteristics are analyzed, and the area, the average distance from the three vertices to the centroid of the suspected leakage area, and the shape factor are calculated to obtain the geometric distribution characteristic parameters of each second-level rectangular sub-region. For each second-level rectangular sub-region, based on the characteristic parameters of the detection area corresponding to the sub-region position in the multi-index feature matrix, the statistical distribution characteristic quantity of the characteristic parameters within the sub-region is calculated.

[0084] Step 334 : Generate a dynamic analysis adjustment value based on the geometric distribution characteristic parameters of each second-level rectangular sub-region and the internal feature difference representation of the calculation sub-region.

[0085] In an embodiment of the present invention, with the support of machine vision, the multi-index feature data of each detection area of ​​the roof is first obtained through the image acquisition equipment, and a multi-index feature matrix is ​​constructed. These feature parameters cover the moisture distribution of the roof, surface texture changes, structural defects, etc. Based on this multi-index feature matrix, the correlation between each feature parameter and the leakage risk is analyzed, and the image is processed by machine vision to identify the performance of each feature parameter in the leakage case, and these feature parameters are correlated with the known leakage situation for analysis. The Harris Hawk optimization algorithm plays a creative role in this link, simulating the Harris Hawk's precise locking and evaluation mechanism of the target when hunting, treating each feature parameter as an "eagle" and the leakage risk target as "prey", and tracking the "moving trajectory" of each feature parameter in different leakage cases through machine vision, that is, the change of the feature parameter, and analyzing it. Analyze the degree of correlation between these "trajectories" and the occurrence of leakage risks to determine the correlation. When determining the weight coefficient, just like a group of Harris's hawks working together to hunt prey, let each characteristic parameter undergo a similar "collaborative" evaluation. For characteristic parameters with high correlation with leakage risk, give them a higher initial weight, just like Harris's hawks give priority to the direction of prey that is easier to capture. Then, by adjusting the hunting strategy in a similar way to a group of hawks, the initial weight is optimized and adjusted, and finally the weight coefficient of each characteristic parameter is determined, and a characteristic parameter weight system is established. Afterwards, based on this weight system, a weighted calculation is performed on the characteristic parameters of each detection area in the multi-indicator feature matrix. Machine vision locates each detection area, extracts the numerical values ​​of the characteristic parameters corresponding to the area, and then multiplies them by their respective weight coefficients. All products are added together to obtain the comprehensive risk assessment value of each detection area.

[0086] Determine preset thresholds of different risk levels. These thresholds are set based on a large amount of roof leakage data and engineering experience, and by analyzing the characteristics of leakage areas and non-leakage areas. Then, compare the comprehensive risk evaluation value of each detection area obtained in step 330 with these preset thresholds. Machine vision compares the comprehensive risk evaluation value of each detection area with different levels of thresholds one by one. According to the comparison results, the roof detection area is divided into suspected leakage areas of different risk levels. For example, when the comprehensive risk evaluation value is higher than the highest threshold, it is divided into a high-risk suspected leakage area; when it is in the middle threshold range, it is divided into medium risk; when it is lower than the lowest threshold, it is divided into low risk. Finally, based on the location information of the suspected leakage area identified by machine vision and the determined risk level, a risk assessment result containing the specific location of the leakage and the corresponding risk level information is generated.

[0087] For each suspected leakage area, machine vision first conducts a comprehensive scan and analysis of the area. When three reference detection points with fixed spatial positions and non-collinearity are determined in the area, machine vision uses image recognition and positioning technology to find that the first detection point is located in the internal area of ​​the boundary of the suspected leakage area. This position is selected in the internal area surrounded by the boundary by identifying the boundary line of the area; the second detection point is in the non-leakage area, and machine vision compares the characteristics of the leakage area and selects it in the area without leakage characteristics; the third detection point is relatively evenly distributed in the suspected leakage area, and the uniformity of the feature distribution in the area is analyzed to select a position with balanced distribution; after these three points are constructed into a spatial reference frame covering the suspected leakage area, the spatial range of the spatial reference frame is segmented into multiple sub-regions. During the segmentation process, the idea of ​​the Harris Hawk optimization algorithm is integrated. Just like the Harris Hawk carefully searches for prey in the hunting area, machine vision gradually refines the spatial range. It first performs preliminary segmentation to obtain larger sub-regions, and then further segments these sub-regions to finally obtain a second-level rectangular sub-region set.

[0088] For each second-level rectangular sub-region, machine vision begins to analyze its geometric distribution characteristics. When calculating the area, it identifies the boundary pixels of the sub-region, counts the number of pixels surrounded by the boundary, and then calculates the actual area of ​​the sub-region based on the conversion ratio between pixels and actual area. When calculating the average distance from the three vertices to the center of mass of the suspected leakage area, the center of mass position of the suspected leakage area is first determined by calculating the average value of all pixel coordinates in the area. Then, the distance from the three vertices of the sub-region to the center of mass is calculated separately. These three distances are added and divided by 3 to obtain the average distance. When calculating the shape factor, it is calculated based on the area and perimeter of the sub-region through a specific proportional relationship. The shape factor can reflect the degree of shape regularity of the sub-region. For the calculation of the statistical distribution feature quantity of the characteristic parameters within the sub-region, machine vision locates the characteristic parameters of the detection area corresponding to the sub-region position based on the multi-index feature matrix. By performing statistical analysis on these characteristic parameters, the statistical distribution feature quantities such as maximum value, minimum value, average value, and variance are calculated to characterize the distribution of the characteristic parameters within the sub-region.

[0089] Based on the geometric distribution characteristic parameters and internal feature difference representation of each second-level rectangular sub-area, machine vision generates dynamic analysis adjustment values ​​in combination with predefined rules. The Harris Hawk optimization algorithm is integrated here. Just like the Harris Hawk adjusts its hunting strategy according to the dynamic changes of its prey, machine vision dynamically adjusts the analysis value according to the geometric characteristics and feature differences of the sub-area. The predefined rules are based on a large number of experiments and data summaries. When the area in the geometric distribution characteristic parameters is large, the average distance is far, and the shape factor is small, combined with the large internal feature difference, a larger dynamic analysis adjustment value is generated; conversely, when the geometric distribution characteristic parameters are relatively stable and the internal feature difference is small, a smaller dynamic analysis adjustment value is generated. In this way, the generation of dynamic analysis adjustment values ​​is achieved.

[0090] Through comprehensive and detailed image acquisition and analysis of the roof inspection area through machine vision, the characteristic parameters of each inspection area can be accurately extracted. The characteristic parameter weight coefficient is determined by combining the Harris Eagle optimization algorithm, so that the weight system is more in line with the actual leakage risk situation. The comprehensive risk evaluation value obtained by weighted calculation can more truly reflect the leakage risk of the inspection area, thereby improving the accuracy of risk assessment, making the division of different risk levels more reasonable and reliable, and being able to accurately identify the boundaries and various features of the roof inspection area. When dividing the suspected leakage area, the position of the suspected leakage area of ​​different risk levels can be accurately determined by comparison with the preset threshold value. The determination of the three benchmark detection points is based on the positioning function of machine vision to ensure the accuracy and coverage of the spatial benchmark framework, making the positioning of the leakage position more accurate, and the multi-level sub-area segmentation in the machine With the support of vision, suspected leakage areas can be divided into detailed areas, which facilitates in-depth analysis of each sub-area. The calculation of the geometric distribution characteristic parameters and the statistical distribution characteristic quantities of the internal characteristic parameters allows the characteristics of each sub-area to be fully displayed. The dynamic analysis adjustment value generated by the Harris Eagle optimization algorithm can be dynamically adjusted according to the actual situation of the sub-area, which improves the reliability and pertinence of the risk assessment. The application of machine vision realizes the automated collection and analysis of the roof inspection process, reduces manual intervention, and improves inspection efficiency. From feature parameter extraction, weight system establishment to risk assessment result generation, the entire process is automatically completed through machine vision and related algorithms, which not only saves a lot of manpower and time costs, but also avoids the subjective errors that may occur in manual inspection, making the inspection results more objective and stable.

[0091] In a preferred embodiment of the present invention, step 4, analyzing pixel-level changes in the visual data based on the dynamic analysis adjustment value to obtain dynamic information on the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area, may include:

[0092] Step 440 , based on the dynamic analysis adjustment value, performing frame-by-frame differential operations on pixel grayscale values ​​and texture features on the time series visual data to generate a pixel-level change feature matrix representing changes in the leakage area;

[0093] Step 441 , performing gradient direction statistics on the pixel-level change feature matrix, determining the main expansion direction of the suspected leakage area based on the spatial distribution density of the gradient vector, and obtaining the expansion direction parameter;

[0094] Step 442 , based on the expansion direction parameter, tracking the displacement trajectory of the leakage boundary point in the temporal visual sequence, and calculating the diffusion rate parameter by the ratio of the boundary point displacement to the time interval between adjacent frames;

[0095] In step 443 , the expansion direction parameter is combined with the diffusion rate parameter, and a curvature sequence comparison is performed on the temporal variation of the leakage profile to generate a morphological evolution law parameter describing the profile deformation law.

[0096] In an embodiment of the present invention, continuously captured time-series visual data are collected. These data are images of the same monitoring area at different time points. Each image contains complete pixel information of the area. At the same time, a dynamic analysis adjustment value is determined. This value is a correction value pre-calculated based on interference factors such as changes in light intensity and slight shaking of the equipment that may exist in the monitoring environment, and is used to reduce the impact of these interferences on the calculation. Then, a frame-by-frame pixel grayscale value difference operation is performed, and two adjacent frames of images are taken from the time-series image, such as the Nth frame and the N+1th frame. For each pixel point in the Nth frame image, the pixel point with a completely corresponding position in the N+1th frame image is found, and the grayscale values ​​of these two pixel points are read respectively (the grayscale value is a value indicating the brightness of the pixel, and the range is usually between 0-255). Then, the grayscale value of the pixel in the N+1th frame image is subtracted from the grayscale value of the corresponding pixel in the Nth frame image to obtain the grayscale value difference result of this pixel point.

[0097] In terms of frame-by-frame differential operation of texture features, the texture features of each pixel in each frame of the image are first extracted. Specifically, the image details within the 3x3 pixel range around each pixel are observed, such as counting the maximum and minimum grayscale values ​​of the pixels in this range and the difference between them. Attention is also paid to the changes in the grayscale values ​​of adjacent pixels to describe the texture features of the pixel. Afterwards, for two adjacent frames of images, the texture features of the corresponding pixels are compared and the difference between the two is calculated. For example, the texture feature data of the pixels in the N+1 frame is subtracted from the texture feature data of the corresponding pixels in the N frame to obtain the texture feature differential result. Finally, the grayscale value differential result and the texture feature differential result of each pixel are arranged in the order of the pixel position in the image to form a matrix with the same number of pixels as the original image, namely the pixel-level change feature matrix. Each element in the matrix accurately reflects the grayscale and texture changes of the corresponding pixel.

[0098] First, the pixel-level change feature matrix generated in step 440 is obtained. This matrix completely records the change feature data of each pixel. Then, gradient direction statistics are performed on the pixel-level change feature matrix. For each pixel in the matrix, the change feature values ​​of its eight adjacent pixels (upper and lower, left and right, upper left, upper right, lower left, and lower right) are checked. By comparing the difference in the change feature values ​​of the current pixel with those of the adjacent pixels, the direction in which the change feature value of the pixel increases fastest is determined. This direction is the gradient direction of the pixel. Then, the gradient directions of all pixels are counted. The entire matrix is ​​divided into multiple small areas, for example, the image can be divided into 10x10 small square areas. In each small area, the number of pixels with different gradient directions is counted to determine the spatial distribution of the gradient vector in the area. Then, the main expansion direction is determined based on the spatial distribution density of the gradient vectors. The proportion of the number of gradient vectors in each possible direction is calculated. The direction with the highest proportion is the direction with the most densely distributed gradient vectors in the area. By combining the conditions of all small areas, the direction with the most densely distributed gradient vectors in the entire suspected leakage area is found. This direction is determined as the main expansion direction of the suspected leakage area, and the expansion direction parameter is obtained.

[0099] First, obtain the expansion direction parameter obtained in step 441. This parameter clarifies the main expansion direction of the suspected leakage area. In the temporal visual sequence, first determine the leakage boundary point, observe the boundary line between the suspected leakage area and the normal area in each frame image, and each pixel point on the boundary line is the leakage boundary point. Based on the expansion direction parameter, track the displacement trajectory of these boundary points in two adjacent frames. For example, mark the position of a boundary point in the Nth frame image, and then find the position of the boundary point after moving along the main expansion direction in the N+1th frame image, record this displacement trajectory, and calculate Calculate the displacement of the boundary points between adjacent frames. That is, measure the pixel distance between the position of the boundary point in the N+1th frame image and the corresponding boundary point in the Nth frame image. At the same time, determine the time interval between the two adjacent frames, that is, the time difference between the shooting of the Nth frame and the N+1th frame image. For example, the shooting interval between the two frames is 0.5 seconds. Finally, divide the displacement of the boundary point between adjacent frames (pixel distance) by the time interval (seconds) to obtain the diffusion rate of the boundary point. Calculate the diffusion rates of multiple boundary points and take the average of these rates as the diffusion rate parameter for the entire suspected leakage area.

[0100] First, the expansion direction parameter obtained in step 441 and the diffusion rate parameter obtained in step 442 are integrated. These two parameters reflect the change of leakage in terms of direction and speed, respectively. Time-series variation data of the leakage profile are extracted. In each frame of the image, the boundary points of the suspected leakage area are connected into a line to form a leakage profile. The position information of each point on the leakage profile in each frame of the image is recorded to obtain the leakage profile shape in different time frames. The time-series variation of the leakage profile is then compared by performing curvature sequence comparison. For the leakage profile of each time frame, multiple points on the profile are selected. For each point, the positions of its preceding and following adjacent points are observed. The curvature of the point is determined by comparing the degree of curvature of the curve formed by these three points. For example, if the curve formed by the three points is more curved, the curvature is larger; if the curve is more gentle, the curvature is smaller. The curvature sequences of different time frames are compared to observe the change of the curvature of each point on the profile in the main expansion direction as the diffusion rate changes. For example, is the curvature change more obvious in the direction with faster diffusion rate? Based on these comparison results, morphological evolution law parameters that can describe the temporal variation of the leakage profile shape are generated.

[0101] The calculation process based on machine vision can realize pixel-level analysis, and carefully calculate the grayscale and texture changes of each pixel point, without missing any subtle changes, reducing missed detections and misjudgments caused by careless manual observation. By accurately determining the expansion direction, calculating the diffusion rate and generating parameters of the morphological evolution law, it can accurately capture the various changing characteristics of the leakage, making the detection results more reliable. The frame-by-frame analysis method of time-series visual data can track the changes in the leakage area in real time. From generating the pixel-level change feature matrix to calculating various parameters, the entire process is carried out quickly based on a continuous image sequence, which can obtain the latest dynamic information of the leakage in a timely manner, allowing staff to understand the development of the leakage at the first time, providing a powerful tool for taking timely treatment measures. Support, traditional manual inspection requires staff to view images frame by frame, which consumes a lot of time and energy, while the method based on machine vision can automatically complete the processing and analysis of large amounts of image data without excessive human intervention. It can quickly calculate and analyze each pixel point and complete the processing of multiple frames of images in a short time, which improves the efficiency of leakage detection and saves labor costs. By generating parameters such as expansion direction, diffusion rate and morphological evolution law, it can comprehensively and clearly display the development process and change law of leakage. These parameters provide detailed data support for analyzing the cause of leakage and predicting the future development trend of leakage, so that staff can formulate more targeted and effective governance plans based on this accurate information, thereby improving the treatment effect of leakage problems.

[0102] In a preferred embodiment of the present invention, the above step 5, based on the dynamic information of the suspected leakage area, constructing the three-dimensional spatial structure of the roof surface by three-dimensional laser scanning to generate the spatial location data of the leakage point, may include:

[0103] Step 550: Determine the target roof area for 3D laser scanning based on the expansion direction parameter and morphological evolution law parameter in the dynamic information of the suspected leakage area, and perform 3D laser scanning on the target roof area to obtain original dense point cloud data of the area.

[0104] Step 551 , pre-processing the original dense point cloud data to generate a processed point cloud dataset, and generating a three-dimensional surface structure of the target roof area through a surface reconstruction algorithm based on the point cloud dataset;

[0105] Step 552 maps the dynamic information of the suspected leakage area onto the three-dimensional curved surface structure, locates the three-dimensional spatial distribution position of the leakage activity on the structure, and calculates the precise three-dimensional spatial coordinates of the leakage point based on the three-dimensional spatial distribution position and the geometric coordinate information of the three-dimensional curved surface structure to generate the spatial position data of the leakage point.

[0106] In an embodiment of the present invention, when determining the target roof area range for three-dimensional laser scanning, the roof is firstly captured in all directions with the help of machine vision, and a high-definition industrial camera is used to shoot roof images according to a preset path to obtain a two-dimensional image sequence covering the entire roof. These images are then stitched together to extract feature points of each image, such as corner points and edge points. The similarity between the feature points is calculated to determine the overlapping areas between the images, and then the multiple images are stitched together into a complete panoramic image of the roof based on the corresponding relationship between the pixel coordinates of the overlapping areas.

[0107] Based on the expansion direction parameters and morphological evolution law parameters in the dynamic information of the suspected leakage area, the area is delineated on the panoramic image. The machine vision will analyze the expansion direction of the suspected leakage area in the image. For example, by comparing the position changes of the edge pixels of the leakage area in images taken at different times, the main and secondary directions of its expansion are determined. At the same time, according to the evolution law of the leakage area morphology over time, such as the rate of area increase and shape change characteristics, the possible range boundaries of the leakage area are calculated. On the panoramic image, the pixel area of ​​the target roof area is determined according to the pixel coordinates corresponding to these boundaries. Then, combined with the intrinsic and extrinsic parameters of the camera, the pixel area is converted into the area range of the actual physical space, thereby guiding the three-dimensional laser scanning equipment to scan the area and obtain the original dense point cloud data.

[0108] When preprocessing the original dense point cloud data, the image denoising method based on machine vision analyzes the image performance corresponding to the original point cloud data. By detecting the noise points in the image, which are usually isolated points with large grayscale values ​​different from the surrounding pixels, the grayscale average value of each pixel and its neighboring pixels is calculated, and the pixels whose grayscale values ​​deviate from the average value by more than a preset threshold are marked as noise points and removed. Outliers appear in the image as pixels far away from the main point cloud cluster. By calculating the average distance from each pixel to a certain number of neighboring pixels around it, the pixels with a distance greater than a preset threshold are judged as outliers and removed. Redundant data mainly refers to repeated or overly dense points. By analyzing the distribution density of pixels in the image, the distance between adjacent pixels is calculated. When the distance is less than the preset value, the point cloud data corresponding to one of the pixels is retained, and other redundant data are removed to generate a processed point cloud dataset.

[0109] When generating a 3D surface structure based on a point cloud dataset, machine vision first extracts features from the image corresponding to the processed point cloud data, acquiring features such as the point cloud data's contours and texture. These features are then used to determine the spatial connections between the point cloud data. A surface reconstruction algorithm is then applied to gradually construct the 3D surface structure of the target roof area based on the spatial coordinates and connections of the point cloud data. During the construction process, the surface shape is continuously adjusted to match the distribution of the point cloud data, ensuring that the generated surface structure accurately reflects the actual roof's form.

[0110] When mapping the dynamic information of a suspected leakage area onto a three-dimensional surface structure, machine vision first converts the position information of the suspected leakage area in the two-dimensional image into a coordinate reference in three-dimensional space. By analyzing the pixel coordinates of the suspected leakage area in the two-dimensional image and combining them with the camera's calibration parameters, the approximate coordinate range of these pixel points in three-dimensional space is calculated. Then, based on the geometric characteristics of the three-dimensional surface structure, the dynamic information of the suspected leakage area is accurately mapped onto the surface structure to locate the three-dimensional spatial distribution of the leakage activity on the structure. To calculate the precise three-dimensional spatial coordinates of the leakage point, machine vision determines the center position of the leakage point by comparing the image features of the leakage area on the surface structure with the feature differences of the surrounding normal area. Then, combined with the geometric coordinate information of the three-dimensional surface structure, the precise coordinates of this center position in three-dimensional space, i.e., the precise three-dimensional spatial coordinates of the leakage point, are calculated, and the spatial position data of the leakage point is generated. During the calculation process, the coordinate calculation results are continuously optimized, and the accuracy of the coordinates is ensured through multiple verifications.

[0111] The calculation process based on machine vision can perform detailed analysis and processing of roof images, accurately identify and remove noise points, outliers and redundant data, and reduce the impact of interference factors on the detection results. In the process of determining the scope of the target roof area, generating three-dimensional spatial surface structures, and locating leakage points, accurate pixel analysis and coordinate conversion can be used to accurately locate the leakage point, greatly improving the accuracy of the spatial position data of the leakage point, making the detection results more reliable, and being able to quickly collect and process roof image data, realizing automatic scanning and analysis of the roof area. Compared with traditional manual detection methods, there is no need to manually check each area of ​​the roof one by one, saving a lot of manpower and time costs, and in data prediction During the processing and surface reconstruction process, through automated algorithm operations, the processed point cloud data set and three-dimensional spatial surface structure can be quickly generated, which speeds up the detection process and improves the overall detection efficiency. The roof is fully imaged, covering every corner of the roof, avoiding missed areas that may occur in manual detection. When analyzing the dynamic information of suspected leakage areas, the expansion direction and morphological evolution law of the leakage area can be comprehensively considered to fully grasp the development trend of leakage activities and ensure that no possible leakage points are missed. The generated three-dimensional spatial surface structure can fully present the shape of the roof, making the positioning of leakage points more comprehensive and intuitive, which is conducive to the overall assessment of roof leakage conditions.

[0112] In a preferred embodiment of the present invention, the above step 6 integrates the leakage risk assessment results, the dynamic information of the suspected leakage area and the three-dimensional spatial positioning data to generate a comprehensive report, which may include:

[0113] Step 660: Obtain leakage risk assessment results, dynamic information of suspected leakage areas, and spatial location data of leakage points;

[0114] Step 661: Coordinate analysis is performed on the spatial location data of the leakage point to extract the three-dimensional spatial coordinate set of the leakage point; classification and mapping of the leakage risk assessment results are performed to generate risk level distribution data; and time series feature extraction is performed on the dynamic information of the suspected leakage area to generate dynamic evolution trajectory data.

[0115] In step 662, the three-dimensional spatial coordinate set, risk level distribution data, and dynamic evolution trajectory data are spatially aligned and attribute-associated to construct a structured leakage data set. Based on the structured leakage data set, a comprehensive report is automatically generated, including the three-dimensional spatial distribution index of the leakage point, the risk level spatial distribution index, and the time series record of the dynamic evolution process of the leakage area.

[0116] In an embodiment of the present invention, when obtaining the leakage risk assessment results, the machine vision system first calls the image acquisition device to continuously shoot the monitored area at a fixed frequency to obtain a series of original images. Then, each original image is preprocessed, the brightness value of the image is checked pixel by pixel, and pixels with brightness values ​​lower than the standard brightness threshold are marked as dark area pixels. The number and proportion of dark area pixels in each image are counted. At the same time, the color characteristics of the pixels in the image are analyzed, and pixels with colors close to the preset leakage characteristic color (such as dark color, water stain color) are screened out. The distribution density of these pixels in the image is calculated, and the dark area pixel ratio and the characteristic color pixel distribution density are comprehensively compared with the preset risk assessment benchmark value to preliminarily determine the risk assessment parameters of each area, which are summarized to form the initial data of the leakage risk assessment results.

[0117] When obtaining dynamic information of suspected leakage areas, the continuously shot image sequence is numbered in chronological order and a fixed time interval is set, for example, one frame of image is extracted as a key frame every 10 seconds. For two adjacent key frame images, the pixels at corresponding positions are compared row by row and column by column. The numerical difference of each pixel is calculated. When the sum of the numerical differences of the three channels exceeds the set color difference threshold, the pixel is marked as a changed pixel. The total number of changed pixels in each key frame image is counted. When the number of changed pixels exceeds the regional area threshold in a certain continuous area, the area is determined to be a suspected dynamic area. The pixel coordinates of the upper left and lower right corners of the area in the image, the time of appearance, and the number of changed pixels compared with the previous key frame are recorded. All such information is collected to form the dynamic information of suspected leakage areas.

[0118] When obtaining the spatial position data of the leakage point, the machine vision system activates the binocular camera synchronous shooting function to ensure that the two cameras shoot the leakage area at the same time, obtaining two left and right images with different perspectives, and correcting the two images to eliminate the error caused by camera lens distortion. By identifying fixed reference points in the image (such as pipe interfaces, equipment corners, etc.), the pixel coordinates of the reference points in the left and right images are determined. For the identified leakage point, its pixel coordinates in the left image and the pixel coordinates in the right image are found, and the horizontal difference between the two coordinates is calculated. According to the known parameters of the camera, such as the actual distance between the two cameras (baseline length), the focal length of the camera, etc., the offset distance of the leakage point in the horizontal direction relative to the camera is determined. Then, by analyzing the vertical position of the leakage point in the image and combining the installation height and pitch angle of the camera, the vertical height of the leakage point is calculated, and finally the spatial position data of the leakage point is obtained through integration.

[0119] When performing coordinate analysis on the spatial position data of the leakage point, the pixel coordinates corresponding to each leakage point in the image are first extracted from the spatial position data of the leakage point. Then, based on the intrinsic parameter data of the camera, including the focal length of the lens, pixel size, etc., the pixel coordinates are converted into coordinates in the camera coordinate system. For example, the horizontal coordinates are calculated according to the number of columns of pixels in the image, and the vertical coordinates are calculated according to the number of rows of pixels in the image. Then, combined with the focal length parameters, they are converted into actual length units. Next, using the external parameter data of the camera, such as the installation position coordinates and rotation angle of the camera in the world coordinate system, the coordinates in the camera coordinate system are converted into three-dimensional coordinates in the world coordinate system. This conversion operation is performed on each leakage point, and all the obtained three-dimensional coordinates are organized into an ordered set, that is, a three-dimensional spatial coordinate set of the leakage point is generated.

[0120] When mapping the leakage risk assessment results into grade classifications, first set the interval range for the risk level division, such as the numerical range of the assessment parameters corresponding to low risk, medium risk, and high risk respectively. Then, extract each assessment parameter value in the leakage risk assessment result one by one and compare it with the set interval range. If an assessment parameter value falls within the low-risk interval, the area corresponding to the parameter is marked as a low-risk level; if it falls within the medium-risk interval, it is marked as a medium-risk level; similarly, determine the high-risk level area, and count the number, area proportion, and other data of each risk level area within the monitoring range. Associate these data with the corresponding regional location information to generate risk level distribution data.

[0121] When extracting temporal features from the dynamic information of suspected leakage areas, the machine vision system arranges the dynamic information of the suspected leakage areas in chronological order. For each suspected area, its image features at different time points are extracted, such as the boundary pixel coordinates of the area, the average brightness value of the pixels in the area, etc., and the change in the boundary pixel coordinates at adjacent time points is calculated to determine the movement direction and distance of the area; the change difference in the average brightness value is calculated to judge the light and dark change trend of the area, and these change amounts, change trends and other data are recorded in chronological order. At the same time, the shape change characteristics of the area at each time point are marked, such as whether it is expanded, shrunk, deformed, etc., and the dynamic evolution trajectory data is generated after integration.

[0122] When performing spatial alignment, the three-dimensional spatial coordinate set is used as the reference coordinate system. For risk level distribution data, according to the pixel position of each risk level area in the image, the coordinate conversion formula (based on camera parameters) is used to convert the pixel position into the spatial position in the reference coordinate system to ensure that the spatial position of the risk level area matches the position in the three-dimensional spatial coordinate set. For dynamic evolution trajectory data, the pixel position of the suspected area corresponding to each time point is also converted into the spatial position in the reference coordinate system, so that the area positions at different time points are unified into the same spatial coordinate system, completing the spatial alignment operation.

[0123] During the attribute association process, a unique identification code is assigned to each coordinate point in the three-dimensional spatial coordinate set, and the risk level information of the corresponding spatial position in the risk level distribution data, such as low risk, medium risk, and high risk, is bound and stored with the identification code of the coordinate point at that position. For the dynamic evolution trajectory data, the change feature data of each time point, such as the moving distance, shape change description, etc., are associated with the identification code of the coordinate point in the area at the corresponding time point. Through this binding association method, the various data of three-dimensional spatial coordinates, risk level and dynamic evolution trajectory are integrated to construct a structured leakage data set.

[0124] When automatically generating a comprehensive report based on a structured leakage data set, the three-dimensional spatial coordinate set is first sorted and arranged in order according to the x-, y-, and z-axis values ​​of the coordinates to establish an index directory. Each index item corresponds to the three-dimensional spatial coordinates and related basic information of a leakage point, forming a three-dimensional spatial distribution index of the leakage point. The risk level distribution data is classified according to the risk level, and the corresponding area information is listed in spatial position order under each risk level to generate a risk level spatial distribution index. For the dynamic evolution trajectory data, the position changes, shape changes, risk level changes, etc. of the leakage area at each time point are recorded in chronological order to form a time series record of the dynamic evolution process of the leakage area. Finally, these three parts are integrated and formatted to automatically generate a comprehensive report.

[0125] Through meticulous analysis and calculation of image pixels, various leakage-related features, such as pixel color changes and regional shape changes, can be accurately captured. Compared with manual observation, this avoids errors caused by visual fatigue and subjective judgment bias, making the obtained leakage risk assessment results, suspected area dynamic information and other data more accurate. Traditional leakage assessment requires manual review of images and recording of data one by one, which consumes a lot of time and manpower. The machine vision system can automatically process large amounts of image data continuously. The entire process from data acquisition to report generation does not require human intervention, shortening the data processing time. At the same time, it can complete the simultaneous monitoring and analysis of multiple areas in a short period of time, improving the overall efficiency of leakage assessment work; it can capture and analyze the monitoring area in all directions and without blind spots, without missing any subtle signs of leakage. Through time series feature extraction, it can track the dynamic changes of suspected leakage areas in real time and accurately record the evolution trajectory of the area. This comprehensive and dynamic monitoring method enables staff to grasp the development trend of leakage in a timely manner and make prevention and treatment preparations in advance.

[0126] like Figure 2 As shown, an embodiment of the present invention further provides a roof leakage point intelligent positioning system based on machine vision, comprising:

[0127] The data acquisition module is used to collect roof image data and surface texture, temperature field and ambient light parameters through a dual-channel sensor device of visible light and infrared thermal imaging, construct a multi-dimensional data set, and generate visual data based on the multi-dimensional data set through dynamic range correction;

[0128] The feature extraction module is used to divide the collected images into detection areas based on visual data, extract local temperature gradients, surface texture mutation rates, and liquid flow trajectory parameters, and establish a multi-index feature matrix;

[0129] The assessment and division module is used to establish a characteristic parameter weight system based on a multi-index characteristic matrix, combine threshold judgment to conduct leakage risk analysis on the roof area, divide the suspected leakage areas into different levels, and generate risk assessment results. For the suspected leakage areas, three fixed reference detection points are determined to construct a spatial reference framework. The coverage area of ​​the spatial reference framework is divided into multiple sub-regions, and dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions.

[0130] A dynamic analysis module is used to analyze pixel-level changes in visual data based on dynamic analysis adjustment values ​​to obtain dynamic information such as the expansion direction, diffusion rate, and morphological evolution law of the suspected leakage area;

[0131] The spatial positioning module is used to construct the three-dimensional spatial structure of the roof surface using three-dimensional laser scanning based on the dynamic information of the suspected leakage area, and generate the spatial location data of the leakage point;

[0132] The comprehensive information module is used to integrate leakage risk assessment results, dynamic information of suspected leakage areas and three-dimensional spatial positioning data to generate a comprehensive report.

[0133] It should be noted that this system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0134] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the above-described method. All implementations in the above-described method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0135] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the above-described method. All implementations in the above-described method embodiment are applicable to this embodiment and can achieve the same technical effects.

[0136] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for intelligently locating roof leakage points based on machine vision, characterized in that: The method comprises: Step 1: Use a dual-channel sensor device for visible light and infrared thermal imaging to collect roof image data, surface texture, temperature field, and ambient lighting parameters, build a multidimensional dataset, and generate visual data based on the multidimensional dataset through dynamic range correction; Step 2: Based on the visual data, the collected image is divided into detection areas, local temperature gradient, surface texture mutation rate and liquid flow trajectory parameters are extracted, and a multi-index feature matrix is ​​established; Step 3: Based on the multi-index feature matrix, a feature parameter weight system is established. This is combined with threshold determination to analyze the leakage risk of the roof area, classifying suspected leakage areas into different levels and generating risk assessment results. For suspected leakage areas, three fixed benchmark detection points are identified to construct a spatial benchmark framework. The coverage area of ​​the spatial benchmark framework is segmented into multiple sub-regions. Dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions. Step 4: Based on the dynamic analysis adjustment value, analyze the pixel-level changes in the visual data to obtain dynamic information about the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area; Step 5: Based on the dynamic information of the suspected leakage area, three-dimensional laser scanning is used to construct the three-dimensional spatial structure of the roof surface and generate the spatial location data of the leakage point; Step 6: Integrate the leakage risk assessment results, dynamic information of suspected leakage areas, and three-dimensional spatial positioning data to generate a comprehensive report.

2. The method for intelligently locating roof leakage points based on machine vision according to claim 1, characterized in that: Step 2 includes: Based on visual data, the collected roof fusion image is divided into multiple rectangular grid detection areas, and the infrared image sub-block and visible light image sub-block corresponding to each detection area are obtained; For each infrared image sub-block in the detection area, the temperature values ​​of the pixels in the detection area are extracted, and the temperature difference between adjacent pixels is calculated based on the pixel temperature values ​​to obtain the local temperature gradient parameter that characterizes the severity of the temperature change in the detection area; For each visible light image sub-block of the detection area, the grayscale information of the pixels in the detection area is extracted. Based on the grayscale information, the texture feature difference value between the pixel point and the neighboring pixels is calculated to obtain the surface texture mutation rate parameter that characterizes the surface structural abnormality of the detection area. Based on the visible light image sub-block of each detection area, a multi-frame image sequence including the detection area is obtained, the displacement vectors of the pixel areas of suspected liquid in the image sequence between adjacent frames are identified and tracked, the liquid flow characteristics are calculated based on the displacement vector sequence, and the liquid flow trajectory parameters that characterize the flow direction and diffusion dynamic characteristics of the liquid in the detection area are obtained; For each detection area, the local temperature gradient parameters, surface texture mutation rate parameters and liquid flow trajectory parameters are combined in a predetermined order to obtain a multi-dimensional feature vector representing the comprehensive state of the detection area. The feature vectors are arranged in the order of the detection areas to construct a two-dimensional matrix, and a multi-index feature matrix is ​​established to characterize the state of the entire roof detection area.

3. The method for intelligently locating roof leakage points based on machine vision according to claim 2, characterized in that: Step 3 includes: Based on the multi-index characteristic matrix, the correlation between each characteristic parameter and leakage risk is analyzed, and a characteristic parameter weight system is established. Based on the characteristic parameter weight system, each characteristic parameter of each detection area in the multi-index characteristic matrix is ​​calculated to obtain the comprehensive risk assessment value of each detection area; The comprehensive risk assessment value is compared with the preset risk level threshold, and the roof inspection area is divided into suspected leakage areas of different risk levels according to the judgment result. The risk assessment result is generated based on the risk level; For each suspected leakage area, three fixed and non-collinear reference detection points are determined within the area to construct a spatial reference frame covering the suspected leakage area. The spatial range of the spatial reference frame covering the suspected leakage area is then segmented into multiple sub-regions to obtain a set of second-level rectangular sub-regions. For each second-level rectangular sub-region, the geometric distribution characteristics are analyzed, and the area, the average distance from the three vertices to the centroid of the suspected leakage area, and the shape factor are calculated to obtain the geometric distribution characteristic parameters of each second-level rectangular sub-region. For each second-level rectangular sub-region, based on the characteristic parameters of the detection area corresponding to the sub-region position in the multi-index feature matrix, the statistical distribution characteristic quantity of the characteristic parameters within the sub-region is calculated. Based on the geometric distribution characteristic parameters of each second-level rectangular sub-region and the internal characteristic difference representation of the calculation sub-region, a dynamic analysis adjustment value is generated.

4. The method for intelligently locating roof leakage points based on machine vision according to claim 3, characterized in that: The reference detection points include a first detection point located in the inner area of ​​the boundary of the suspected leakage area; a second detection point located in the non-leakage area; and a third detection point located in a relatively uniformly distributed area within the suspected leakage area.

5. The method for intelligently locating roof leakage points based on machine vision according to claim 4, characterized in that: Based on the dynamic analysis adjustment value, the pixel-level changes in the visual data are analyzed to obtain dynamic information about the expansion direction, diffusion rate, and morphological evolution of the suspected leakage area, including: Based on the dynamic analysis adjustment value, a frame-by-frame differential operation of pixel grayscale values ​​and texture features is performed on the temporal visual data to generate a pixel-level change feature matrix that represents the changes in the leakage area; Perform gradient direction statistics on the pixel-level change feature matrix, determine the main expansion direction of the suspected leakage area based on the spatial distribution density of the gradient vector, and obtain the expansion direction parameter; Based on the expansion direction parameter, the displacement trajectory of the leakage boundary point is tracked in the temporal visual sequence, and the diffusion rate parameter is calculated by the ratio of the boundary point displacement to the time interval between adjacent frames. The expansion direction parameter is combined with the diffusion rate parameter, and the curvature sequence comparison is performed on the temporal change of the leakage profile to generate the morphological evolution law parameters that describe the profile deformation law.

6. The method for intelligently locating roof leakage points based on machine vision according to claim 5, characterized in that: Based on the dynamic information of the suspected leakage area, 3D laser scanning is used to construct the 3D spatial structure of the roof surface and generate the spatial location data of the leakage point, including: Based on the expansion direction parameters and morphological evolution law parameters in the dynamic information of the suspected leakage area, the target roof area range of the 3D laser scanning is determined, and the 3D laser scanning is performed on the target roof area to obtain the original dense point cloud data of the area; Preprocess the original dense point cloud data to generate a processed point cloud dataset, and then generate the three-dimensional surface structure of the target roof area through a surface reconstruction algorithm based on the point cloud dataset; The dynamic information of the suspected leakage area is mapped onto the three-dimensional spatial surface structure to locate the three-dimensional spatial distribution position of the leakage activity on the structure. Based on the three-dimensional spatial distribution position and combined with the geometric coordinate information of the three-dimensional spatial surface structure, the precise three-dimensional spatial coordinates of the leakage point are calculated to generate the spatial position data of the leakage point.

7. The method for intelligently locating roof leakage points based on machine vision according to claim 6, characterized in that: Integrate leakage risk assessment results, dynamic information on suspected leakage areas, and 3D spatial positioning data to generate a comprehensive report, including: Obtain leakage risk assessment results, dynamic information of suspected leakage areas, and spatial location data of leakage points; Perform coordinate analysis on the spatial location data of leakage points to extract the three-dimensional spatial coordinate set of leakage points; perform grade classification mapping on the leakage risk assessment results to generate risk grade distribution data; perform time series feature extraction on the dynamic information of suspected leakage areas to generate dynamic evolution trajectory data; The three-dimensional spatial coordinate set, risk level distribution data and dynamic evolution trajectory data are spatially aligned and attribute-associated to construct a structured leakage dataset. Based on the structured leakage dataset, a comprehensive report is automatically generated, including the three-dimensional spatial distribution index of the leakage point, the spatial distribution index of the risk level and the time series record of the dynamic evolution process of the leakage area.

8. A machine vision-based intelligent positioning system for roof leakage points, the system implementing the method according to any one of claims 1 to 7, characterized in that: include: The data acquisition module is used to collect roof image data and surface texture, temperature field and ambient light parameters through a dual-channel sensor device of visible light and infrared thermal imaging, construct a multi-dimensional data set, and generate visual data based on the multi-dimensional data set through dynamic range correction; The feature extraction module is used to divide the collected images into detection areas based on visual data, extract local temperature gradients, surface texture mutation rates, and liquid flow trajectory parameters, and establish a multi-index feature matrix; The assessment and classification module is used to establish a characteristic parameter weight system based on a multi-index characteristic matrix, conduct leakage risk analysis on the roof area in combination with threshold judgment, divide suspected leakage areas into different levels, and generate risk assessment results; For suspected leakage areas, three fixed benchmark detection points are determined to construct a spatial benchmark framework. The coverage area of ​​the spatial benchmark framework is divided into multiple sub-regions. Dynamic analysis adjustment values ​​are generated based on the geometric distribution characteristics and internal feature differences of the sub-regions. A dynamic analysis module is used to analyze pixel-level changes in visual data based on dynamic analysis adjustment values ​​to obtain dynamic information such as the expansion direction, diffusion rate, and morphological evolution law of the suspected leakage area; The spatial positioning module is used to construct the three-dimensional spatial structure of the roof surface using three-dimensional laser scanning based on the dynamic information of the suspected leakage area, and generate the spatial location data of the leakage point; The comprehensive information module is used to integrate leakage risk assessment results, dynamic information of suspected leakage areas and three-dimensional spatial positioning data to generate a comprehensive report.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images