Wind turbine tower foundation detection method and system based on industrial vision

Abstract

This invention relates to the field of foundation inspection technology, specifically to a method and system for inspecting wind turbine pile foundations based on industrial vision. The method includes: acquiring multi-view industrial vision images of the wind turbine pile foundation; identifying the main pile foundation objects in the multi-view industrial vision images and analyzing the surface texture features and structural deformation features corresponding to the main pile foundation objects; determining target detection areas, including those with abnormal surface texture features and structural deformation features, from multiple detection areas corresponding to the multi-view industrial vision images; constructing a three-dimensional point cloud model of the pile foundation based on panoramic grayscale images and depth images from each viewpoint; and determining the elevation data corresponding to each area of ​​the foundation surface based on the three-dimensional point cloud model. This invention, through the acquisition and processing of multi-view industrial vision images, can accurately identify the texture features and structural deformation features of the pile foundation surface, thereby effectively detecting initial defect areas.

Description

Technical Field

[0001] This invention relates to the field of foundation testing technology, specifically to a method and system for testing the foundation of wind turbine pile foundations based on industrial vision. Background Technology

[0002] Currently, traditional methods often require a lot of manual intervention. Operators need to manually perform visual inspections and record data, which increases the impact of human factors on the results and is inefficient. Moreover, traditional detection methods lack automation technology support, making it impossible to achieve real-time and continuous monitoring, resulting in long detection cycles and slow response speeds. Furthermore, traditional methods usually rely on visual observation and simple tools, making it difficult to achieve high-precision defect identification, which is prone to missed or false detections, leading to difficulties in subsequent maintenance and management.

[0003] Furthermore, traditional methods are difficult to conduct effective inspections under complex or harsh environmental conditions, which limits their application scope. Moreover, traditional methods often rely on single images or simple measurement tools, lacking the ability to deeply analyze image data and failing to fully utilize the collected information. Additionally, for different types of pile foundations or environmental conditions, traditional methods usually require redesigning and adjusting the inspection process, resulting in poor adaptability and low efficiency. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting the foundation of wind turbine pile bases based on industrial vision, comprising: Acquire multi-view industrial visual images of wind turbine pile foundations, identify the main objects of the pile foundations in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main objects of the pile foundations. Target detection areas, including surface texture features and structural deformation features, are identified from multiple detection areas corresponding to multi-view industrial vision images; a three-dimensional point cloud model of the pile foundation is constructed based on panoramic grayscale images and depth images from various viewpoints, and the elevation data corresponding to each area of ​​the foundation surface is determined based on the three-dimensional point cloud model; Based on the spatial position sequence of multiple target detection regions in a multi-view industrial vision image, the adjacent second target detection regions of each first target detection region are determined, and based on the feature distribution position range corresponding to each target detection region, the corresponding first feature distribution position range in the first target detection region and the corresponding second feature distribution position range in the second target detection region are determined. The second feature distribution location range is determined to correspond to the third feature distribution location range in the first target detection area, and the first feature distribution location range and the third feature distribution location range are determined as the key detection focus location range in the first target detection area; High-precision scanning analysis is performed on the key detection focus area, and the detection area other than the key detection focus area is subjected to hierarchical scanning analysis according to the corresponding scanning analysis precision; based on the hierarchical scanning analysis results and the elevation data corresponding to the three-dimensional point cloud model, the initial defect area and defect depth in each target detection area are determined.

[0005] Preferably, after determining the initial defect region and defect depth in each target detection region, the method further includes: In a continuous multi-view image sequence, determine multiple key image frames corresponding to each initial defect region and a first preset number of adjacent image frames corresponding to each key image frame. Determine the surface texture similarity between each adjacent image frame and the corresponding key image frame, and determine the reference image frame corresponding to each key image frame in the first preset number of adjacent image frames; In a continuous multi-view image sequence, determine the comparison verification image frame corresponding to the benchmark reference image frame for each key image frame; Based on the surface texture similarity and elevation data between the benchmark reference image frame and the corresponding comparison verification image frame corresponding to each key image frame, the initial defect areas are verified to determine the real defect areas.

[0006] Preferably, determining the comparison verification image frame corresponding to the benchmark reference image frame for each key image frame in a continuous multi-view image sequence includes: In a continuous multi-view image sequence, at least one complementary view image frame is identified where the shooting view difference between the reference image frame corresponding to each key image frame satisfies the first view condition. In at least one complementary viewpoint image frame, identify the corresponding contrast verification image frame with the highest surface texture similarity to the reference image frame corresponding to each key image frame. The acquisition time of each complementary image frame is later than the acquisition time of the corresponding reference image frame. Furthermore, when the shooting height of the complementary image frame is the same as or similar to that of the corresponding reference image frame, the time difference between the acquisition time of the complementary image frame and the acquisition time of the corresponding reference image frame satisfies the preset time condition.

[0007] Preferably, based on the surface texture similarity and elevation data between the baseline reference image frame and the corresponding comparison verification image frame corresponding to each key image frame, each initial defect region is verified to determine the true defect region, including: Under the condition that the surface texture similarity between the baseline reference image frame and the corresponding comparison verification image frame for each key image frame satisfies the second similarity condition; Determine the first texture similarity between the previous image frame of the reference image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; Determine the first elevation difference between the reference image frame corresponding to each key image frame and the previous image frame of the corresponding reference image frame; Determine the second elevation difference between the comparison verification image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; And determine the third elevation difference between the previous image frame of the reference image frame corresponding to each key image frame and the corresponding comparison verification image frame; When at least one of the following conditions is met: the first texture similarity satisfies the third similarity condition; the difference between the first elevation difference and the second elevation difference satisfies the second elevation condition; and the difference between the first elevation difference and the third elevation difference satisfies the third elevation condition, the corresponding initial defect region is confirmed as a real defect region.

[0008] Preferably, acquiring multi-view industrial visual images of the wind turbine pile foundation includes: The target scanning camera and its scanning resolution are determined from industrial cameras fixed at different heights around the foundation based on panoramic grayscale images. Based on the scanning parameters of the target scanning camera, acquire depth images of the pile foundation from various viewpoints; Control the drone equipped with industrial vision acquisition equipment to collect a continuous multi-view image sequence of the pile foundation according to a preset flight path.

[0009] Preferably, determining the target scanning camera and its scanning resolution from industrial cameras fixed at different heights around the foundation based on panoramic grayscale images includes: Texture detection is performed on the panoramic grayscale image of the pile foundation to determine the surface condition of the foundation. If there are no obvious cracks or peeling on the foundation surface, the target scanning camera shall include at least low-height and medium-height industrial cameras; otherwise, the target scanning camera shall include at least low-height, medium-height and high-height industrial cameras. Calculate the texture complexity of the foundation surface based on panoramic grayscale images; If the texture complexity is less than or equal to the preset texture complexity threshold, the scanning resolution of the target scanning camera is set to the preset resolution value; otherwise, the target scanning resolution value of each target scanning camera is adaptively calculated based on the panoramic grayscale image.

[0010] Preferably, after determining the target detection region, including surface texture features and structural deformation feature anomalies, from multiple detection regions corresponding to multi-view industrial visual images, the method further includes: The target detection region is segmented to obtain the image segmentation result, and the interference object in the target detection region is judged according to the image segmentation result to determine whether it meets the preset exclusion rule; If the interfering object meets the preset exclusion rules, the interfering object will be removed to obtain the target detection area after removal.

[0011] Preferably, high-precision scanning analysis is performed on the key detection focus area, and graded scanning analysis is performed on the detection position areas other than the key detection focus area according to the corresponding scanning analysis precision, including: The comprehensive scan analysis task to be executed is broken down into multiple sub-scan tasks, and these sub-scan tasks are set in the task queue. The sub-scanning tasks include a first sub-scanning task that scans and analyzes the key detection focus area according to a first specified scanning method, and a second sub-scanning task that scans and analyzes the detection area other than the key detection focus area in the first target detection area according to a second specified scanning method. Based on task priority and dependencies, multiple sub-scan tasks are dynamically allocated to multiple processing units, and these sub-scan tasks are executed by the multiple processing units.

[0012] Preferably, a 3D point cloud model of the pile foundation is constructed based on panoramic grayscale images and depth images from various viewpoints, including: Feature points are extracted from depth images at each viewpoint, and the feature points are evenly distributed in the convex and concave regions of the depth images. The feature points of the depth images from each viewpoint are used for registration to obtain the transformation matrix corresponding to each depth image from each viewpoint. The point cloud data corresponding to the depth images from different viewpoints are stitched together according to the transformation matrix to obtain a three-dimensional point cloud model.

[0013] The industrial vision-based wind turbine pile foundation inspection system is applicable to the aforementioned industrial vision-based wind turbine pile foundation inspection method, including: The feature extraction module is configured to acquire multi-view industrial visual images of wind turbine pile foundations, identify the main objects of the pile foundations in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main objects of the pile foundations. The 3D modeling module is configured to determine target detection areas, including surface texture features and structural deformation features, from multiple detection areas corresponding to multi-view industrial vision images; construct a 3D point cloud model of the pile foundation based on panoramic grayscale images and depth images from various viewpoints, and determine the elevation data corresponding to each area of ​​the foundation surface based on the 3D point cloud model; The range detection module is configured to determine the adjacent second target detection regions of each first target detection region based on the spatial position sequence of multiple target detection regions in a multi-view industrial vision image, and to determine the first feature distribution position range in the first target detection region and the second feature distribution position range in the second target detection region based on the feature distribution position range corresponding to each target detection region. The detection focusing module is configured to determine the third feature distribution location range corresponding to the second feature distribution location range in the first target detection area, and to determine the first feature distribution location range and the third feature distribution location range as the key detection focusing location range in the first target detection area; The scanning analysis module is configured to perform high-precision scanning analysis on the key detection focus area and to perform graded scanning analysis on the detection location area other than the key detection focus area according to the corresponding scanning analysis precision; based on the graded scanning analysis results and the elevation data corresponding to the 3D point cloud model, the initial defect area and defect depth in each target detection area are determined.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention acquires and processes multi-view industrial vision images, enabling accurate identification of texture and structural deformation features on the surface of pile foundations. This effectively detects initial defect areas, and high-precision scanning analysis and hierarchical scanning further verify and confirm the accuracy of the actual defect areas. Furthermore, by combining industrial vision technology and automated image processing, it can automatically perform detection and analysis with the assistance of equipment such as drones, reducing manual intervention and improving detection efficiency. Moreover, by decomposing the comprehensive scanning analysis task into multiple sub-tasks and dynamically allocating them, it improves resource utilization efficiency and processing speed. This invention, through its target scanning camera selection and scanning resolution adjustment mechanism, enables the detection system to adaptively adjust according to actual conditions, thereby adapting to different wind turbine pile foundation environments and improving detection flexibility. Furthermore, through multi-level analysis methods such as image segmentation and feature similarity comparison, it can effectively screen and eliminate interfering objects, focusing on the true defect areas and ensuring the reliability of detection results. Additionally, by comparing key image frames and benchmark reference image frames, combined with surface texture similarity and elevation data, a reasonable defect verification mechanism is established, which helps improve the accuracy and reliability of defect identification. Attached Figure Description

[0015] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture in one embodiment of the present invention.

[0016] In the diagram: 1. Feature extraction module; 2. 3D modeling module; 3. Range detection module; 4. Detection focusing module; 5. Scanning analysis module. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1, please refer to Figure 1 This invention provides a technical solution: a method for detecting the foundation of wind turbine piles based on industrial vision, comprising: S1. Obtain multi-view industrial visual images of the wind turbine pile foundation, identify the main pile foundation objects in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main pile foundation objects. S2. Determine the target detection area, including surface texture features and structural deformation features, from multiple detection areas corresponding to multi-view industrial vision images; construct a three-dimensional point cloud model of the pile foundation based on panoramic grayscale images and depth images from various viewpoints, and determine the elevation data corresponding to each area of ​​the foundation surface based on the three-dimensional point cloud model. S3. Determine the adjacent second target detection regions of each first target detection region based on the spatial position sequence of multiple target detection regions in the multi-view industrial vision image, and determine the first feature distribution position range in the first target detection region and the second feature distribution position range in the second target detection region based on the feature distribution position range corresponding to each target detection region. S4. Determine the third feature distribution location range corresponding to the second feature distribution location range in the first target detection area, and determine the first feature distribution location range and the third feature distribution location range as the key detection focus location range in the first target detection area; S5. Perform high-precision scanning analysis on the key detection focus area, and perform graded scanning analysis on the detection location area other than the key detection focus area according to the corresponding scanning analysis precision; based on the graded scanning analysis results and the elevation data corresponding to the three-dimensional point cloud model, determine the initial defect area and defect depth in each target detection area.

[0019] It should be noted that industrial cameras were used to capture images of the wind turbine pile foundation from multiple angles. These images captured various aspects of the pile foundation, including its surface texture, shape, and structural features. For example, if the pile foundation has some cracks or corrosion, these problems may appear in different details from different angles. Image processing algorithms were used to identify the main objects of the pile foundation in these images. For each identified object, the system analyzed its surface texture features (such as color and graininess) and structural deformation features (such as cracks and dents). For example, one image might show a significant crack on one side of the pile foundation, while the other side appears intact. From images viewed from multiple perspectives, the system identifies target detection areas containing anomalous features. For example, if irregular surface textures or deformations are observed in a part of the pile foundation from certain perspectives, these areas will be marked as detection areas requiring special attention. A three-dimensional point cloud model of the pile foundation is constructed using panoramic grayscale images and depth images. This model can intuitively display the shape and elevation changes of the pile foundation, helping to determine the stability of its structure. For example, the model may show that the height of a certain area is lower than the expected value, indicating a possible settlement problem. Based on the spatial location of the target detection area in the image, the system identifies adjacent detection areas. For example, if the first detection area is part of a pile foundation, the second detection area might be another adjacent part. The system analyzes the relationship between these two areas to understand whether there are any chain reactions or related problems. Within each detection area, the system analyzes the location range of feature distribution. For example, if a crack is found in the first target detection area, the system determines the specific location of the crack and its surrounding features. At the same time, it also looks for factors that may affect the adjacent second target detection area and integrates this information to form a key detection area. The system performs more precise scanning analysis on key inspection areas. This means using more detailed inspection methods, such as high-definition cameras or laser scanning, to capture more detailed information within identified critical areas. Meanwhile, for other less important inspection areas, the system employs tiered scanning analysis, conducting different levels of inspection based on their importance and risk level. Based on the results of the scanning analysis and the elevation data of the 3D point cloud model, the system identifies the initial defect areas and their depths. For example, the inspection results may show that the crack depth in a certain area has reached a certain level, exceeding safety standards, thus requiring remedial measures.

[0020] In an optional embodiment, after determining the initial defect region and defect depth in each target detection region, the method further includes: In a continuous multi-view image sequence, determine multiple key image frames corresponding to each initial defect region and a first preset number of adjacent image frames corresponding to each key image frame. Determine the surface texture similarity between each adjacent image frame and the corresponding key image frame, and determine the reference image frame corresponding to each key image frame in the first preset number of adjacent image frames; In a continuous multi-view image sequence, determine the comparison verification image frame corresponding to the benchmark reference image frame for each key image frame; Based on the surface texture similarity and elevation data between the benchmark reference image frame and the corresponding comparison verification image frame corresponding to each key image frame, the initial defect areas are verified to determine the real defect areas.

[0021] It should be noted that in a continuous multi-view image sequence, the system identifies multiple key image frames associated with the initial defect area. Key image frames are those images that clearly show the characteristics of the defect. For example, if a crack in a pile foundation appears repeatedly and is clearly visible in a set of images, then these images will be marked as key image frames. For each key image frame, the system will select a certain number (e.g., five) of adjacent image frames, which are images that are close to the key image frame in time. The purpose of this is to collect more contextual information in order to better understand the nature and development of the defect. For example, if a crack is observed in a key image frame, the system will simultaneously view images of the preceding and following frames to understand whether the crack is expanding or whether there are other related changes. The system analyzes the surface texture similarity between each adjacent image frame and its corresponding key image frame. This similarity calculation can be based on factors such as color, texture mode, and lighting. If the texture features of an adjacent image frame are very similar to those of the key image frame, it indicates that the state of the pile foundation has not changed significantly at these two time points. After calculating the similarity, the system determines a reference image frame from a first preset number of adjacent image frames. This reference image frame is considered to be the image that best represents the state of the key image frame. For example, if the surface texture of one of the adjacent image frames is closest to that of the key image frame, then this frame is selected as the reference image frame. The system identifies a comparison verification image frame corresponding to a baseline reference image frame within a continuous multi-view image sequence. This comparison verification image frame is one that is temporally close to the baseline reference image but not directly adjacent to it. The purpose is to further verify the stability and development trend of the defects. Based on the surface texture similarity between the baseline reference image frame and the comparison verification image frame corresponding to each key image frame, as well as the elevation data, the system verifies each initial defect region. For example, the cracks in the key image frames are compared with the baseline reference image and the comparison verification image to observe whether the cracks exhibit consistent behavior or whether their shape and location have changed significantly.

[0022] In an optional embodiment, determining the comparison verification image frame corresponding to the baseline reference image frame for each key image frame in a continuous multi-view image sequence includes: In a continuous multi-view image sequence, at least one complementary view image frame is identified where the shooting view difference between the reference image frame corresponding to each key image frame satisfies the first view condition. In at least one complementary viewpoint image frame, identify the corresponding contrast verification image frame with the highest surface texture similarity to the reference image frame corresponding to each key image frame. The acquisition time of each complementary image frame is later than the acquisition time of the corresponding reference image frame. Furthermore, when the shooting height of the complementary image frame is the same as or similar to that of the corresponding reference image frame, the time difference between the acquisition time of the complementary image frame and the acquisition time of the corresponding reference image frame satisfies the preset time condition.

[0023] It should be noted that the system analyzes the shooting angles between each key image frame and its corresponding reference image frame. If the two images are shot at different angles, and this difference satisfies a preset first viewpoint condition (e.g., the viewpoint difference is within a certain range), then the system will determine at least one viewpoint complementary image frame from the continuous image sequence. These viewpoint complementary image frames provide different viewpoints from the reference image frame in order to obtain more information. For example, if a key image frame shows damage to a building and its shooting angle is frontal, and the corresponding reference image frame is also frontal, then the system may select an image shot from the side or obliquely above as a viewpoint complementary image frame. This different viewpoint can help observe the details of the damage. After identifying the complementary viewpoint image frames, the system searches among these images for the comparison verification image frame with the highest surface texture similarity to the baseline reference image frame corresponding to each key image frame. The purpose of this step is to find an image that can reflect the features of the baseline image as much as possible. Suppose that an image from a side view (complementary viewpoint image frame) is found to show a part of the damage features of the building well, and its surface texture is very similar to the texture of the baseline reference image frame. Among these complementary image frames, the image from this side view is selected as the comparison verification image frame because its texture is closest to the baseline image. Ensure that the selected complementary viewpoint image frames were acquired later than the corresponding reference image frames. This condition is to ensure that the comparison and verification image frames reflect a more recent state. Simultaneously, the shooting height of these complementary viewpoint image frames must be the same as or similar to that of the reference image frames to ensure spatial comparability. If the reference image frame was captured at 10:00 AM, the system will search for all complementary viewpoint image frames captured after 10:00 AM. For example, if a complementary viewpoint image frame was captured at 10:30 AM and its shooting height is the same as the reference image frame (e.g., both were captured from a height of 3 meters), then this image meets the criteria. Check whether the time difference meets the preset time conditions, such as the time difference should not exceed 30 minutes; this is to ensure that the comparison between images is reasonable and can truly reflect the development state of defects.

[0024] In an optional embodiment, based on the surface texture similarity and elevation data between the baseline reference image frame and the corresponding comparison verification image frame corresponding to each key image frame, each initial defect region is verified to determine the true defect region, including: Under the condition that the surface texture similarity between the baseline reference image frame and the corresponding comparison verification image frame for each key image frame satisfies the second similarity condition; Determine the first texture similarity between the previous image frame of the reference image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; Determine the first elevation difference between the reference image frame corresponding to each key image frame and the previous image frame of the corresponding reference image frame; Determine the second elevation difference between the comparison verification image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; And determine the third elevation difference between the previous image frame of the reference image frame corresponding to each key image frame and the corresponding comparison verification image frame; When at least one of the following conditions is met: the first texture similarity satisfies the third similarity condition; the difference between the first elevation difference and the second elevation difference satisfies the second elevation condition; and the difference between the first elevation difference and the third elevation difference satisfies the third elevation condition, the corresponding initial defect region is confirmed as a real defect region.

[0025] It should be noted that in each key image frame, the system compares the surface texture similarity between its corresponding baseline reference image frame and the comparison verification image frame. If this similarity meets the preset second similarity condition (for example, the similarity must reach a certain threshold), then the defect area can be further confirmed. Suppose that the wall of a building has obvious cracks; there is a key image frame (such as a frontal image taken at 10:00 AM), a baseline reference image frame (such as a frontal image taken at 10:05 AM), and a comparison verification image frame (such as a side image taken at 10:15 AM); through texture analysis, it is found that the texture similarity between the baseline reference image and the comparison verification image is 0.85, which meets the second similarity condition. The system compares the previous image frame (i.e., the image captured slightly earlier) of the reference image frame corresponding to each key image frame with the previous image frame of the corresponding verification image frame to determine the first texture similarity. This step aims to confirm the continuity of historical data. If the previous image frame of the reference image was captured at 9:55 AM, and the previous image frame of the verification image was captured at 10:05 AM, the system calculates the texture similarity between the two images. If the calculation result is 0.80 and meets the condition for the first texture similarity, the system proceeds to the next step. The following analysis will focus on three elevation differences: First elevation difference: the elevation difference between the reference image frame corresponding to each key image frame and its preceding image frame; Second elevation difference: the elevation difference between each comparison verification image frame and its preceding image frame; Third elevation difference: the elevation difference between the preceding image frame of the reference image frame corresponding to each key image frame and its corresponding comparison verification image frame. Calculating these elevation differences helps determine the height consistency between images, thereby ensuring the spatial comparability of the measured defect features. Assuming the elevation of the preceding image frame of the reference image frame is 3 meters, and the corresponding reference image frame's elevation is 3.1 meters, then the first elevation difference is 0.1 meters; the elevation of the preceding image frame of the comparison verification image frame is 3.2 meters, and the comparison verification image frame's elevation is 3.3 meters, then the second elevation difference is 0.1 meters; assuming the elevation difference between the preceding image frame of the reference image frame and its corresponding comparison verification image frame is 0.2 meters, then the third elevation difference is 0.2 meters. After obtaining the above data, the system will check the following conditions: whether the first texture similarity meets the third similarity condition (e.g., similarity greater than 0.75); whether the difference between the first elevation difference and the second elevation difference meets the second elevation condition (e.g., difference less than 0.05 meters); whether the difference between the first elevation difference and the third elevation difference meets the third elevation condition (e.g., difference less than 0.1 meters); if at least one of the above conditions is met, the corresponding initial defect area can be confirmed as a real defect area.

[0026] In an optional embodiment, acquiring multi-view industrial visual images of the wind turbine pile foundation includes: The target scanning camera and its scanning resolution are determined from industrial cameras fixed at different heights around the foundation based on panoramic grayscale images. Based on the scanning parameters of the target scanning camera, acquire depth images of the pile foundation from various viewpoints; Control the drone equipped with industrial vision acquisition equipment to collect a continuous multi-view image sequence of the pile foundation according to a preset flight path.

[0027] It is important to note that before starting image acquisition, it is necessary to first determine a suitable target scanning camera. This can be done by analyzing panoramic grayscale images. Panoramic grayscale images are acquired from industrial cameras fixed at different heights around the foundation, which can capture the complete view around the foundation. For example, assuming several industrial cameras are installed around the pile foundation of a wind turbine at different heights (e.g., 1 meter, 3 meters, and 5 meters), the panoramic grayscale images acquired by these cameras can be used to analyze which areas are the target areas and determine a suitable target scanning camera, such as selecting a camera located at a height of 3 meters for detailed scanning. At the same time, based on the camera's optical parameters, the scanning resolution of the camera can also be determined, for example, set to a resolution of 1 pixel per millimeter to ensure sufficient detail is captured. Once the target scanning camera and its resolution are determined, the next step is to acquire depth images of the pile foundation from various perspectives based on these parameters. Depth images are images that contain distance information between each point in the scene and the camera, which is crucial for subsequent defect detection and 3D reconstruction. Using the determined target scanning camera, the pile foundation is photographed from multiple angles; for example, the camera is rotated at 10-degree intervals in the vertical direction to capture depth images from multiple perspectives. Assuming that at a specific location, depth data from 0 to 5 meters is obtained after scanning, including information on key areas such as the top, sides, and bottom of the pile foundation, these depth images can be used to further analyze the structural characteristics of the pile foundation, such as whether there are cracks or other defects. To obtain more comprehensive image information, a drone equipped with industrial vision acquisition equipment can be used to collect data along a preset flight path. This method can capture the entire view of the pile foundation from the air and take pictures at different heights and angles. The drone's flight path is set so that it flies around the pile foundation at a certain height (e.g., 10 meters) and moves at a stable speed. The camera on the drone can capture a continuous sequence of multi-view images in real time. Assuming the drone is set to take a picture every 1 meter, the camera will slightly adjust its angle each time to obtain a different perspective. This ensures that different sides of the entire pile foundation are covered and rich image data is obtained. These images can be used for subsequent analysis to check the stability and safety of the pile foundation.

[0028] In an optional embodiment, determining the target scanning camera and its scanning resolution from industrial cameras fixed at different heights around the foundation based on a panoramic grayscale image includes: Texture detection is performed on the panoramic grayscale image of the pile foundation to determine the surface condition of the foundation. If there are no obvious cracks or peeling on the foundation surface, the target scanning camera shall include at least low-height and medium-height industrial cameras; otherwise, the target scanning camera shall include at least low-height, medium-height and high-height industrial cameras. Calculate the texture complexity of the foundation surface based on panoramic grayscale images; If the texture complexity is less than or equal to the preset texture complexity threshold, the scanning resolution of the target scanning camera is set to the preset resolution value; otherwise, the target scanning resolution value of each target scanning camera is adaptively calculated based on the panoramic grayscale image.

[0029] It should be noted that multiple industrial cameras installed around the pile foundation capture panoramic grayscale images from fixed positions at different heights (e.g., low height 1 meter, medium height 3 meters, and high height 5 meters). These images are then used for texture detection to assess the condition of the foundation surface. Suppose that a panoramic grayscale image of the pile foundation is captured, and after texture detection, the surface is found to be smooth and uniform, without obvious cracks or peeling. In this case, the foundation can be judged to be in good condition. Based on the surface condition of the foundation, determine the required target scanning camera configuration: if the foundation surface is found to be in good condition, only low- and medium-height industrial cameras are needed for detailed scanning; if cracks or spalling are found on the foundation surface, at least low-, medium-, and high-height industrial cameras are required to obtain a more comprehensive view and detail; if no cracks are found on the foundation surface, select one 1-meter and one 3-meter high camera for target scanning; if minor cracks are found, it may be necessary to add a 5-meter high camera for comprehensive monitoring. By analyzing panoramic grayscale images, the texture complexity of the foundation surface can be calculated. Texture complexity reflects the richness of surface details and is usually quantified using image processing algorithms. Assuming the calculated texture complexity is 0.5, this indicates that the foundation surface is relatively flat and simple. The calculated texture complexity is compared with a preset threshold: the preset texture complexity threshold is 0.7. Since 0.5 is less than 0.7, the scanning resolution of the target scanning camera can be set to a preset value, such as 1 pixel per millimeter, which can meet the image quality requirements. If the texture complexity exceeds the set threshold, for example, if the calculated result is 0.8, adaptive adjustment is required. Based on the richness of detail in the panoramic grayscale image, the optimal scanning resolution of each target scanning camera is dynamically calculated to ensure that the necessary details can be captured. Since the texture complexity is 0.5, which is less than the threshold of 0.7, the scanning resolution of the target scanning camera is set to the preset value of 1 mm 1 pixel. This can effectively guarantee image quality.

[0030] In an optional embodiment, after determining a target detection region including surface texture features and structural deformation feature anomalies from multiple detection regions corresponding to multi-view industrial vision images, the method further includes: The target detection region is segmented to obtain the image segmentation result, and the interference object in the target detection region is judged according to the image segmentation result to determine whether it meets the preset exclusion rule; If the interfering object meets the preset exclusion rules, the interfering object will be removed to obtain the target detection area after removal.

[0031] It's important to note that image segmentation is the process of decomposing an image into multiple parts (or "regions"), which typically represent pixels with similar properties. In practical applications, the purpose of image segmentation is to distinguish true features (such as cracks, peeling, etc.) from interfering objects (such as dust, weeds, etc.) within a target detection region. Suppose a clear crack is found as a target feature within a detection region, but this region also contains dust and small stones, which are interfering objects that need to be removed. Through image segmentation, the crack can be separated from the dust and stones, making the crack's boundaries clearer. This may involve algorithms such as thresholding, edge detection, or watershed segmentation. After image segmentation is completed, interference objects in the image need to be judged according to preset exclusion rules. These preset exclusion rules may include features such as the size, shape, and color of the objects to determine which objects should be considered interference and removed from the target detection area. Suppose the preset exclusion rules stipulate that any object with an area smaller than a certain threshold (e.g., 10 square millimeters) should be considered interference objects. Through image segmentation, several small stones and a clump of dust were identified near the crack. After size measurement, it was found that the area of ​​these interference objects was less than 10 square millimeters, so they met the preset exclusion rules. If the interfering objects meet the preset exclusion rules, they can be removed. This means that these interfering objects are removed from the image of the target detection area, resulting in a cleaner and more accurate target detection area. After removal, all small stones and dust in the image are deleted, leaving only the image of the cracks. In this way, the final image only contains the relevant target features, which is convenient for subsequent analysis and evaluation, such as the length and width of the cracks.

[0032] In an optional embodiment, high-precision scanning analysis is performed on the key detection focus area, and graded scanning analysis is performed on the detection position areas other than the key detection focus area according to the corresponding scanning analysis precision, including: The comprehensive scan analysis task to be executed is broken down into multiple sub-scan tasks, and these sub-scan tasks are set in the task queue. The sub-scanning tasks include a first sub-scanning task that scans and analyzes the key detection focus area according to a first specified scanning method, and a second sub-scanning task that scans and analyzes the detection area other than the key detection focus area in the first target detection area according to a second specified scanning method. Based on task priority and dependencies, multiple sub-scan tasks are dynamically allocated to multiple processing units, and these sub-scan tasks are executed by the multiple processing units.

[0033] It is important to clarify which areas are the focus of inspection; these areas may be those that require special attention due to potential structural problems (such as cracks or deformation) revealed by previous monitoring results; for example, if a local crack exists in the foundation of a wind turbine, the area where the crack is located is the focus of inspection. The comprehensive scanning analysis task to be performed is decomposed into multiple sub-scanning tasks; each sub-scanning task is set around a different detection target; for key detection focus areas, a high-precision first-specified scanning method is used for detailed scanning analysis; for example, a high-resolution camera and sophisticated image processing algorithms are used to capture subtle changes in cracks; For the detection location range other than the key detection focus area, a graded scanning analysis is performed according to the second specified scanning method with lower precision; a lower resolution camera can also be used to save time and resources while ensuring sufficient information is obtained; the first sub-scan task will take high-precision pictures of the crack, while the second sub-scan task will take conventional pictures of other parts of the foundation to check for new or undiscovered problems. Multiple sub-scan tasks are integrated into a task queue for subsequent scheduling and execution. The order in the task queue can be adjusted based on the importance, priority, and dependencies between tasks. For example, the first sub-scan task (high precision for cracks) needs to be completed first before the second sub-scan task can be performed on other areas, because it is desirable to conduct a more extensive inspection after obtaining key data. Based on task priority and dependencies, multiple sub-scanning tasks in the task queue are dynamically allocated to multiple processing units (such as different cameras or computing devices); this enables parallel processing and improves overall scanning efficiency; if three cameras are available for scanning, the first camera focuses on high-precision scanning of cracks, while the second and third cameras simultaneously perform routine scanning of other areas; the system will automatically assign the first sub-scanning task to the first camera, and the second sub-scanning task to the other two cameras. Multiple sub-scanning tasks are executed in parallel by multiple processing units; this not only speeds up the overall scanning process but also ensures that all tasks are completed within the specified time. In actual operation, the first camera takes high-precision pictures of the crack, while the second and third cameras scan other areas. All data is uploaded to the central processing system for analysis in real time after it is generated.

[0034] In an optional embodiment, a three-dimensional point cloud model of the pile foundation is constructed based on a panoramic grayscale image and depth images from various viewpoints, including: Feature points are extracted from depth images at each viewpoint, and the feature points are evenly distributed in the convex and concave regions of the depth images. The feature points of the depth images from each viewpoint are used for registration to obtain the transformation matrix corresponding to each depth image from each viewpoint. The point cloud data corresponding to the depth images from different viewpoints are stitched together according to the transformation matrix to obtain a three-dimensional point cloud model.

[0035] It should be noted that feature points are extracted for the depth image from each viewpoint. Feature points are points that can represent the salient features in the image, and they play a key role in the subsequent registration and stitching process. Feature points need to be evenly distributed in the convex areas (such as the edges or protruding parts of the pile foundation) and concave areas (such as cracks or grooves) of the depth image. This distribution ensures that the model can accurately reflect the characteristics of the structure as a whole. Suppose that the pile foundation of a wind turbine is scanned, and several depth images from different viewpoints are obtained. In each depth image, multiple feature points are identified using feature extraction algorithms (such as SIFT or ORB). For example, in the depth image of a certain viewpoint, the top edge and bottom crack of the pile foundation will be detected as feature points. Feature points from depth images at different viewpoints are used for registration to obtain the corresponding transformation matrix. The purpose of registration is to align image data from different viewpoints so that they can be accurately stitched into a complete 3D model. The transformation matrix defines how to transform the coordinate system of one viewpoint to the coordinate system of another viewpoint. Typically, this process involves solving for rotation and translation parameters. After extracting feature points from different viewpoints, a set of matching feature points is selected (e.g., the same pile foundation edge point in both viewpoints). By calculating the relationship between these matching points, a transformation matrix can be determined, indicating how to transform the depth information of the first viewpoint to the coordinate system of the second viewpoint. The point cloud data corresponding to the depth images from each viewpoint are stitched together according to the previously obtained transformation matrix to form a 3D point cloud model. A point cloud is a collection of points defined in 3D space, each point containing spatial coordinates (x, y, z) and possible color information. By stitching, a complete 3D structure can be obtained. Assuming there are three depth images from three viewpoints and their respective transformation matrices, the corresponding point cloud data can be obtained from each depth image. Then, these point cloud data are transformed into a unified coordinate system according to the transformation matrix and merged. For example, the point cloud data of the first viewpoint is transformed and stitched onto the point cloud data of the third viewpoint to form a complete 3D model of the pile foundation.

[0036] Example 2, please refer to Figure 2 This invention provides a technical solution: a wind turbine pile foundation inspection system based on industrial vision, applicable to the aforementioned wind turbine pile foundation inspection method based on industrial vision, comprising: Feature extraction module 1 is configured to acquire multi-view industrial visual images of wind turbine pile foundations, identify the main objects of the pile foundations in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main objects of the pile foundations. 3D modeling module 2 is configured to determine target detection areas, including surface texture features and structural deformation features, from multiple detection areas corresponding to multi-view industrial vision images; construct a 3D point cloud model of the pile foundation based on panoramic grayscale images and depth images from various viewpoints; and determine the elevation data corresponding to each area of ​​the foundation surface based on the 3D point cloud model. The range detection module 3 is configured to determine the adjacent second target detection region of each first target detection region based on the spatial position sequence of multiple target detection regions in a multi-view industrial vision image, and to determine the first feature distribution position range in the first target detection region and the second feature distribution position range in the second target detection region based on the feature distribution position range corresponding to each target detection region. The detection focusing module 4 is configured to determine the third feature distribution location range corresponding to the second feature distribution location range in the first target detection area, and to determine the first feature distribution location range and the third feature distribution location range as the key detection focusing location range in the first target detection area; The scanning analysis module 5 is configured to perform high-precision scanning analysis on the key detection focus area and to perform hierarchical scanning analysis on the detection position area other than the key detection focus area according to the corresponding scanning analysis precision; based on the hierarchical scanning analysis results and the elevation data corresponding to the three-dimensional point cloud model, the initial defect area and defect depth in each target detection area are determined.

[0037] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for detecting the foundation of wind turbine pile bases based on industrial vision, characterized in that, include: Acquire multi-view industrial visual images of wind turbine pile foundations, identify the main objects of the pile foundations in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main objects of the pile foundations. Target detection areas, including surface texture features and structural deformation features, are identified from multiple detection areas corresponding to multi-view industrial vision images; a three-dimensional point cloud model of the pile foundation is constructed based on panoramic grayscale images and depth images from various viewpoints, and the elevation data corresponding to each area of ​​the foundation surface is determined based on the three-dimensional point cloud model; Based on the spatial position sequence of multiple target detection regions in a multi-view industrial vision image, the adjacent second target detection regions of each first target detection region are determined, and based on the feature distribution position range corresponding to each target detection region, the corresponding first feature distribution position range in the first target detection region and the corresponding second feature distribution position range in the second target detection region are determined. The second feature distribution location range is determined to correspond to the third feature distribution location range in the first target detection area, and the first feature distribution location range and the third feature distribution location range are determined as the key detection focus location range in the first target detection area; High-precision scanning analysis is performed on the key detection focus area, and the detection area other than the key detection focus area is subjected to hierarchical scanning analysis according to the corresponding scanning analysis precision; based on the hierarchical scanning analysis results and the elevation data corresponding to the three-dimensional point cloud model, the initial defect area and defect depth in each target detection area are determined.

2. The method for detecting wind turbine pile foundations based on industrial vision according to claim 1, characterized in that, After determining the initial defect region and defect depth in each target detection region, the method further includes: In a continuous multi-view image sequence, determine multiple key image frames corresponding to each initial defect region and a first preset number of adjacent image frames corresponding to each key image frame. Determine the surface texture similarity between each adjacent image frame and the corresponding key image frame, and determine the reference image frame corresponding to each key image frame in the first preset number of adjacent image frames; In a continuous multi-view image sequence, determine the comparison verification image frame corresponding to the benchmark reference image frame for each key image frame; Based on the surface texture similarity and elevation data between the benchmark reference image frame and the corresponding comparison verification image frame corresponding to each key image frame, the initial defect areas are verified to determine the real defect areas.

3. The method for detecting wind turbine pile foundations based on industrial vision according to claim 2, characterized in that, In a continuous multi-view image sequence, the comparison verification image frame corresponding to the benchmark reference image frame for each key image frame is determined, including: In a continuous multi-view image sequence, at least one complementary view image frame is identified where the shooting view difference between the reference image frame corresponding to each key image frame satisfies the first view condition. In at least one complementary viewpoint image frame, identify the corresponding contrast verification image frame with the highest surface texture similarity to the reference image frame corresponding to each key image frame. The acquisition time of each complementary image frame is later than the acquisition time of the corresponding reference image frame. Furthermore, when the shooting height of the complementary image frame is the same as or similar to that of the corresponding reference image frame, the time difference between the acquisition time of the complementary image frame and the acquisition time of the corresponding reference image frame satisfies the preset time condition.

4. The method for detecting wind turbine pile foundations based on industrial vision according to claim 3, characterized in that, Based on the surface texture similarity and elevation data between the baseline reference image frame and the corresponding comparison verification image frame for each key image frame, the initial defect regions are verified to determine the true defect regions, including: Under the condition that the surface texture similarity between the baseline reference image frame and the corresponding comparison verification image frame for each key image frame satisfies the second similarity condition; Determine the first texture similarity between the previous image frame of the reference image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; Determine the first elevation difference between the reference image frame corresponding to each key image frame and the previous image frame of the corresponding reference image frame; Determine the second elevation difference between the comparison verification image frame corresponding to each key image frame and the previous image frame of the corresponding comparison verification image frame; And determine the third elevation difference between the previous image frame of the reference image frame corresponding to each key image frame and the corresponding comparison verification image frame; When at least one of the following conditions is met: the first texture similarity satisfies the third similarity condition; the difference between the first elevation difference and the second elevation difference satisfies the second elevation condition; and the difference between the first elevation difference and the third elevation difference satisfies the third elevation condition, the corresponding initial defect region is confirmed as a real defect region.

5. The method for detecting wind turbine pile foundations based on industrial vision according to claim 4, characterized in that, Acquire multi-view industrial visual images of wind turbine pile foundations, including: The target scanning camera and its scanning resolution are determined from industrial cameras fixed at different heights around the foundation based on panoramic grayscale images. Based on the scanning parameters of the target scanning camera, acquire depth images of the pile foundation from various viewpoints; Control the drone equipped with industrial vision acquisition equipment to collect a continuous multi-view image sequence of the pile foundation according to a preset flight path.

6. The method for detecting wind turbine pile foundations based on industrial vision according to claim 5, characterized in that, The target scanning camera and its scanning resolution are determined from industrial cameras fixed at different heights around the foundation based on panoramic grayscale images, including: Texture detection is performed on the panoramic grayscale image of the pile foundation to determine the surface condition of the foundation. If there are no obvious cracks or peeling on the foundation surface, the target scanning camera shall include at least low-height and medium-height industrial cameras; otherwise, the target scanning camera shall include at least low-height, medium-height and high-height industrial cameras. Calculate the texture complexity of the foundation surface based on panoramic grayscale images; If the texture complexity is less than or equal to the preset texture complexity threshold, the scanning resolution of the target scanning camera is set to the preset resolution value; otherwise, the target scanning resolution value of each target scanning camera is adaptively calculated based on the panoramic grayscale image.

7. The method for detecting wind turbine pile foundations based on industrial vision according to claim 6, characterized in that, After determining the target detection region, including surface texture features and structural deformation feature anomalies, from multiple detection regions corresponding to multi-view industrial vision images, the method further includes: The target detection region is segmented to obtain the image segmentation result, and the interference object in the target detection region is judged according to the image segmentation result to determine whether it meets the preset exclusion rule; If the interfering object meets the preset exclusion rules, the interfering object will be removed to obtain the target detection area after removal.

8. The method for detecting wind turbine pile foundations based on industrial vision according to claim 7, characterized in that, High-precision scanning analysis is performed on the key detection focus area, and graded scanning analysis is performed on the detection position areas other than the key detection focus area according to the corresponding scanning analysis precision, including: The comprehensive scan analysis task to be executed is broken down into multiple sub-scan tasks, and these sub-scan tasks are set in the task queue. The sub-scanning tasks include a first sub-scanning task that scans and analyzes the key detection focus area according to a first specified scanning method, and a second sub-scanning task that scans and analyzes the detection area other than the key detection focus area in the first target detection area according to a second specified scanning method. Based on task priority and dependencies, multiple sub-scan tasks are dynamically allocated to multiple processing units, and these sub-scan tasks are executed by the multiple processing units.

9. The method for detecting wind turbine pile foundations based on industrial vision according to claim 8, characterized in that, A 3D point cloud model of the pile foundation is constructed based on panoramic grayscale images and depth images from various viewpoints, including: Feature points are extracted from depth images at each viewpoint, and the feature points are evenly distributed in the convex and concave regions of the depth images. The feature points of the depth images from each viewpoint are used for registration to obtain the transformation matrix corresponding to each depth image from each viewpoint. The point cloud data corresponding to the depth images from different viewpoints are stitched together according to the transformation matrix to obtain a three-dimensional point cloud model.

10. A wind turbine pile foundation inspection system based on industrial vision, applicable to the wind turbine pile foundation inspection method based on industrial vision as described in any one of claims 1-9, characterized in that, include: The feature extraction module is configured to acquire multi-view industrial visual images of wind turbine pile foundations, identify the main objects of the pile foundations in the multi-view industrial visual images, and analyze the surface texture features and structural deformation features corresponding to the main objects of the pile foundations. The 3D modeling module is configured to determine target detection areas, including surface texture features and structural deformation features, from multiple detection areas corresponding to multi-view industrial vision images; construct a 3D point cloud model of the pile foundation based on panoramic grayscale images and depth images from various viewpoints, and determine the elevation data corresponding to each area of ​​the foundation surface based on the 3D point cloud model; The range detection module is configured to determine the adjacent second target detection regions of each first target detection region based on the spatial position sequence of multiple target detection regions in a multi-view industrial vision image, and to determine the first feature distribution position range in the first target detection region and the second feature distribution position range in the second target detection region based on the feature distribution position range corresponding to each target detection region. The detection focusing module is configured to determine the third feature distribution location range corresponding to the second feature distribution location range in the first target detection area, and to determine the first feature distribution location range and the third feature distribution location range as the key detection focusing location range in the first target detection area; The scanning analysis module is configured to perform high-precision scanning analysis on the key detection focus area and to perform graded scanning analysis on the detection location area other than the key detection focus area according to the corresponding scanning analysis precision; based on the graded scanning analysis results and the elevation data corresponding to the 3D point cloud model, the initial defect area and defect depth in each target detection area are determined.

Images