OpenCV and YOLO-based metal roof weld defect intelligent detection and positioning system

Abstract

The invention relates to a metal roof weld defect intelligent detection and positioning system based on OpenCV and YOLO, and the system comprises a detection robot, and an industrial control module, a solid-state laser radar module, a positioning module, an ink jet module and a power module which are arranged on the detection robot. The industrial control module is used for performing work control on the detection robot, the solid-state laser radar module, the positioning module and the ink jet module, the solid-state laser radar module is used for performing refined modeling on a detected metal roof, and the ink jet module is used for performing ink jet marking on detected weld defects. The positioning module is used for providing high-precision coordinates for detected weld defects, and the power supply module is used for supplying power to the detection robot, the industrial control module, the solid-state laser radar module, the positioning module and the ink jet module. The airport building metal roof welding seam detection efficiency can be improved, and the metal roof operation and maintenance cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of metal roof weld defect detection, and in particular to an intelligent metal roof weld defect detection and positioning system based on OpenCV and YOLO. Background Art

[0002] With the rapid development of my country's aviation industry, the scale of airport construction continues to expand. Metal roofs are widely used in large buildings such as airport terminals due to their superior durability and lightweight characteristics. However, due to long-term exposure to harsh natural environments such as wind, rain, temperature differences, and ultraviolet rays, these metal roofs are prone to structural defects such as corrosion, weld cracking, weld penetration, and dislocation. If not detected and maintained in time, the defects will gradually expand, affecting the overall structural stability and service life of the roof, and even posing a safety hazard.

[0003] Traditional manual weld inspection methods are inefficient, complicated to operate, and have limited coverage effects on complex and large-area metal roofs. Manual inspection not only consumes a lot of manpower and time, but is also easily affected by the professional level of inspectors and environmental factors, resulting in inaccurate inspection results, which are difficult to meet the needs of efficient and accurate maintenance of airports. As the scale of airport construction continues to expand, the requirements for inspection speed, accuracy and cost control are gradually increasing. The development of an intelligent and automated weld defect detection and positioning system has become an inevitable trend in the development of the industry. Summary of the invention

[0004] The purpose of this invention is to provide an intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO, to improve the efficiency of airport building weld detection, to reduce the operation and maintenance cost of metal roofs, to promote the development of metal roof operation and maintenance in the direction of intelligence, and to provide technical support for ensuring the safe operation and maintenance of metal roofs of airport buildings throughout their life cycle.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The invention discloses an intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO, comprising: a detection robot, an industrial control module, a solid-state laser radar module, a positioning module, an inkjet module and a power module, wherein the detection robot is used to detect metal roof weld defects during movement, the industrial control module, the solid-state laser radar module, the positioning module, the inkjet module and the power module are arranged on the detection robot, the industrial control module is used to control the operation of the detection robot, the solid-state laser radar module, the positioning module and the inkjet module, the solid-state laser radar module is used to perform refined modeling of the detected metal roof, the inkjet module is used to perform inkjet marking on the detected weld defects, the positioning module is used to provide high-precision coordinates for the detected weld defects, and the power module is used to power the detection robot, the industrial control module, the solid-state laser radar module, the positioning module and the inkjet module.

[0007] Optionally, the detection robot includes a robot body, a control receiving unit and a driving unit, the robot body is connected to the control receiving unit and the driving unit respectively, the control receiving unit is used to receive control instructions from the industrial control module; the driving unit is used to control the motor through the control instructions to drive the movement of the robot body.

[0008] Optionally, the industrial control module includes an image acquisition unit, a mobile control unit, a solid-state laser radar scanning control unit, an RTK positioning control unit and an inkjet control unit, wherein the image acquisition unit is used to acquire track images during the movement of the inspection robot; the mobile control unit is used to detect the track position based on the track image, and control the real-time movement of the inspection robot through the track position; the solid-state laser radar scanning control unit is used to control the solid-state laser radar module to perform refined modeling through scanning during the real-time movement of the inspection robot; the RTK positioning control unit is used to control the positioning module to output high-precision coordinates during the real-time movement of the inspection robot; the inkjet control unit is used to detect track weld defects based on the track image detection, and obtain the coordinates of the track weld defect points through the RTK positioning control unit, and then control the inkjet module to perform inkjet marking.

[0009] Optionally, the image acquisition unit includes a first camera device and a second camera device, the first camera device is arranged in front of the detection robot to acquire a front track image; the second camera device is arranged on the side of the detection robot to acquire a side track image.

[0010] Optionally, the mobile control unit includes an image processing subunit, a track detection subunit, and a control generation subunit, wherein the image processing subunit is used to grayscale, denoise, and enhance the front track image; the track detection subunit is used to use the OpenCV method to perform track edge detection and track end point detection on the processed front track image, extract track edge information and track end point information, and determine the offset of the detection robot through the track edge information, and determine the track end point position through the track end point information; the control generation subunit is used to generate control instructions based on the offset and track end point position to control the movement and steering of the detection robot.

[0011] Optionally, performing track edge detection through the processed front track image to extract track edge information, and determining the offset of the detection robot through the track edge information includes:

[0012] Extracting the track edge in the processed front track image using a Canny operator;

[0013] Extracting straight line segments from the track edge, selecting left and right track boundary lines from the straight line segments, and selecting the left and right track boundary lines with the longest straight line length as the final track boundary;

[0014] The midpoint is calculated based on the edge point coordinates of the final track boundary to obtain the track centerline, and the track centerline is compared with the theoretical center coordinates of the image to obtain the offset.

[0015] Optionally, performing track end point detection through the processed forward track image to extract track end point information, and determining the track end point position through the track end point information includes:

[0016] Binarizing the processed front track image to extract candidate reflective strip areas;

[0017] Perform pixel scanning on the candidate area of ​​the reflective stripe to screen out effective stripes;

[0018] Counting the number of valid stripes, and extracting a target reflective strip area from the reflective strip candidate area according to a number threshold;

[0019] The average row coordinates of the target reflective strip area are calculated to determine the position of the reflective strip, that is, the end position of the track.

[0020] Optionally, the inkjet control unit includes a defect detection subunit and an inkjet marking subunit, wherein the defect detection subunit is used to input the side track image into a defect detection model to obtain track weld defect information, and simultaneously obtain the track weld defect point coordinates through the RTK positioning control unit, wherein the defect detection model is obtained through improved YOLOv5 neural network training; the inkjet marking subunit is used to generate inkjet instructions and inkjet marks based on the track weld defect information and the track weld defect point coordinates, and send them to the inkjet module.

[0021] Optionally, the solid-state laser radar scanning control unit controls the solid-state laser radar module to scan and establish a refined model of the metal roof, and the RTK positioning control unit controls the high-precision coordinates recorded by the positioning module, as the map for the inspection robot to conduct the next metal roof inspection.

[0022] The beneficial effects of the present invention are:

[0023] The present invention uses OpenCV edge detection technology to quickly identify the specific location of the roof weld track, and uses a deep learning algorithm through an improved YOLOv5 target detection model to accurately classify and locate the weld defects on both sides, ensuring the comprehensiveness and accuracy of the detection. The crawler design enhances the adaptability of the trolley in the inclined terrain of the metal roof, and provides a new solution for the intelligent and automated detection of airport metal roofs. The present invention can improve the efficiency of weld detection in metal roof buildings such as airports, reduce the operation and maintenance costs of metal roofs, promote the development of metal roof operation and maintenance in the direction of intelligence, and provide technical support for ensuring the safe operation and maintenance of metal roofs of airport buildings throughout their life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a schematic diagram of the structure of the intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to an embodiment of the present invention;

[0026] Figure 2 This is a circuit diagram of a detection robot chassis according to an embodiment of the present invention;

[0027] Figure 3 Schematic diagrams of the movement of the detection robot according to an embodiment of the present invention, wherein (a) is a schematic diagram of straight-line movement, and (b) is a schematic diagram of turning and obstacle-crossing movement;

[0028] Figure 4 It is the STM32F407VET7 algorithm flow chart of the embodiment of the present invention;

[0029] Figure 5 is a flowchart of image preprocessing according to an embodiment of the present invention;

[0030] Figure 6 A flow chart of a track endpoint detection algorithm according to an embodiment of the present invention;

[0031] Figure 7 This is a track detection effect diagram of an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the improved YOLOv5 neural network structure of an embodiment of the present invention;

[0033] Fig. 9 This is a flow chart of the solid-state laser radar and RTK combined scanning modeling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] The roofs of the airport terminal and piers are made of stainless steel, and all welds are welded using a continuous resistance pressure welding process, with the overall structure sealed through seamless welding of the interface. However, during the metal roof welding process, various welding defects are prone to occur due to factors such as changes in the operating temperature of the welding equipment and thermal expansion of the metal. These defects include discontinuous welding (broken welds), incomplete penetration of welds, misaligned welding, and local over-welding. These welding defects not only affect the appearance and stability of the structure, but more seriously, they may cause hidden dangers such as water leakage during future use. Therefore, this embodiment provides an intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO, such as Figure 1As shown, it includes: a detection robot, an industrial control module, a solid-state laser radar module, a positioning module, an inkjet module and a power module, wherein the detection robot is used to detect metal roof weld defects during movement, and the industrial control module, the solid-state laser radar module, the positioning module, the inkjet module and the power module are arranged on the detection robot, the industrial control module is used to control the detection robot, the solid-state laser radar module, the positioning module and the inkjet module, the solid-state laser radar module is used to perform refined modeling of the detected metal roof, the inkjet module is used to inkjet mark the detected weld defects, the positioning module is used to provide high-precision coordinates for the detected weld defects, and the power module is used to power the detection robot, the industrial control module, the solid-state laser radar module, the positioning module and the inkjet module.

[0037] Furthermore, the detection robot includes a robot body, a control receiving unit and a driving unit. The robot body is connected to the control receiving unit and the driving unit respectively. The control receiving unit is used to receive control instructions from the industrial control module; the driving unit is used to control the motor through the control instructions to drive the movement of the robot body.

[0038] Specifically, in this embodiment, the robot body adopts a chassis trolley design, and the chassis trolley adopts a crawler design, which has excellent terrain adaptability and stability. The control receiving unit adopts the STM32F407 control chip, and the drive unit adopts a motor drive. The chassis PCB board integrates the STM32F407 control chip and the motor drive, and receives control instructions through the communication interface with the industrial control module to achieve precise differential movement. The crawler design enhances the mobility of the trolley in a complex metal roof environment, enabling it to flexibly cope with slopes, joints and other uneven terrains, ensuring the reliable operation of the detection equipment in harsh environments.

[0039] In order to design a metal roof structure with an average weld height of 2.7CM, a weld spacing of 40CM, and a slope of 0-30° in the terminal building metal roof, this embodiment designs a two-wheel drive crawler chassis with a size of 330*255*100 (mm) and a combined aluminum alloy shell with a deadweight of 2.2Kg. It performs steering operations through the differential of two motors. Under normal working conditions, it can perform body actions such as straight-line driving and obstacle crossing at a speed of 0-0.7M / S between two vertical locking welds with a slope of less than 40°.

[0040] Chassis circuit structure Figure 2 As shown in the figure, the circuit is composed of STM32F403VET7 microcontroller as the core. The STM32F407VET7 is integrated on a PCB board with a motor drive circuit. By receiving the UART communication field transmitted by the industrial control module, PWM pulse width modulation is used to control the left and right motors to achieve the following Figure 3 Straight-line driving, turning, and obstacle crossing as shown in (a) and (b).

[0041] like Figure 4 As shown, the STM32F407VET7 implements the tasks of running the motor, receiving actual data sent back by the encoder, displaying speed information, receiving speed instructions from the STM32F407ZET6_2, and calculating the PWM wave for controlling the motor in the system.

[0042] Furthermore, the industrial control module includes an image acquisition unit, a mobile control unit, a solid-state laser radar scanning control unit, an RTK positioning control unit and an inkjet control unit, wherein the image acquisition unit is used to acquire track images during the movement of the detection robot; the mobile control unit is used to detect the track position based on the track image, and control the real-time movement of the detection robot through the track position; the solid-state laser radar scanning control unit is used to control the solid-state laser radar module to perform refined modeling through scanning during the real-time movement of the detection robot; the RTK positioning control unit is used to control the positioning module to output high-precision coordinates during the real-time movement of the detection robot; the inkjet control unit is used to detect track weld defects based on the track image detection, and obtain the coordinates of the track weld defect points through the RTK positioning control unit, and then control the inkjet module to perform inkjet marking.

[0043] Specifically, in this embodiment, the industrial control module performs core control of the system and adopts a vehicle-mounted micro-industrial computer, which has a rich expansion interface and computing performance far exceeding that of traditional single-chip microcomputers. The industrial control module performs environmental perception and weld defect detection through industrial cameras integrated in the front and both sides of the vehicle body. By embedding deep learning algorithms, the industrial control module can analyze images in real time, identify weld edges and defect types, and generate control instructions to coordinate the movement of the detection robot and the work of the inkjet module. The industrial control module also realizes efficient data interaction with other modules through a dedicated communication interface, providing guarantee for the intelligent operation of the whole vehicle.

[0044] Furthermore, the image acquisition unit includes a first camera device and a second camera device. The first camera device is arranged in front of the detection robot to acquire the front track image; the second camera device is arranged on the side of the detection robot to acquire the side track image.

[0045] Furthermore, the mobile control unit includes an image processing subunit, a track detection subunit, and a control generation subunit, wherein the image processing subunit is used to grayscale, denoise, and enhance the front track image; the track detection subunit is used to use the OpenCV method to perform track edge detection and track end point detection respectively through the processed front track image, extract track edge information and track end point information, and determine the offset of the detection robot through the track edge information, and determine the track end point position through the track end point information; the control generation subunit is used to generate control instructions based on the offset and the track end point position, and control the movement and steering of the detection robot.

[0046] Among them, performing track edge detection through the processed front track image to extract track edge information, and determining the offset of the detection robot through the track edge information includes:

[0047] The Canny operator is used to extract the track edge in the processed forward track image;

[0048] Extract straight line segments from the track edge, select left and right track boundary lines from the straight line segments, and select the left and right track boundary lines with the longest straight line length as the final track boundary;

[0049] The midpoint is calculated based on the edge point coordinates of the final track boundary to obtain the track centerline, which is then compared with the theoretical center coordinates of the image to obtain the offset.

[0050] Among them, performing track end point detection through the processed front track image to extract track end point information, and determining the track end point position through the track end point information includes:

[0051] Binarize the processed front track image and extract the candidate area of ​​the reflective strip;

[0052] Perform pixel scanning on the candidate areas of reflective stripes to screen out effective stripes;

[0053] Count the number of valid stripes and extract the target reflective stripe area from the reflective stripe candidate area according to the number threshold;

[0054] Calculate the average row coordinates of the target reflective strip area to determine the position of the reflective strip, that is, the end point of the track.

[0055] Specifically, this embodiment implements image processing based on the OpenCV library. The core is that the front camera collects track images, and by calculating the track boundary line and center line, the position of the detection robot is adjusted in real time to keep it running in the center of the track. At the same time, by detecting the horizontal end line of the track, the detection robot is guided to perform off-track operations at the end position. The following is the specific design and implementation method of the algorithm:

[0056] The front camera collects images of the metal roof track in real time, and the images need to be preprocessed to improve the detection accuracy of key features. Figure 5 As shown in the figure, the color image is first converted into a grayscale image to reduce the computational complexity and enhance the significance of the track structure. A 5×5 Gaussian kernel is used to smooth the grayscale image to reduce the impact of noise on edge detection. The Canny operator is then used to extract the edge of the track, and the thresholds are set to 50 and 150 to ensure that the track boundary lines are clear. The edge coherence is strengthened by the dilation operation, the kernel size is 4×4, and it is iterated twice to repair the broken edge features. Thus, the image after edge detection is obtained.

[0057] The track information in the image after edge detection is extracted by using Hough transform line detection and angle screening. The specific implementation is as follows: the algorithm extracts all qualified straight line segments by setting the parameters of Hough transform (accumulator resolution is 1 pixel, angle resolution is 1°, threshold is 50). Then the left and right track boundary lines are screened out using the straight line inclination angle and midpoint position. The left boundary requires the inclination angle to be less than -30° and the midpoint position to be located in the left half of the image; the right boundary requires the inclination angle to be greater than 30° and the midpoint to be located in the right half of the image. Finally, by calculating the length of the straight line, the longest left and right boundary lines are selected as the final track boundaries.

[0058] In order to determine the specific position of the inspection robot between the two metal vertical edges, the algorithm needs to calculate the center line of the two vertical edge tracks. Specifically, for each row of the image, the edge point coordinates of the left and right boundaries in the row are calculated respectively, and the horizontal coordinates of the two are averaged to generate the midpoint of the center line. Then the midpoint sequence is connected into a track center line, and up to 10 midpoints are selected at the bottom of the center line, and the average of their horizontal coordinates is calculated. The offset is compared with the theoretical center coordinates of the image to obtain the offset. The offset is used to determine whether the robot has deviated from the track center. Then, the chassis control command is sent based on the offset to adjust its driving direction using the PID control algorithm.

[0059] In order to ensure that the inspection robot can accurately identify the end point of the track and perform steering operations, a target recognition algorithm based on reflective strip detection is proposed. Figure 6As shown in the figure, the image of the end point of the track is collected by the front camera, and the image processing technology is used to realize the automatic recognition of the specific reflective strip features, so as to provide a clear turning signal for the robot. First, the collected image is converted into a grayscale image to reduce the data dimension and highlight the brightness information. On this basis, the grayscale image is binarized by the Otsu threshold method, the highlight area in the image is extracted, and the candidate area that may contain the reflective strip is screened out. Then the program scans each row of pixels in the binary image column by column to detect the boundary change of the black and white stripes. During the scanning process, the starting and ending positions of the stripes are determined by judging the change of pixel values ​​(from white to black or from black to white), and the valid stripe information is screened out according to the set stripe width range. At the same time, the number of valid stripes in each row will be counted. By judging whether the number of stripes exceeds the set threshold, it can be judged whether the row belongs to the target reflective strip area. This avoids misjudgment caused by noise or other interference. Finally, the position of the reflective strip is determined by calculating the average row coordinates of all detected valid stripe areas. When the algorithm detects a reflective strip that meets the conditions, the car will enter the steering mode.

[0060] In the algorithm design, in order to fully consider the real-time requirements of robot driving, the efficient OpenCV method is used to avoid complex calculations. In addition, by controlling the time interval between data acquisition and processing, the algorithm operation frequency is kept at around 20Hz, thereby ensuring the real-time collaboration between image processing and robot control. The detection effect is shown in Figure 2. Figure 7 .

[0061] Furthermore, the inkjet control unit includes a defect detection subunit and an inkjet marking subunit, wherein the defect detection subunit is used to input the side track image into a defect detection model to obtain track weld defect information, and simultaneously obtain the track weld defect point coordinates through the RTK positioning control unit, wherein the defect detection model is obtained through improved YOLOv5 neural network training; the inkjet marking subunit is used to generate inkjet instructions and inkjet marks based on the track weld defect information and the track weld defect point coordinates, and send them to the inkjet module.

[0062] The improved YOLOv5 neural network structure is as follows Figure 8 As shown, it includes: Backbone (backbone network), which is used to extract image features; Neck (neck network), which is used to enhance the representation of features, mainly through feature pyramid network and path aggregation network; Detect (detection network), which is used to output the final target detection results, including bounding boxes, class labels and confidence levels.

[0063] Specifically, in order to achieve real-time rail weld damage detection, this embodiment proposes a target detection algorithm based on a pre-trained YOLOv5 neural network, using cameras on both sides of the robot to detect and identify the welds on the edge of the track in real time. The algorithm accurately identifies the characteristics of weld damage, providing an important basis for the robot's subsequent path planning and maintenance decisions. The specific algorithm steps are as follows:

[0064] (1) Image acquisition and preprocessing:

[0065] The cameras installed on both sides capture the image information around the track in real time, and the image resolution is uniformly set to 640×480 pixels. Since the YOLOv5 model has been pre-trained for target detection tasks, the input images do not need to be pre-processed. The collected original images are directly input into the YOLOv5 model, and the model's powerful feature extraction and classification capabilities are used to complete the detection and identification of welds.

[0066] (2) Target Detection

[0067] The input image is processed as follows Figure 8 After the YOLOv5 neural network shown in the figure is used, the model will automatically output the detection box containing the weld and related classification information. Specifically, it includes the coordinates of the rectangular box where each weld is located in the image (that is, the position and size of the detection box), the confidence value of the model for each weld target (indicating the certainty of the model's detection results), and the classification label of the weld damage. Through its convolutional neural network, YOLOv5 can quickly and accurately identify the weld area in the image and mark the specific damage type.

[0068] The weld inspection results output by the model will be further processed and classified. Specifically, according to the information of the detection frame, the weld location information (including coordinates), damage type, detection confidence and other data will be stored. This information will be used to generate a detailed weld data collection report. The weld data in the report includes the specific location of the weld, damage type and confidence, which can provide an effective reference for subsequent maintenance personnel.

[0069] Furthermore, the solid-state laser radar scanning control unit controls the solid-state laser radar module to scan and establish a refined model of the metal roof, and the RTK positioning control unit controls the high-precision coordinates recorded by the positioning module to serve as a map for the inspection robot to conduct the next metal roof inspection.

[0070] Specifically, in this embodiment, the solid-state laser radar control unit uses the LIO-SAM algorithm to accurately process the point cloud data collected by the solid-state laser radar module, and then establishes a high-precision and detailed three-dimensional map. The environment is dynamically modeled through real-time positioning and mapping technology (SLAM). The generated three-dimensional map can accurately reflect the structural distribution of the metal roof.

[0071] The RTK positioning control unit controls the positioning module to provide high-precision real-time positioning data, aligning the map generated by the lidar with the actual coordinate system to further improve the positioning accuracy of the map. By combining these data, the established three-dimensional map can not only serve as the navigation and positioning basis for the inspection robot, but also provide a stable and reliable inspection path for the robot in subsequent inspection tasks, ensuring that the robot can accurately inspect the metal roof and identify potential problems and defects.

[0072] In order to achieve high-precision three-dimensional mapping, this embodiment proposes the following Fig. 9 The 3D mapping and positioning method based on the LIO-SAM algorithm and RTK positioning technology shown in the figure uses solid-state laser radar to scan and obtain point cloud data, and corrects the point cloud data with high-precision coordinates provided by the RTK positioning control unit to generate a refined 3D map for the inspection robot to use during the inspection process. By combining the high-precision point cloud information of the laser radar with the RTK positioning data, the accuracy and reliability of the map in positioning and navigation are ensured. The specific implementation steps are as follows:

[0073] (1) LiDAR scanning and point cloud data collection: Solid-state LiDAR collects point cloud data on the metal roof in real time by scanning the surrounding environment. LiDAR emits laser beams at a certain frequency, captures the reflected signals of surrounding objects, calculates the distance and forms point cloud data. To ensure the comprehensiveness and accuracy of the scan, the LiDAR is installed at an appropriate position on the inspection robot, and the scanning range covers the entire target area of ​​the metal roof. The point cloud data has high spatial resolution and accuracy, and can truly reflect the three-dimensional structure of the environment.

[0074] (2) LIO-SAM algorithm processes point cloud data and builds maps: The LIO-SAM (LiDAR Inertial Odometry and Mapping) algorithm uses the point cloud data provided by the LiDAR to build real-time maps of the environment. The algorithm aligns and optimizes the point cloud data obtained by the LiDAR scan, and calculates the position of the robot in real time while building the map. LIO-SAM can handle dynamic environments under high-speed motion and generate high-precision three-dimensional maps through efficient data fusion. This map can accurately reflect the structural characteristics of the metal roof and other environmental factors.

[0075] (3) RTK positioning technology provides high-precision coordinate correction: The RTK positioning control unit provides centimeter-level high-precision coordinate data by receiving satellite signals. The unit can record the precise position of the inspection robot on the roof in real time, ensuring that each position update has very high accuracy. The RTK data is combined with the map data generated by LIO-SAM, and coordinate correction is used to correct possible positioning deviations, further improving the spatial accuracy of the three-dimensional map. RTK technology uses high-precision coordinates transmitted in real time to perform positioning correction and optimization on the map generated by LIO-SAM.

[0076] (4) Map optimization and fusion: By combining LIO-SAM and RTK data, the inspection robot can create an accurate 3D map on the metal roof. Specifically, by using RTK positioning data as a global positioning correction reference, LIO-SAM performs spatial correction on the point cloud data to ensure that the generated 3D map can reflect the real environment of the roof. During this optimization process, deviations in the map will be corrected, ensuring that the data for subsequent tasks can be more accurate.

[0077] Furthermore, the inkjet module is used to mark the detected weld defects. The module uses an electrically controlled kinetic energy push rod as a power device and carries a high-precision spray bottle as the inkjet body. The industrial control module directly controls the inkjet module and completes the marking operation of the weld defects by accurately calculating the inkjet position. The design of this module not only meets the high requirements for marking accuracy, but also has the characteristics of being lightweight and easy to maintain, providing a clear and intuitive reference for subsequent defect repair work.

[0078] Furthermore, the power module supplies power to the detection robot, industrial control module, inkjet module, solid-state laser radar module, and positioning module. Specifically, a 12000mAh DC battery is used to provide a stable 12V output voltage to power other hardware modules. In order to simplify wiring and improve reliability, the power module is designed with three 12V DC male output interfaces and two DC wires, which can simultaneously meet the power needs of the detection robot, industrial control module, solid-state laser radar module, positioning module, and inkjet module. The high-capacity design of this module ensures that the car can still provide stable power supply even when working continuously for a long time, and is suitable for high-energy consumption scenarios of large-area metal roof inspection tasks.

[0079] Specifically, this embodiment uses OpenCV edge detection technology to quickly identify the specific location of the roof weld track, and uses a deep learning algorithm through the YOLOv5 target detection model to accurately classify and locate the weld defects on both sides to ensure the comprehensiveness and accuracy of the detection. The crawler design enhances the adaptability of the trolley in the inclined terrain of the metal roof, and provides a new solution for the intelligent and automated detection of the airport metal roof. It can improve the efficiency of airport building weld detection, reduce the cost of metal roof operation and maintenance, promote the development of metal roof operation and maintenance in the direction of intelligence, and provide technical support for ensuring the safe operation and maintenance of the metal roof of the airport building throughout the life cycle.

[0080] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. Intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO, characterized by: include: An inspection robot, an industrial control module, a solid-state laser radar module, a positioning module, an inkjet module and a power module, wherein the inspection robot is used to detect metal roof weld defects during movement, the industrial control module, the solid-state laser radar module, the positioning module, the inkjet module and the power module are arranged on the inspection robot, the industrial control module is used to control the inspection robot, the solid-state laser radar module, the positioning module and the inkjet module, the solid-state laser radar module is used to perform refined modeling of the inspected metal roof, the inkjet module is used to inkjet mark the detected weld defects, the positioning module is used to provide high-precision coordinates for the detected weld defects, and the power module is used to power the inspection robot, the industrial control module, the solid-state laser radar module, the positioning module and the inkjet module.

2. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 1 is characterized in that: The detection robot includes a robot body, a control receiving unit and a driving unit. The robot body is connected to the control receiving unit and the driving unit respectively. The control receiving unit is used to receive control instructions from the industrial control module; the driving unit is used to control the motor through the control instructions to drive the movement of the robot body.

3. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 1 is characterized in that: The industrial control module includes an image acquisition unit, a mobile control unit, a solid-state laser radar scanning control unit, an RTK positioning control unit and an inkjet control unit, wherein the image acquisition unit is used to acquire track images during the movement of the detection robot; the mobile control unit is used to detect the track position based on the track image, and control the real-time movement of the detection robot through the track position; the solid-state laser radar scanning control unit is used to control the solid-state laser radar module to perform refined modeling through scanning during the real-time movement of the detection robot; the RTK positioning control unit is used to control the positioning module to output high-precision coordinates during the real-time movement of the detection robot; the inkjet control unit is used to detect track weld defects based on the track image detection, and obtain the coordinates of the track weld defect points through the RTK positioning control unit, and then control the inkjet module to perform inkjet marking.

4. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 3 is characterized in that: The image acquisition unit includes a first camera device and a second camera device. The first camera device is arranged in front of the detection robot to acquire a front track image; the second camera device is arranged on the side of the detection robot to acquire a side track image.

5. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 4 is characterized in that: The mobile control unit includes an image processing subunit, a track detection subunit, and a control generation subunit, wherein the image processing subunit is used to grayscale, denoise, and enhance the front track image; the track detection subunit is used to use the OpenCV method to perform track edge detection and track end point detection on the processed front track image, extract track edge information and track end point information, and determine the offset of the detection robot through the track edge information, and determine the track end point position through the track end point information; the control generation subunit is used to generate control instructions based on the offset and track end point position to control the movement and steering of the detection robot.

6. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 5 is characterized in that: Performing track edge detection through the processed front track image to extract track edge information, and determining the offset of the detection robot through the track edge information includes: Extracting the track edge in the processed front track image using a Canny operator; Extracting straight line segments from the track edge, selecting left and right track boundary lines from the straight line segments, and selecting the left and right track boundary lines with the longest straight line length as the final track boundary; The midpoint is calculated based on the edge point coordinates of the final track boundary to obtain the track centerline, and the track centerline is compared with the theoretical center coordinates of the image to obtain the offset.

7. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 5 is characterized in that: Performing track end point detection through the processed front track image to extract track end point information, and determining the track end point position through the track end point information includes: Binarizing the processed front track image to extract candidate reflective strip areas; Perform pixel scanning on the candidate area of ​​the reflective stripe to screen out effective stripes; Counting the number of valid stripes, and extracting a target reflective strip area from the reflective strip candidate area according to a number threshold; The average row coordinates of the target reflective strip area are calculated to determine the position of the reflective strip, that is, the end position of the track.

8. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to claim 4 is characterized in that: The inkjet control unit includes a defect detection subunit and an inkjet marking subunit, wherein the defect detection subunit is used to input the side track image into a defect detection model to obtain track weld defect information, and simultaneously obtain the coordinates of the track weld defect point through the RTK positioning control unit, wherein the defect detection model is obtained through improved YOLOv5 neural network training; the inkjet marking subunit is used to generate inkjet instructions and inkjet marks based on the track weld defect information and the track weld defect point coordinates, and send them to the inkjet module.

9. The intelligent detection and positioning system for metal roof weld defects based on OpenCV and YOLO according to any one of claims 1 to 8, characterized in that: The solid-state laser radar scanning control unit controls the solid-state laser radar module to scan and establish a refined model of the metal roof, and the RTK positioning control unit controls the high-precision coordinates recorded by the positioning module as the map for the inspection robot to conduct the next metal roof inspection.

Images