Multi-modal data fusion unmanned aerial vehicle structure three-dimensional detection and defect evaluation equipment

Abstract

The invention provides a multi-modal data fusion unmanned aerial vehicle structure three-dimensional detection and defect evaluation device, and the device comprises an unmanned aerial vehicle body, wings installed at the two sides of the unmanned aerial vehicle body, and a camera body installed at the bottom of the unmanned aerial vehicle body. The defect evaluation equipment comprises a concrete defect detection mechanism fixed to the surface of the unmanned aerial vehicle body, a driving assembly is arranged at the top of the camera body, and scales used for detecting the telescopic length of the driving assembly are arranged at the top of the camera body. According to the multi-modal data fusion unmanned aerial vehicle structure three-dimensional detection and defect evaluation equipment provided by the invention, a linkage mode is adopted, the aperture size is synchronously adjusted and the optical filter is switched through the linkage driving assembly, and adjustment can be quickly made according to the real-time illumination condition, so that detailed information of a concrete structure is more accurately captured, and the detection accuracy is improved. And the accuracy of defect detection is further enhanced.

Description

Technical Field

[0001] The present invention relates to the field of concrete defect detection, and in particular to a multi-modal data fusion unmanned aerial vehicle (UAV) structure three-dimensional detection and defect assessment device. Background Art

[0002] The safety and stability of concrete structures are crucial in infrastructure such as buildings, bridges, and tunnels. Over time, concrete structures are subject to a variety of factors, including environmental erosion such as wind and rain, temperature fluctuations, chemical corrosion, loads, and material aging, resulting in defects such as cracks, spalling, holes, and steel corrosion.

[0003] Traditional manual inspection methods mainly rely on inspectors to check the surface of the structure one by one based on their experience and visual observation. This method is not only labor-intensive and inefficient, but the inspection results are also greatly affected by the subjective factors of the inspectors, which is prone to missed inspections and false inspections. For some high-altitude, narrow or complex structures, manual inspection also has great safety risks, such as: Destructive testing: Destructive testing methods such as core drilling and pull-out methods can directly obtain defect information inside the structure, but will cause certain damage to the structure, affecting its normal use and durability. In addition, destructive testing methods can usually only be used to inspect local areas, making it difficult to fully reflect the overall condition of the structure;

[0004] Existing nondestructive testing methods, such as ultrasonic testing and radar testing, have overcome the shortcomings of manual and destructive testing to a certain extent, but they also have some limitations. For example, ultrasonic testing requires high operator skills, and the test results are easily affected by factors such as the distribution of aggregates and steel bars within the concrete. Although radar testing can quickly obtain information inside the structure, its detection accuracy for some shallow defects is limited.

[0005] With the rapid development of drone technology, drones have been increasingly widely used in the field of structural inspection. When conducting inspections using drone technology, cameras are key data acquisition components in drone inspection equipment. However, the functions of cameras currently carried by drones are relatively simple and lack adaptability to different lighting conditions and inspection requirements. For example, in strong light environments, visible light images may be overexposed, resulting in loss of image details. In addition, fixed aperture lenses are generally used to detect concrete defects, which cannot be adjusted according to different inspection scenarios, affecting the image acquisition quality.

[0006] Therefore, it is necessary to provide a new multimodal data fusion UAV structure three-dimensional detection and defect assessment equipment to solve the above technical problems. Summary of the Invention

[0007] In order to solve the above technical problems, the present invention provides a three-dimensional detection and defect assessment device for UAV structures with multimodal data fusion.

[0008] The multimodal data fusion unmanned aerial vehicle (UAV) structure three-dimensional detection equipment provided by the present invention includes: an UAV body, wings installed on both sides of the UAV body, and a camera body installed at the bottom of the UAV body. The defect assessment equipment includes a concrete defect detection mechanism fixed to the surface of the UAV body, a drive assembly is provided on the top of the camera body, and a scale for detecting the telescopic length of the drive assembly is provided on the top of the camera body. A filter ring and an aperture are provided inside the camera body, and a filter is rotatably provided inside the filter ring. The filter ring is located on one side of the aperture. One end of the drive assembly passes through the surface of the camera body and is fixedly connected to the surfaces of the filter ring and the aperture, respectively.

[0009] When the driving assembly is started, the driving assembly drives the filter ring and the aperture to rotate, so that the filter ring and the aperture are in the filter switching state and the aperture aperture adjustment state respectively.

[0010] Preferably, the concrete defect detection mechanism includes a GPS positioning sensor, a visible light + thermal imaging sensor, a lidar sensor, a voiceprint collection system, a millimeter wave radar and an IMU sensor. The GPS positioning sensor is fixed on the top of the drone body, the visible light + thermal imaging sensor and the lidar sensor are respectively fixed on the bottom of the two wings, the voiceprint collection system and the millimeter wave radar are respectively fixed on the bottom of the drone body, and the IMU sensor is fixed inside the drone body.

[0011] Preferably, a base is further provided between the camera body and the drone body, the camera body is fixed on the base, the other end of the base is fixedly connected to the bottom of the drone body, side panels are fixed on both sides of the camera body, the two side panels are fixedly connected via a mounting plate, the mounting plate is located on the top of the camera body, two slide grooves 1 are provided on the surface of the mounting plate, the inner walls of the two slide grooves 1 are provided with a limiting groove 1, a laser ranging sensor is fixed on the inner side wall of one of the side panels, and a drive assembly is fixed on the inner side wall of the other side panel;

[0012] The driving assembly includes an electric push rod, which is fixed on the inner wall of the other side plate, and two vertical plates are sliding on the top of the two slide grooves, and the bottom of the two vertical plates is provided with connecting parts. Both sides of the two connecting parts slide on the inner wall of the limiting groove one, and one end of the two connecting parts passes through the surface of the camera body and is fixedly connected to the filter ring and the surface of the aperture respectively. The two vertical plates are fixedly connected by a horizontal plate, and the horizontal plate is located above the mounting plate. The output end of the electric push rod is fixedly connected to the middle part of the horizontal plate, and a scale for detecting the telescopic length of the electric push rod is also provided at the edge of one of the slide grooves.

[0013] Preferably, the two connecting members include connecting rod 1, connecting rod 2, telescopic plate 1, telescopic plate 2 and connecting plate, the fixed ends of telescopic plate 1 and telescopic plate 2 are respectively fixedly connected to the bottoms of the two vertical plates, the movable ends of telescopic plate 1 and telescopic plate 2 are respectively fixedly connected to one end of connecting rod 1 and connecting rod 2, sliders are fixed on both sides of the fixed ends of telescopic plate 1 and telescopic plate 2, the telescopic plate 1 and telescopic plate 2 slide on the inner side walls of the two limit grooves 1 respectively through the sliders, two through grooves are provided on the top of the drone body, the other ends of connecting rod 1 and connecting rod 2 respectively pass through the two through grooves and are fixedly connected to the surfaces of the aperture and the filter.

[0014] Preferably, an optical unit is fixed inside the camera body, and the optical unit is fixed with an image processing circuit, a photoelectric sensor, a rear lens group, a front lens group and a front lens group in sequence from the inside to the outside, and the image processing circuit, the photoelectric sensor, the rear lens group, the front lens group and the front lens group are all fixedly connected to the bottom wall of the camera body, the photoelectric sensor is electrically connected to the image processing circuit, the filter is located between the photoelectric sensor and the rear lens group, the aperture is located between the rear lens group and the front lens group, and the filter and the aperture are respectively fixedly connected to the bottom wall of the camera body.

[0015] Preferably, the outer side of the filter ring is fixedly connected to the bottom wall of the camera body through a fixing block, the bottom wall of the filter ring is provided with a light outlet, and a turntable 1 is provided at the center of the bottom wall of the filter ring by rotating through a rotating shaft, and the turntable 1 is located in the middle of the rotating shaft, the size of the turntable 1 is smaller than the filter ring, and a gap is formed between the turntable 1 and the filter ring, a connecting rod 3 is fixed at the edge of the turntable 1, the surface of the connecting rod 3 slides in the gap, the other end of the connecting rod 3 is fixedly connected to the other end of the connecting rod 2, the surface of the turntable 1 is provided with a plurality of array-distributed grooves, and the inner walls of the plurality of grooves are fixed with filters, a cover plate is provided on the top of the turntable 1, the cover plate is connected to the rotating shaft by a knob thread, and a light inlet and an observation port are provided on the surface of the cover plate, the observation port is located on one side of the light inlet, and the light inlet is located directly above the light outlet.

[0016] Preferably, a second turntable is provided on the top of the aperture through a rotating rod, the other end of the connecting rod is fixedly connected to the side of the second turntable, a guide groove is provided on the surface of the aperture, and a plurality of equally spaced limiting grooves are provided on the surface ring of the second turntable, a plurality of rotating posts are further provided between the aperture and the second turntable, one end of the plurality of rotating posts slides on the bottom wall of the guide groove, and the other end of the plurality of rotating posts slides on the inner side wall of the limiting groove, a light shielding plate is fixed to the middle of the plurality of rotating posts, and the light shielding plate is located between the second turntable and the aperture;

[0017] When the connecting rod 1 is driven to drive the turntable 2 to rotate at the top of the aperture, the two ends of the multiple rotating columns slide on the guide groove and the inner wall of the limit groove respectively. At this time, one end of the multiple light-shielding plates converges or expands toward the middle, so that the aperture of the aperture surface is in an adjusted state.

[0018] Preferably, the centers of the light inlet, the light outlet and the aperture are aligned with the center of the surface of the rear lens group.

[0019] Defect assessment equipment for three-dimensional inspection of structures using drones with multimodal data fusion, including a sensor module for collecting multimodal data of concrete structures;

[0020] Data acquisition and preprocessing module, used to collect and preliminarily process the concrete structure data collected by sensors, including data cleaning, noise and abnormal data removal and data synchronization;

[0021] Data fusion module, used to fuse concrete structure data of different modes;

[0022] A 3D reconstruction and model generation module is used to generate a 3D model of the concrete structure based on the fused concrete structure data;

[0023] Defect assessment and diagnosis module, used to assess and diagnose defects in concrete structures in 3D models;

[0024] Flight control and path planning module, used to control the flight of the UAV and plan the flight path;

[0025] The human-computer interaction and result display module is used to control the UAV, satellite and ground receiving equipment to exchange information, and feed back the results to the ground receiving equipment.

[0026] Preferably, in the data fusion module, the Canny algorithm is used to extract edge information of the concrete structure in the image and identify the contours of cracks and holes on the surface of the concrete structure.

[0027] Compared with related technologies, the multimodal data fusion UAV structure 3D detection and defect assessment equipment provided by the present invention has the following beneficial effects:

[0028] 1. The present invention provides a rotatable filter ring inside the camera body, and a filter is rotatably arranged inside the filter ring. Under different lighting conditions, the filter ring is driven to rotate by a driving component to switch the filters, which can effectively filter out interfering light of different wavelengths, improve the clarity and contrast of the image, and thus more accurately capture the details of the concrete structure, further enhancing the accuracy of defect detection. An aperture is also provided inside the camera body, and the driving component can drive the aperture to rotate to adjust the aperture. In a strong light environment, the aperture can be appropriately reduced to reduce the amount of light entering the lens, avoid image overexposure, and retain more image details; in a weak light environment, the aperture can be appropriately increased to increase the amount of light entering the lens, improve the brightness of the image, and make the texture, cracks and other details of the concrete surface clearly visible. This solves the problems that the current cameras carried by drones have relatively single functions and lack adaptability to different lighting conditions and detection requirements.

[0029] 2. The present invention adopts a linkage approach, with the linkage drive assembly synchronously adjusting the aperture size and switching the filter. This allows for rapid adjustments based on real-time lighting conditions. Under strong direct sunlight, the linkage adjustment reduces the aperture and switches to a filter that filters out strong light, effectively preventing image overexposure and preserving detailed information on the surface of the structure. In low-light environments, the aperture is increased and a filter suitable for low light is switched to improve image brightness and clarity, ensuring the stable and reliable quality of the collected image data and providing a high-quality data foundation for subsequent defect detection and assessment.

[0030] 3. The linkage method adopted by the present invention eliminates the need for manual filter switching and aperture adjustment, which greatly reduces the time and cost of manual intervention and improves the degree of automation of detection. When inspecting large-scale concrete buildings, manual operation may require multiple workers to spend a lot of time adjusting parameters. The linkage method can automatically complete filter switching and aperture adjustment according to the preset flight path and detection parameters, quickly obtain high-quality image data, and improve detection efficiency.

[0031] 4. The present invention provides a scale for detecting the telescopic length of the drive assembly on the top of the camera body. The scale serves as a reference standard for equipment calibration to ensure that the adjustment parameters between different devices are comparable. During the use of the equipment, the aperture and filter are regularly calibrated according to the scale to ensure the stability of the equipment performance, improve the accuracy and consistency of the test results, and facilitate the standardization and normalization of the detection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the overall structure of the multimodal data fusion UAV structure three-dimensional detection equipment provided by the present invention;

[0033] Figure 2 This is a schematic diagram of the structure of the bottom of the drone body;

[0034] Figure 3 Schematic diagram of the structure of the camera body;

[0035] Figure 4 It is a structural diagram of the top of the camera body;

[0036] Figure 5 This is a schematic diagram of the main view structure inside the camera body;

[0037] Figure 6 for Figure 5 Enlarged view of point A in the middle;

[0038] Figure 7 This is a schematic diagram of the aperture installation;

[0039] Figure 8 Schematic diagram of the aperture structure Figure 1 ;

[0040] Figure 9 Schematic diagram of the aperture structure Figure 2 ;

[0041] Figure 10 Schematic diagram of the structure of the filter body;

[0042] Figure 11 Schematic diagram of the split structure of the filter body;

[0043] Figure 12 Schematic diagram of the overall structure of the multimodal data fusion UAV structure 3D detection equipment provided by the present invention Figure 2 ;

[0044] Figure 13 A schematic diagram of the process flow of the defect assessment equipment for three-dimensional detection of UAV structures using multimodal data fusion provided by the present invention.

[0045] Numbers in the figure: 1. UAV body; 11. GPS positioning sensor; 12. Wing; 121. Visible light + thermal imaging sensor; 122. LiDAR sensor; 13. Roller; 14. Voiceprint acquisition system; 15. Millimeter wave radar; 16. IMU sensor; 2. Base; 21. Camera body; 211. Through slot; 22. Side panel; 23. Mounting plate; 231. Slide slot 1; 232. Limit slot 1; 233. Scale; 24. Laser ranging sensor; 25. Electric push rod; 26. Vertical plate; 261. Connecting rod 1; 262. Connecting rod 2; 263. Slider; 264. Telescopic Plate 1; 265, telescopic plate 2; 27, horizontal plate; 28, connecting plate; 3, image processing circuit; 31, photoelectric sensor; 4, filter ring; 41, fixing block; 42, light outlet; 43, rotating shaft; 431, turntable 1; 432, connecting rod 3; 433, groove; 434, filter; 44, cover plate; 441, light inlet; 442, observation port; 45, knob; 5, rear lens group; 6, aperture; 61, turntable 2; 611, guide groove; 62, limit groove; 63, sunshade; 64, rotating column; 7, front lens group; 71, front lens group; 8, satellite; 9, ground receiving equipment. DETAILED DESCRIPTION

[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0047] Please refer to Figures 1 to 13 ,in, Figure 1 This is a schematic diagram of the overall structure of the multimodal data fusion UAV structure three-dimensional detection equipment provided by the present invention; Figure 2 This is a schematic diagram of the structure of the bottom of the drone body; Figure 3 Schematic diagram of the structure of the camera body; Figure 4 It is a structural diagram of the top of the camera body; Figure 5 This is a schematic diagram of the main view structure inside the camera body; Figure 6 for Figure 5 Enlarged view of point A in the middle; Figure 7 This is a schematic diagram of the aperture installation; Figure 8 Schematic diagram of the aperture structure Figure 1 ; Figure 9 Schematic diagram of the aperture structure Figure 2 ; Figure 10Schematic diagram of the structure of the filter body; Figure 11 Schematic diagram of the split structure of the filter body; Figure 12 Schematic diagram of the overall structure of the multimodal data fusion UAV structure 3D detection equipment provided by the present invention Figure 2 ; Figure 13 A schematic diagram of the process flow of the defect assessment equipment for three-dimensional detection of UAV structures using multimodal data fusion provided by the present invention.

[0048] In some embodiments, such as Figures 1 to 11 As shown, a multimodal data fusion unmanned aerial vehicle structure three-dimensional detection device includes a drone body 1, wings 12 installed on both sides of the drone body 1, and a camera body 21 installed at the bottom of the drone body 1. The defect assessment device includes a concrete defect detection mechanism fixed to the surface of the drone body 1, a drive assembly is provided on the top of the camera body 21, and a scale 233 for detecting the telescopic length of the drive assembly is provided on the top of the camera body 1. The interior of the camera body 21 is provided with a filter ring 4 and an aperture 6. The interior of the filter ring 4 is rotatably provided with a filter 434. The filter ring 4 is located on one side of the aperture 6. One end of the drive assembly passes through the surface of the camera body 21 and is fixedly connected to the surfaces of the filter ring 4 and the aperture 6 respectively.

[0049] When the driving assembly is started, the driving assembly drives the filter 4 and the aperture 6 to rotate, so that the filter 4 and the aperture 6 are in the filter 434 switching and aperture adjustment states respectively;

[0050] The concrete defect detection mechanism includes a GPS positioning sensor 11, a visible light + thermal imaging sensor 121, a lidar sensor 122, a voiceprint collection system 14, a millimeter wave radar 15, and an IMU sensor 16. The GPS positioning sensor 11 is fixed to the top of the drone body 1, the visible light + thermal imaging sensor 121 and the lidar sensor 122 are respectively fixed to the bottom of the two wings 12, the voiceprint collection system 14 and the millimeter wave radar 15 are respectively fixed to the bottom of the drone body 1, and the IMU sensor 16 is fixed inside the drone body 1;

[0051] The camera body 21 is installed at the bottom of the drone body 1, which can directly capture the surface image of the concrete structure at close range to obtain detailed information such as color and texture, meeting the inspection requirements. The bottom installation has little effect on the flight stability of the drone body 1, making it easy to operate and maintain.

[0052] IMU sensor 16: located inside the drone body 1, used to measure the attitude, acceleration and angular velocity of the drone body 1. Installing it inside the body can avoid external interference and ensure measurement accuracy;

[0053] GPS sensor 11: used to receive satellite signals;

[0054] LiDAR sensor 122: emits a laser beam and measures the reflected light. Installation on the side of wing 12 can expand the detection range and is suitable for different detection scenarios;

[0055] Millimeter-wave radar 15: The millimeter-wave radar 15 can penetrate vegetation and light obstructions. Installed on the bottom of the drone body 1, it ensures effective detection of defects on the surface of concrete structures. The installation position and angle of the millimeter-wave radar 15 can be flexibly adjusted according to detection requirements.

[0056] Visible light + thermal imaging sensor 121: captures infrared radiation from the concrete surface;

[0057] The voiceprint collection system 14 is a microphone array used to capture abnormal vibration noise inside or on the surface of concrete.

[0058] Specifically, a rotatable filter ring 4 is provided inside the camera body 21, and a filter 434 is rotatably provided inside the filter ring 4. Under different lighting conditions, the filter ring 4 is driven by the driving component to rotate to realize the switching of the filter 434, which can effectively filter out interfering light of different wavelengths, improve the clarity and contrast of the image, thereby more accurately capturing the details of the structure, and further enhancing the accuracy of defect detection. An aperture 6 is also provided inside the camera body 21, and the driving component can drive the aperture 6 to rotate to adjust the aperture. In a strong light environment, the aperture of the aperture 6 can be appropriately reduced to reduce the amount of light entering the lens, avoid image overexposure, and retain more image details; in a weak light environment, the aperture of the aperture 6 can be appropriately increased to increase the amount of light entering the lens, improve the brightness of the image, and make the texture, cracks and other details of the concrete surface clearly visible, thereby solving the problems that the current cameras carried by drones have relatively single functions and lack adaptability to different lighting conditions and detection requirements.

[0059] Furthermore, a linkage method is adopted to synchronously adjust the size of aperture 6 and switch filter 434 through the linkage drive component, which can quickly make adjustments according to the real-time lighting conditions. Under direct strong light, the linkage adjustment shrinks the aperture of aperture 6 and switches to the filter 434 that filters strong light, effectively avoiding image overexposure and retaining detailed information on the surface of the structure; in a low-light environment, the aperture of aperture 6 is increased and switched to the filter 434 suitable for low light, improving the brightness and clarity of the image, ensuring the quality of the collected image data is stable and reliable, and providing a high-quality data foundation for subsequent defect detection and evaluation.

[0060] Furthermore, with the linkage method, there is no need to manually switch the filter 434 and adjust the aperture 6, which greatly reduces the time and cost of manual intervention and improves the degree of automation of detection. When inspecting large-scale concrete buildings, manual operation may require multiple workers to spend a lot of time adjusting parameters. The linkage method can automatically complete the switching of the filter 434 and the adjustment of the aperture 6 according to the preset flight path and detection parameters, quickly obtaining high-quality image data and improving detection efficiency.

[0061] In addition, by providing a scale 233 for detecting the telescopic length of the drive assembly on the top of the camera body 21, the scale 233 serves as a reference standard for equipment calibration, ensuring that the adjustment parameters between different devices are comparable. During the use of the equipment, the aperture 6 and the filter 434 are regularly calibrated according to the scale 233, which can ensure the stability of the equipment's performance, improve the accuracy and consistency of the test results, and facilitate the standardization and normalization of the detection work.

[0062] In some embodiments, reference Figures 1 to 6 As shown, a base 2 is further provided between the camera body 21 and the drone body 1, and the camera body 21 is fixed on the base 2, and the other end of the base 2 is fixedly connected to the bottom of the drone body 1. The provision of the base 2 enhances the connection stability between the camera body 21 and the drone body 1, ensuring that the camera body 21 can remain stable during the flight of the drone, thereby avoiding the accuracy of data acquisition being affected by shaking. Side panels 22 are fixed on both sides of the camera body 21, and the two side panels 22 are fixedly connected by a mounting plate 23. The mounting plate 23 is located at the top of the camera body 21, and two slide grooves 231 are provided on the surface of the mounting plate 23. The inner walls of the two slide grooves 231 are both provided with a limiting groove 232. A laser ranging sensor 24 is fixed on the inner side wall of one of the side panels 22, and a driving component is fixed on the inner side wall of the other side panel 22.

[0063] The slide groove 1 231 and the limiting groove 1 232 on the mounting plate 23 provide a motion track for the riser 26 and the connector. The sliding of the slider 263 in the limiting groove 1 232 ensures the stability of the connector during movement, allowing the thrust of the electric push rod 25 to be accurately transmitted to the aperture 6 and the filter 4, thereby adjusting the size of the aperture 6 and the switching of the filter 434.

[0064] The driving assembly includes an electric push rod 25, which is fixed to the inner wall of the other side plate 22. Two vertical plates 26 slide on the top of the two slide grooves 231. The bottom of the two vertical plates 26 is provided with a connecting piece. Both sides of the two connecting pieces slide on the inner wall of the limit groove 232. One end of the two connecting pieces passes through the surface of the camera body 1 and is fixedly connected to the surface of the filter ring 4 and the aperture 6 respectively. The two vertical plates 26 are fixedly connected by a horizontal plate 27. The horizontal plate 27 is located above the mounting plate 23. The output end of the electric push rod 25 is fixedly connected to the middle of the horizontal plate 27. A scale 233 for detecting the telescopic length of the electric push rod 25 is also provided on the edge of one of the slide grooves 231.

[0065] The laser distance sensor 24 is used to measure the extension and retraction distance of the electric push rod 25, that is, to measure the distance that the electric push rod 25 drives the vertical plate 26 and the horizontal plate 27 to slide on the scale 233 on the surface of the mounting plate 23, thereby facilitating the understanding of the specific adjustment range of the aperture 6 and the specific position of the filter 434 when the electric push rod 25 is pushed;

[0066] Specifically, a linkage method is adopted, whereby the electric push rod 25 in the driving assembly drives the telescopic plate 1 264 and the telescopic plate 265 at the bottom of the vertical plate 26 to slide along the inner wall of the slide groove 1 231, thereby adjusting the size of the aperture 6 in the camera body 1 at the bottom thereof. While the size of the aperture 6 is adjusted, the filter 434 in the filter ring 4 is switched synchronously. For example, when the ND filter is switched to reduce the amount of light entering, the aperture 6 is linked to increase to increase the amount of light entering. When the polarization filter is switched to reduce the reflection of the water surface but the light intensity does not change much, the aperture 6 is adjusted to optimize the depth of field and dynamic range of the image.

[0067] The two connecting parts include a connecting rod 1 261, a connecting rod 262, a telescopic plate 1 264, a telescopic plate 265 and a connecting plate 28. The fixed ends of the telescopic plate 1 264 and the telescopic plate 265 are respectively fixedly connected to the bottoms of the two vertical plates 26, and the movable ends of the telescopic plate 1 264 and the telescopic plate 265 are respectively fixedly connected to one end of the connecting rod 1 261 and the connecting rod 2 262. Sliders 263 are fixed on both sides of the fixed ends of the telescopic plate 1 264 and the telescopic plate 265. The telescopic plate 1 264 and the telescopic plate 265 slide on the inner side walls of the two limiting grooves 1 232 through the sliders 263. Two through grooves 211 are provided on the top of the drone body 1. The other ends of the connecting rod 1 261 and the connecting rod 2 262 respectively pass through the two through grooves 211 and are fixedly connected to the surfaces of the aperture 6 and the filter ring 4.

[0068] Specifically, when the electric push rod 25 pushes the horizontal plate 27 to move, the telescopic plate 1 264 and the telescopic plate 2 265 are extended and retracted in length, and the fixed ends of the telescopic plate 1 264 and the telescopic plate 2 265 slide on the inner walls of the two limiting grooves 1 232 respectively through the slider 263.

[0069] In some embodiments, reference Figure 5 As shown, an optical unit is fixed inside the camera body 21, and the optical unit is fixed with an image processing circuit 3, a photoelectric sensor 31, a rear lens group 5, a front lens group 7 and a front lens group 71 from the inside to the outside. The image processing circuit 3, the photoelectric sensor 31, the rear lens group 5, the front lens group 7 and the front lens group 71 are all fixedly connected to the bottom wall of the camera body 21. The photoelectric sensor 31 is electrically connected to the image processing circuit 3. The photoelectric sensor 31 is responsible for converting the received light signal into an electrical signal, and then transmitting it to the image processing circuit 3 for further processing. The filter ring 4 is located between the photoelectric sensor 31 and the rear lens group 5, and the aperture 6 is located between the rear lens group 5 and the front lens group 7. The filter ring 4 and the aperture 6 are respectively fixedly connected to the bottom wall of the camera body 21. The filter ring 4 is located between the photoelectric sensor 31 and the rear lens group 5. By rotating the filter ring to switch different filters 434, interference light of different wavelengths can be effectively filtered out.

[0070] In some embodiments, reference Figures 4 to 11 As shown, the outer side of the filter ring 4 is fixedly connected to the bottom wall of the camera body 21 through a fixing block 41, thereby ensuring the stability of the filter ring 4 inside the camera body 21. A light outlet 42 is provided on the bottom wall of the filter ring 4. A turntable 1 431 is provided at the center of the bottom wall of the filter ring 4 via a rotating shaft 43, and the turntable 1 431 is located in the middle of the rotating shaft 43. The size of the turntable 1 431 is smaller than the filter ring 4, and a gap is formed between the turntable 1 431 and the filter ring 4. A connecting rod 3 432 is fixed to the edge of the turntable 1 431. The surface slides in the gap, the other end of the connecting rod 3 432 is fixedly connected to the other end of the connecting rod 262, the surface of the turntable 1 431 is provided with a plurality of grooves 433 distributed in an array, the inner walls of the plurality of grooves 433 are fixed with filters 434, a cover plate 44 is provided on the top of the turntable 1 431, and the cover plate 44 is threadedly connected to the rotating shaft 43 via a knob 45, and a light inlet 441 and an observation port 442 are provided on the surface of the cover plate 44, the observation port 442 is located on one side of the light inlet 441, and the light inlet 441 is located directly above the light outlet 42;

[0071] A second turntable 61 is provided at the top of the aperture 6, which is rotatable by a rotating rod. The other end of the connecting rod 261 is fixedly connected to the side of the second turntable 61. A guide groove 611 is provided on the surface of the aperture 6, and a plurality of equally spaced limiting grooves 62 are provided on the surface ring of the second turntable 61. A plurality of rotating posts 64 are also provided between the aperture 6 and the second turntable 61. One end of each of the rotating posts 64 slides on the bottom wall of the guide groove 611, and the other end of each of the rotating posts 64 slides on the inner side wall of the limiting groove 62. A light shielding plate 63 is fixed to the middle of each of the rotating posts 64, and the light shielding plate 63 is located between the second turntable 61 and the aperture 6.

[0072] When the driving connecting rod 1 261 drives the turntable 2 61 to rotate on the top of the aperture 6, the ends of the multiple rotating pillars 64 slide on the inner side walls of the guide groove 611 and the limiting groove 62 respectively. At this time, one end of the multiple light shielding plates 63 converges or spreads toward the middle, so that the aperture of the aperture 6 surface is in an adjustable state;

[0073] The centers of the light inlet 441 , the light outlet 42 and the aperture 6 are aligned with the center of the surface of the rear lens group 5 .

[0074] Specifically, when the electric push rod 25 is driven, it drives the connecting rod 1 261 to move, thereby causing the turntable 2 61 to rotate around the rotating rod at the top of the aperture 6. A guide groove 611 is provided on the surface of the aperture 6, and a plurality of equally spaced limiting grooves 62 are provided on the surface ring of the turntable 2 61. One end of the plurality of rotating posts 64 slides on the bottom wall of the guide groove 611, and the other end slides on the inner side wall of the limiting groove 62. When the turntable 2 61 rotates, due to the polygonal shape of the guide groove 611 and the limitation of the limiting groove 62, the two ends of the rotating posts 64 slide on the guide groove 611. The groove 611 and the inner wall of the limit groove 62 slide, causing the rotating column 64 to move and change its angle. A light shielding plate 63 is fixed in the middle of the multiple rotating columns 64. The light shielding plate 63 is located between the turntable 2 61 and the aperture 6. As the rotating column 64 moves and the angle changes, one end of the multiple light shielding plates 63 converges or expands toward the middle. When the light shielding plates 63 converge toward the middle, the area of ​​the aperture 6 surface not blocked by the light shielding plates 63 decreases, that is, the aperture diameter becomes smaller; when the light shielding plates 63 expand, the aperture 6 surface not blocked by the light shielding plates 63 The area of ​​the part increases, that is, the aperture becomes larger, thereby realizing the adjustment of the size of the aperture 6. When the electric push rod 25 drives the connecting rod 1 261 to move, the connecting rod 262 also produces corresponding movement. The other end of the connecting rod 262 is fixedly connected to the other end of the connecting rod 3 432. The surface of the connecting rod 3 432 slides in the gap formed by the filter ring 4 and the turntable 1 431, and the connecting rod 3 432 is fixed at the edge of the turntable 1 431. When the connecting rod 262 moves, it drives the connecting rod 3 432 to move, and the filter ring 4 and the turntable 1 431 are adjusted. The turntable 431 is rotated about the rotation axis 43. The surface of the turntable 431 is provided with a plurality of grooves 433 distributed in an array. The inner walls of the plurality of grooves 433 are fixed with filters 434. As the turntable 431 rotates, the different grooves 433 and the filters 434 fixed therein are rotated in sequence to the optical path between the light inlet 441 and the light outlet 42. When a filter 434 required for a specific purpose is rotated to the optical path, light passes through the filter 434 and enters the subsequent optical system, thereby realizing the switching of the filter 434.

[0075] Furthermore, by using the electric push rod 25 as a driving source, the automatic operation of adjusting the size of the aperture 6 and switching the filter 434 is achieved without the need for manual adjustment, thereby improving the detection efficiency and convenience of operation and reducing human errors.

[0076] In some embodiments, reference Figure 12 as well as Figure 13As shown, a defect assessment device for three-dimensional detection of structures using drones with multimodal data fusion includes a sensor module for collecting multimodal data of concrete structures, including position data obtained by a GPS positioning sensor 11, visible light + thermal imaging sensor 121 collecting visible light and thermal imaging image data, lidar sensor 122 collecting point cloud data, voiceprint collection system 14 collecting voiceprint data, millimeter wave radar 15 collecting radar data, IMU sensor 16 collecting posture data, and camera body 21 collecting image data;

[0077] The data acquisition and preprocessing module is used to collect and preliminarily process the concrete structure data collected by sensors, including data cleaning, removing noise and abnormal data, and data synchronization to ensure that the data from different sensors are aligned in time, and converting data in different formats into a unified format for subsequent processing;

[0078] The data fusion module is used to fuse concrete structure data of different modes. First, features are extracted from concrete structure data of different modes, such as color and texture features from visible light images, temperature features from thermal images, spatial geometric features from point cloud data, frequency and amplitude features from voiceprint data, distance and speed features from radar data, and angle and acceleration features from posture data. According to actual needs, the Canny algorithm is used to extract edge information of concrete structures in images and identify the contours of cracks and holes on the surface of concrete structures. Alternatively, a feature-level fusion algorithm is used to fuse features of different modes in the early stage of data processing. A decision-level fusion algorithm is used to fuse the detection results of each mode to generate fused feature data of concrete structures.

[0079] The 3D reconstruction and model generation module is used to generate a 3D model of the concrete structure based on the fused concrete structure data. It selects a point cloud-based 3D reconstruction algorithm and uses the point cloud data collected by the LiDAR to perform 3D reconstruction. The fused data is processed using the selected point cloud-based 3D reconstruction algorithm to generate a 3D model, including a point cloud model and a mesh model. At this time, the generated 3D model is optimized, such as using the PMVS algorithm module to densify the sparse point cloud model, remove noise and redundant points, and improve the accuracy and quality of the model.

[0080] The defect assessment and diagnosis module is used to evaluate and diagnose defects in concrete structures in 3D models. It extracts possible defect features from the 3D model, such as the shape, size, direction of cracks, and the degree of surface unevenness. It then uses deep learning algorithms to identify and classify the extracted defect features and determine the type of defect, such as cracks, corrosion, and deformation. Based on the type, size, and location of the defect, it then assesses and diagnoses the safety and stability of the structure, determining the severity and impact of the defect.

[0081] The flight control and path planning module is used to control the UAV's flight and plan its flight path. It first uses data obtained by the sensor module to perceive the environment, including the location, size, shape of obstacles, and the height of the terrain. It then selects a task allocation algorithm based on a genetic algorithm or an auction mechanism. Based on the environmental perception results and mission requirements, it uses the selected task allocation algorithm based on the genetic algorithm or auction mechanism to plan the UAV's flight path. The flight control system controls the UAV's flight attitude, speed, and altitude to ensure that the UAV completes its mission safely and efficiently.

[0082] The human-computer interaction and result display module is used to control the information exchange between the UAV, satellite 8 and ground receiving equipment 9, and to display the three-dimensional reconstruction model, defect assessment results, flight path and other information in a three-dimensional graphical manner on the result display interface on the ground receiving equipment 9 in an intuitive manner.

[0083] The circuits and controls involved in the present invention are all prior art and will not be described in detail here.

[0084] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. The multimodal data fusion UAV structure 3D detection equipment is characterized by: The invention comprises an unmanned aerial vehicle (UAV) body (1), wings (12) mounted on both sides of the UAV body (1), and a camera body (21) mounted on the bottom of the UAV body (1), characterized in that the defect assessment device comprises a concrete defect detection mechanism fixed on the surface of the UAV body (1), a driving assembly is provided on the top of the camera body (21), a scale (233) for detecting the telescopic length of the driving assembly is provided on the top of the camera body (1), a filter ring (4) and an aperture (6) are provided inside the camera body (21), a filter (434) is rotatably provided inside the filter ring (4), the filter ring (4) is located on one side of the aperture (6), and one end of the driving assembly passes through the surface of the camera body (21) and is fixedly connected to the surfaces of the filter ring (4) and the aperture (6) respectively; When the driving component is started, the driving component drives the filter ring (4) and the aperture (6) to rotate, so that the filter ring (4) and the aperture (6) are respectively in the filter (434) switching state and the aperture (6) aperture adjustment state.

2. The multimodal data fusion 3D detection device for UAV structures according to claim 1, characterized in that: The concrete defect detection mechanism comprises a GPS positioning sensor (11), a visible light + thermal imaging sensor (121), a laser radar sensor (122), a voiceprint collection system (14), a millimeter wave radar (15) and an IMU sensor (16), wherein the GPS positioning sensor (11) is fixed on the top of the drone body (1), the visible light + thermal imaging sensor (121) and the laser radar sensor (122) are respectively fixed on the bottom of the two wings (12), the voiceprint collection system (14) and the millimeter wave radar (15) are respectively fixed on the bottom of the drone body (1), and the IMU sensor (16) is fixed inside the drone body (1).

3. The multimodal data fusion 3D detection device for UAV structures according to claim 2, characterized in that: A base (2) is further provided between the camera body (21) and the drone body (1), the camera body (21) is fixed on the base (2), the other end of the base (2) is fixedly connected to the bottom of the drone body (1), side panels (22) are fixedly provided on both sides of the camera body (21), the two side panels (22) are fixedly connected via a mounting plate (23), the mounting plate (23) is located at the top of the camera body (21), two slide grooves (231) are provided on the surface of the mounting plate (23), the inner walls of the two slide grooves (231) are provided with a limiting groove (232), a laser ranging sensor (24) is fixed on the inner side wall of one of the side panels (22), and a driving assembly is fixed on the inner side wall of the other side panel (22); The driving assembly includes an electric push rod (25), which is fixed on the inner wall of the other side plate (22). Two vertical plates (26) are slid on the top of the two slide grooves (231). The bottom of the two vertical plates (26) is provided with a connecting piece. Both sides of the two connecting pieces slide on the inner wall of the limit groove (232). One end of the two connecting pieces passes through the surface of the camera body (1) and is fixedly connected to the surface of the filter ring (4) and the surface of the aperture (6) respectively. The two vertical plates (26) are fixedly connected by a horizontal plate (27). The horizontal plate (27) is located above the mounting plate (23). The output end of the electric push rod (25) is fixedly connected to the middle of the horizontal plate (27). A scale (233) for detecting the telescopic length of the electric push rod (25) is also provided at the edge of one of the slide grooves (231).

4. The multimodal data fusion 3D detection device for UAV structures according to claim 3 is characterized in that: The two connecting members include a connecting rod 1 (261), a connecting rod 2 (262), a telescopic plate 1 (264), a telescopic plate 2 (265) and a connecting plate (28), wherein the fixed ends of the telescopic plate 1 (264) and the telescopic plate 2 (265) are respectively fixedly connected to the bottoms of the two vertical plates (26), and the movable ends of the telescopic plate 1 (264) and the telescopic plate 2 (265) are respectively fixedly connected to one end of the connecting rod 1 (261) and the connecting rod 2 (262). Sliders (263) are fixed on both sides of the fixed end of the telescopic plate 2 (265), and the telescopic plate 1 (264) and the telescopic plate 2 (265) slide on the inner side walls of the two limiting grooves 1 (232) respectively through the slides (263). Two through grooves (211) are provided on the top of the drone body (1), and the other ends of the connecting rod 1 (261) and the connecting rod 2 (262) pass through the two through grooves (211) respectively and are fixedly connected to the surfaces of the aperture (6) and the filter ring (4).

5. The multimodal data fusion 3D detection device for UAV structures according to claim 4 is characterized in that: An optical unit is fixed inside the camera body (21), and an image processing circuit (3), a photoelectric sensor (31), a rear lens group (5), a front lens group (7), and a front lens group (71) are fixed to the optical unit in sequence from the inside to the outside. The image processing circuit (3), the photoelectric sensor (31), the rear lens group (5), the front lens group (7), and the front lens group (71) are all fixedly connected to the bottom wall of the camera body (21). The photoelectric sensor (31) is electrically connected to the image processing circuit (3). The filter ring (4) is located between the photoelectric sensor (31) and the rear lens group (5). The aperture (6) is located between the rear lens group (5) and the front lens group (7). The filter ring (4) and the aperture (6) are respectively fixedly connected to the bottom wall of the camera body (21).

6. The multimodal data fusion 3D detection device for UAV structures according to claim 5, characterized in that: The outer side of the filter ring (4) is fixedly connected to the bottom wall of the camera body (21) through a fixed block (41), the bottom wall of the filter ring (4) is provided with a light outlet (42), a turntable (431) is provided at the center of the bottom wall of the filter ring (4) through a rotating shaft (43), and the turntable (431) is located in the middle of the rotating shaft (43), the size of the turntable (431) is smaller than the filter ring (4), a gap is formed between the turntable (431) and the filter ring (4), a connecting rod (432) is fixed at the edge of the turntable (431), the surface of the connecting rod (432) slides in the gap, and the connecting rod (432) is provided at the bottom wall of the filter ring (4). The other end of the rod three (432) is fixedly connected to the other end of the connecting rod two (262); a plurality of array-distributed grooves (433) are provided on the surface of the turntable one (431); filters (434) are fixed on the inner walls of the plurality of grooves (433); a cover plate (44) is provided on the top of the turntable one (431); the cover plate (44) is threadedly connected to the rotating shaft (43) via a knob (45); a light inlet (441) and an observation port (442) are provided on the surface of the cover plate (44); the observation port (442) is located on one side of the light inlet (441), and the light inlet (441) is located directly above the light outlet (42).

7. The multimodal data fusion 3D drone structure device according to claim 6, characterized in that: A turntable 2 (61) is provided on the top of the aperture (6) through a rotating rod, and the other end of the connecting rod 1 (261) is fixedly connected to the side of the turntable 2 (61). A guide groove (611) is provided on the surface of the aperture (6), and a plurality of equally spaced limiting grooves (62) are provided on the surface ring of the turntable 2 (61). A plurality of rotating columns (64) are also provided between the aperture (6) and the turntable 2 (61), and one end of the plurality of rotating columns (64) slides on the bottom wall of the guide groove (611), and the other end of the plurality of rotating columns (64) slides on the inner side wall of the limiting groove (62). A light shielding plate (63) is fixed to the middle of the plurality of rotating columns (64), and the light shielding plate (63) is located between the turntable 2 (61) and the aperture (6); When the connecting rod 1 (261) is driven to drive the turntable 2 (61) to rotate at the top of the aperture (6), the two ends of the multiple rotating columns (64) slide on the inner side walls of the guide groove (611) and the limiting groove (62) respectively. At this time, one end of the multiple light shielding plates (63) converges or expands toward the middle, so that the aperture of the aperture (6) surface is in an adjusted state.

8. The multimodal data fusion 3D detection device for UAV structures according to claim 7, characterized in that: The centers of the light inlet (441), the light outlet (42) and the aperture (6) are aligned with the center of the surface of the rear lens group (5).

9. A defect assessment device for three-dimensional detection of structures using multimodal data fusion using unmanned aerial vehicles, applied to the three-dimensional detection device for structures using multimodal data fusion using unmanned aerial vehicles according to any one of claims 1 to 8, characterized in that: It includes a sensor module for collecting multimodal data of concrete structures; Data acquisition and preprocessing module, used to collect and preliminarily process the concrete structure data collected by sensors, including data cleaning, noise and abnormal data removal and data synchronization; Data fusion module, used to fuse concrete structure data of different modes; A 3D reconstruction and model generation module is used to generate a 3D model of the concrete structure based on the fused concrete structure data; Defect assessment and diagnosis module, used to assess and diagnose defects in concrete structures in 3D models; Flight control and path planning module, used to control the flight of the UAV and plan the flight path; The human-computer interaction and result display module is used to control the UAV, satellite (8) and ground receiving equipment (9) to perform information interaction and feed back the results to the ground receiving equipment (9).

10. The defect assessment device for three-dimensional detection of unmanned aerial vehicle structures using multimodal data fusion according to claim 9, characterized in that: In the data fusion module, the Canny algorithm is used to extract edge information of the concrete structure in the image and identify the contours of cracks and holes on the surface of the concrete structure.

Images