Highway slope efficient inspection method and system using unmanned aerial vehicle

Abstract

The invention discloses a road slope efficient inspection method and system using an unmanned aerial vehicle, and relates to the field of unmanned aerial vehicle inspection. The method is applied to an inspection platform, and comprises the following steps: obtaining a first road slope image collected by an unmanned aerial vehicle; dividing the first road slope image into a plurality of overlapped sub-images with the same size; calculating a local entropy and a gradient mean value of the plurality of overlapped sub-images; according to the local entropies and the gradient mean values of the plurality of overlapped sub-images, calculating illumination compensation coefficients of the plurality of overlapped sub-images; performing brightness compensation on the plurality of overlapped sub-images based on the illumination compensation coefficients of the plurality of overlapped sub-images to obtain a second road slope image; and carrying out disease identification on the second road slope image to generate an inspection report. By implementing the technical scheme provided by the invention, the problems of non-uniform brightness and color distortion of the image shot by the unmanned aerial vehicle in the current complex illumination environment are solved.

Description

Technical Field

[0001] This application relates to the technical field of drone inspection, specifically to a method and system for efficient inspection of highway slopes using drones. Background Technology

[0002] Due to its advantages such as high flight speed, wide coverage, and low cost, drone inspection has been widely used in highway slope engineering monitoring.

[0003] In existing technologies, drones use cameras to capture slope images, and then image processing algorithms identify defects such as cracks and landslides. However, actual inspections often face complex lighting environments, leading to uneven brightness and color distortion in the images, severely affecting the accuracy of subsequent defect feature extraction. Current mainstream illumination compensation methods mostly employ fixed-window global histogram equalization or Gamma correction, but these methods are prone to over-enhancement or under-compensation in non-uniform lighting scenarios caused by terrain undulations, resulting in fine features such as cracks being masked by noise and causing false detections.

[0004] Therefore, there is an urgent need for an efficient method and system for unmanned aerial vehicle (UAV) inspection of highway slopes that can adapt to complex lighting conditions. Summary of the Invention

[0005] To address the issues of uneven brightness and color distortion in images captured by drones under complex lighting conditions, this application provides a method and system for efficient inspection of highway slopes using drones.

[0006] In a first aspect, this application provides a method for efficient inspection of highway slopes using unmanned aerial vehicles (UAVs), applied to an inspection platform, the method comprising:

[0007] Acquire the first highway slope image captured by the drone;

[0008] The first highway slope image is divided into multiple overlapping sub-images of the same size;

[0009] Calculate the local entropy and gradient mean of the multiple overlapping sub-images;

[0010] The illumination compensation coefficients of the multiple overlapping sub-images are calculated based on the local entropy and gradient mean of the multiple overlapping sub-images.

[0011] Based on the illumination compensation coefficients of the multiple overlapping sub-images, brightness compensation is performed on the multiple overlapping sub-images to obtain a second highway slope image;

[0012] The second highway slope image is used to identify defects and generate an inspection report.

[0013] Optionally, before acquiring the first highway slope image collected by the UAV, the process may further include:

[0014] Obtain the topographic features of the target highway slope, including the slope variance and the mean slope curvature.

[0015] Calculate the terrain complexity of the target highway slope based on the terrain features.

[0016] The terrain complexity is matched with a preset shooting density database to obtain the shooting density corresponding to the terrain complexity of the target highway slope, so that the UAV can acquire the first highway slope image according to the shooting density corresponding to the terrain complexity of the target highway slope.

[0017] Optionally, calculating the terrain complexity of the target highway slope based on the terrain features further includes:

[0018] Calculate the vegetation coverage of the target highway slope;

[0019] When the vegetation coverage rate is greater than or equal to the preset vegetation coverage rate, the near-infrared band distribution map of the target highway slope is obtained.

[0020] Calculate the infrared band variance of the near-infrared band distribution map;

[0021] Based on the variance of the infrared band, determine the terrain complexity compensation coefficient of the target highway slope;

[0022] The terrain complexity is compensated for based on the terrain complexity compensation coefficient to obtain the target terrain complexity.

[0023] Optionally, dividing the first highway slope image into multiple overlapping sub-images of the same size specifically includes:

[0024] Assess the computational resources required for the first highway slope image;

[0025] The overlap ratio is determined based on the ratio between the computational resources required for the first highway slope image and the total computational resources of the inspection platform.

[0026] Convert the first highway slope image into a gradient image;

[0027] Identify multiple high gradient regions in the gradient image;

[0028] The region size is determined based on the region size within the multiple high gradient regions;

[0029] Based on the overlap ratio and the region size, the first highway slope image is divided into multiple overlapping sub-images of the same size.

[0030] Optionally, the step of calculating the illumination compensation coefficients of the multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images specifically involves:

[0031]

[0032] in, Let be the illumination compensation coefficient for the i-th overlapping sub-image. Let be the local entropy of the i-th overlapping sub-image. The maximum local entropy among multiple overlapping sub-images. Let be the mean gradient of the i-th overlapping sub-image. The maximum gradient mean among multiple overlapping sub-images. and These are the local entropy weighting coefficient and the gradient mean weighting coefficient, respectively. > , + =1.

[0033] Optionally, the step of performing brightness compensation on the multiple overlapping sub-images based on the illumination compensation coefficients of the multiple overlapping sub-images to obtain the second highway slope image specifically includes:

[0034] Multiply the pixel matrix of the multiple overlapping sub-images by their respective illumination compensation coefficients to obtain multiple overlapping sub-compensated images;

[0035] Extract the overlapping region between the first overlapping sub-compensation image and the second overlapping sub-compensation image, wherein the first overlapping sub-compensation image and the second overlapping sub-compensation image are any two adjacent overlapping sub-compensation images among the plurality of overlapping sub-compensation images;

[0036] Convert multiple pixels in the overlapping region into a coordinate matrix;

[0037] The fused pixel value of the multiple pixels in the overlapping region is calculated based on the pixel values ​​of the multiple pixels in the first overlapping sub-compensation image, the pixel values ​​of the second overlapping sub-compensation image, and the coordinate matrix.

[0038] Based on the fused pixel values ​​of multiple pixels in the overlapping region, the multiple overlapping sub-compensation images are fused and stitched together to obtain the second highway slope image.

[0039] Optionally, the step of identifying defects in the second highway slope image and generating an inspection report specifically includes:

[0040] The second highway slope image is downsampled at multiple levels to obtain multiple secondary images;

[0041] The multiple secondary images are fused to obtain multiple fused images;

[0042] Multiple fused images are input into the disease detection model to obtain small-scale and large-scale disease areas, and an inspection report is generated.

[0043] Secondly, this application provides a high-efficiency highway slope inspection platform using unmanned aerial vehicles (UAVs). The system is an inspection platform, which includes an acquisition module, a processing module, and an output module, wherein:

[0044] The acquisition module is used to acquire the first highway slope image collected by the UAV;

[0045] The processing module is used to divide the first highway slope image into multiple overlapping sub-images of the same size; calculate the local entropy and gradient mean of the multiple overlapping sub-images; calculate the illumination compensation coefficient of the multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images; and perform brightness compensation on the multiple overlapping sub-images based on the illumination compensation coefficient of the multiple overlapping sub-images to obtain the second highway slope image.

[0046] The output module is used to identify defects in the second highway slope image and generate an inspection report.

[0047] Thirdly, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0048] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any one of the first aspects.

[0049] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0050] This application divides a first highway slope image captured by a drone into multiple overlapping sub-images, and then calculates the local entropy and gradient mean of each overlapping sub-image to dynamically generate an illumination compensation coefficient. The local entropy reflects the pixel distribution complexity of the image (overall image quality), and the gradient mean reflects the richness of edge information (local detail quality). Then, brightness compensation is applied to each overlapping sub-image based on the illumination compensation coefficient, effectively suppressing problems such as uneven brightness and color distortion while preserving the detailed features of the slope structure. Finally, by identifying defects in the compensated second highway slope image, the accuracy and reliability of defect identification during highway slope inspections under complex lighting conditions are significantly improved. Ultimately, a comprehensive and accurate inspection report is generated, providing a scientific basis for slope safety assessment and maintenance decisions. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an efficient highway slope inspection method using unmanned aerial vehicles (UAVs) provided in an embodiment of this application.

[0052] Figure 2 This is a structural schematic diagram of a high-efficiency highway slope inspection platform using unmanned aerial vehicles (UAVs) provided in an embodiment of this application.

[0053] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0054] Explanation of reference numerals in the attached drawings: 1. Acquisition module; 2. Processing module; 3. Output module; 300. Electronic device; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] Currently, when drone inspections encounter complex lighting environments that lead to uneven brightness and color distortion in images, fixed-window global histogram equalization or gamma correction is often used to optimize image brightness and improve image quality. However, these two methods cannot finely adjust for the differences in reflectivity of different materials such as vegetation and soil on slope surfaces, as well as local shadows caused by terrain undulations. Specifically, global histogram equalization can cause saturation of details in bright areas and insufficient contrast in shadow areas due to forced stretching of global grayscale. Gamma correction, on the other hand, cannot balance the brightness requirements of different areas due to a single power function transformation. This can lead to overexposure in bright areas due to excessive brightening and noise in shadow areas due to excessive darkening. Consequently, this can result in missed detections or misjudgments when identifying defects.

[0057] To address the aforementioned problems, this application provides a method for efficient highway slope inspection using unmanned aerial vehicles (UAVs). This method is applied to an inspection platform, such as... Figure 1 As shown, the method includes steps S101 to S106, which are as follows:

[0058] S101. Acquire the first highway slope image collected by the drone.

[0059] In the above steps, the drone, equipped with a high-pixel visible light camera, continuously photographs the target highway slope according to a preset route, generating a high-resolution first highway slope image that reflects the complete structural features of the target highway slope, and then transmits the first highway slope image to the inspection platform.

[0060] In one possible implementation, since the terrain of highway slopes varies significantly across different regions, a preliminary survey of the target highway slope is necessary before setting the preset flight path to ensure the path matches the terrain complexity. Specifically, this involves using radar and a multispectral camera mounted on a UAV to acquire the terrain features of the target highway slope, including slope variance and mean slope curvature. The terrain complexity of the target highway slope is then calculated based on these features using the following formula:

[0061]

[0062] Where C represents terrain complexity. For slope variance, The mean curvature of the slope. This represents the influence coefficient of slope variance on slope disease identification. The coefficient representing the influence of the mean slope curvature on slope disease identification.

[0063] In the above formula, a larger slope variance indicates a more severe terrain undulation, more complex lighting occlusion and image acquisition perspective changes, and a larger mean slope curvature indicates a greater local curvature of the ground surface, a more complex terrain surface geometry, and a greater susceptibility to shadow occlusion or image distortion, thus increasing terrain complexity. Furthermore, since slope and curvature have different accuracies in identifying slope defects, separate parameters are also set. and This allows for a more accurate description of the slope's topographical complexity. In real-world scenarios, abrupt changes in slope gradient are more crucial for disease identification. Greater than Among them, the slope variance and curvature variance are normalized results.

[0064] In addition, the vegetation coverage on slopes also changes with climate. For example, in winter, when vegetation withers, it is easier to photograph the slope surface. In summer, however, when vegetation grows lushly, the slope surface is obscured, making it difficult for drones to photograph the slope surface. In this case, in order to more accurately describe the complexity of the slope topography, this application uses a multispectral camera mounted on a drone to capture multispectral images of the slope, and then extracts the near-infrared band distribution map (vegetation characteristics) and the red band distribution map (bare soil / rock characteristics). Based on the near-infrared band distribution map and the red band distribution map, the vegetation coverage is calculated.

[0065] Then, when the vegetation coverage reaches the vegetation coverage threshold, it indicates that the currently calculated terrain complexity is difficult to describe the real situation. At this time, although the excessively high vegetation coverage leads to inaccurate terrain complexity of the slope, the growth of the vegetation itself can also reflect the changes in terrain. For example, when the terrain changes, the groundwater level will change, and the water absorption of the vegetation roots will be affected, resulting in a shift in the vegetation spectrum. Therefore, this application calculates the variance of the near-infrared band in the multispectral image based on the near-infrared band distribution map, and then normalizes the near-infrared band variance to obtain the terrain complexity compensation coefficient. Finally, the terrain complexity is compensated based on the terrain complexity compensation coefficient to obtain the target terrain complexity. The specific calculation method is as follows:

[0066]

[0067] in, To determine the complexity of the target terrain, For the original terrain complexity, For near-infrared band variance, This represents the influence coefficient of topographic change on vegetation growth.

[0068] In the above formula, the larger the variance of the near-infrared band, the greater the terrain variation and the greater the terrain complexity.

[0069] In the above process, although excessive vegetation cover can interfere with the direct identification of slope topographic complexity, abundant vegetation provides diverse data support for the indirect interpretation of topographic complexity. Therefore, the vegetation cover threshold can be defined as a basic critical value to ensure the accuracy of indirect identification of topographic complexity. Through this quantitative standard, a more accurate descriptive system for slope topographic complexity can be constructed.

[0070] Then, the terrain complexity is matched with a preset shooting density database to obtain the shooting density corresponding to the terrain complexity of the target highway slope. The preset shooting density database stores the correspondence between shooting density and terrain complexity. Shooting density can be understood as the frequency of UAV image acquisition per unit area of ​​the slope. Finally, based on the shooting density, the inspection speed and inspection route of the UAV are planned to improve the image quality of the first highway slope image. The higher the shooting density, the slower the inspection speed and the denser the inspection route. For example, a grid-like inspection route can be used, and the higher the shooting density, the denser the inspection grid.

[0071] S102. Divide the first highway slope image into multiple overlapping sub-images of the same size.

[0072] In the above steps, since the reflectivity of different surface materials varies significantly, directly performing global illumination compensation on the first highway slope image will cause overexposure in high-reflectivity surface areas and loss of detail in low-reflectivity surface areas. Therefore, this application divides the first highway slope image into multiple sub-images of the same size, restricts different surface materials within the sub-image range, and then performs adaptive illumination compensation independently on each sub-image, thereby avoiding global parameters being misled by local outliers.

[0073] When dividing the first highway slope image into multiple sub-images, some image features may exist at the boundary between two sub-images. In this case, after adaptive illumination compensation of the two sub-images, the brightness of the boundary area of ​​the two sub-images may change abruptly when they are fused and stitched together. Therefore, in order to make the edge transition of adjacent sub-images smoother, this application divides the first highway slope into multiple overlapping sub-images. When performing illumination compensation on each overlapping sub-image, since the image features of the overlapping part are considered, the features of the overlapping part will not be cut off during fusion, thus more completely preserving the information.

[0074] In one possible implementation, dividing the first highway slope into multiple overlapping sub-images can improve the fusion effect. However, excessively small overlapping regions result in poor fusion of areas with large image feature ranges, while excessively large overlapping regions lead to a surge in computational load. Therefore, to address this issue, this application first assesses the computational resources required for the first highway slope image. The required computational resources can be understood as the number of basic operations required for the algorithm to execute. The required computational resources for an image are primarily determined by the number of times the algorithm accesses pixels in the image. Therefore, the higher the resolution of the first highway slope image, the more computational resources are required. To maximize the utilization of system computational resources while improving the fusion effect of subsequent overlapping regions, the ratio between the required computational resources of the first highway slope image and the total computational resources of the inspection platform is used to determine the relationship between each overlapping sub-image and its adjacent overlapping sub-image. The overlap ratio between images is as follows: for example, if the computational resources required for the first highway slope image are 0.8 and the total computational resources of the inspection platform are 2, and each overlapping sub-image has 4 adjacent overlapping sub-images, then the overlap ratio between two adjacent overlapping sub-images is 0.1. In addition, in order to meet the fusion requirements of some image feature ranges with large ranges, this application converts the first highway image into a gradient image, then identifies multiple high gradient regions in the gradient image, then selects the high gradient region with the largest range from the multiple high gradient regions, then converts the high gradient region with the largest range into a standard reference region range, then uses the reference region range as the region size of the overlapping sub-region, and finally divides the first highway slope image into multiple overlapping sub-images of the same size according to the overlap ratio and region size, thereby improving the fusion effect of multiple overlapping sub-images after brightness compensation.

[0075] S103. Calculate the local entropy and gradient mean of multiple overlapping sub-images.

[0076] S104. Calculate the illumination compensation coefficients of multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images.

[0077] In steps S103 to S104 above, the local entropy of the overlapping sub-image is calculated using the Shannon entropy formula to obtain the complexity of the pixel distribution of the overlapping sub-image. It can be understood that the greater the difference in pixel values, the higher the entropy value and the stronger the unevenness of illumination. Then, for the gradient mean of the overlapping sub-image, the Sobel operator can be used to calculate the gradient image of the overlapping sub-image, and then the gradient image is averaged to obtain the gradient mean, thereby clarifying the richness of edge information of the overlapping sub-image.

[0078] Then, the maximum local entropy and the maximum gradient mean among the local entropies of multiple overlapping sub-images are traversed to ensure that the illumination compensation coefficients have a uniform scale standard across the entire image. The illumination compensation coefficients of each overlapping sub-image are then calculated using the following formula:

[0079]

[0080] in, Let be the illumination compensation coefficient for the i-th overlapping sub-image. Let be the local entropy of the i-th overlapping sub-image. The maximum local entropy among multiple overlapping sub-images. Let be the mean gradient of the i-th overlapping sub-image. The maximum gradient mean among multiple overlapping sub-images. and These are the local entropy weighting coefficient and the gradient mean weighting coefficient, respectively. > , + =1.

[0081] In the above formula, local entropy reflects the uniformity of illumination in the overlapping sub-images. The higher the local entropy, the more uneven the illumination. In this case, the illumination compensation coefficient increases to enhance the brightness of details in low-illumination areas. However, to prevent overexposure in already bright areas, which would obscure details, the gradient mean of the overlapping sub-images is introduced as a constraint term to control the magnitude of illumination compensation. That is, when local entropy is high but details are clear (low gradient mean), the illumination compensation coefficient is suppressed, thus achieving a balance between illumination compensation and detail preservation. In addition, since the primary task of slope inspection is to improve image detectability through illumination adjustment, and the gradient mean is used as a constraint term to avoid loss of details during illumination compensation, the local entropy weight coefficient is greater than the gradient mean weight coefficient.

[0082] S105. Based on the illumination compensation coefficients of multiple overlapping sub-images, brightness compensation is performed on multiple overlapping sub-images to obtain the second highway slope image.

[0083] In the above steps, the pixel matrix of multiple overlapping sub-images is multiplied by their respective illumination compensation coefficients to obtain multiple overlapping sub-compensated images. Then, the multiple sub-compensated images are stitched together and fused to obtain the second highway slope image.

[0084] During the stitching and fusion process, although the overlapping area can eliminate boundary effects, if the difference in illumination compensation coefficients between two adjacent overlapping sub-compensation images used for stitching is large, the pixel values ​​at the boundary will change abruptly, still forming a relatively obvious dividing line. Therefore, the continuity of pixel values ​​in the overlapping area must also be considered during the fusion and stitching process to ensure a smooth transition. Specifically, for pixels in the overlapping area, their pixel values ​​are weighted and fused based on their coordinates in the overlapping area. This ensures that the pixel values ​​achieve the optimal balance of detail representation in the two adjacent overlapping sub-compensation images while eliminating visual interference caused by boundary effects, thus obtaining the fused pixel value. The following calculation formula can be used for this purpose:

[0085]

[0086]

[0087] in, The fused pixel value of the i-th pixel. , Let be the pixel values ​​of the i-th pixel in the j-th and k-th overlapping sub-compensation images, respectively. Let i be the weight value of the i-th pixel. Let be the boundary distance between the i-th pixel and the k-th overlapping sub-compensation image, and D be the total width of the overlapping region between the j-th and k-th overlapping sub-compensation images.

[0088] In the above formula, when the boundary distance from the i-th pixel to the overlapping region of the k-th overlapping sub-compensation image is 0, then... =1, at this time the i-th pixel fully adopts the illumination compensation result of the j-th overlapping sub-compensation image. When the boundary distance of the overlapping region from the i-th pixel to the k-th overlapping sub-compensation image is D, then =0. At this time, the i-th pixel fully adopts the illumination compensation result of the k-th overlapping sub-compensation image. When the boundary distance between the i-th pixel and the overlapping area of ​​the k-th overlapping sub-compensation image is between 0 and D, if the i-th pixel is close to the boundary of the overlapping area of ​​the k-th overlapping sub-compensation image, the pixel value of the i-th pixel mainly inherits the illumination compensation effect of the k-th overlapping sub-compensation image. If the i-th pixel is close to the boundary of the overlapping area of ​​the j-th overlapping sub-compensation image, the pixel value of the i-th pixel mainly inherits the illumination compensation effect of the j-th overlapping sub-compensation image. This realizes the gradual transition of pixels in the overlapping area between the two overlapping sub-compensation images, thereby eliminating the boundary effect.

[0089] S106. Identify defects in the images of the second highway slope and generate an inspection report.

[0090] In the above steps, by inputting the images of the second highway slope into the disease detection model, various disease features are obtained, and inspection reports are generated from these features. The disease monitoring model is a neural network model trained from a large number of slope disease images.

[0091] In one possible implementation, the disease characteristics of highway slopes can be divided into small-scale disease characteristics and large-scale disease characteristics based on the size of the disease. Small-scale disease characteristics manifest as minute cracks in the slope, while large-scale disease characteristics manifest as collapses or landslides. Therefore, to improve the recall rate of small-scale disease and the localization accuracy of large-scale disease, this application performs multi-level downsampling on the second highway slope image before inputting it into the disease detection model, obtaining multiple secondary images. Since the main small-scale disease characteristics of the slope are 1-50mm cracks, and the main large-scale disease characteristics are 1-10mm cracks... Due to slope deformation, multiple secondary images were generated, namely, 1 / 2 resolution (64 channels), 1 / 4 resolution (128 channels), and 1 / 8 resolution (256 channels). Among them, the 1 / 2 resolution (64 channels) secondary image can more accurately reflect the edge details (small-scale diseases) in the image, the 1 / 4 resolution (128 channels) secondary image can more accurately reflect the texture information (some small-scale diseases and some large-scale diseases) in the image, and the 1 / 8 resolution (256 channels) secondary image can more accurately reflect the semantic features (large-scale diseases) in the image.

[0092] Then, the 1 / 2 resolution highway slope image and the 1 / 4 resolution highway slope image are fused by channel fusion to obtain a 1 / 2 resolution (96 channels) highway slope image. This fusion method retains high resolution while incorporating feature information from more channels, thereby enhancing the shallow network's ability to recognize small targets and improving the recall rate of small-scale disease areas. The 1 / 4 resolution highway slope image and the 1 / 8 resolution highway slope image are then fused by channel fusion to obtain a 1 / 4 resolution (192 channels) highway slope image. This fusion method utilizes lower resolution... It provides a global view, and then integrates multi-scale semantic features through channel fusion to assist the deep network in locating the overall boundary of large targets, thereby improving the positioning accuracy of large-scale disease areas. Finally, the 1 / 2 resolution (96 channels) highway slope image is input into the disease detection model to output small-scale disease features, and the 1 / 4 resolution (192 channels) highway slope image is input into the disease detection model to output large-scale disease features. Finally, the small-scale disease features and large-scale disease features are combined to generate an inspection report, which includes disease images, types, sizes, grades and coordinates.

[0093] Reference Figure 2This application also provides a high-efficiency highway slope inspection platform using unmanned aerial vehicles (UAVs). The system is an inspection platform, which includes an acquisition module 1, a processing module 2, and an output module 3, wherein:

[0094] Module 1 is used to acquire the first highway slope image collected by the UAV;

[0095] Processing module 2 is used to divide the first highway slope image into multiple overlapping sub-images of the same size; calculate the local entropy and gradient mean of the multiple overlapping sub-images; calculate the illumination compensation coefficient of the multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images; and perform brightness compensation on the multiple overlapping sub-images based on the illumination compensation coefficient of the multiple overlapping sub-images to obtain the second highway slope image.

[0096] Output module 3 is used to identify defects in the second highway slope image and generate an inspection report.

[0097] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0098] This application also discloses an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0099] The communication bus 302 is used to enable communication between these components.

[0100] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0101] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0102] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0103] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for an efficient highway slope inspection method using unmanned aerial vehicles (UAVs).

[0104] exist Figure 3In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call an application stored in the memory 305 for an efficient highway slope inspection method using drones. When executed by one or more processors 301, the electronic device 300 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0106] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0110] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0111] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for efficient inspection of highway slopes using unmanned aerial vehicles (UAVs), characterized in that, The method, applied to an inspection platform, includes: Acquire the first highway slope image captured by the drone; The first highway slope image is divided into multiple overlapping sub-images of the same size; Calculate the local entropy and gradient mean of the multiple overlapping sub-images; The illumination compensation coefficients of the multiple overlapping sub-images are calculated based on the local entropy and gradient mean of the multiple overlapping sub-images. Based on the illumination compensation coefficients of the multiple overlapping sub-images, brightness compensation is performed on the multiple overlapping sub-images to obtain a second highway slope image; The second highway slope image is used to identify defects and generate an inspection report.

2. The method according to claim 1, characterized in that, Before acquiring the first highway slope image collected by the drone, the process specifically includes: Obtain the topographic features of the target highway slope, including the slope variance and the mean slope curvature. Calculate the terrain complexity of the target highway slope based on the terrain features. The terrain complexity is matched with a preset shooting density database to obtain the shooting density corresponding to the terrain complexity of the target highway slope, so that the UAV can acquire the first highway slope image according to the shooting density corresponding to the terrain complexity of the target highway slope.

3. The method according to claim 2, characterized in that, The step of calculating the terrain complexity of the target highway slope based on the terrain features specifically includes: Calculate the vegetation coverage of the target highway slope; When the vegetation coverage rate is greater than or equal to the preset vegetation coverage rate, the near-infrared band distribution map of the target highway slope is obtained. Calculate the infrared band variance of the near-infrared band distribution map; Based on the variance of the infrared band, determine the terrain complexity compensation coefficient of the target highway slope; The terrain complexity is compensated for based on the terrain complexity compensation coefficient to obtain the target terrain complexity.

4. The method according to claim 1, characterized in that, The step of dividing the first highway slope image into multiple overlapping sub-images of the same size specifically includes: Assess the computational resources required for the first highway slope image; The overlap ratio is determined based on the ratio between the computational resources required for the first highway slope image and the total computational resources of the inspection platform. Convert the first highway slope image into a gradient image; Identify multiple high gradient regions in the gradient image; The region size is determined based on the region size within the multiple high gradient regions; Based on the overlap ratio and the region size, the first highway slope image is divided into multiple overlapping sub-images of the same size.

5. The method according to claim 1, characterized in that, The step of calculating the illumination compensation coefficients of the multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images is specifically as follows: in, Let be the illumination compensation coefficient for the i-th overlapping sub-image. Let be the local entropy of the i-th overlapping sub-image. The maximum local entropy among multiple overlapping sub-images. Let be the mean gradient of the i-th overlapping sub-image. The maximum gradient mean among multiple overlapping sub-images. and These are the local entropy weighting coefficient and the gradient mean weighting coefficient, respectively. > , + =1.

6. The method according to claim 1, characterized in that, The step of performing brightness compensation on the multiple overlapping sub-images based on the illumination compensation coefficients to obtain the second highway slope image specifically includes: Multiply the pixel matrix of the multiple overlapping sub-images by their respective illumination compensation coefficients to obtain multiple overlapping sub-compensated images; Extract the overlapping region between the first overlapping sub-compensation image and the second overlapping sub-compensation image, wherein the first overlapping sub-compensation image and the second overlapping sub-compensation image are any two adjacent overlapping sub-compensation images among the plurality of overlapping sub-compensation images; Convert multiple pixels in the overlapping region into a coordinate matrix; The fused pixel value of the multiple pixels in the overlapping region is calculated based on the pixel values ​​of the multiple pixels in the first overlapping sub-compensation image, the pixel values ​​of the second overlapping sub-compensation image, and the coordinate matrix. Based on the fused pixel values ​​of multiple pixels in the overlapping region, the multiple overlapping sub-compensation images are fused and stitched together to obtain the second highway slope image.

7. The method according to claim 1, characterized in that, The step of identifying defects in the second highway slope image and generating an inspection report specifically includes: The second highway slope image is downsampled at multiple levels to obtain multiple secondary images; The multiple secondary images are fused to obtain multiple fused images; Multiple fused images are input into the disease detection model to obtain small-scale disease areas and large-scale disease areas, and an inspection report is generated.

8. A high-efficiency highway slope inspection platform using unmanned aerial vehicles (UAVs), characterized in that, The system is an inspection platform, which includes an acquisition module (1), a processing module (2), and an output module (3), wherein: The acquisition module (1) is used to acquire the first highway slope image collected by the UAV; The processing module (2) is used to divide the first highway slope image into multiple overlapping sub-images of the same size; calculate the local entropy and gradient mean of the multiple overlapping sub-images; calculate the illumination compensation coefficient of the multiple overlapping sub-images based on the local entropy and gradient mean of the multiple overlapping sub-images; and perform brightness compensation on the multiple overlapping sub-images based on the illumination compensation coefficient of the multiple overlapping sub-images to obtain the second highway slope image. The output module (3) is used to identify defects in the second highway slope image and generate an inspection report.

9. An electronic device, characterized in that, The device includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are used to communicate with other devices. The processor (301) is used to execute the instructions stored in the memory (305) to cause the electronic device (300) to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 7.

Images