A highway slope efficient inspection method and system using a drone

Abstract

The application discloses a highway slope efficient inspection method and system using a UAV, and relates to the field of UAV inspection. The method is applied to an inspection platform, and the method comprises the following steps: acquiring a first highway slope image collected by a UAV; dividing the first highway slope image into a plurality of overlapping sub-images with the same size; calculating local entropy and gradient mean value of the plurality of overlapping sub-images; calculating illumination compensation coefficients of the plurality of overlapping sub-images according to the local entropy and the gradient mean value of the plurality of overlapping sub-images; performing brightness compensation on the plurality of overlapping sub-images based on the illumination compensation coefficients of the plurality of overlapping sub-images to obtain a second highway slope image; and performing disease identification on the second highway slope image to generate an inspection report. The technical scheme provided by the application solves the problem that the brightness of the image collected by the UAV is uneven and the color of the image is distorted under a complex illumination environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle inspection, in particular to a highway slope efficient inspection method and system applying unmanned aerial vehicle. BACKGROUND

[0002] Unmanned aerial vehicle inspection has been widely applied in highway slope engineering monitoring due to its advantages of fast flight speed, wide coverage and low cost.

[0003] In the prior art, the unmanned aerial vehicle collects slope images by carrying a camera, and then identifies diseases such as cracks and collapses through image processing algorithms. However, in actual inspection, complex lighting environments are often encountered, which leads to problems such as uneven brightness and color distortion in images, seriously affecting the accuracy of subsequent disease feature extraction. However, the current mainstream illumination compensation methods mostly use fixed window global histogram equalization or Gamma correction, but these two methods are prone to over-enhancement or insufficient compensation in non-uniform illumination scenes caused by terrain fluctuations, which leads to the problem that cracks and other subtle features are submerged in noise, resulting in false detection.

[0004] Therefore, there is an urgent need for a highway slope efficient inspection method and system applying unmanned aerial vehicle that can adapt to complex lighting conditions. SUMMARY

[0005] In view of the problem of uneven brightness and color distortion in images taken by unmanned aerial vehicles in complex lighting environments, the present application provides a highway slope efficient inspection method and system applying unmanned aerial vehicle.

[0006] In a first aspect, the present application provides a highway slope efficient inspection method applying unmanned aerial vehicle, applied to an inspection platform, the method comprising:

[0007] obtaining a first highway slope image collected by an unmanned aerial vehicle;

[0008] dividing the first highway slope image into a plurality of overlapping sub-images of the same size;

[0009] calculating the local entropy and gradient mean of a plurality of overlapping sub-images;

[0010] calculating the illumination compensation coefficient of a plurality of overlapping sub-images according to the local entropy and gradient mean of a plurality of overlapping sub-images;

[0011] performing brightness compensation on a plurality of overlapping sub-images based on the illumination compensation coefficient of a plurality of overlapping sub-images to obtain a second highway slope image;

[0012] performing disease identification on the second highway slope image to generate an inspection report.

[0013] Optionally, before the acquiring the first road slope image collected by the unmanned aerial vehicle, the method further comprises:

[0014] acquiring a topographic feature of the target road slope, the topographic feature comprising a slope variance and a mean slope curvature;

[0015] calculating a topographic complexity of the target road slope according to the topographic feature;

[0016] matching the topographic complexity with a preset shooting density database to obtain a shooting density corresponding to the topographic complexity of the target road slope, so that the unmanned aerial vehicle collects the first road slope image according to the shooting density corresponding to the topographic complexity of the target road slope.

[0017] Optionally, the calculating the topographic complexity of the target road slope according to the topographic feature further comprises:

[0018] calculating a vegetation coverage of the target road slope;

[0019] when the vegetation coverage is greater than or equal to a preset vegetation coverage, acquiring a near-infrared band distribution map of the target road slope;

[0020] calculating an infrared band variance of the near-infrared band distribution map;

[0021] determining a topographic complexity compensation coefficient of the target road slope according to the infrared band variance;

[0022] compensating the topographic complexity of the target road slope according to the topographic complexity compensation coefficient to obtain a target topographic complexity.

[0023] Optionally, the dividing the first road slope image into a plurality of overlapping sub-images of the same size comprises:

[0024] evaluating required computing resources of the first road slope image;

[0025] determining an overlap ratio according to a ratio between the required computing resources of the first road slope image and total computing resources of the inspection platform;

[0026] converting the first road slope image into a gradient image;

[0027] identifying a plurality of high gradient regions in the gradient image;

[0028] determining a region size according to region sizes of the plurality of high gradient regions;

[0029] dividing the first road slope image into a plurality of overlapping sub-images of the same size according to the overlap ratio and the region size.

[0030] Optionally, the illumination compensation coefficient of each of the plurality of overlapping sub-images is calculated according to a local entropy and a gradient mean value of each of the plurality of overlapping sub-images, and specifically includes:

[0031]

[0032] wherein, is the illumination compensation coefficient of the i-th overlapping sub-image, is the local entropy of the i-th overlapping sub-image, is the maximum local entropy in the plurality of overlapping sub-images, is the gradient mean value of the i-th overlapping sub-image, is the maximum gradient mean value in the plurality of overlapping sub-images, and are a local entropy weight coefficient and a gradient mean value weight coefficient, respectively, wherein, , =1.

[0033] Optionally, the brightness compensation is performed on the plurality of overlapping sub-images based on the illumination compensation coefficient of each of the plurality of overlapping sub-images, to obtain a second highway slope image, and specifically includes:

[0034] The pixel matrix of each of the plurality of overlapping sub-images is multiplied by the corresponding illumination compensation coefficient to obtain a plurality of overlapping sub-compensation images;

[0035] The overlapping area between a first overlapping sub-compensation image and a second overlapping sub-compensation image is extracted, wherein the first overlapping sub-compensation image and the second overlapping sub-compensation image are any two adjacent overlapping sub-compensation images in the plurality of overlapping sub-compensation images;

[0036] The plurality of pixel points in the overlapping area are converted into a coordinate matrix;

[0037] The fusion pixel value of each of the plurality of pixel points in the overlapping area is calculated according to the pixel value of each of the plurality of pixel points in the overlapping area in the first overlapping sub-compensation image, the pixel value of each of the plurality of pixel points in the overlapping area in the second overlapping sub-compensation image, and the coordinate matrix;

[0038] The plurality of overlapping sub-compensation images are fused and spliced according to the fusion pixel value of each of the plurality of pixel points in the overlapping area to obtain the second highway slope image.

[0039] Optionally, the disease identification is performed on the second highway slope image to generate an inspection report, and specifically includes:

[0040] The second highway slope image is multi-level down-sampled to obtain a plurality of secondary images.​​

[0041] fusing the plurality of secondary images to obtain a plurality of fused images;

[0042] inputting the plurality of fused images into the disease detection model to obtain small-scale disease areas and large-scale disease areas, and generating an inspection report.

[0043] In a second aspect, the present application provides a highway slope efficient inspection platform using a UAV, the system is an inspection platform, the inspection platform comprises an acquisition module, a processing module and an output module, wherein:

[0044] The acquisition module is configured to acquire a first highway slope image collected by a UAV.

[0045] The processing module is configured to divide the first highway slope image into a plurality of overlapping sub-images of the same size, calculate local entropy and gradient mean of the plurality of overlapping sub-images, calculate illumination compensation coefficients of the plurality of overlapping sub-images according to the local entropy and the gradient mean of the plurality of overlapping sub-images, and perform brightness compensation on the plurality of overlapping sub-images based on the illumination compensation coefficients of the plurality of overlapping sub-images to obtain a second highway slope image.

[0046] The output module is configured to perform disease identification on the second highway slope image to generate an inspection report.

[0047] In a third aspect, the present application provides an electronic device, comprising a processor, a memory, a user interface and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of the first aspect.

[0048] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores instructions, when the instructions are executed, the method of any one of the first aspect is executed.

[0049] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0050] The application divides the first road slope image collected by the unmanned aerial vehicle into multiple overlapping sub-images, and then calculates the local entropy and gradient mean value of each overlapping sub-image to dynamically generate an illumination compensation coefficient, wherein the local entropy reflects the pixel distribution complexity (overall image quality) of the image, and the gradient mean value reflects the edge information richness (local detail quality) of the image. At this time, the brightness of each overlapping sub-image is compensated according to the illumination compensation coefficient, so as to effectively suppress the problems of uneven brightness, color distortion and the like, while retaining the structural detail features of the slope. Finally, the disease identification is performed on the compensated second road slope image, so as to significantly improve the disease identification accuracy and reliability of the road slope inspection under complex illumination conditions, and finally generate a comprehensive and accurate inspection report, thereby providing a scientific basis for the slope safety evaluation and maintenance decision. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 Fig. 1 is a flow diagram of a road slope efficient inspection method using an unmanned aerial vehicle provided by an embodiment of the application.

[0052] Figure 2 Fig. 2 is a structural diagram of a road slope efficient inspection platform using an unmanned aerial vehicle provided by an embodiment of the application.

[0053] Figure 3 Fig. 3 is a structural diagram of an electronic device provided by an embodiment of the application.

[0054] The following items are explained: 1, acquisition module; 2, processing module; 3, output module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the present application will be described clearly and completely in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0056] Currently, in the process of unmanned aerial vehicle inspection, complex lighting environment leads to uneven brightness, color distortion and other problems in images. Global histogram equalization or Gamma correction with fixed window is usually used to optimize the brightness of images to improve image quality. However, these two methods cannot make fine adjustment according to the reflectivity difference of different materials such as slope surface vegetation, rock and soil, and local shadows caused by terrain undulations. Specifically, global histogram equalization will cause saturation of details in strong light area and insufficient contrast in shadow area due to forced stretching of global gray scale. Gamma correction cannot balance the brightness requirements of different areas due to single power function transformation, which may cause overexposure in strong light area and noise drowning in shadow area, resulting in missed detection or misjudgment in disease identification.

[0057] To solve the above problems, the present application provides a high-efficiency highway slope inspection method using unmanned aerial vehicle. The method is applied to an inspection platform, as shown in the figure, which includes steps S101 to S106. The steps are as follows: Figure 1

[0058] S101, obtaining a first highway slope image collected by an unmanned aerial vehicle.

[0059] In the above steps, the unmanned aerial vehicle carries a high-pixel visible light camera and continuously photographs the target highway slope according to the preset flight route to generate a high-resolution first highway slope image that can reflect the complete structural characteristics of the target highway slope, and then transmits the first highway slope image to the inspection platform.

[0060] In one possible implementation, since highway slopes in different regions differ greatly in terrain, the target highway slope needs to be pre-investigated before setting the preset flight route, so that the preset flight route matches the terrain complexity. Specifically, the terrain features of the target highway slope are obtained by the radar device and multispectral camera carried by the unmanned aerial vehicle, including slope variance, slope curvature mean value, and then the terrain complexity of the target highway slope is calculated according to the terrain features, specifically using the following formula:

[0061]

[0062] Where C is the terrain complexity, is the slope variance, is the slope curvature mean value, is the influence coefficient of slope variance on slope disease identification, is the influence coefficient of slope curvature mean value on slope disease identification.

[0063] ​In the above formula, the greater the slope variance, the more intense the terrain undulation, the more complex the changes in illumination shielding and image acquisition viewing angle, the greater the average slope curvature, the greater the local curvature of the ground surface, the more complex the geometric structure of the terrain surface, and the more likely to produce shadow shielding or image distortion. At this time, the terrain complexity is also greater. In addition, since the slope and curvature of the slope are different in accuracy for slope disease identification, the terrain complexity is more accurately described by and In actual scenarios, since slope mutation is more critical for disease identification, the terrain complexity is more accurately described by greater than wherein the slope variance and the curvature variance are normalized results.

[0064] In addition, with changes in the climate, the vegetation coverage on the slope also changes. For example, in winter, the grass and trees wither, and it is easier to capture the surface conditions of the slope. In summer, the vegetation grows luxuriantly, and the slope surface is shielded by the vegetation, making it difficult for the unmanned aerial vehicle to capture the surface of the slope. In this case, in order to more accurately describe the terrain complexity of the slope, the present application uses the multispectral camera carried by the unmanned aerial vehicle to capture the multispectral image of the slope, and then extracts the near-infrared band distribution graph (vegetation feature) and the red band distribution graph (bare soil / rock feature) therefrom. According to the near-infrared band distribution graph and the red band distribution graph, the vegetation coverage is calculated.

[0065] When the vegetation coverage reaches the vegetation coverage threshold, it is indicated that the terrain complexity calculated at the moment cannot accurately describe the real situation. Although the high vegetation coverage makes the terrain complexity of the slope inaccurate, the growth of the vegetation itself can also reflect the change in the terrain. For example, when the terrain changes, the underground water level changes, and the root system of the vegetation is affected, causing the vegetation spectrum to shift. Therefore, according to the near-infrared band distribution graph, the present application calculates the near-red band variance in the multispectral image, normalizes the near-red band variance to obtain a terrain complexity compensation coefficient, and finally compensates the terrain complexity according to the terrain complexity compensation coefficient to obtain a target terrain complexity. The specific calculation method is as follows:

[0066]

[0067] wherein, is the target terrain complexity, is the original terrain complexity, is the near-red band variance, is an influence coefficient of the terrain change on the growth of the vegetation.

[0068] In the above formula, the greater the near-red band variance, the greater the terrain change, and the greater the terrain complexity.

[0069] In the above process, although excessive vegetation coverage will interfere with the direct identification of the complexity of the slope terrain, the lush vegetation community provides multi-data support for the indirect interpretation of the complexity of the terrain. Therefore, the vegetation coverage threshold can be defined as the basic critical value to ensure the accuracy of the indirect identification of the complexity of the terrain. Through this quantitative standard, the description system of the complexity of the slope terrain can be more accurately constructed.

[0070] Then the terrain complexity is matched with a preset shooting density database to obtain the shooting density corresponding to the terrain complexity of the target highway slope, wherein the preset shooting density database stores the corresponding relationship between the shooting density and the terrain complexity. The shooting density can be understood as the frequency of unmanned aerial vehicle image collection in a unit area of the slope region. Finally, according to the shooting density, the inspection speed and the inspection route of the planned unmanned aerial vehicle are adjusted to improve the image quality of the first highway slope image. The greater the shooting density, the slower the inspection speed and the more intensive the inspection route. For example, a grid-shaped inspection route can be used. The greater the shooting density, the more intensive the inspection grid.

[0071] S102, divide the first highway slope image into a plurality of overlapping sub-images of the same size.

[0072] In the above step, since the reflectivity of different surface materials is significantly different, directly performing global illumination compensation on the first highway slope image will cause the problem of overexposure of high reflectivity surface regions and loss of details of low reflectivity surface regions. Therefore, the first highway slope image is divided into a plurality of sub-images of the same size in the present application, different surface materials are limited within the sub-image range, and then adaptive illumination compensation is independently performed on each sub-image, thereby avoiding the global parameters being misled by local outliers.

[0073] When the first highway slope image is divided into a plurality of sub-images, part of the image features will exist at the junction of two sub-images. At this time, after adaptive illumination compensation is performed on the two sub-images, fusion splicing will cause the boundary region of the two sub-images to have a sudden change in brightness. Therefore, in order to make the edge transition of adjacent sub-images smoother, the first highway slope is divided into a plurality of overlapping sub-images in the present application. When the illumination compensation is performed on each overlapping sub-image, since the image features of the overlapping part are considered, the features of the overlapping part will not be cut off during fusion, thereby more complete information can be retained.

[0074] In a possible implementation, although dividing the first road slope into multiple overlapping sub-images can improve the fusion effect, the fusion effect of an area with a large image feature range is poor when the overlap area is too small, and the calculation amount increases sharply when the overlap area is too large. Therefore, to solve this problem, the application first evaluates the required computing resources of the first road slope image. The required computing resources can be understood as the number of basic operations required for algorithm execution, and the required computing resources of the image are mainly determined by the number of times of accessing the pixel points in the image by the algorithm. Therefore, the higher the resolution of the first road slope image, the more computing resources are required. At this time, in order to maximize the use of system computing resources and improve the fusion effect of the subsequent overlap area, the overlap ratio between each overlapping sub-image and the adjacent overlapping sub-image is determined according to the ratio between the required computing resources of the first road slope image and the total computing resources of the inspection platform. For example, if the required computing resources of the first road slope image are 0.8, the total computing resources of the inspection platform are 2, and each overlapping sub-image has 4 adjacent overlapping sub-images, the overlap ratio between the two adjacent overlapping sub-images is 0.1. In addition, in order to meet the fusion requirements of the area with a large image feature range, the application converts the first road image into a gradient image, identifies multiple high-gradient areas in the gradient image, selects a high-gradient area with the largest area range from the multiple high-gradient areas, converts the high-gradient area with the largest area range into a standard reference area range, and then takes the reference area range as the area size of the overlapping sub-area. Finally, according to the overlap ratio and the area size, the first road slope image is divided into multiple overlapping sub-images with the same size, so as to improve the fusion effect of the multiple overlapping sub-images after brightness compensation.

[0075] S103, calculate the local entropy and gradient mean value of the multiple overlapping sub-images.

[0076] S104, calculate the illumination compensation coefficient of the multiple overlapping sub-images according to the local entropy and gradient mean value of the multiple overlapping sub-images.

[0077] In the above steps S103 to S104, for the local entropy of the overlapping sub-image, the application calculates it by using the Shannon entropy calculation formula, so as to obtain the complexity of the pixel distribution of the overlapping sub-image. It can be understood that the greater the difference between the pixel values, the higher the entropy value, and the stronger the illumination non-uniformity. Then, for the gradient mean value of the overlapping sub-image, the Sobel operator is used to calculate the gradient image of the overlapping sub-image, and then the mean value of the gradient image is calculated to obtain the gradient mean value, so as to clearly determine the edge information richness of the overlapping sub-image.

[0078] Then, the maximum local entropy in the local entropy of the plurality of overlapping sub-images and the maximum gradient mean in the gradient mean are traversed again to ensure that the illumination compensation coefficient has a unified scale standard in the global range of the image, and then the illumination compensation coefficient of each overlapping sub-image is calculated by using the following calculation formula:

[0079]

[0080] wherein, is the illumination compensation coefficient of the i-th overlapping sub-image, is the local entropy of the i-th overlapping sub-image, is the maximum local entropy in the plurality of overlapping sub-images, is the gradient mean of the i-th overlapping sub-image, is the maximum gradient mean in the plurality of overlapping sub-images, and are the local entropy weight coefficient and the gradient mean weight coefficient respectively, wherein, , =1.

[0081] In the above formula, the local entropy reflects the uniformity of the illumination of the overlapping sub-image, and the larger the local entropy is, the more uneven the illumination is, at this time, the illumination compensation coefficient is increased to enhance the brightness of the detail features in the low-illumination area, but in order to prevent the overexposure of the area with high brightness, which leads to the loss of detail features, the gradient mean of the overlapping sub-image is introduced as a constraint term to control the compensation amplitude of the illumination, that is, for the local entropy which is high but the detail features are clear (the gradient mean is low), the illumination compensation coefficient is suppressed, so as to realize the balance between the illumination compensation and the detail protection. In addition, since the primary task of the slope inspection is to improve the image detectability through the illumination adjustment, and the gradient mean is used as a constraint term to avoid the loss of detail features in the illumination compensation process, therefore, the local entropy weight coefficient is greater than the gradient mean weight coefficient.

[0082] S105, based on the illumination compensation coefficient of the plurality of overlapping sub-images, performing brightness compensation on the plurality of overlapping sub-images to obtain a second highway slope image.

[0083] In the above steps, the pixel matrix of the plurality of overlapping sub-images is multiplied by the corresponding illumination compensation coefficient to obtain a plurality of overlapping sub-compensation images, and then the plurality of sub-compensation images are spliced and fused to obtain a second highway slope image.

[0084] ​​In the splicing fusion process, although the overlapping area can eliminate the boundary effect, if the difference of illumination compensation coefficients of two adjacent overlapping sub-compensation images used for splicing is large, the pixel value at the boundary will change suddenly, and a clear boundary line will still be formed. Therefore, the continuity of the pixel value in the overlapping area needs to be considered in the fusion splicing process, so as to make the pixel value in the overlapping area smoothly transition. Specifically, for the pixel points in the overlapping area, the pixel value is weighted and fused according to the coordinates of the pixel points in the overlapping area, so that the pixel value of the pixel points reaches the optimal balance of detail expression in two adjacent overlapping sub-compensation images, and the visual interference caused by the boundary effect is eliminated, so as to obtain the fused pixel value. The following calculation formula can be used for calculation:

[0085]

[0086]

[0087] wherein, is the fused pixel value of the i-th pixel point, , are the pixel values of the i-th pixel point in the j-th overlapping sub-compensation image and the k-th overlapping sub-compensation image respectively, is the weight value of the i-th pixel point, is the boundary distance of the i-th pixel point to the overlapping area of the k-th overlapping sub-compensation image, and D is the total width of the overlapping area of the j-th overlapping sub-compensation image and the k-th overlapping sub-compensation image.

[0088] In the above formula, when the boundary distance of the i-th pixel point to the overlapping area of the k-th overlapping sub-compensation image is 0, then =1, at this time, the i-th pixel point completely adopts the illumination compensation result of the j-th overlapping sub-compensation image, when the boundary distance of the i-th pixel point to the overlapping area of the k-th overlapping sub-compensation image is D, then =0, at this time, the i-th pixel point completely adopts the illumination compensation result of the k-th overlapping sub-compensation image, when the boundary distance of the i-th pixel point to the overlapping area of the k-th overlapping sub-compensation image is between 0 and D, at this time, if the i-th pixel point is close to the boundary of the overlapping area of the k-th overlapping sub-compensation image, the pixel value of the i-th pixel point mainly inherits the illumination compensation effect of the k-th overlapping sub-compensation image, if the i-th pixel point is close to the boundary of the overlapping area of the j-th overlapping sub-compensation image, the pixel value of the i-th pixel point mainly inherits the illumination compensation effect of the j-th overlapping sub-compensation image, thereby realizing the gradual transition of the pixel points in the overlapping area between the two overlapping sub-compensation images, and eliminating the boundary effect.

[0089] S106, disease identification is performed on the second highway slope image to generate an inspection report.

[0090] In the above steps, by inputting the second highway slope image into the disease detection model, multiple disease characteristics are obtained, and the multiple disease characteristics generate an inspection report. The disease monitoring model is a neural network model trained by a large number of slope disease images.

[0091] In a possible implementation, the disease characteristics of the highway slope can be divided into small-scale disease characteristics and large-scale disease characteristics according to the disease size, wherein the small-scale disease characteristics represent fine cracks of the slope, and the large-scale disease characteristics represent collapse or collapse of the slope. Therefore, in order to improve the small-scale disease recall rate and the large-scale disease positioning accuracy, the second highway slope image is subjected to multi-level downsampling before being input into the disease detection model, to obtain multiple secondary images. Since the small-scale disease characteristics of the slope are mainly 1-50 mm cracks, and the large-scale disease characteristics are mainly 1-10 m slope deformation, the multiple secondary images are 1 / 2 resolution (64 channel) secondary images, 1 / 4 resolution (128 channel) secondary images, and 1 / 8 resolution (256 channel) secondary images. Among them, the 1 / 2 resolution (64 channel) secondary image can more accurately reflect the edge details (small-scale disease) in the image, the 1 / 4 resolution (128 channel) secondary image can more accurately reflect the texture information (part of the small-scale disease and part of the large-scale disease) in the image, and the 1 / 8 resolution (256 channel) secondary image can more accurately reflect the semantic characteristics (large-scale disease) in the image.

[0092] Then, the 1 / 2 resolution highway slope image and the 1 / 4 resolution highway slope image are channel fused to obtain a 1 / 2 resolution (96 channel) highway slope image. This fusion method retains high resolution while fusing more channel feature information, thereby enhancing the recognition ability of the shallow network for small targets and improving the recall rate of the small-scale disease area. The 1 / 4 resolution highway slope image and the 1 / 8 resolution highway slope image are channel fused to obtain a 1 / 4 resolution (192 channel) highway slope image. This fusion method uses low resolution to provide a global view, and then integrates multi-scale semantic features by channel fusion to assist the deep network in positioning the overall boundary of large targets, thereby improving the positioning accuracy of the large-scale disease area. Finally, the 1 / 2 resolution (96 channel) highway slope image is input into the disease detection model to output small-scale disease characteristics, and the 1 / 4 resolution (192 channel) highway slope image is input into the disease detection model to output large-scale disease characteristics. Finally, the small-scale disease characteristics and the large-scale disease characteristics generate an inspection report, which includes disease images, types, sizes, grades, and coordinates.

[0093] Reference Figure 2The application further provides a highway slope efficient inspection platform using a UAV, the system is the inspection platform, the inspection platform comprises an acquisition module 1, a processing module 2 and an output module 3, wherein:

[0094] The acquisition module 1 is used for acquiring a first highway slope image collected by the UAV.

[0095] The processing module 2 is used for dividing the first highway slope image into a plurality of overlapping sub-images of the same size; calculating local entropy and gradient mean values of the plurality of overlapping sub-images; calculating illumination compensation coefficients of the plurality of overlapping sub-images according to the local entropy and the gradient mean values of the plurality of overlapping sub-images; and performing brightness compensation on the plurality of overlapping sub-images based on the illumination compensation coefficients of the plurality of overlapping sub-images to obtain a second highway slope image.

[0096] The output module 3 is used for performing disease identification on the second highway slope image to generate an inspection report.

[0097] It should be noted that the device provided in the above embodiment is used to implement its functions, and the above-mentioned division of each functional module is only used as an example for illustration, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0098] The application further discloses an electronic device. Referring to Figure 3 , Figure 3 is a structural schematic diagram of an electronic device disclosed by the embodiment of the application. The electronic device 300 can include at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0099] The communication bus 302 is used to realize the connection and communication between the components.

[0100] The user interface 303 can include a display screen (Display) and a camera (Camera), and the optional user interface 303 can further include a standard wired interface and a wireless interface.

[0101] The network interface 304 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0102] The processor 301 can include one or more processing cores. The processor 301 connects various parts within the server through various interfaces and lines, performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Alternatively, the processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 301 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 301, but can be realized by a separate chip.

[0103] The memory 305 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 305 can also be at least one storage device located away from the aforementioned processor 301. Referring to Figure 3 The memory 305 as a kind of computer storage medium can include an operating system, a network communication module, a user interface module and an application program of the application of the highway slope efficient inspection method of unmanned aerial vehicle.

[0104] In Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 301 can be used to invoke an application program stored in the memory 305, which is an application of the method for efficiently patrolling a highway slope by using a UAV, and when executed by one or more processors 301, causes the electronic device 300 to perform the method described in one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0105] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0106] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some service interface, device or unit, and can be electrical or other forms.

[0107] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0109] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0110] The above-described are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure.

[0111] The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for efficient inspection of highway slope by using a UAV, characterized in that, The method is applied to a patrol platform and comprises the following steps: acquiring a first highway slope image collected by a UAV; dividing the first highway slope image into a plurality of overlapping sub-images of the same size; calculating local entropy and gradient mean values of the plurality of overlapping sub-images; calculating illumination compensation coefficients of the plurality of overlapping sub-images according to the local entropy and gradient mean values of the plurality of overlapping sub-images, and the calculation specifically comprises the following steps: ； wherein, is a local entropy of the i-th overlapping sub-image, is a local entropy of the i-th overlapping sub-image, is a maximum local entropy among the plurality of overlapping sub-images, is a gradient mean of the i-th overlapping sub-image, is a maximum gradient mean among the plurality of overlapping sub-images, and are a local entropy weight coefficient and a gradient mean weight coefficient, respectively, wherein, , + = 1.​ performing brightness compensation on the plurality of overlapping sub-images based on the illumination compensation coefficients of the plurality of overlapping sub-images to obtain a second highway slope image, and the brightness compensation specifically comprises the following steps: multiplying pixel matrices of the plurality of overlapping sub-images by the respective corresponding illumination compensation coefficients to obtain a plurality of overlapping sub-compensation images; extracting an overlapping area between a first overlapping sub-compensation image and a second overlapping sub-compensation image, the first overlapping sub-compensation image and the second overlapping sub-compensation image being any two adjacent overlapping sub-compensation images in the plurality of overlapping sub-compensation images; converting a plurality of pixel points in the overlapping area into a coordinate matrix; calculating fusion pixel values of the plurality of pixel points in the overlapping area according to pixel values of the plurality of pixel points in the overlapping area in the first overlapping sub-compensation image, pixel values of the plurality of pixel points in the overlapping area in the second overlapping sub-compensation image, and the coordinate matrix; fusing and splicing the plurality of overlapping sub-compensation images according to the fusion pixel values of the plurality of pixel points in the overlapping area to obtain the second highway slope image; performing disease identification on the second highway slope image to generate a patrol report.

2. The method of claim 1, wherein, Before the step of acquiring the first highway slope image collected by the UAV, the method further comprises the following steps: acquiring terrain features of a target highway slope, the terrain features comprising a slope variance and a slope curvature mean value; calculating a terrain complexity of the target highway slope according to the terrain features; matching the terrain complexity with a preset shooting density database to obtain a shooting density corresponding to the terrain complexity of the target highway slope, so that the UAV collects the first highway slope image according to the shooting density corresponding to the terrain complexity of the target highway slope.

3. The method of claim 2, wherein, The step of calculating the terrain complexity of the target highway slope according to the terrain features further comprises the following steps: calculating a vegetation coverage rate of the target highway slope; when the vegetation coverage rate is greater than or equal to a preset vegetation coverage rate, acquiring a near-infrared band distribution map of the target highway slope; calculating an infrared band variance of the near-infrared band distribution map; determining a terrain complexity compensation coefficient of the target highway slope according to the infrared band variance; compensating the terrain complexity of the target highway slope according to the terrain complexity compensation coefficient to obtain a target terrain complexity.

4. The method of claim 1, wherein, The step of dividing the first highway slope image into a plurality of overlapping sub-images of the same size comprises the following steps: evaluating required computing resources of the first highway slope image; determining an overlapping ratio according to a ratio between the required computing resources of the first highway slope image and total computing resources of the patrol platform; converting the first highway slope image into a gradient image; identifying a plurality of high-gradient areas in the gradient image; determining a region size according to a region size in a plurality of the high gradient regions; dividing the first highway slope image into a plurality of overlapping sub-images of the same size according to the overlap ratio and the region size.

5. The method of claim 1, wherein, The disease identification on the second highway slope image generates an inspection report, and specifically includes: multi-level down-sampling the second highway slope image to obtain a plurality of secondary images; fusing a plurality of the secondary images to obtain a plurality of fused images; inputting a plurality of the fused images into a disease detection model to obtain small-scale disease regions and large-scale disease regions and generate an inspection report.

6. A highway slope efficient inspection system using a UAV, characterized in that, The system is an inspection platform, and the inspection platform includes an acquisition module (1), a processing module (2), and an output module (3), wherein: The acquisition module (1) is configured to acquire a first highway slope image collected by a UAV. The processing module (2) is configured to divide the first highway slope image into a plurality of overlapping sub-images of the same size, calculate local entropy and gradient mean of a plurality of the overlapping sub-images, and calculate illumination compensation coefficients of a plurality of the overlapping sub-images according to the local entropy and the gradient mean of a plurality of the overlapping sub-images. The processing module (2) is configured to divide the first highway slope image into a plurality of overlapping sub-images of the same size, calculate local entropy and gradient mean of a plurality of the overlapping sub-images, and calculate illumination compensation coefficients of a plurality of the overlapping sub-images according to the local entropy and the gradient mean of a plurality of the overlapping sub-images. ； wherein, is a local entropy of the i-th overlapping sub-image, is a local entropy of the i-th overlapping sub-image, is a maximum local entropy among the plurality of overlapping sub-images, is a gradient mean of the i-th overlapping sub-image, is a maximum gradient mean among the plurality of overlapping sub-images, and are a local entropy weight coefficient and a gradient mean weight coefficient, respectively, wherein, , + = 1.​ The processing module (2) is configured to perform brightness compensation on a plurality of the overlapping sub-images based on the illumination compensation coefficients of a plurality of the overlapping sub-images to obtain a second highway slope image, and specifically includes: The processing module (2) is configured to perform brightness compensation on a plurality of the overlapping sub-images based on the illumination compensation coefficients of a plurality of the overlapping sub-images to obtain a second highway slope image, and specifically includes: extracting an overlapping region between a first overlapping sub-compensation image and a second overlapping sub-compensation image, the first overlapping sub-compensation image and the second overlapping sub-compensation image being any two adjacent overlapping sub-compensation images in a plurality of the overlapping sub-compensation images; converting a plurality of pixel points in the overlapping region into a coordinate matrix; calculating a fused pixel value of the plurality of pixel points in the overlapping region according to pixel values of the plurality of pixel points in the overlapping region in the first overlapping sub-compensation image and the second overlapping sub-compensation image and the coordinate matrix; fusing and splicing a plurality of the overlapping sub-compensation images according to the fused pixel value of the plurality of pixel points in the overlapping region to obtain the second highway slope image. The output module (3) is configured to perform disease identification on the second highway slope image to generate an inspection report.

7. An electronic device, comprising: The electronic device (300) includes a processor (301), a memory (305), a user interface (303), and a network interface (304), the memory (305) is configured to store instructions, the user interface (303) and the network interface (304) are configured to communicate with other devices, and the processor (301) is configured to execute the instructions stored in the memory (305) to enable the electronic device (300) to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method of any one of claims 1 to 5.

Images