Line target enhancement method, device and equipment based on unmanned aerial vehicle inspection and medium

Abstract

The present application relates to the technical field of image processing, and provides a line target enhancement method, device and equipment based on unmanned aerial vehicle inspection and a medium. The method can collect polarization images of multiple preset polarization directions, eliminate motion blur and channel misplacement, generate a line polarization degree map and a line polarization angle map according to the polarization images of each preset polarization direction, so as to convert the polarization characteristics of light into computable image features, generate an initial candidate mask according to the line polarization degree map and the line polarization angle map, thereby locking the line target area and reducing the detection range, perform line continuity enhancement on the initial candidate mask, and output a continuous and clear line polarization enhancement response map, and finally fuse the line polarization enhancement response map and the original optical image, so as to take into account polarization enhancement and original texture, and obtain a high-quality line target enhancement image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for enhancing linear targets based on unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] Currently, the following methods are mainly used for visual detection of linear targets such as power lines, guy wires, and cables in drone inspection scenarios: (1) Target detection method based on ordinary RGB (Red Green Blue) camera: Color images are acquired using a conventional visible light camera, and wire regions in the images are identified through Convolutional Neural Network (CNN) and target detection networks (such as YOLO (You Only Look Once) and Faster R-CNN). Features such as brightness contrast, edge gradient, and geometric elongated shape are usually utilized.

[0003] (2) Traditional image enhancement methods based on edge or ridge operators: edge detection is performed on the image using operators such as Canny, Sobel, and Laplacian, and then thin line structures are extracted through morphological operations (dilation, erosion, thinning), or ridge enhancement techniques such as Hessian matrix features and Frangi filter are used to highlight the linear structures.

[0004] (3) Linear target segmentation method based on deep learning: introduce semantic segmentation networks such as U-Net (U-shaped convolutional neural network) and DeepLab (deep laboratory) to perform pixel-level segmentation of the wire region in the image. It requires supervised training through a large amount of wire annotation data.

[0005] The methods described above are all based on ordinary intensity imaging, which only utilizes the brightness or color information of each pixel without taking into account the polarization characteristics of light. However, in the UAV inspection environment, linear targets such as power lines, guy wires, and cables typically have the following characteristics: complex backgrounds (including sky, clouds, mountains, trees, buildings, etc.), strong lighting (such as backlighting, sidelighting, strong sunlight causing background overexposure or high contrast), extremely fine targets (the line width is only 1-3 pixels under long-distance conditions), and complex materials (metals, coatings, cable sheaths have strong specular reflection or a certain degree of polarization characteristics).

[0006] Under the aforementioned inspection environment characteristics, existing methods mainly have the following problems: (1) Extremely low contrast, the wires are almost submerged in the background: against a backlit sky background, the wires are close to or completely submerged in the sky, making them difficult to distinguish in ordinary RGB imaging.

[0007] (2) Changes in illumination lead to extremely unstable detection: The same wire will produce very different imaging effects when photographed at different times and angles, requiring a huge amount of training data to adapt.

[0008] (3) The polarization information was not utilized, wasting the distinguishability brought about by the material difference: the reflection characteristics of metal or coated materials such as wires and cables to polarized light are significantly different from those of backgrounds such as clouds, sky, and mountains. If only brightness information is used, a very important distinguishing dimension is lost.

[0009] (4) Low recall rate for detecting small targets: For small linear targets at long distances, ordinary convolutional features are difficult to separate from noise, resulting in serious missed detections.

[0010] Therefore, there is a need for an enhanced imaging method that can fully utilize the principle of polarization imaging and combine the morphological characteristics of linear targets, so as to significantly improve the visibility and separability of linear targets such as power lines in images under complex lighting, strong reflection and extremely low contrast environments, and provide more reliable input for subsequent automatic recognition and obstacle avoidance. Summary of the Invention

[0011] In view of the above, it is necessary to provide a method, device, equipment and medium for enhancing linear targets based on UAV inspection, which aims to solve the problem of weak imaging of linear targets in UAV inspection missions.

[0012] A method for enhancing linear targets based on UAV inspection, the method comprising: In response to a linear target enhancement command triggered by a UAV inspection mission, polarization images with multiple preset polarization directions are acquired. Generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction; An initial candidate mask is generated based on the linear polarization degree diagram and the linear polarization angle diagram; Linear continuity enhancement is applied to the initial candidate mask to obtain a linear polarization enhancement response map; Obtain the original optical image corresponding to the polarization image for each preset polarization direction; The linear polarization enhancement response map is fused with the original optical image to obtain the target enhancement image.

[0013] A linear target enhancement device based on UAV inspection, characterized in that the linear target enhancement device based on UAV inspection comprises: The acquisition unit is used to acquire polarization images with multiple preset polarization directions in response to the linear target enhancement command triggered by the UAV inspection mission. The generation unit is used to generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction; The generation unit is further configured to generate an initial candidate mask based on the linear polarization degree map and the linear polarization angle map; An enhancement unit is used to perform linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map; The acquisition unit is used to acquire the original optical image corresponding to the polarization image of each preset polarization direction; The fusion unit is used to fuse the linear polarization enhancement response map with the original optical image to obtain the target enhancement image.

[0014] A computer device, the computer device comprising: A memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the UAV-based linear target enhancement method.

[0015] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the UAV-based linear target enhancement method.

[0016] As can be seen from the above technical solutions, the present invention can acquire polarization images with multiple preset polarization directions, eliminating motion blur and channel misalignment; generate linear polarization degree map and linear polarization angle map based on the polarization image with each preset polarization direction, so as to convert the polarization characteristics of light into computable image features; generate initial candidate masks based on the linear polarization degree map and linear polarization angle map, thereby locking the linear target region and reducing the detection range; enhance the linear continuity of the initial candidate mask, and output a continuous and clear linear polarization enhancement response map; fuse the linear polarization enhancement response map with the original optical image, which can take into account both polarization enhancement and original texture, and obtain a high-quality linear target enhancement image. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the linear target enhancement method based on UAV inspection of the present invention.

[0018] Figure 2 This is a functional block diagram of a preferred embodiment of the linear target enhancement device based on UAV inspection of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of a computer device that implements a preferred embodiment of the linear target enhancement method based on UAV inspection according to the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the linear target enhancement method based on UAV inspection according to the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0022] The aforementioned method for enhancing linear targets based on UAV inspection is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0023] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), interactive network television (IPTV), smart wearable device, etc.

[0024] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0025] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0026] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0027] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0028] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0029] S10, in response to the linear target enhancement command triggered by the UAV inspection mission, acquires polarization images with multiple preset polarization directions.

[0030] In this embodiment, the drone inspection task can be a drone-based inspection task in various scenarios such as power line inspection, wind turbine, bridge cable inspection, and petrochemical plant pipe gallery.

[0031] In this embodiment, the linear target may include small linear targets such as power lines, guy wires, and cables.

[0032] In this embodiment, the linear target enhancement command can be triggered synchronously when the UAV inspection task is performed.

[0033] In this embodiment, a polarization camera rigidly fixed to the belly of the UAV or a front / side gimbal can be used. The field of view covers the inspection flight path and areas where linear targets may appear. This camera calibrates intrinsic parameters, distortion, and extrinsic parameters with RGB (Red, Green, Blue) / depth sensors to acquire polarized images. The polarization camera should have vibration reduction or electronic image stabilization capabilities to prevent misalignment of the micro-polarization array channels. The exposure time should ensure motion blur is less than approximately 0.5-1 pixel, while avoiding saturation in the sky or reflective areas. Typically, it can be adaptively set within the range of 1 / 2000-1 / 8000 seconds based on flight speed, focal length, and pixel angular resolution. The acquisition resolution must ensure that the target linewidth or linear response is at least approximately 1-3 pixels in the image. For long-distance inspection, a resolution of 1920×1080 or higher is preferred, with 4K being even better.

[0034] In this embodiment, the preset polarization direction can be a 0°, 45°, 90°, or 135° polarization direction.

[0035] Using four angles—0°, 45°, 90°, and 135°—can improve noise immunity and engineering availability. In principle, as long as at least three different polarization directions are collected without angle degradation, linear polarization information can also be calculated.

[0036] The above embodiments enable the acquisition of raw multi-dimensional data required for polarization detection, eliminating motion blur and channel misalignment, and providing stable input for subsequent calculations.

[0037] In this embodiment, if a single-chip micro-polarization array camera is used, four channels of polarization information (each 2×2 pixels is a polarization unit) can be obtained in a single shot.

[0038] Specifically, if a micro-polarization array camera is used, these images are obtained by decoding, interpolating and resampling the original mosaic polarization image, and it is necessary to ensure that the four channels are aligned in the same spatial position to generate polarization images of the same resolution and coordinates in each direction.

[0039] If the camera is a polarization camera or an RGB camera with a binocular structure, then perform geometric calibration and extrinsic parameter alignment to ensure that the spatial positions of the four polarization channels and the RGB channels are completely consistent.

[0040] The above embodiments ensure that the same pixel corresponds to the same physical position in the four polarization images, thereby eliminating polarization channel misalignment and geometric deviation and ensuring the accuracy of subsequent polarization information calculation.

[0041] S11, Generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction.

[0042] In this embodiment, generating a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction includes: Based on the pixel values ​​in the polarization image of each preset polarization direction, calculate the degree of linear polarization and the angle of linear polarization at each pixel position. The linear polarization degree map is obtained by stitching together the degree of linear polarization (DoLP) at each pixel position. The linear polarization angle map is obtained by stitching together the linear polarization angle (AoLP) of each pixel position.

[0043] The linear polarization degree is used to represent the degree of polarization of each pixel, and its value ranges from [0,1]. Linear targets such as wires are long and continuous structures, which appear as continuous, narrow, and oriented linear regions in the image. When the wire is under specific viewing angles and lighting conditions, its linear polarization degree appears as a long and thin highly polarized band in the image, which can form a clear contrast with the background.

[0044] For example, taking polarization directions of 0°, 45°, 90°, and 135° as examples, the degree of linear polarization at each pixel position can be calculated using the following formula: ; Where DoLP represents the degree of linear polarization; I0 represents the light intensity value of the polarized image at the corresponding pixel position in the 0° polarization direction; I 45 This represents the light intensity value at the corresponding pixel location in a polarized image with a 45° polarization direction; I 90 This represents the light intensity value at the corresponding pixel location of a polarized image with a 90° polarization direction; I 135 This represents the light intensity value at the corresponding pixel position of a polarized image with a 135° polarization direction. Represents a very small positive number (such as 10). -6 The purpose of this is to prevent the denominator from being zero and to avoid calculation errors.

[0045] The linear polarization angle represents the polarization direction and is related to the attitude of linear targets such as wires and the direction of incident light, with a value range of [0, π).

[0046] For example, taking polarization directions of 0°, 45°, 90°, and 135° as examples, the linear polarization angle at each pixel position can be calculated using the following formula: ; Where AoLP represents the linear polarization angle.

[0047] Through the above embodiments, the polarization characteristics of light can be transformed into computable image features, establishing a material differentiation dimension.

[0048] S12, Generate an initial candidate mask based on the linear polarization degree diagram and the linear polarization angle diagram.

[0049] Based on real-world scenario data, the DoLP of linear metal wires under strong light is often greater than a certain threshold (i.e., the polarization threshold). The typical threshold range is DoLP > 0.10–0.25, which can be adaptively adjusted based on calibration data (e.g., based on the environment and material).

[0050] Furthermore, when the AoLP standard deviation When the polarization angle consistency threshold (e.g., 10°–25°) is exceeded, the local polarization direction can be considered consistent, indicating the possibility of a linear target. In this case, even if the background area (e.g., clouds, sky) has a high DoLP, the AoLP spatial variation is large, and it can still be excluded.

[0051] Therefore, candidate regions can be selected based on the aforementioned polarization properties of linear targets to reduce the detection workload.

[0052] Specifically, generating the initial candidate mask based on the linear polarization degree map and the linear polarization angle map includes: A polarization degree threshold is obtained, and pixels with a linear polarization degree greater than the polarization degree threshold are obtained from the linear polarization degree map to construct a first mask; Configure a local window; Taking each pixel in the linear polarization angle map as the center, calculate the standard deviation of the linear polarization angle of all pixels within the local window, and use it as the standard deviation for each pixel. A polarization angle consistency threshold is obtained, and pixels with a standard deviation less than the polarization angle consistency threshold are obtained from the linear polarization angle map to construct a second mask; The first mask and the second mask are ANDed pixel by pixel to obtain the initial candidate mask.

[0053] The local window can be configured as a 5×5 window.

[0054] Through the above embodiments, the mask retains only the area of ​​highly polarized wires with consistent orientation, which can filter out clutter such as clouds and sky, thereby locking the approximate area of ​​linear targets and greatly reducing the detection range.

[0055] S13, perform linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map.

[0056] In this embodiment, based on the initial candidate mask, further filtering and enhancement are needed by utilizing the characteristics of the linear structure.

[0057] Specifically, the step of performing linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map includes: Perform a morphological opening operation on the initial candidate mask to obtain a denoised mask; The denoising mask is processed based on a thinning algorithm to obtain multiple first candidate skeletons; Connectivity analysis is performed on the multiple first candidate skeletons to obtain multiple skeleton branches; Delete skeleton branches whose length is less than the length threshold from the plurality of skeleton branches to obtain a plurality of second candidate skeletons; Estimate the local orientation of each second candidate skeleton, and select skeletons with continuous orientation, curvature change less than a preset threshold and skeleton length within a preset length range from the plurality of second candidate skeletons based on the local orientation of each second candidate skeleton, to obtain the third candidate skeleton; Obtain the directional consistency weight and skeleton distance weight; Linear enhancement is performed on the third candidate skeleton based on the directional consistency weight and the skeleton distance weight to obtain the linear polarization enhancement response map.

[0058] Among them, 3×3 or 5×5 morphological opening operations can be used to remove isolated noise points and small patches in the initial candidate mask.

[0059] The denoising mask can be processed using Zhang-Suen or Guo-Hall to obtain the plurality of first candidate skeletons with a width of one pixel.

[0060] The length threshold can be configured to be 20–50 pixels.

[0061] Principal component analysis or structural tensor estimation can be used to estimate the local orientation of each second candidate skeleton.

[0062] Specifically, based on the local direction of each second candidate skeleton, skeletons with continuous direction, curvature change less than a preset threshold, and skeleton length within a preset length range (e.g., 20–80 pixels) are selected from the plurality of second candidate skeletons. This can suppress candidate regions with drastic direction changes and severe breaks.

[0063] Specifically, the third candidate skeleton is linearly enhanced according to the orientation consistency weight and the skeleton distance weight. Pixels within a certain width range near the third candidate skeleton are given an enhanced response, that is, pixels near the skeleton are given a high response value that is proportional to the consistency of DoLP and AoLP, and the response is 0 in non-candidate areas, thereby forming the final linear polarization enhancement response map.

[0064] For example, the following formula can be used to generate a linear polarization enhancement response map: ; in, This represents the response value at pixel coordinates (x, y) in the linear polarization enhancement response map; This represents the degree of linear polarization at pixel coordinates (x, y). Indicates the weight of directional consistency; This represents the skeleton distance weight.

[0065] The directional consistency weight can be calculated based on AoLP fluctuations. Specifically, since AoLP is an angular quantity and has π-periodicity, local fluctuations can be calculated using circular statistics. Within a local window centered at pixel coordinates (x, y), for each AoLP value... calculate and Calculate the average value , , then calculate Local angular fluctuation (i.e., the standard deviation of AoLP at pixel coordinates (x,y)) ) can be represented as The closer R is to 1, the more consistent the angles and the smaller the fluctuations. The weight for directional consistency can be chosen as follows: Alternatively, it can be normalized to 0-1 according to the polarization angle consistency threshold. This is a preset reference fluctuation threshold (such as 15° or its radian value).

[0066] Among them, the skeleton distance weight indicates whether a pixel is close to the center of the skeleton (the closer the distance, the greater the weight).

[0067] Through the above embodiments, it is possible to combine linear geometric priors to purify the target and suppress noise, thereby outputting continuous and clear linear enhancement features.

[0068] S14, acquire the original optical image corresponding to the polarization image for each preset polarization direction.

[0069] In this embodiment, obtaining the original optical image corresponding to the polarization image for each preset polarization direction includes: The original optical image is obtained by simultaneously acquiring an image using a color camera; or The polarization images of each preset polarization direction are synthesized to obtain the original optical image.

[0070] The color camera can be a separate camera, and it has undergone extrinsic parameter calibration and spatial alignment with the polarization camera.

[0071] In the process of synthesizing polarized images in each preset polarization direction, the corresponding pixel values ​​of the polarized images in each preset polarization direction can be added together or averaged to obtain a synthesized grayscale image.

[0072] S15, the linear polarization enhancement response map is fused with the original optical image to obtain the target enhancement image.

[0073] In this embodiment, fusing the linear polarization enhancement response map with the original optical image to obtain the target enhancement image includes: When the original optical image is a color image, the original optical image is converted into a grayscale image; The grayscale image is normalized to obtain the first image; The linear polarization enhancement response map is normalized to obtain a second image; The first image and the second image are fused according to the fusion weights to obtain the target enhanced image.

[0074] For example, when the original optical image is a color image, the grayscale image G(x,y) can be constructed using the following formula: G(x,y)=0.299R+0.587G+0.114B.

[0075] Furthermore, the first image and the second image can be fused using the following formula: ; in, α represents the pixel value at pixel coordinates (x, y) in the target enhanced image; α represents the fusion weight. This represents the response value at pixel coordinates (x, y) in the second image; This represents the grayscale value at pixel coordinates (x, y) in the first image.

[0076] The normalization operation can normalize the value to [0,1].

[0077] The typical value of the fusion weight is between 0.4 and 0.7. When α is larger, the polarization characteristics of linear targets are more strongly reflected. By adjusting α, a trade-off can be made between visual readability and the degree of linear prominence.

[0078] In the above embodiments, the final target enhancement image is a grayscale image that has a significant enhancement effect on linear targets such as power lines, taking into account both polarization enhancement and original texture details, and can be directly used as input data for subsequent tasks such as detection, segmentation, and obstacle avoidance algorithms.

[0079] In this embodiment, to improve the operability of implementation, parameters that are most sensitive to changes in different lighting, background and materials, such as polarization degree threshold, polarization angle consistency threshold and fusion weight, can be optimized by online adaptive algorithms.

[0080] Specifically, after obtaining the enhanced target image, the method further includes: At preset time intervals, acquire frame images within the previous preset duration, starting from the current timestamp; Obtain the linear polarization degree distribution of the candidate line region and the linear polarization degree distribution of the background region within the frame image; The initial polarization threshold that maximizes the difference in linear polarization distribution between the candidate line region and the background region is calculated according to the maximum inter-class difference principle. The initial polarization threshold is then smoothed with a first preset numerical range as a constraint to obtain the updated polarization threshold. Calculate the average skeleton length, false detection rate, and detection confidence of the frame image, and adjust the polarization angle consistency threshold according to the average skeleton length, the false detection rate, and the detection confidence to obtain the updated polarization angle consistency threshold; The noise level of the background region of the frame image is calculated, the initial fusion weight that maximizes the detection confidence and minimizes the noise level is calculated, and the initial fusion weight is smoothed with a second preset numerical range as a constraint to obtain the updated fusion weight.

[0081] For example, in the most recent N frames, the DoLP distribution, AoLP local fluctuations, and linear continuity scores of the candidate line region and background region are statistically analyzed. The polarization threshold is updated according to quantiles, Otsu's thresholding method, or the maximum inter-class difference principle, and constrained to the range of 0.10-0.25. Further, the polarization angle consistency threshold is updated based on the false detection rate, skeleton continuity length, and detection confidence. Even further, within the range of 0.4-0.7, the fusion weights that maximize subsequent line detection confidence and minimize background response are selected, and exponential moving averages are used to update the parameters to avoid inter-frame abrupt changes.

[0082] Other parameters in this embodiment can be optimized through on-site calibration and historical data statistics.

[0083] The above embodiments enable adaptation to complex lighting, background, or material changes, thereby improving robustness across all scenarios.

[0084] This embodiment utilizes the physical properties of polarization to separate linear targets such as power lines from the background in terms of material properties, significantly improving contrast. By combining the degree of linear polarization, the angle of linear polarization, and geometric priors of linear structures, highly robust line enhancement is achieved. This ensures that linear targets remain prominent even under backlighting, high contrast, and complex background conditions, significantly reducing the false negative rate. Furthermore, the resulting enhanced image is easily integrated with existing deep learning detection frameworks, resulting in low engineering implementation costs.

[0085] As can be seen from the above technical solutions, the present invention can acquire polarization images with multiple preset polarization directions, eliminating motion blur and channel misalignment; generate linear polarization degree map and linear polarization angle map based on the polarization image with each preset polarization direction, so as to convert the polarization characteristics of light into computable image features; generate initial candidate masks based on the linear polarization degree map and linear polarization angle map, thereby locking the linear target region and reducing the detection range; enhance the linear continuity of the initial candidate mask, and output a continuous and clear linear polarization enhancement response map; fuse the linear polarization enhancement response map with the original optical image, which can take into account both polarization enhancement and original texture, and obtain a high-quality linear target enhancement image.

[0086] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the linear target enhancement device based on UAV inspection of the present invention. The UAV-based linear target enhancement device 11 includes a data acquisition unit 110, a generation unit 111, an enhancement unit 112, an acquisition unit 113, and a fusion unit 114. The module / unit referred to in this invention is a series of computer program segments that can be executed by a processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0087] The acquisition unit 110 is used to acquire polarization images with multiple preset polarization directions in response to a linear target enhancement command triggered by a UAV inspection mission. The generation unit 111 is used to generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction. The generation unit 111 is further configured to generate an initial candidate mask based on the linear polarization degree map and the linear polarization angle map; The enhancement unit 112 is used to perform linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map; The acquisition unit 113 is used to acquire the original optical image corresponding to the polarization image of each preset polarization direction; The fusion unit 114 is used to fuse the linear polarization enhancement response map with the original optical image to obtain the target enhancement image.

[0088] As can be seen from the above technical solutions, the present invention can acquire polarization images with multiple preset polarization directions, eliminating motion blur and channel misalignment; generate linear polarization degree map and linear polarization angle map based on the polarization image with each preset polarization direction, so as to convert the polarization characteristics of light into computable image features; generate initial candidate masks based on the linear polarization degree map and linear polarization angle map, thereby locking the linear target region and reducing the detection range; enhance the linear continuity of the initial candidate mask, and output a continuous and clear linear polarization enhancement response map; fuse the linear polarization enhancement response map with the original optical image, which can take into account both polarization enhancement and original texture, and obtain a high-quality linear target enhancement image.

[0089] like Figure 3 The diagram shown is a schematic representation of the computer device used to implement a preferred embodiment of the method for enhancing linear targets based on unmanned aerial vehicle (UAV) inspection according to the present invention.

[0090] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a linear target enhancement program based on UAV inspection.

[0091] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0092] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0093] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal and external storage units of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of a linear target enhancement program based on UAV inspection, but also to temporarily store data that has been output or will be output.

[0094] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing a linear target enhancement program based on UAV inspection) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0095] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the various embodiments of the above-described method for enhancing linear targets based on UAV inspection, for example... Figure 1 The steps are shown.

[0096] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a data acquisition unit 110, a generation unit 111, an enhancement unit 112, an acquisition unit 113, and a fusion unit 114.

[0097] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the UAV-based linear target enhancement method described in the various embodiments of this invention.

[0098] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0099] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0100] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0101] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0102] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0103] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0104] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish a communication connection between the computer device 1 and other computer devices.

[0105] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0106] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0107] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0108] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement a linear target enhancement method based on UAV inspection, and the processor 13 can execute the multiple instructions to achieve the following: In response to a linear target enhancement command triggered by a UAV inspection mission, polarization images with multiple preset polarization directions are acquired. Generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction; An initial candidate mask is generated based on the linear polarization degree diagram and the linear polarization angle diagram; Linear continuity enhancement is applied to the initial candidate mask to obtain a linear polarization enhancement response map; Obtain the original optical image corresponding to the polarization image for each preset polarization direction; The linear polarization enhancement response map is fused with the original optical image to obtain the target enhancement image.

[0109] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0110] It should be noted that all the data involved in this case was legally obtained.

[0111] If any AI models, software tools, or components not belonging to this company appear in the embodiments of this invention, they are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this invention has been obtained by an entity authorized (with the knowledge and consent) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0112] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0113] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0114] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0116] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0117] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0118] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for enhancing linear targets based on unmanned aerial vehicle (UAV) inspection, characterized in that, The method for enhancing linear targets based on UAV inspection includes: In response to a linear target enhancement command triggered by a UAV inspection mission, polarization images with multiple preset polarization directions are acquired. Generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction; An initial candidate mask is generated based on the linear polarization degree map and the linear polarization angle map; Linear continuity enhancement is applied to the initial candidate mask to obtain a linear polarization enhancement response map; Obtain the original optical image corresponding to the polarization image for each preset polarization direction; The linear polarization enhancement response map is fused with the original optical image to obtain the target enhancement image.

2. The linear target enhancement method based on UAV inspection as described in claim 1, characterized in that, The step of generating a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction includes: Based on the pixel values ​​in the polarization image of each preset polarization direction, calculate the degree of linear polarization and the angle of linear polarization at each pixel position. The linear polarization degree map is obtained by stitching together the linear polarization degree of each pixel position according to the pixel position. The linear polarization angle diagram is obtained by stitching together the linear polarization angles of each pixel position.

3. The linear target enhancement method based on UAV inspection as described in claim 1, characterized in that, The step of generating an initial candidate mask based on the linear polarization degree map and the linear polarization angle map includes: A polarization degree threshold is obtained, and pixels with a linear polarization degree greater than the polarization degree threshold are obtained from the linear polarization degree map to construct a first mask; Configure a local window; Taking each pixel in the linear polarization angle map as the center, calculate the standard deviation of the linear polarization angle of all pixels within the local window, and use it as the standard deviation for each pixel; A polarization angle consistency threshold is obtained, and pixels with a standard deviation less than the polarization angle consistency threshold are obtained from the linear polarization angle map to construct a second mask; The first mask and the second mask are ANDed pixel by pixel to obtain the initial candidate mask.

4. The linear target enhancement method based on UAV inspection as described in claim 1, characterized in that, The step of performing linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map includes: Perform a morphological opening operation on the initial candidate mask to obtain a denoised mask; The denoising mask is processed based on a thinning algorithm to obtain multiple first candidate skeletons; Connectivity analysis is performed on the multiple first candidate skeletons to obtain multiple skeleton branches; Delete skeleton branches whose length is less than the length threshold from the plurality of skeleton branches to obtain a plurality of second candidate skeletons; Estimate the local orientation of each second candidate skeleton, and select skeletons with continuous orientation, curvature change less than a preset threshold and skeleton length within a preset length range from the plurality of second candidate skeletons based on the local orientation of each second candidate skeleton, to obtain the third candidate skeleton; Obtain the directional consistency weight and skeleton distance weight; Linear enhancement is performed on the third candidate skeleton based on the directional consistency weight and the skeleton distance weight to obtain the linear polarization enhancement response map.

5. The linear target enhancement method based on UAV inspection as described in claim 1, characterized in that, The process of acquiring the original optical image corresponding to the polarization image for each preset polarization direction includes: The original optical image is obtained by simultaneously acquiring an image using a color camera; or The polarization images of each preset polarization direction are synthesized to obtain the original optical image.

6. The linear target enhancement method based on UAV inspection as described in claim 3, characterized in that, The process of fusing the linear polarization enhancement response map with the original optical image to obtain the target enhancement image includes: When the original optical image is a color image, the original optical image is converted into a grayscale image; The grayscale image is normalized to obtain the first image; The linear polarization enhancement response map is normalized to obtain a second image; The first image and the second image are fused according to the fusion weights to obtain the target enhanced image.

7. The method for enhancing linear targets based on UAV inspection as described in claim 6, characterized in that, After obtaining the enhanced target image, the method further includes: At preset time intervals, acquire frame images within the previous preset duration, starting from the current timestamp; Obtain the linear polarization degree distribution of the candidate line region and the linear polarization degree distribution of the background region within the frame image; The initial polarization threshold that maximizes the difference in linear polarization distribution between the candidate line region and the background region is calculated according to the maximum inter-class difference principle. The initial polarization threshold is then smoothed with a first preset numerical range as a constraint to obtain the updated polarization threshold. Calculate the average skeleton length, false detection rate, and detection confidence of the frame image, and adjust the polarization angle consistency threshold according to the average skeleton length, the false detection rate, and the detection confidence to obtain the updated polarization angle consistency threshold; The noise level of the background region of the frame image is calculated, the initial fusion weight that maximizes the detection confidence and minimizes the noise level is calculated, and the initial fusion weight is smoothed with a second preset numerical range as a constraint to obtain the updated fusion weight.

8. A linear target enhancement device based on UAV inspection, characterized in that, The linear target enhancement device based on UAV inspection includes: The acquisition unit is used to acquire polarization images with multiple preset polarization directions in response to the linear target enhancement command triggered by the UAV inspection mission. The generation unit is used to generate a linear polarization degree map and a linear polarization angle map based on the polarization image of each preset polarization direction; The generation unit is further configured to generate an initial candidate mask based on the linear polarization degree map and the linear polarization angle map; An enhancement unit is used to perform linear continuity enhancement on the initial candidate mask to obtain a linear polarization enhancement response map; The acquisition unit is used to acquire the original optical image corresponding to the polarization image of each preset polarization direction; The fusion unit is used to fuse the linear polarization enhancement response map with the original optical image to obtain the target enhancement image.

9. A computer device, characterized in that, The computer device includes: A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the linear target enhancement method based on UAV inspection as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the linear target enhancement method based on UAV inspection as described in any one of claims 1 to 7.

Images