A surface-oriented surveying and mapping unmanned aerial vehicle image correction method and system

Abstract

This invention discloses a method and system for correcting UAV images of water surfaces, relating to the field of image correction technology. The method includes: determining the target position relationship based on the original UAV image corresponding to the waterline region of a bridge pier; cropping the original UAV image according to the candidate connected regions and the target position relationship to obtain a near-waterline local image; performing edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; determining folded edge candidate edge chains based on the chain length and direction change of the anode adjacent edge chains, and generating folded edge candidate images; and performing mirroring and folding processing on the second neighboring image block corresponding to the folded edge candidate image based on the folded edge candidate edge chain to obtain a folded neighboring image block. This invention can improve the accuracy of near-water region boundary identification.

Description

Technical Field

[0001] This invention relates to the field of image correction technology, and in particular to a method and system for correcting images of a surveying UAV facing the water surface. Background Technology

[0002] With the increasing demand for digital mapping of bridges, piers, wharves, cross-river transportation facilities, and riverside engineering structures, surveying drones are widely used in surveying operations of bridge piers, pile foundations, and their surrounding areas due to their advantages such as high mobility, wide coverage, and high data acquisition efficiency. In actual engineering scenarios, surveying drones typically need to continuously photograph the near-water area of ​​bridge piers, the water-land interface area, and the distribution area of ​​auxiliary protective components to generate orthophotos for topographic mapping, structural monitoring, facility positioning, and data database construction. Because bridge piers are in a water environment for a long time, their outer sides are often equipped with anti-corrosion auxiliary components such as sacrificial anode blocks, and these components are usually... Installed near the waterline of the bridge pier, when a drone images the area near the water of the bridge pier, the actual surface of the bridge pier, the surface of the water, and the sacrificial anode block will all enter the imaging range at the same time. The water surface is affected by specular reflection, wave undulation, local highlights, and changes in ambient light, which can easily form a reflective boundary in the image that is continuously connected to the outline of the actual structure. This makes the actual area of ​​the bridge pier and the reflective area of ​​the water surface appear as a complex splicing relationship in the image, thus forming a pseudo-boundary phenomenon that extends continuously along the area adjacent to the sacrificial anode block. This pseudo-boundary has strong visual continuity and structural integrity, and is easy to be mixed with the actual boundary of the structure, which will affect subsequent surveying and mapping processing.

[0003] Existing technologies for processing UAV imagery typically utilize edge and texture information, as well as image matching relationships, to perform geometric correction, aerial triangulation, and orthorectification. However, they lack effective identification and elimination mechanisms for the phenomenon of mixed and spliced ​​solid areas and water surface reflection areas caused by the proximity of sacrificial anode blocks in the water-near area of ​​bridge piers. This can easily lead to the misjudgment of such pseudo-boundaries as real structural boundaries in subsequent image matching and geometric constraint establishment. Consequently, the matching relationships in local areas deviate from the actual correspondence of ground features, resulting in local geometric distortion, boundary misalignment, or orthorectification distortion in the area near the waterline of bridge piers. This, in turn, affects the spatial accuracy of the orthorectified images from UAVs and the quality of the surveying results. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies that easily misjudge such pseudo-boundaries as real structural boundaries and participate in subsequent image matching and geometric constraint establishment, and to propose a method and system for correcting images of UAVs for water surface mapping.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution: A method for correcting images of a surveying UAV facing a water surface, comprising: S1. Based on the original UAV imagery corresponding to the waterline area of ​​the bridge pier, determine the target position relationship; based on the candidate connected regions and the target position relationship, crop the original UAV imagery to obtain a local image near the waterline. S2. Perform edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; determine the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain, and generate the fold edge candidate image; S3. Based on the candidate edge chain of the candidate edge of the folded edge, the second neighboring image block corresponding to the candidate edge of the folded edge is mirrored to obtain the flipped neighboring image block; based on the flipped neighboring image block, the near waterline local image is expanded to generate a mirrored domain mask image. S4. Based on the mirror domain mask image, extract non-mirror adjacent pixels in the near-waterline local image; generate a corrected near-waterline local image based on the non-mirror adjacent pixels. S5. Based on the corrected near-waterline local image, obtain the preprocessed UAV image; perform geometric correction on the preprocessed UAV image to obtain the orthophoto result image of the surveying UAV.

[0006] Preferably, determining the target positional relationship includes: Obtain the original UAV imagery corresponding to the waterline area of ​​the bridge pier; The original UAV imagery is converted to grayscale to obtain a grayscale image. A smoothing filter is applied to the grayscale image to obtain a filtered grayscale image; Edge extraction is performed on the filtered grayscale image to obtain the edge image; Extract multiple candidate connected regions from the edge image; The main edge is extracted from the edge image to obtain the edge region of the bridge pier; Determine the target positional relationship between the candidate connected region and the pier edge region.

[0007] Preferably, obtaining a local image near the waterline includes: Based on the circumscribed contour size, contour shape, and target position relationship of the candidate connected regions, the anode candidate region is determined from multiple candidate connected regions; An anode anchor mask image is generated based on the pixel position of the anode candidate region in the original UAV image; the pixels in the anode anchor mask image corresponding to the anode candidate region are marked as anode pixels. Based on the bounding rectangle of the anode candidate region, the original UAV image is cropped to obtain a local image near the waterline.

[0008] Preferably, a plurality of anode adjacent edge chains are obtained, including: Edge extraction is performed on the local image near the waterline to obtain the edge image of the waterline; Boundary extraction is performed on the anode anchor point mask image to obtain the anode boundary image; Based on the position of the anode boundary pixels in the anode boundary image, the edge pixels in the near-waterline edge image are filtered by adjacency to obtain the initial edge pixel set; Starting from the edge pixels in the initial edge pixel set, edge connectivity tracking is performed in the near-waterline edge image to obtain multiple anode adjacent edge chains.

[0009] Preferably, generating candidate images of the folded edge includes: Determine the chain length and direction variation of the anode adjacent edge chain; Calculate the adjacency length between the anode adjacent edge chain and the anode boundary image; Based on the chain length, direction change, and adjacency length of the anode adjacent edge chains, multiple anode adjacent edge chains are sorted to obtain the edge chain sorting result; Based on the edge chain sorting results, candidate edge chains for the fold edge are determined from multiple adjacent edge chains of the anode. Candidate images of the fold edge are generated based on the pixel positions of the candidate edge chains in the local image near the waterline.

[0010] Preferably, the obtained flipped neighborhood image patch includes: Based on the position of the candidate pixels of the fold edge in the candidate image of the fold edge, determine the candidate edge chain of the fold edge; Determine the tangent and normal directions of each candidate pixel on the candidate edge chain of the fold edge; Based on the normal direction of the candidate pixels of the fold edge, the second neighboring image block located on the second side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; Based on the tangent direction of the candidate edge chain of the fold edge, the second neighboring image block is mirrored to obtain the folded neighboring image block.

[0011] Preferably, generating a mirror domain mask image includes: Based on the normal direction of the candidate pixels of the fold edge, the first neighboring image block located on the first side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; The flipping similarity is determined based on the gray values ​​of each pixel in the first neighboring image block and the gray values ​​of the corresponding pixels in the flipped neighboring image block. The mirror side is determined based on the fold similarity corresponding to the first side of the candidate edge chain and the fold similarity corresponding to the second side of the candidate edge chain. Starting from the mirror side, a region expansion is performed in the local image near the waterline to obtain the mirror region; A mirror domain mask image is generated based on the pixel position of the mirror region in the local image near the waterline.

[0012] Preferably, generating a corrected near-waterline local image includes: Based on the mirror domain mask image, determine the image region to be repaired in the near-waterline local image; The pixels in the local image near the waterline that correspond to the area to be repaired are marked as the pixels to be repaired. Based on the candidate images of the folded edge, determine the boundary guiding pixels of the image region to be repaired; Based on the location of the pixel to be repaired and the location of the boundary guiding pixels, extract non-mirror adjacent pixels that are adjacent to the region of the image to be repaired in the local image near the waterline; Based on the gray values ​​of the non-mirror adjacent pixels, the gray values ​​of the pixels to be repaired are filled to obtain the repaired local image of the waterline. Based on the repaired local image of the waterline, a corrected local image of the waterline is generated.

[0013] Preferably, the orthophoto image obtained from the mapping UAV includes: The corrected near-waterline local image is embedded into the original UAV image to obtain the preprocessed UAV image. Feature points are extracted from the preprocessed UAV images to obtain a set of feature points; Feature matching is performed on the set of feature points to obtain the set of image matching points; Based on the image matching point set, geometric correction is performed on the preprocessed UAV image to obtain the geometrically corrected UAV image; Based on the geometrically corrected UAV imagery, generate orthorectified imagery; Image mosaicking is performed on the orthorectified image to obtain the orthorectified result image of the surveying UAV.

[0014] To address the aforementioned problems, the present invention also provides an image correction system for a surveying UAV facing a water surface, the system comprising: The anchor point positioning module determines the target position relationship based on the original UAV imagery corresponding to the waterline area of ​​the bridge pier; and performs cropping processing on the original UAV imagery based on the candidate connected regions and the target position relationship to obtain a local image near the waterline. The edge chain filtering module performs edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; it determines the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain and generates the fold edge candidate image; The folding and expansion module performs mirror folding processing on the second neighboring image block corresponding to the folding edge candidate image based on the folding edge candidate edge chain to obtain the folded neighboring image block; based on the folded neighboring image block, the near-waterline local image is expanded to generate a mirrored domain mask image. The deconstruction and repair module extracts non-mirror neighboring pixels in the near-waterline local image based on the mirror domain mask image; and generates a corrected near-waterline local image based on the non-mirror neighboring pixels. The orthophoto output module obtains preprocessed UAV images based on the corrected near-waterline local images; it then performs geometric correction on the preprocessed UAV images to obtain orthophoto results images from the surveying UAV.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention constructs a local image of the near-waterline by analyzing the spatial adjacency relationship between the sacrificial anode block and the edge of the pier, and extracts the continuous edge chain associated with the anode block. It identifies the pseudo-boundary formed by the solid area and the mirror area in the complex water surface reflection environment, avoids such boundaries from being mistakenly used as real structural boundaries in the mapping process, and improves the accuracy of near-water area boundary identification.

[0016] 2. This invention utilizes the correspondence between the image content on both sides of the candidate edge chain of the folded edge to identify and locate the mirror region, and generates a mirror domain mask image. Based on this, the image region affected by mirror interference is repaired in a targeted manner, so that the mixed splicing relationship between the real region and the mirror region is eliminated, improving the integrity and continuity of the image content in the near-water area and enhancing the image realism.

[0017] 3. This invention embeds the corrected near-waterline local image into the original UAV image, and then performs geometric correction, orthorectification and image mosaicking processing, so that the image data involved in the surveying and mapping calculation is closer to the real scene, reduces the impact of false boundaries on image matching and spatial position calculation, and improves the positioning accuracy and quality of the orthorectified results of the surveying and mapping UAV. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a method for correcting images of a water-facing UAV, as provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] This embodiment provides a method for correcting images of a surveying UAV facing a water surface. See [link to relevant documentation]. Figure 1 Specifically, including: S1. Based on the original UAV imagery corresponding to the waterline area of ​​the bridge pier, determine the target position relationship; based on the candidate connected regions and the target position relationship, crop the original UAV imagery to obtain a local image near the waterline. In embodiments of the present invention, determining the target position relationship includes: Obtain the original UAV imagery corresponding to the waterline area of ​​the bridge pier; The waterline area of ​​a bridge pier refers to the transitional area formed by the long-term or periodic alternating contact between the outer surface of the bridge pier and the surrounding water. This area is located between the physical structure of the bridge pier and the water body and is affected by water level changes, wave erosion, water flow impact, and tidal fluctuations. It usually includes the submerged part of the outer wall of the bridge pier, the adjacent part exposed above the water surface, and the boundary zone between the two. The corresponding image content is the area where the texture of the bridge pier surface, the texture of the water surface, and the outline of the boundary between the two coexist.

[0021] Control the surveying drone to fly along the outer side of the bridge pier waterline area, so that the optical camera on the drone is aimed at the bridge pier waterline area to image it. Save the pixel array formed by the optical camera at the same shooting moment as the original drone image, and retain the image content of the bridge pier outer wall, the water surface boundary area, and the auxiliary components near the bridge pier waterline area in the original drone image.

[0022] The original UAV imagery is converted to grayscale to obtain a grayscale image. The original UAV imagery refers to two-dimensional image data obtained by taking pictures of the waterline area of ​​the bridge piers using optical imaging equipment carried by a surveying UAV. This image data records the spatial distribution and surface brightness information of the bridge piers, the water surface, and the auxiliary components near the bridge piers. Grayscale images refer to image data formed by converting the color information in the original UAV imagery into single brightness information, where each pixel uses a grayscale value to represent the brightness of the corresponding location. Extract the red, green, and blue channel values ​​of each pixel in the original UAV image. Convert the red, green, and blue channel values ​​into grayscale values ​​for that pixel according to the brightness weighting relationship. Arrange the grayscale values ​​corresponding to all pixels in the original UAV image according to the original pixel positions to obtain a grayscale image.

[0023] A smoothing filter is applied to the grayscale image to obtain a filtered grayscale image; Filtered grayscale images refer to image data formed after smoothing random noise and local brightness fluctuations in grayscale images, which preserves the main structural contours and reduces noise interference. Each pixel in the grayscale image is taken as the pixel to be processed. A filtering neighborhood is formed by selecting the pixels adjacent to the pixel to be processed. The average grayscale value is calculated based on the grayscale values ​​of each pixel in the filtering neighborhood. The average grayscale value is assigned to the corresponding position of the pixel to be processed in the filtered grayscale image. The same processing is performed on all pixels in the grayscale image to obtain the filtered grayscale image.

[0024] Edge extraction is performed on the filtered grayscale image to obtain the edge image; The grayscale value of each pixel in the filtered grayscale image is differentially calculated along the horizontal and vertical directions of adjacent pixels to obtain the horizontal and vertical grayscale changes corresponding to each pixel. The grayscale change intensity of each pixel is determined based on the horizontal and vertical grayscale changes. Pixels whose grayscale change intensity is a local maximum along the grayscale change direction are identified as edge pixels. An edge image is generated based on the pixel positions of all edge pixels in the filtered grayscale image. The positions in the edge image corresponding to the edge pixels are marked as edge positions, and the positions in the edge image other than edge positions are marked as non-edge positions.

[0025] Extract multiple candidate connected regions from the edge image; Edge image refers to image data extracted from locations with significant brightness changes in a filtered grayscale image, used to characterize the boundary distribution between different regions; candidate connected region refers to a continuous region in the edge image composed of multiple interconnected edge pixels, which corresponds to the contours of local entities or attached components in the original scene. Edge locations in the edge image are connected according to the eight-neighborhood connection relationship. Interconnected edge locations are divided into the same connected region, and edge locations not connected to the connected region are divided into other connected regions. For each connected region, the edge locations, circumscribed contour, circumscribed contour size and number of pixels are recorded. The recorded connected regions are used as candidate connected regions to obtain multiple candidate connected regions.

[0026] The main edge is extracted from the edge image to obtain the edge region of the bridge pier; For each candidate connected region in the edge image, extract all edge position coordinates that make up the candidate connected region. Determine the leftmost, rightmost, topmost, and bottommost boundaries of the candidate connected region based on all edge position coordinates. Determine the horizontal span based on the pixel distance between the leftmost and rightmost boundaries, and the vertical span based on the pixel distance between the topmost and bottommost boundaries. Count the occurrence frequency of each connecting direction between any two edge positions in the candidate connected region, and determine the connecting direction with the highest occurrence frequency as the main extension direction of the candidate connected region. Determine whether the candidate connected region extends along the pier height direction based on the correspondence between the vertical and horizontal spans. Sort the candidate connected regions according to the size of their vertical spans, and determine the candidate connected region extending along the pier height direction with the largest vertical span as the pier edge region. Record the set of edge position coordinates, boundary coordinates, and contour coordinates corresponding to the pier edge region.

[0027] Determine the target positional relationship between the candidate connected region and the pier edge region.

[0028] The pier edge region refers to the continuous boundary region in the edge image corresponding to the outer contour of the pier. This region reflects the spatial boundary between the pier and the surrounding environment. The target position relationship refers to the spatial positional correspondence between the candidate connected region and the pier edge region. This spatial positional correspondence is used to describe the distance distribution and adjacency distribution of the candidate connected region relative to the pier edge region.

[0029] For each edge coordinate in each candidate connected region, the distance between each edge coordinate in the pier edge region and the edge coordinate in the pier edge region is calculated point by point. The minimum distance among all the distances is determined as the shortest pixel spacing between the candidate connected region and the pier edge region. A connection vector is determined based on the center coordinates of the outer contours of the candidate connected region and the pier edge region. The direction of the connection vector in the image coordinate system determines whether the candidate connected region is located to the left, right, top, or bottom of the pier edge region. The adjacency state is determined based on whether there are directly contacting edge positions between the boundary coordinates of the candidate connected region and the boundary coordinates of the pier edge region. The overlap state is determined based on whether there are overlapping areas between the boundary coordinates of the candidate connected region and the boundary coordinates of the pier edge region. A spatial correspondence between the candidate connected region and the pier edge region is established based on the shortest pixel spacing, the orientation information corresponding to the connection vector, the adjacency state, and the overlap state. This spatial correspondence is then determined as the target position relationship.

[0030] In an embodiment of the present invention, obtaining a local image of the near-waterline includes: Based on the circumscribed contour size, contour shape, and target position relationship of the candidate connected regions, the anode candidate region is determined from multiple candidate connected regions; The circumscribed contour size refers to the spatial size information corresponding to the smallest closed contour used to enclose all edge positions of the candidate connected region. This spatial size information includes the span of the circumscribed contour in the horizontal direction and the span in the vertical direction of the image, which is used to characterize the area occupied by the candidate connected region in the image. The contour shape refers to the geometric shape information formed by the outer boundary of the candidate connected region. This geometric shape information is composed of the extension direction of the boundary of the candidate connected region, the boundary turning situation, and the boundary closure state, which is used to reflect the appearance shape of the entity contour corresponding to the candidate connected region. The anode candidate region refers to the candidate connected region among multiple candidate connected regions that meets the requirements of circumscribed contour size, contour shape, and target position relationship. This region corresponds to the projection area of ​​the sacrificial anode block near the waterline of the bridge pier in the image. It should be noted that sacrificial anode blocks refer to electrochemical protective components fixedly installed on the surface of bridge piers, pile foundations, wharves, or other metal structures. They are made of metal materials with electrode potential lower than that of the protected metal components. When they form an electrical connection with the protected metal components and are both in an aquatic environment, the sacrificial anode blocks preferentially undergo oxidation reactions and continuously release electrons. After the protected metal components receive electrons, their surface corrosion reactions are inhibited, thereby reducing the corrosion rate of the protected metal components. Sacrificial anode blocks are usually installed in the waterline area, tidal zone, or long-term immersion area of ​​the structure and are attached to the surface of the protected components in the form of blocks, plates, or strips.

[0031] For each candidate connected region, extract all pixel coordinates contained within that region. Based on these coordinates, determine the leftmost, rightmost, topmost, and bottommost pixel rows of the candidate connected region. Determine the horizontal span of the circumscribed contour based on the leftmost and rightmost pixel columns, and the vertical span based on the topmost and bottommost pixel rows. Determine whether the contour forms a closed boundary based on the connection order of the boundary pixels of the candidate connected region. Determine the contour inflection points based on the change in the extension direction of adjacent boundary pixels. Determine the contour shape based on the number and distribution of the contour inflection points. Finally, determine the shape based on the coordinates of the center pixel of the candidate connected region. The orientation of the candidate connected region relative to the pier edge region is determined by the coordinates of the center pixel of the pier edge region and the marker. The proximity relationship is determined by the minimum pixel distance between the boundary pixels of the candidate connected region and the boundary pixels of the pier edge region. The attachment relationship is determined by whether there are adjacent pixels between the boundary pixels of the candidate connected region and the boundary pixels of the pier edge region. The candidate connected region with both the lateral span and the longitudinal span of the outer contour being smaller than the corresponding span of the pier edge region, having a turning point at the contour boundary, and being located near the pier edge region and having an attachment relationship with the pier edge region is determined as the anode candidate region.

[0032] An anode anchor mask image is generated based on the pixel position of the anode candidate region in the original UAV image; the pixels in the anode anchor mask image corresponding to the anode candidate region are marked as anode pixels. An anode anchor mask image refers to a marker image generated based on the anode candidate region. This marker image is used to record the spatial distribution of the anode candidate region in the original UAV image. The position corresponding to the anode candidate region is given anode identification information, and the remaining positions are given background identification information. An anode pixel refers to the pixel in the anode anchor mask image that corresponds to the anode candidate region. This pixel corresponds to the image position within the projection range of the sacrificial anode block in the original UAV image. A blank marker image is created based on the number of pixel rows and columns of the original UAV image, so that each pixel position in the blank marker image corresponds to a pixel position in the original UAV image. Based on all pixel coordinates contained in the anode candidate region, the pixel position corresponding to all pixel coordinates is found in the blank marker image. The found pixel position is assigned an anode pixel marker, and the pixel position not found is assigned a background pixel marker. Boundary connectivity checks are performed on the marker positions corresponding to the boundary pixels of the anode candidate region. Isolated marker positions not connected to the main body of the anode candidate region are restored to background pixel markers. Unmarked pixel positions inside the main body of the anode candidate region that are surrounded by anode pixel markers are supplemented as anode pixel markers, thus obtaining the anode anchor point mask image.

[0033] Based on the bounding rectangle of the anode candidate region, the original UAV image is cropped to obtain a local image near the waterline.

[0034] The circumscribed rectangle refers to the smallest rectangular area that can completely enclose all the boundary positions of the anode candidate area. This rectangular area is used to determine the coverage of the anode candidate area in the original UAV image. The near-waterline local image refers to the local image data formed by cropping from the original UAV image based on the circumscribed rectangle. This local image data includes the anode candidate area, the pier edge area, and the waterline area adjacent to the pier edge area.

[0035] Based on the coordinates of all pixels contained in the anode candidate region, determine the leftmost pixel column, rightmost pixel column, topmost pixel row, and bottommost pixel row of the anode candidate region. Generate an outer rectangle that can enclose the anode candidate region based on the leftmost pixel column, rightmost pixel column, topmost pixel row, and bottommost pixel row. Based on the upper boundary pixel row, lower boundary pixel row, left boundary pixel column, and right boundary pixel column of the outer rectangle, determine the corresponding cropping range in the original UAV image. Copy all pixels within the cropping range to the new image data in the row and column order of the original UAV image. The new image data is determined as a near-waterline local image. The near-waterline local image retains the image content corresponding to the anode candidate region in the original UAV image as well as the image content of the bridge pier waterline adjacent to the anode candidate region.

[0036] S2. Perform edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; determine the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain, and generate the fold edge candidate image; In embodiments of the present invention, a plurality of anode adjacent edge chains are obtained, including: Edge extraction is performed on the local image near the waterline to obtain the edge image of the waterline; The near-waterline local image is converted to grayscale to obtain a near-waterline grayscale image. The near-waterline grayscale image is then smoothed and filtered to obtain a near-waterline filtered grayscale image. The horizontal and vertical grayscale changes of each pixel in the near-waterline filtered grayscale image are calculated. The grayscale change intensity and direction of each pixel are determined based on the horizontal and vertical grayscale changes. The grayscale change intensity of the pixel is compared with that of its adjacent pixels in the same direction based on the grayscale change direction. Pixels with a grayscale change intensity greater than that of their adjacent pixels in the same direction are identified as near-waterline edge pixels. The near-waterline edge image is then generated based on the pixel positions of the near-waterline edge pixels in the near-waterline local image.

[0037] Boundary extraction is performed on the anode anchor point mask image to obtain the anode boundary image; The near-waterline edge image refers to the image data formed after edge extraction of a local near-waterline image. This image data is used to represent the location distribution of obvious grayscale changes in the local near-waterline image. The anode boundary image refers to the image data formed after extracting the outer boundary of the anode pixels in the anode anchor point mask image. This image data is used to represent the outer contour position of the anode candidate region. For each anode pixel in the anode anchor point mask image, a neighborhood check is performed. Anode pixels with background pixels in their neighborhood are identified as anode boundary pixels, and anode pixels without background pixels in their neighborhood are identified as anode interior pixels. An anode boundary image is generated based on the pixel positions of all anode boundary pixels in the anode anchor point mask image. The positions in the anode boundary image corresponding to the anode boundary pixels are marked as anode boundary positions, and the positions in the anode boundary image other than the anode boundary positions are marked as non-anode boundary positions.

[0038] Based on the position of the anode boundary pixels in the anode boundary image, the edge pixels in the near-waterline edge image are filtered by adjacency to obtain the initial edge pixel set; Using the row and column numbers of each anode boundary pixel in the anode boundary image as a reference position, find edge pixels in the near-waterline edge image that have the same row and column numbers as the reference position, as well as edge pixels adjacent to the reference position. Determine the found edge pixels as adjacent edge pixels. Merge all the adjacent edge pixels corresponding to the anode boundary pixels, delete duplicate adjacent edge pixels, and obtain the initial edge pixel set.

[0039] Starting from the edge pixels in the initial edge pixel set, edge connectivity tracking is performed in the near-waterline edge image to obtain multiple anode adjacent edge chains.

[0040] Anode boundary pixels refer to pixels located on the outer contour of the anode candidate region in the anode boundary image, which correspond to the boundary position of the sacrificial anode block in the image; the starting edge pixel set refers to the set of edge pixels adjacent to the anode boundary pixels in the near-waterline edge image, which is used to determine the position for edge connectivity tracking starting from the vicinity of the anode candidate region; the anode adjacent edge chain refers to the sequence of edge pixels obtained by continuously tracking interconnected edge pixels in the near-waterline edge image, starting from the edge pixels in the starting edge pixel set.

[0041] Each edge pixel in the initial edge pixel set is used as a tracking starting point. Starting from each tracking starting point, edge pixels adjacent to the current edge pixel in the near-waterline edge image that are not recorded by the current edge chain are searched. The found edge pixels are added to the current edge chain. The newly added edge pixels are used as the new current edge pixels to continue searching for adjacent edge pixels. The tracking of the current edge chain is stopped when there are no adjacent edge pixels that can be connected. The edge pixel sequence continuously recorded by the same tracking starting point is determined as the anode adjacent edge chain. The same edge connectivity tracking process is performed on all tracking starting points. Edge chains containing the same edge pixels are merged to obtain multiple anode adjacent edge chains.

[0042] In an embodiment of the present invention, generating a candidate image for the folded edge includes: Determine the chain length and direction variation of the chain adjacent to the anode edge; Chain length refers to the cumulative length information formed along the edge connection path between the first and last edge pixels in the anode adjacent edge chain. This length information reflects the extension range of the anode adjacent edge chain in the near-waterline edge image. Directional change refers to the cumulative degree of directional change between adjacent edge pixels along the extension path of the anode adjacent edge chain. This directional change reflects the deflection and bending of the anode adjacent edge chain during spatial extension. Extract all edge pixel coordinates from the anode adjacent edge chain arranged in the edge connectivity tracking order. The edge pixel coordinate that appears first is taken as the previous coordinate, and the edge pixel coordinate adjacent to the previous coordinate and appearing later is taken as the next coordinate. Subtract the row number of the previous coordinate from the row number of the next coordinate to obtain the row coordinate difference, and subtract the column number of the previous coordinate from the column number of the next coordinate to obtain the column coordinate difference. Determine the pixel connection distance between the previous and next coordinates based on the row and column coordinate differences. Add the pixel connection distances between all adjacent edge pixels in the same anode adjacent edge chain to obtain the chain length of the anode adjacent edge chain. Determine the connection direction between the previous and next coordinates based on the row and column coordinate differences. All connection directions obtained in the order of arrangement form a connection direction sequence. Two adjacent connection directions in the connection direction sequence are compared. When the previous connection direction is the same as the next connection direction, the local direction change corresponding to the two connection directions is recorded as no direction change. When the previous connection direction is different from the next connection direction, the angle change between the two connection directions in the image coordinate plane is determined according to the row coordinate difference and column coordinate difference corresponding to the previous connection direction and the row coordinate difference and column coordinate difference corresponding to the next connection direction. The angle change is taken as the local direction change corresponding to the two connection directions. All local direction changes in the adjacent edge chain of the same anode are added together to obtain the direction change of the adjacent edge chain of the anode.

[0043] It should be noted that the anode adjacent edge chain is essentially a discrete path formed by continuous edge pixels according to spatial connection relationships. The pixel connection distance between adjacent edge pixels reflects the actual extension length of the path in the image plane. Therefore, the sum of the pixel connection distances between all adjacent edge pixels can characterize the overall extension degree of the anode adjacent edge chain, thus obtaining the chain length. The connection direction reflects the extension direction of the edge path in the image plane, while the change in the angle between adjacent connection directions reflects the turning degree of the edge path. The sum of the angle changes between all adjacent connection directions can characterize the overall deflection and bending of the anode adjacent edge chain, thus obtaining the direction change amount.

[0044] Calculate the adjacency length between the anode adjacent edge chain and the anode boundary image; The adjacency length refers to the continuous length information that maintains the adjacent state between the anode adjacent edge chain and the anode boundary image. This length information reflects the degree of closeness between the anode adjacent edge chain and the anode boundary. For each edge pixel coordinate in each anode adjacent edge chain, a neighborhood search is performed. The anode boundary pixel adjacent to the edge pixel coordinate is found in the anode boundary image. Edge pixels that can be found to be adjacent to anode boundary pixels are identified as adjacent pixels. Adjacent pixels that are continuously distributed in the edge connection order in the same anode adjacent edge chain are formed into adjacent pixel segments. The segment length of the adjacent pixel segment is determined according to the pixel connection distance between adjacent adjacent pixels in each adjacent pixel segment. The segment lengths of all adjacent pixel segments corresponding to the same anode adjacent edge chain are added together to obtain the adjacency length between the anode adjacent edge chain and the anode boundary image.

[0045] It should be noted that the adjacency length is used to characterize the degree of continuity in the spatial proximity between the anode adjacent edge chain and the anode boundary. When the edge pixels in the anode adjacent edge chain are continuously adjacent to the anode boundary pixels, it indicates that the edge chain extends along the anode boundary over a long range. Therefore, by accumulating the connection distances between all consecutive adjacent pixels, the actual attachment range and the degree of extension between the anode adjacent edge chain and the anode boundary can be reflected, thus obtaining the adjacency length between the anode adjacent edge chain and the anode boundary image.

[0046] Based on the chain length, direction change, and adjacency length of the anode adjacent edge chains, multiple anode adjacent edge chains are sorted to obtain the edge chain sorting result; The edge chain sorting result refers to the arrangement result formed after comparing multiple anode adjacent edge chains based on chain length, direction change amount and adjacency length. This arrangement result is used to represent the degree of association between multiple anode adjacent edge chains and the anode boundary. For each anode adjacent edge chain, obtain its chain length, direction change, and adjacency length. Sort multiple anode adjacent edge chains in descending order of adjacency length. For anode adjacent edge chains with the same adjacency length, sort them in descending order of chain length. For anode adjacent edge chains with the same adjacency length and chain length, sort them in ascending order of direction change. The edge chain sorting result is obtained.

[0047] Based on the edge chain sorting results, candidate edge chains for the fold edge are determined from multiple adjacent edge chains of the anode. The candidate edge chain of the fold edge refers to the edge chain selected from multiple anode adjacent edge chains based on the edge chain sorting results. This edge chain corresponds to the edge region in the near waterline local image that maintains a continuous adjacent relationship with the anode boundary and has a stable extension state. The anode adjacent edge chain ranked first in the edge chain sorting results is determined as the fold edge candidate edge chain. The pixel positions of all edge pixels in the fold edge candidate edge chain are extracted in the near-waterline local image. A blank marker image is established based on the number of pixel rows and columns in the near-waterline local image. The pixels in the blank marker image corresponding to all edge pixel positions of the fold edge candidate edge chain are marked as fold edge candidate pixels. The pixels in the blank marker image other than the fold edge candidate pixels are marked as background pixels, thus obtaining the fold edge candidate image.

[0048] Candidate images of the fold edge are generated based on the pixel positions of the candidate edge chains in the local image near the waterline.

[0049] A fold edge candidate image refers to a labeled image generated based on the pixel positions of the fold edge candidate edge chains in the near-waterline local image. This labeled image is used to record the spatial distribution of the fold edge candidate edge chains in the near-waterline local image.

[0050] A blank marker image is created based on the number of pixel rows and columns in the near-waterline local image, so that each pixel position in the blank marker image corresponds to a pixel position in the near-waterline local image. The row and column numbers of all edge pixels in the candidate edge chain of the fold edge are extracted. Pixel positions in the blank marker image that have the same row and column numbers as all edge pixels are found, and these pixel positions are marked as candidate pixels for the fold edge. The remaining pixel positions in the blank marker image that are not marked as candidate pixels for the fold edge are marked as background pixels. The blank marker image after marking is determined as the candidate image for the fold edge.

[0051] S3. Based on the candidate edge chain of the candidate edge of the folded edge, the second neighboring image block corresponding to the candidate edge of the folded edge is mirrored to obtain the flipped neighboring image block; based on the flipped neighboring image block, the near waterline local image is expanded to generate a mirrored domain mask image. In an embodiment of the present invention, obtaining a flipped neighborhood image patch includes: Based on the position of the candidate pixels of the fold edge in the candidate image of the fold edge, determine the candidate edge chain of the fold edge; The candidate pixel of the fold edge refers to the pixel in the candidate image of the fold edge used to characterize the spatial position of the candidate edge chain of the fold edge. This pixel corresponds to the position traversed by the candidate edge chain of the fold edge in the local image near the waterline. The candidate edge chain of the fold edge refers to the continuous edge path formed by connecting multiple interconnected candidate pixels of the fold edge in the order of edge extension. This continuous edge path corresponds to the edge region in the local image near the waterline that maintains a continuous adjacency relationship with the anode boundary. It is used to characterize the continuous edge distribution state formed by the boundary of the solid structure and the mirror boundary of the water surface near the anode candidate region.

[0052] Extract the row and column numbers of all candidate pixels for the folded edge in the candidate image. Arrange all candidate pixels for the folded edge in a connected manner according to the adjacency relationship between them. Divide the interconnected candidate pixels for the folded edge into the same edge pixel sequence. Arrange the candidate pixels for the folded edge in the same edge pixel sequence according to the edge extension order to obtain the candidate edge chain for the folded edge. Record the row and column numbers of each candidate pixel for the folded edge in the candidate edge chain for the folded edge.

[0053] Determine the tangent and normal directions of each candidate pixel on the candidate edge chain of the fold edge; The tangent direction refers to the direction in which the candidate edge chain of the fold edge extends along the edge at a certain candidate pixel position of the fold edge. This direction is used to represent the local extension state of the candidate edge chain of the fold edge in the image plane; the normal direction refers to the direction perpendicular to the tangent direction. This direction is used to represent the distribution direction of the regions on both sides of the candidate edge chain of the fold edge. The preceding and following adjacent candidate pixels of the current candidate pixel in the candidate edge chain are used as the direction calculation pixels. The row direction change is determined by the difference between the row number of the following adjacent candidate pixel and the row number of the preceding adjacent candidate pixel. The column direction change is determined by the difference between the column number of the following adjacent candidate pixel and the column number of the preceding adjacent candidate pixel. The direction jointly represented by the row direction change and the column direction change is determined as the tangent direction of the current candidate pixel. The direction perpendicular to the tangent direction is determined as the normal direction of the current candidate pixel.

[0054] Based on the normal direction of the candidate pixels of the fold edge, the second neighboring image block located on the second side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; The first neighboring image block refers to a local image region located on the first side of the candidate edge chain of the fold edge, based on the candidate pixels of the fold edge and along the normal direction. This local image region contains pixel information of the first side of the candidate edge chain of the fold edge. The second neighboring image block refers to a local image region located on the second side of the candidate edge chain of the fold edge, based on the candidate pixels of the fold edge and along the normal direction. This local image region contains pixel information of the second side of the candidate edge chain of the fold edge. Using the row and column numbers of the current folded edge candidate pixels as the truncation reference positions, the extension direction of the folded edge candidate edge chain at that position is determined according to the tangent direction corresponding to the current folded edge candidate pixel. The distribution direction of the second side relative to the folded edge candidate edge chain is determined according to the normal direction corresponding to the current folded edge candidate pixel. In the near-waterline local image, consecutive folded edge candidate pixels that pass through the current folded edge candidate pixel and belong to the same folded edge candidate edge chain are selected along the tangent direction. The positions corresponding to the selected consecutive folded edge candidate pixels are used as the boundary reference positions of the second neighboring image block. For each boundary reference position, consecutive image pixels adjacent to the boundary reference position are selected along the normal direction of the second side. The boundary reference positions arranged along the tangent direction and the image pixels selected along the normal direction of the second side are combined to form the second neighboring image block. The original row number, original column number, normal offset position relative to the corresponding boundary reference position, and tangential arrangement position relative to the current folded edge candidate pixel are recorded for each image pixel in the second neighboring image block. The gray value of each image pixel in the near-waterline local image is retained.

[0055] Based on the tangent direction of the candidate edge chain of the fold edge, the second neighboring image block is mirrored to obtain the folded neighboring image block.

[0056] The folded neighbor image block refers to the image region formed by mirroring the second neighbor image block with reference to the tangent direction corresponding to the fold edge candidate edge chain. This image region is used to reflect the symmetrical correspondence between the second neighbor image block and the fold edge candidate edge chain.

[0057] Using the tangent direction of the candidate edge chain at the current candidate pixel of the fold edge as the fold reference direction, and the boundary reference position sequence corresponding to the second neighboring image block as the fold reference line, the tangential arrangement position of each image pixel in the second neighboring image block relative to the fold reference line is kept unchanged. The normal offset position of the image pixel relative to the fold reference line is transformed into the normal offset position in the opposite direction. The row and column numbers of the image pixel after folding are determined according to the maintained tangential arrangement position and the transformed normal offset position. The gray value of the image pixel is written to the position corresponding to the row and column numbers after folding. If multiple image pixels are written to the same position after folding, the multiple gray values ​​corresponding to the position are averaged and written to the position. For image pixels whose position after folding is outside the local image boundary near the waterline, they are not written to the folded area image block. After completing the position transformation and gray value writing of all image pixels, the folded area image block is obtained.

[0058] In an embodiment of the present invention, generating a mirror domain mask image includes: Based on the normal direction of the candidate pixels of the fold edge, the first neighboring image block located on the first side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; The first side refers to the image region on one side, which is divided along the normal direction with the candidate edge chain of the fold edge as the boundary. This image region and the second side image region are located on opposite sides of the candidate edge chain of the fold edge. The first region image block refers to the local image data located on the first side of the candidate edge chain of the fold edge in the local image near the waterline. This local image data is composed of pixels adjacent to the candidate pixels of the fold edge within the range of the first side. Using the pixel position of the current fold edge candidate pixel as the reference position for region extraction, the extension direction of the fold edge candidate edge chain at this position is determined according to the tangent direction corresponding to the current fold edge candidate pixel. The distribution direction of the first side relative to the fold edge candidate edge chain is determined according to the normal direction corresponding to the current fold edge candidate pixel. In the near-waterline local image, consecutive fold edge candidate pixels that pass through the current fold edge candidate pixel and belong to the same fold edge candidate edge chain are selected along the tangent direction. The position corresponding to the selected consecutive fold edge candidate pixels is used as the boundary reference position of the first region image block. For each boundary reference position, consecutive image pixels adjacent to the boundary reference position are selected along the normal direction of the first side. The boundary reference positions arranged along the tangent direction and the image pixels selected along the normal direction of the first side are combined to form the first region image block. The original row number, original column number, normal offset position relative to the corresponding boundary reference position, and tangential arrangement position relative to the current fold edge candidate pixel are recorded for each image pixel in the first region image block. The gray value of each image pixel in the near-waterline local image is also retained.

[0059] The flipping similarity is determined based on the gray values ​​of each pixel in the first neighboring image block and the gray values ​​of the corresponding pixels in the flipped neighboring image block. Folding similarity refers to the degree of similarity between the first region image block and the folded region image block in the corresponding pixel grayscale distribution. This degree of similarity is used to represent the corresponding state of the image content on both sides of the candidate edge chain of the folded edge after mirror folding. Based on the tangential arrangement position and normal offset position of each image pixel in the first region image block, find the corresponding image pixel with the same tangential arrangement position and the same normal offset position in the flipped neighbor image block, calculate the gray value difference between the gray value of the image pixel in the first region image block and the gray value of the corresponding image pixel in the flipped neighbor image block, accumulate all the gray values ​​of the corresponding image pixels in the first region image block and divide by the number of pixels involved in the calculation to obtain the average gray value difference, and determine the flipping similarity based on the average gray value difference; The gray values ​​of all image pixels involved in the calculation in the first region image block are summed and divided by the number of pixels involved in the calculation to obtain the first average gray value. The gray values ​​of all corresponding image pixels involved in the calculation in the flipped neighbor image block are summed and divided by the number of pixels involved in the calculation to obtain the second average gray value. The larger value between the first average gray value and the second average gray value is used as the gray-level normalization reference value. The average gray-level difference is divided by the gray-level normalization reference value to obtain the normalized gray-level difference. Then, the flipped similarity is calculated, where flipped similarity = 1 - normalized gray-level difference.

[0060] It should be noted that the folding similarity is used to characterize the degree of correspondence between the first region image block and the folded neighboring image blocks. Since mirror reflection satisfies the principle of optical path reversibility, when the image content on one side of the candidate edge chain of the fold edge originates from the mirror mapping of the image content on the other side, the gray-level distributions of the corresponding positions on both sides have a high degree of correspondence. Therefore, the gray-level difference is used to characterize the degree of difference between the two, and the influence of the different overall brightness levels is eliminated by normalization processing. When the normalized gray-level difference decreases, it indicates that the brightness distribution of the corresponding pixels on both sides is closer, and their mirror correspondence is stronger, thus obtaining a higher folding similarity. When the normalized gray-level difference increases, it indicates that the brightness distribution of the corresponding pixels on both sides deviates more, and their mirror correspondence is weakened, thus obtaining a lower folding similarity.

[0061] The mirror side is determined based on the fold similarity corresponding to the first side of the candidate edge chain and the fold similarity corresponding to the second side of the candidate edge chain. The mirror side refers to the side of the image regions on both sides of the candidate edge chain of the fold that corresponds to the image region on the other side after mirror folding. This side of the image region corresponds to the mirror content formed by the reflection of the water surface. The folding similarity of each candidate pixel on the first side of the folding edge candidate edge chain is summed and divided by the number of candidate pixels involved in the calculation to obtain the average folding similarity of the first side. The folding similarity of each candidate pixel on the second side of the folding edge candidate edge chain is summed and divided by the number of candidate pixels involved in the calculation to obtain the average folding similarity of the second side. The average folding similarity of the first side and the average folding similarity of the second side are compared, and the side with the higher average folding similarity value is determined as the mirror side.

[0062] Starting from the mirror side, a region expansion is performed in the local image near the waterline to obtain the mirror region; The mirror region refers to the image region obtained by extending along the continuous gray-level region in the local image near the waterline, starting from the mirror side. This region corresponds to the pixel distribution range in the local image near the waterline that is affected by water surface reflection. The pixel position adjacent to the mirror side in the candidate edge chain of the fold edge is used as the expansion start position. In the local image near the waterline, an image pixel adjacent to the expansion start position but not recorded is searched. Based on the difference between the gray value of the image pixel and the gray value of the corresponding pixel at the expansion start position, the connectivity between the image pixel and the candidate edge chain of the fold edge, and whether the image pixel crosses the candidate edge chain of the fold edge, it is determined whether the image pixel should be added to the mirror region. The image pixel added to the mirror region continues to be used as a new expansion position, and the search for adjacent image pixels is repeated until there are no adjacent image pixels that can be added to the mirror region, thus obtaining the mirror region.

[0063] A mirror domain mask image is generated based on the pixel position of the mirror region in the local image near the waterline.

[0064] A mirror domain mask image is a labeled image generated based on the pixel positions of the mirror region in the local image near the waterline. In this labeled image, the pixels corresponding to the mirror region are labeled as mirror pixels, and the remaining pixels are labeled as background pixels.

[0065] A blank marker image is created based on the number of pixel rows and columns in the local image near the waterline. Each pixel position in the blank marker image corresponds to a pixel position in the local image near the waterline. The row and column numbers of all image pixels in the mirror region are extracted. Pixel positions in the blank marker image that have the same row and column numbers as all image pixels are found and marked as mirror pixels. The remaining pixel positions in the blank marker image that are not marked as mirror pixels are marked as background pixels. The blank marker image after marking is determined as the mirror domain mask image.

[0066] S4. Based on the mirror domain mask image, extract non-mirror adjacent pixels in the near-waterline local image; generate a corrected near-waterline local image based on the non-mirror adjacent pixels. In an embodiment of the present invention, generating a corrected near-waterline local image includes: Based on the mirror domain mask image, determine the image region to be repaired in the near-waterline local image; The process iterates through all pixels in the mirror mask image row by row and column by column, determining the labeling state of each pixel. Pixels labeled as mirror pixels are extracted, and their corresponding row and column numbers are recorded. Based on these recorded row and column numbers, the corresponding image pixels are located in the near-waterline local image. All corresponding image pixels are aggregated according to their original spatial distribution to form the image region to be repaired. A connectivity check is performed on all image pixels in the image region to be repaired. Image pixels with pixel connections are grouped into the same image region to be repaired, while those without pixel connections are grouped into different image regions to be repaired. Boundary pixels of all image regions to be repaired are extracted, and the row and column numbers of each boundary pixel are recorded. A set of pixel positions for the image region to be repaired is then established.

[0067] The pixels in the local image near the waterline that correspond to the area to be repaired are marked as the pixels to be repaired. The image region to be repaired refers to the region in the near-waterline local image that corresponds to the mirror region and requires image content reconstruction. This region corresponds to the image portion that has been removed due to mirror reflection. The pixel to be repaired refers to the pixel located inside the image region to be repaired and whose grayscale value needs to be redefined. This pixel corresponds to the position in the near-waterline local image where no valid image content has been retained. Based on the set of pixel locations in the image region to be repaired, image pixels with the same row number and column number are searched one by one in the near-waterline local image. The status of the found image pixels is changed to the state to be repaired, and the original gray value, row number and column number of the image pixel are stored together. For image pixels in the near-waterline local image that are not included in the set of pixel locations, their status is changed to the state to be retained. Boundary contact checks are performed on all image pixels in the state to be repaired. Image pixels in the state to be repaired that are adjacent to image pixels in the state to be retained are recorded as image pixels in the state to be repaired as boundary pixels in the state to be repaired, and image pixels in the state to be repaired that are not adjacent to image pixels in the state to be retained are recorded as internal pixels in the state to be repaired. A distribution map of the pixels to be repaired is then established.

[0068] Based on the candidate images of the folded edge, determine the boundary guiding pixels of the image region to be repaired; Boundary guide pixels refer to pixels located on the candidate edge chain of the fold edge, which are used to characterize the boundary position between the area to be repaired and the non-mirror area; Extract all candidate pixels of the folded edge in the candidate image of the folded edge, and record the row number and column number of each candidate pixel. Based on the recorded row number and column number, locate the corresponding image pixel in the near waterline local image, and use the located image pixel as a candidate guiding pixel. Calculate the spatial adjacency relationship between each candidate guiding pixel and the boundary pixel to be repaired. When a candidate guiding pixel has a direct adjacency relationship with at least one boundary pixel to be repaired, the candidate guiding pixel is determined as a boundary guiding pixel. When there is no direct adjacency relationship between a candidate guiding pixel and the boundary pixel to be repaired, continue to search for adjacent candidate pixels of the folded edge along the extension direction of the candidate edge chain until a candidate pixel of the folded edge with an adjacency relationship with the boundary pixel to be repaired is determined, and it is determined as a boundary guiding pixel. Record the row number, column number, and corresponding edge chain position order of all boundary guiding pixels to form a set of boundary guiding pixels.

[0069] Based on the location of the pixel to be repaired and the location of the boundary guiding pixels, extract non-mirror adjacent pixels that are adjacent to the region of the image to be repaired in the local image near the waterline; Extract the row and column numbers of all pixels to be repaired, and extract the row and column numbers of all boundary guide pixels. Construct a boundary guide path based on the arrangement order of the boundary guide pixels in the fold edge candidate edge chain. Divide the near-waterline local image into the side to be repaired and the non-mirror side based on the boundary guide path. For each pixel to be repaired, find the surrounding image pixels that have a direct pixel connection relationship with it. Image pixels located on the non-mirror side that are not marked as pixels to be repaired are identified as candidate non-mirror adjacent pixels. For each candidate non-mirror adjacent pixel, further check its positional relationship with the boundary guide path. Candidate non-mirror adjacent pixels that do not have a fold edge candidate edge chain blocking relationship with the pixel to be repaired are identified as non-mirror adjacent pixels.

[0070] Based on the gray values ​​of the non-mirror adjacent pixels, the gray values ​​of the pixels to be repaired are filled to obtain the repaired local image of the waterline. Non-mirror adjacent pixels refer to image pixels located outside and adjacent to the image area to be repaired. These image pixels correspond to the image content in the near-waterline local image that is not affected by mirror reflection. The repaired near-waterline local image refers to the near-waterline local image formed after the image area to be repaired is filled with gray values. This image retains the original image content of the non-mirror area and completes the image content of the image area to be repaired. For each pixel to be repaired, extract all its corresponding non-mirror neighboring pixels. Calculate the pixel distance between each non-mirror neighboring pixel and the pixel to be repaired. Establish a correspondence based on the pixel distance. Weight the gray values ​​of all non-mirror neighboring pixels, with the closer the non-mirror neighboring pixel is to the pixel to be repaired, the greater its participation degree. Divide the weighted sum by the total weighted participation degree of all pixels to obtain the filling gray value corresponding to the pixel to be repaired. Write the filling gray value to the location of the pixel to be repaired. Remove the repair mark from the pixels to be repaired after filling and update them as valid pixels that can participate in filling. Repeat the same process for the remaining pixels to be repaired. Continue to include the newly generated valid pixels into the set of non-mirror neighboring pixels. The filling process gradually advances from the edge of the image region to be repaired towards the interior of the image region to be repaired until all pixels to be repaired in the image region to be repaired obtain the corresponding filling gray value, thus obtaining the repaired near-waterline local image.

[0071] Based on the repaired local image of the waterline, a corrected local image of the waterline is generated.

[0072] The corrected near-waterline local image refers to the image data obtained after eliminating the influence of the mirror region on the repaired near-waterline local image. This image data is used to characterize the true image content of the bridge pier waterline area and its adjacent areas after removing mirror interference.

[0073] Extract all filled pixels and adjacent valid pixels from the repaired near-waterline local image. Calculate the grayscale change between each filled pixel and its surrounding valid pixels, as well as the grayscale change along the boundary guide path and perpendicular to the boundary guide path. Based on the calculation results, check the grayscale transition between the filled area and the non-mirror area. Filled pixels with significant grayscale jumps relative to their surrounding valid pixels are re-selected as pixels to be adjusted. Re-extract the grayscale information involved in filling from the surrounding valid pixels and recalculate the corresponding grayscale values. Write the recalculated grayscale values ​​to the positions of the pixels to be adjusted. After continuous adjustment of all filled pixels, retain the original image content outside the repaired area and merge the adjusted filled area with the original retained area to generate the corrected near-waterline local image.

[0074] S5. Based on the corrected near-waterline local image, obtain the preprocessed UAV image; perform geometric correction on the preprocessed UAV image to obtain the orthophoto result image of the surveying UAV.

[0075] In an embodiment of the present invention, the orthophoto image of the mapping UAV is obtained, including: The corrected near-waterline local image is embedded into the original UAV image to obtain the preprocessed UAV image. Based on the cropping range recorded when generating the near-waterline local image, the starting pixel row, ending pixel row, starting pixel column, and ending pixel column of the near-waterline local image in the original UAV image are determined. Based on the row and column numbers of each pixel in the corrected near-waterline local image, the corresponding row and column numbers of that pixel in the original UAV image are determined. The grayscale values ​​of the pixels at the corresponding row and column numbers in the original UAV image are replaced with the grayscale values ​​of the corresponding pixels in the corrected near-waterline local image. The pixels of the original UAV image outside the cropping range are left unchanged, resulting in the preprocessed UAV image.

[0076] Feature points are extracted from the preprocessed UAV images to obtain a set of feature points; Preprocessed UAV imagery refers to UAV imagery formed by replacing the corresponding area in the original UAV imagery with a corrected local image of the waterline. The areas in the original UAV imagery affected by mirror interference have been corrected. Feature point set refers to the set of image points with clear spatial location and reproducible identification extracted from the preprocessed UAV imagery. This set of image points corresponds to the edges of ground features, texture intersections, or locations with obvious grayscale changes. The preprocessed UAV image is converted to grayscale to obtain a preprocessed grayscale image. The horizontal and vertical grayscale changes of each pixel in the preprocessed grayscale image are calculated. The grayscale change distribution in the neighborhood of the pixel is determined based on the horizontal and vertical grayscale changes. Based on the grayscale change distribution, image points that can be repeatedly located in adjacent images are determined. The determined image points are recorded as feature points. The pixel row number, pixel column number, and neighborhood grayscale description information of each feature point in the preprocessed UAV image are recorded to obtain a set of feature points.

[0077] Feature matching is performed on the set of feature points to obtain the set of image matching points; Feature point sets are extracted from multiple preprocessed UAV images that participate in the processing of the same survey area. For any two preprocessed UAV images with overlapping areas, a first feature point list and a second feature point list are established. A feature point is selected from the first feature point list, and the corresponding pixel row number, pixel column number, and neighborhood grayscale description information are extracted. The neighborhood grayscale description information is compared item by item with the neighborhood grayscale description information corresponding to each feature point in the second feature point list, and the difference between the grayscale description information is calculated to obtain multiple difference results. The second feature point with the smallest difference is determined from the multiple difference results, and this second feature point is used as the first candidate matching point. Then, based on the first candidate matching point, a reverse comparison is performed with all feature points in the first feature point list to determine the first feature point with the smallest difference. When the first feature point obtained by the reverse comparison is the same as the currently selected first feature point, the two feature points are determined as a group of image matching points. The same process is repeated for all first feature points to delete duplicate matching points and matching points with overlapping positions. The pixel row number and pixel column number corresponding to each group of image matching points are recorded to obtain the image matching point set.

[0078] Based on the image matching point set, geometric correction is performed on the preprocessed UAV image to obtain the geometrically corrected UAV image; The image matching point set refers to the point set established by the positional correspondence between feature point sets. This point set is used to represent image points corresponding to the same ground position in different UAV images. The geometrically corrected UAV image refers to the UAV image after spatial position adjustment based on the image matching point set. The positional deviation caused by shooting posture, lens distortion or imaging angle in the UAV image is corrected. Extract the pixel row and column numbers corresponding to all image matching points in the image matching point set. Establish positional correspondences based on the pixel positions of the same set of image matching points in different preprocessed UAV images. Calculate the spatial transformation relationship between each preprocessed UAV image based on all positional correspondences. Calculate the target position of each pixel in a unified coordinate plane based on the spatial transformation relationship. Perform position mapping on all pixels in the preprocessed UAV image one by one, converting the original row and column numbers of each pixel to the row and column numbers corresponding to the target position. For cases where multiple pixels are mapped to adjacent target positions, determine the grayscale value corresponding to the target position according to the distance relationship between the target position and the original position. For positions with gaps after mapping, perform grayscale completion based on the grayscale values ​​of surrounding mapped pixels. The image after position mapping and grayscale completion is determined as the geometrically corrected UAV image.

[0079] Based on the geometrically corrected UAV imagery, generate orthorectified imagery; The row and column numbers of each pixel in the geometrically corrected UAV image are determined. Based on the camera focal length, principal point coordinates, and pixel size, the pixel row and column numbers are converted into image point coordinates in the camera coordinate system. Based on the position of the camera center in the survey area spatial coordinate system and the camera attitude angle during UAV shooting, the image point coordinates are converted into projection rays in the survey area spatial coordinate system. Based on the ground elevation data of the survey area, the intersection point of the projection ray and the ground elevation surface is determined. The planar coordinates of the intersection point are determined as the ground plane position corresponding to the pixel. The gray value of the pixel is written into the orthorectified image pixel corresponding to the ground plane position. For the case where multiple pixels correspond to the same orthorectified image pixel, the corresponding gray value is calculated based on the distance between the projection intersection point of each pixel and the center of the orthorectified image pixel. For orthorectified image pixels without written pixels, gray level completion is performed based on the adjacent orthorectified image pixels around the orthorectified image pixel that have already written gray values, thus obtaining the orthorectified image.

[0080] Image mosaicking is performed on the orthorectified image to obtain the orthorectified result image of the surveying UAV.

[0081] Orthorectified imagery refers to the image formed by transforming geometrically corrected UAV images according to the horizontal projection relationship of the ground, and establishing a correspondence between the image positions in the image and the actual ground positions; image mosaicking refers to the process of stitching and fusing multiple orthorectified images with overlapping areas according to their spatial positional relationship; orthorectified imagery of surveying UAVs refers to the final surveying image after image mosaicking is completed. This image can continuously reflect the planar distribution of ground features in the survey area and is used for geographic information mapping, topographic surveying, and spatial position analysis.

[0082] Based on the ground plane coordinate range corresponding to each orthorectified image, determine the writing position of each orthorectified image in the same result coordinate plane. Establish a blank result image covering the ground plane coordinate range of all orthorectified images. For each ground plane location, find all orthorectified images covering that location. When a ground plane location is covered by only one orthorectified image, write the pixel grayscale value of that orthorectified image at that location to the corresponding position in the blank result image. When a ground plane location is covered by multiple orthorectified images, extract the values ​​from those multiple orthorectified images. The grayscale value of the pixel at the ground plane position is calculated. The distance from the ground plane position to the effective image boundary of each orthorectified image is calculated. All distances are added together to obtain the total distance. The distance corresponding to each orthorectified image is divided by the total distance to obtain the fusion weight corresponding to the orthorectified image. The grayscale value of the pixel at the ground plane position of each orthorectified image is multiplied by the corresponding fusion weight and then added together to obtain the fused grayscale value of the ground plane position. The fused grayscale value is written to the corresponding position of the blank result image. After the grayscale values ​​of all ground plane positions are written, the orthorectified result image of the surveying UAV is obtained.

[0083] Because there are pseudo-edges near the waterline of the bridge pier, formed by the edges of the sacrificial anode blocks, the water surface reflections, and the actual surface of the bridge pier, these pseudo-edges are easily mistaken for the real boundaries of the bridge pier or components in the original UAV imagery, thus affecting image matching, geometric correction, and orthorectification results. Therefore, it is necessary to convert the local images affected by the pseudo-edges into orthorectified images with a unified ground coordinate relationship through local deflection correction, geometric correction, and orthorectification mosaicking. After obtaining the orthorectified image, it can be used as the base map of the surveying results for further tasks such as extracting the waterline boundary of the bridge pier, determining the shoreline location, locating the anode block ancillary facilities, updating the terrain around the bridge, and adding the data to the geographic information database.

[0084] One embodiment of the present invention also provides an image correction system for a surveying UAV facing the water surface; in this embodiment, the functions of each module / unit are as follows: The anchor point positioning module determines the target position relationship based on the original UAV imagery corresponding to the waterline area of ​​the bridge pier; and performs cropping processing on the original UAV imagery based on the candidate connected regions and the target position relationship to obtain a local image near the waterline. The edge chain filtering module performs edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; it determines the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain and generates the fold edge candidate image; The folding and expansion module performs mirror folding processing on the second neighboring image block corresponding to the folding edge candidate image based on the folding edge candidate edge chain to obtain the folded neighboring image block; based on the folded neighboring image block, the near-waterline local image is expanded to generate a mirrored domain mask image. The deconstruction and repair module extracts non-mirror neighboring pixels in the near-waterline local image based on the mirror domain mask image; and generates a corrected near-waterline local image based on the non-mirror neighboring pixels. The orthophoto output module obtains preprocessed UAV images based on the corrected near-waterline local images; it then performs geometric correction on the preprocessed UAV images to obtain orthophoto results images from the surveying UAV.

[0085] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for correcting images of a surveying UAV facing a water surface, characterized in that, The process includes the following steps: S1. Determine the target position relationship based on the original UAV image corresponding to the waterline area of ​​the bridge pier; and crop the original UAV image according to the candidate connected region and the target position relationship to obtain a local image near the waterline. S2. Perform edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; determine the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain, and generate the fold edge candidate image; S3. Based on the candidate edge chain of the candidate edge of the folded edge, the second neighboring image block corresponding to the candidate edge of the folded edge is mirrored to obtain the flipped neighboring image block; based on the flipped neighboring image block, the near waterline local image is expanded to generate a mirrored domain mask image. S4. Based on the mirror domain mask image, extract non-mirror adjacent pixels in the near-waterline local image; generate a corrected near-waterline local image based on the non-mirror adjacent pixels. S5. Based on the corrected near-waterline local image, obtain the preprocessed UAV image; perform geometric correction on the preprocessed UAV image to obtain the orthophoto result image of the surveying UAV.

2. The method for correcting images of a surveying UAV facing a water surface according to claim 1, characterized in that, Determine the target location relationships, including: Obtain the original UAV imagery corresponding to the waterline area of ​​the bridge pier; The original UAV imagery is converted to grayscale to obtain a grayscale image. A smoothing filter is applied to the grayscale image to obtain a filtered grayscale image; Edge extraction is performed on the filtered grayscale image to obtain the edge image; Extract multiple candidate connected regions from the edge image; The main edge is extracted from the edge image to obtain the edge region of the bridge pier; Determine the target positional relationship between the candidate connected region and the pier edge region.

3. The method for correcting images of a surveying UAV facing a water surface according to claim 1, characterized in that, Obtain a local image of the waterline, including: Based on the circumscribed contour size, contour shape, and target position relationship of the candidate connected regions, the anode candidate region is determined from multiple candidate connected regions; An anode anchor mask image is generated based on the pixel position of the anode candidate region in the original UAV image; the pixels in the anode anchor mask image corresponding to the anode candidate region are marked as anode pixels. Based on the bounding rectangle of the anode candidate region, the original UAV image is cropped to obtain a local image near the waterline.

4. The image correction method for a water-facing UAV as described in claim 3, characterized in that, Multiple anode adjacent edge chains are obtained, including: Edge extraction is performed on the local image near the waterline to obtain the edge image of the waterline; Boundary extraction is performed on the anode anchor point mask image to obtain the anode boundary image; Based on the position of the anode boundary pixels in the anode boundary image, the edge pixels in the near-waterline edge image are filtered by adjacency to obtain the initial edge pixel set; Starting from the edge pixels in the initial edge pixel set, edge connectivity tracking is performed in the near-waterline edge image to obtain multiple anode adjacent edge chains.

5. The method for correcting images of a surveying UAV facing a water surface according to claim 4, characterized in that, Generate candidate images for the folded edge, including: Determine the chain length and direction variation of the anode adjacent edge chain; Calculate the adjacency length between the anode adjacent edge chain and the anode boundary image; Based on the chain length, direction change, and adjacency length of the anode adjacent edge chains, multiple anode adjacent edge chains are sorted to obtain the edge chain sorting result; Based on the edge chain sorting results, candidate edge chains for the fold edge are determined from multiple adjacent edge chains of the anode. Candidate images of the fold edge are generated based on the pixel positions of the candidate edge chains in the local image near the waterline.

6. The method for correcting images of a surveying UAV facing a water surface according to claim 1, characterized in that, The resulting flipped neighborhood image patch includes: Based on the position of the candidate pixels of the fold edge in the candidate image of the fold edge, determine the candidate edge chain of the fold edge; Determine the tangent and normal directions of each candidate pixel on the candidate edge chain of the fold edge; Based on the normal direction of the candidate pixels of the fold edge, the second neighboring image block located on the second side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; Based on the tangent direction of the candidate edge chain of the fold edge, the second neighboring image block is mirrored to obtain the folded neighboring image block.

7. The method for correcting images of a surveying UAV facing a water surface according to claim 6, characterized in that, Generate a mirror domain mask image, including: Based on the normal direction of the candidate pixels of the fold edge, the first neighboring image block located on the first side of the candidate edge chain of the fold edge is extracted from the local image near the waterline; The flipping similarity is determined based on the gray values ​​of each pixel in the first neighboring image block and the gray values ​​of the corresponding pixels in the flipped neighboring image block. The mirror side is determined based on the fold similarity corresponding to the first side of the candidate edge chain and the fold similarity corresponding to the second side of the candidate edge chain. Starting from the mirror side, a region expansion is performed in the local image near the waterline to obtain the mirror region; A mirror domain mask image is generated based on the pixel position of the mirror region in the local image near the waterline.

8. The method for correcting images of a surveying UAV facing a water surface according to claim 1, characterized in that, Generate a corrected local image of the waterline, including: Based on the mirror domain mask image, determine the image region to be repaired in the near-waterline local image; The pixels in the local image near the waterline that correspond to the area to be repaired are marked as the pixels to be repaired. Based on the candidate images of the folded edge, determine the boundary guiding pixels of the image region to be repaired; Based on the location of the pixel to be repaired and the location of the boundary guiding pixels, extract non-mirror adjacent pixels that are adjacent to the region of the image to be repaired in the local image near the waterline; Based on the gray values ​​of the non-mirror adjacent pixels, the gray values ​​of the pixels to be repaired are filled to obtain the repaired local image of the waterline. Based on the repaired local image of the waterline, a corrected local image of the waterline is generated.

9. The image correction method for a water-facing UAV as described in claim 1, characterized in that, The obtained orthophotos from the mapping drone include: The corrected near-waterline local image is embedded into the original UAV image to obtain the preprocessed UAV image. Feature points are extracted from the preprocessed UAV images to obtain a set of feature points; Feature matching is performed on the set of feature points to obtain the set of image matching points; Based on the image matching point set, geometric correction is performed on the preprocessed UAV image to obtain the geometrically corrected UAV image; Based on the geometrically corrected UAV imagery, generate orthorectified imagery; Image mosaicking is performed on the orthorectified image to obtain the orthorectified result image of the surveying UAV.

10. A system for applying the image correction method for a water-facing surveying UAV as described in any one of claims 1-9, characterized in that, The system includes: The anchor point positioning module determines the target position relationship based on the original UAV imagery corresponding to the waterline area of ​​the bridge pier; and performs cropping processing on the original UAV imagery based on the candidate connected regions and the target position relationship to obtain a local image near the waterline. The edge chain filtering module performs edge connectivity tracking in the near-waterline edge image corresponding to the near-waterline local image to obtain multiple anode adjacent edge chains; it determines the fold edge candidate edge chain based on the chain length and direction change of the anode adjacent edge chain and generates the fold edge candidate image; The folding and expansion module performs mirror folding processing on the second neighboring image block corresponding to the folding edge candidate image based on the folding edge candidate edge chain to obtain the folded neighboring image block; based on the folded neighboring image block, the near-waterline local image is expanded to generate a mirrored domain mask image. The deconstruction and repair module extracts non-mirror neighboring pixels in the near-waterline local image based on the mirror domain mask image; and generates a corrected near-waterline local image based on the non-mirror neighboring pixels. The orthophoto output module obtains preprocessed UAV images based on the corrected near-waterline local images; geometric correction is then performed on the preprocessed UAV images to obtain orthophoto results images of the surveying UAV.

Images