A continuous ice field splicing method based on shipborne local field of view images

Abstract

The application relates to the technical field of shipborne local field of view image construction, and discloses a continuous ice field splicing method based on a shipborne local field of view image, which comprises the following steps: generating a bird's-eye ice field image according to a local ice field time sequence image, removing invalid areas to obtain available areas; dividing feature detection areas and splicing writing areas according to the available areas; performing feature point extraction and interframe matching in the feature detection areas, determining local registration transformation of a current frame relative to a previous accepted frame; taking a first frame as an anchor frame, and frame by frame, accumulating to obtain global poses of all frames in a global splicing coordinate system; constructing a global splicing canvas according to projection positions of all frames in the global coordinate system, and frame by frame, mapping and writing effective pixels into the canvas according to the global poses of all frames and the splicing writing areas, so as to obtain an ice field splicing picture which is continuously expanded along a heading direction. The application overcomes the industry pain points of difficult feature point extraction and false matching in a single texture scene, and realizes stable and continuous ice field splicing.

Description

Technical Field

[0001] This invention relates to the field of shipborne local view image construction technology, and in particular to a method for stitching together continuous ice rinks based on shipborne local view images. Background Technology

[0002] In actual navigation scenarios in ice-covered areas, ships need to continuously perceive the state of the ice ahead, to the sides, and to the surrounding area to assist in analyzing ice distribution, areas of broken ice, ice ridges, cracks, ice floe boundaries, navigable areas, and the evolution of the ice surface near the ship's navigation path. Continuous ice field images can also provide basic visual data for subsequent ice condition identification, track tracing, channel assessment, and navigational support decision-making.

[0003] In related technologies, Sandru et al. proposed a sea ice mapping method based on a shipborne monocular camera. This method corrects the perspective of the forward-view image to obtain a top-view ice field image, and then uses the Accelerated Robust Feature Algorithm (SURF) to complete feature matching and continuous stitching of adjacent top-view ice field images, thereby obtaining a two-dimensional sea ice field image along the ship's track direction.

[0004] However, shipborne sea ice images typically exhibit characteristics such as monotonous texture, low contrast, similar ice edge morphology, and significant water surface reflection. When processing viewpoint-corrected frontal sea ice images in such scenarios, the SURF algorithm may encounter insufficient or mismatched effective feature points, leading to registration deviations between adjacent images. This can result in ice edge misalignment, local ghosting, splicing breaks, and cumulative deformation, and in severe cases, failure to stitch consecutive images together. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a continuous ice rink stitching method based on shipborne local view images. This method can utilize continuously acquired local ice rink images to complete spatial registration, continuous stitching, and global fusion between multiple frames, gradually generating large-scale ice rink stitched images that can even be extended to the kilometer level.

[0006] This invention provides a method for stitching together continuous ice rinks based on shipborne local view images, comprising the following steps: Acquire multiple frames of local ice field time-series images continuously collected during navigation in the ice field, generate an overhead ice field image based on the local ice field time-series images, remove invalid areas, and obtain the usable area; The feature detection area and the stitching writing area are divided according to the available area. The feature detection area is the area with clear and stable ice surface texture, which has not been disturbed by ship waves and avoids the blurred areas in the distance. The stitching writing area is the area with clear imaging, reliable projection relationship and can effectively represent the ice field coverage of the current frame. Both the feature detection area and the stitching writing area remove the unstable projection areas at a distance, the bottom area disturbed by the ship, the area blocked by the ship hull, and the invalid areas on the left and right edges. Feature point extraction and inter-frame matching are performed within the feature detection area of ​​each frame of the overhead view of the ice rink image. The matching point pair relationship between the current frame and the previous received frame is established, and the local registration transformation of the current frame relative to the previous received frame is determined based on the matching result. Using the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, for each subsequent frame, the global pose of that frame in the global stitching coordinate system is obtained frame by frame based on its local registration transformation relative to the previous accepted frame. A global stitching canvas is constructed based on the projection position of each frame of the overhead ice rink image in the global coordinate system. Then, according to the global pose and stitching writing area of ​​each frame, the effective pixels in the corresponding overhead ice rink image are mapped frame by frame and written to the global stitching canvas to obtain a stitched ice rink image that extends continuously along the ship's sailing direction.

[0007] Optionally, the feature detection region and the splicing and writing region are divided according to the available area, including the following steps: For the area stitching mode, the minimum and maximum values ​​of all non-zero pixels in the available area in the x / y direction are taken to form the outer area; Remove the top 18% of the outer region to obtain the standard region; The feature detection area is defined as a range of 18%–64% vertically and 10%–90% horizontally from the standard region. The area of ​​44%~98% vertically and 10%~90% horizontally of the standard area is selected as the splicing and writing area.

[0008] Optionally, the feature detection region and the splicing and writing region are divided according to the available area, including the following steps: For strip stitching mode, the minimum and maximum values ​​of all non-zero pixels in the available area in the x / y direction are taken to form the outer region; Remove the top 35% of the outer region to obtain the standard region; The feature detection area is defined as the range of 30%~56% vertically and 24%~76% horizontally of the standard region. The area of ​​80%~95% vertically and 38%~62% horizontally of the standard area is selected as the splicing and writing area.

[0009] Optionally, the feature point extraction adopts a scale-invariant feature extraction algorithm to obtain candidate feature points; the feature detection region is divided into multiple grid units, and the candidate feature points are sorted in descending order according to the feature response intensity, and a preset number of high response feature points are retained in each grid unit according to the feature response intensity.

[0010] Optionally, inter-frame matching includes the following steps: For each feature point within the feature detection region of the current frame, perform a k=2 nearest neighbor search based on L2 distance within the feature detection region of the previous received frame to obtain the nearest neighbor candidate feature point and the second nearest neighbor candidate feature point; Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance. If the nearest neighbor distance is less than the product of the preset matching ratio threshold and the second nearest neighbor distance, then retain the matching point pair; otherwise, discard the matching point pair. The selected matching point pairs are sorted in ascending order of matching distance to serve as the candidate matching point pair set for subsequent local registration transformation estimation.

[0011] Optionally, inter-frame matching also includes the following steps: The current frame is matched with historical keyframes that meet the interval conditions to obtain a set of additional matching point pairs. The matching results of adjacent frames and the additional matching results are used as candidate matching results, and the effectiveness of the candidate matching results is filtered according to the number of feature points, the number of original matching point pairs, the number of inliers, or the proportion of inliers. The candidate matching result with the best quality is selected to determine the local registration transformation. When multiple candidate results are of similar quality, the matching result between the current frame and the previous accepted frame is preferred.

[0012] Optionally, the method also includes: When the current frame fails to obtain a valid matching result that meets the conditions, the predicted motion of the current frame is estimated based on the historical motion of the most recent frames, and the predicted global pose of the current frame is determined based on the predicted motion. The current frame is then written into the global stitching canvas to maintain short-term continuous stitching. If the number of consecutive stitching attempts using predicted motion exceeds the preset limit for consecutive prediction attempts, or if the prediction result is unreliable, the current frame is skipped and not written into the final stitching result.

[0013] Optionally, the global pose of the frame in the global stitching coordinate system is obtained frame by frame based on the local registration transformation, including the following steps: Set the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, and set its global pose as the unit transformation matrix. For each subsequent frame, the global pose of the previous received frame is multiplied by the local registration transformation of the current frame relative to the previous received frame to obtain the global pose of the current frame in the global stitching coordinate system.

[0014] Optionally, a global mosaic canvas is constructed based on the projection positions of each frame of the overhead ice rink image in the global coordinate system, including the following steps: The local boundaries of the top-view ice rink images of each received frame are projected onto the global coordinate system after corresponding global pose transformation. The minimum and maximum values ​​of all projected boundaries in the horizontal and vertical directions are counted to determine the width and height of the global stitching canvas. Construct a global coordinate offset transformation matrix based on the minimum values ​​of all accepted frames in the horizontal and vertical coordinates after global pose transformation; Based on the global pose and offset transformation matrix of each frame, determine the actual transformation matrix of each frame to be written into the global stitching canvas, and map all frames into the same global stitching canvas.

[0015] Optionally, the effective pixels in the corresponding overhead view of the ice rink are mapped frame by frame and written to the global stitching canvas, including the following steps: Based on the stitching writing area of ​​each frame and its mapping position in the global stitching canvas, a global quality map corresponding to each frame is generated. For the pixel position mapped to the global mosaic canvas in the current frame, determine whether that position is already occupied by a pixel in an existing frame: If the position is not yet occupied, the pixel value of the corresponding position in the current frame is directly written to the canvas; If the position is already occupied, compare the quality value of the corresponding pixel in the current frame with the quality value of the existing pixel in the canvas. Only if the quality value of the current frame pixel is higher than the quality value of the existing pixel and the difference exceeds the preset replacement threshold, the current frame pixel will be replaced and written to the canvas. Otherwise, the original pixel in the canvas will be retained.

[0016] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: This invention provides a continuous ice rink stitching method based on shipborne local view images. Instead of full-image matching, it strategically delineates a feature detection region specifically for feature pre-processing based on the physical laws of ice surface imaging and the influence of ship waves. This region deliberately avoids distant areas with simple and blurry textures, the disturbed bottom near-ship area, and areas obscured by the ship's hull and with invalid edges, retaining only ice surfaces with clear, stable textures and reliable projection relationships. This significantly improves the quality and quantity of effective feature point extraction. Simultaneously, the method also independently delineates a stitching writing region, located differently from the feature detection region, designed to cover new ice rink areas. During feature matching, the algorithm operates only within the high-quality feature detection region, supplemented by a matching point pair filtering strategy, greatly reducing the probability of mismatches and ensuring the accuracy of local registration transformation between adjacent frames. Finally, by accumulating global pose frame by frame with the first frame as the anchor frame and only writing effective pixels from the stitching writing region of each frame to the global canvas, this strategy avoids stitching breaks caused by the accumulation of mismatches and ensures continuous expansion of the stitching image along the course through the spatial configuration of the stitching writing region. This invention fundamentally overcomes the industry pain points of difficult feature point extraction and mismatch in single-texture scenes, and achieves a robust, continuous, unbroken, and low-ghosting ice rink splicing effect under complex ice conditions. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a method for stitching together continuous ice rinks based on shipborne local view images, provided in an embodiment of the present invention; Figure 2 This is the original image of the shipborne local ice rink before processing, provided in an embodiment of the present invention. Figure 3 The image of the ice rink obtained by top-view projection processing of the original image provided in the embodiments of the present invention; Figure 4 This is a schematic diagram of a stitching writing region mask for controlling the writing of a top-down ice rink image into a global stitching canvas, provided in an embodiment of the present invention. Figure 5 A schematic diagram of the effective ice rink writing area determined based on the spliced ​​writing area provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the feature matching results of adjacent frame top-view ice rink images provided in an embodiment of the present invention; Figure 7This is a schematic diagram of a large-scale continuous ice rink image obtained by continuously stitching together multiple frames of top-down ice rink images in an embodiment of the present invention; Figure 8 This is a schematic diagram of sea ice thickness observation and spatiotemporal registration of continuous ice fields provided in an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0020] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] First, this invention provides a method for stitching together continuous ice rinks based on shipborne local view images, specifically as follows: Figure 1 As shown, it includes the following steps: The process involves acquiring a series of consecutive local ice field images during navigation in the ice-covered area. A top-down view of the ice field is then generated from these images, and invalid areas are removed to obtain the usable region. First, a sequence of consecutive local ice field images is acquired during navigation in the ice-covered area. These images can be consecutive frames captured by shipborne cameras, underway cameras, or other navigation observation equipment. The specific steps include: 1. [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Figure 2As shown, continuous acquisition of shipborne local ice rink images: The local ice rink image sequence can originate from video data continuously collected by the shipborne platform during navigation, or from continuous image frames extracted from video data at preset intervals. To ensure that each frame of local ice rink image participates in subsequent processing in the actual acquisition order, this invention organizes the input image sequence temporally, determining the sequential relationship between each frame image based on the acquisition order, time order, or preset frame extraction order. Furthermore, this invention can also control the starting position, the number of input images, and the frame extraction interval according to preset parameters to adapt to the ice rink image processing requirements under different sampling frequencies, different navigation durations, and different stitching ranges. Through the above methods, this invention can establish a stable image input order, providing a consistent data foundation for subsequent top-view projection, inter-frame registration, global pose accumulation, and continuous stitching of local ice rink images, and avoiding stitching instability problems caused by chaotic input order or inconsistent sampling. 2. Camera Model Construction, Distortion Reduction, and View Plane Unification: To enable subsequent registration and stitching of continuously acquired local ice rink images in a unified coordinate system, this invention first constructs a camera model and performs distortion reduction and view plane unification processing on the input images. This invention uses pre-calibrated camera intrinsic parameters and distortion parameters. Furthermore, this invention uses previously measured camera mounting attitude parameters and mounting height. The mounting attitude parameters include at least pitch and roll angles, and the mounting height is used to determine the projection relationship between image pixels and the ice surface plane. After determining the camera model parameters, a projection mapping relationship from the original image plane to the ice surface view plane is constructed based on the camera intrinsic parameters, distortion parameters, mounting attitude parameters, mounting height, forward observation range, lateral observation range, and metric resolution. Based on the projection mapping relationship, a view projection is performed on the distortion-reduced local ice rink image to obtain a view of the ice rink at a unified scale, as shown below. Figure 3 As shown. If necessary, the top-view projection results can be cropped to obtain a uniform top-view processing area.

[0022] Through the above processing, subsequent image registration is no longer performed directly on the original oblique view image, but on the top-view ice rink image with a unified scale and coordinate relationship, thereby reducing the impact of lens distortion, angle changes and camera posture changes on the stitching results of continuous ice rinks.

[0023] The available area is divided into a feature detection area and a stitching writing area. The feature detection area is defined as a region with clear and stable ice surface texture, unaffected by ship waves, and avoiding distant blurred areas. The stitching writing area is defined as a region with clear imaging, reliable projection relationships, and the ability to effectively represent the ice rink coverage of the current frame. Both the feature detection area and the stitching writing area exclude distant projection instability areas, the bottommost near-ship disturbance area, ship hull occlusion areas, and invalid areas on the left and right edges. Specifically, the process includes the following steps: 3. Ship hull occlusion masking, invalid area removal, and region division: After obtaining the top-view ice rink image under a unified top-view plane, this invention first removes areas unsuitable for subsequent registration and stitching, and then divides the available area into a feature detection area and a stitching writing area. Specifically, this invention first uses a ship hull occlusion mask corresponding to the acquired local ice rink image sequence to mask the ship hull occlusion areas in the top-view plane to prevent the ship structure, deck edges, and related occlusion content from participating in subsequent processing, such as... Figure 4 As shown. Further, this invention combines brightness information from the overhead view of the ice rink image to remove low-brightness invalid areas. In one embodiment, pixels with grayscale values ​​below a preset threshold can be considered invalid pixels; the preset threshold can be, for example, 8, thereby removing black invalid areas formed during the overhead projection process. Black invalid areas refer to black, data-free background areas in the overhead view of the ice rink image generated by overhead projection. For example... Figure 4 The left and right black borders and black right-angled triangles in the image. Furthermore, this invention further masks the boundary areas and near-ship areas of the top-view image to reduce the impact of unstable projection edge areas on subsequent processing. In one embodiment, 12 pixels can be masked on each of the four edges of the top-view image, and 24 pixels can be masked on the near-ship area at the bottom of the top-view image; if necessary, 5×5 structuring elements can be used to shrink the area boundaries to further remove unstable edge pixels.

[0024] After removing the aforementioned invalid regions, this invention further divides the usable area of ​​the same overhead view of the ice rink into feature detection regions. Within these regions, feature point extraction, feature description, and adjacent frame matching are performed to obtain matching results that characterize the spatial correspondence between the current frame and the reference frame. Based on these matching results, a stitching writing region is determined to write the valid pixels from the current frame's overhead view of the ice rink into the global stitching canvas. This controls the image content retained in the final stitched image and avoids writing black invalid regions, hull-occluded regions, and unstable edge regions into the final result. Figure 5 As shown.

[0025] Feature point extraction and inter-frame matching are performed within the feature detection area of ​​each frame of the overhead view of the ice rink image. The matching point pair relationship between the current frame and the previous received frame is established, and the local registration transformation of the current frame relative to the previous received frame is determined based on the matching result. Using the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, for each subsequent frame, the global pose of that frame in the global stitching coordinate system is obtained frame by frame based on its local registration transformation relative to the previous accepted frame. A global stitching canvas is constructed based on the projection position of each frame of the overhead ice rink image in the global coordinate system. Then, according to the global pose and stitching writing area of ​​each frame, the effective pixels in the corresponding overhead ice rink image are mapped frame by frame and written to the global stitching canvas to obtain a stitched ice rink image that extends continuously along the ship's sailing direction.

[0026] This invention provides a method for stitching continuous ice rink images based on shipborne local view images. First, a top-view image of the ice rink is generated from a time-series front view image acquired by a shipborne camera through viewpoint correction, and invalid areas such as the ship hull and edge distortion are removed to obtain a clean, usable ice surface area. Then, within this usable area, based on the ice surface imaging clarity, the range of ship wave disturbance, and projection reliability, two functionally independent sub-regions are strategically divided: a "feature detection region" dedicated to feature extraction and matching, and a "stitching writing region" dedicated to pixel writing. Based on this, the algorithm only performs scale operations within the high-quality feature detection region. Degree-invariant feature extraction and inter-frame matching are employed. Reliable matching point pairs between the current frame and the previously received frame are established by filtering based on distance ratios, and then the local registration transformation is calculated. Subsequently, the first frame is used as the global anchor frame, and the local transformation is accumulated and passed frame by frame to obtain the accurate global pose of each frame in the global stitching coordinate system. Finally, based on the global pose and stitching writing area of ​​each frame, the effective pixels of each frame are sequentially mapped to the dynamically expanded global canvas, and fused and written using a pixel quality comparison strategy. This achieves continuous, stable, and uninterrupted panoramic stitching of the ice rink along the ship's course while suppressing feature mismatch and cumulative drift.

[0027] As one possible approach, the feature detection region and the splicing and writing region are divided based on the available area, including the following steps: For the area stitching mode, first remove the hull occlusion and black invalid areas to obtain a binary mask of the usable area; then take the minimum and maximum values ​​of all non-zero pixels in the x / y direction of the mask to form a minimum circumscribed axis aligned rectangle, which serves as the basis for the subsequent proportional division of the feature detection area and the stitching writing area.

[0028] By removing the top 18% of the outer region, the top of the image represents the actual, more distant ice rink. Information about the distant ice rink is easily blurred, making matching difficult. Removing the top portion and leaving the bottom portion results in a clearer and more reliable image, yielding the standard region.

[0029] The feature detection region is defined as 18%–64% vertically and 10%–90% horizontally of the standard area. The vertical 18%–64% region can be reduced to the middle portion. The ice surface in the upper part of the image, due to its distance, has less texture and is unstable. The lower part of the image is closer to the ship's hull; the ship's waves disturb the sea ice, causing it to drift and significantly impacting the stitching. The horizontal 10%–90% region avoids the left and right edges of the image, as these areas are susceptible to black borders, projection stretching, and boundary jitter. Retaining the middle 80% ensures sufficient sea ice texture features and maintains higher matching stability.

[0030] The standard area is defined as 44%–98% vertically and 10%–90% horizontally as the stitching and writing area. Vertically (44%–98%): The top area is farther away and more prone to instability after projection, so the writing area doesn't start from 0%, but from 44%, actively avoiding the more unstable parts of the upper half. The lower effective area is preserved as much as possible, as it usually contains more valuable new coverage content in the current frame, so it's preserved up to 98%, writing as much effective ice rink information as possible, but not to 100%, to avoid potential interference from nearby ships and edge instability at the bottom. Horizontally, the left and right edges are still removed. Similar to the feature detection area, the left and right edges are not very reliable, so the middle 80% is the main effective band.

[0031] As another feasible approach, the feature detection region and the concatenation and writing region are divided according to the available area, including the following steps: For the strip stitching mode, first remove the hull occlusion and black invalid areas to obtain a binary mask of the usable area; then take the minimum and maximum values ​​of all non-zero pixels in the x / y direction of the mask to form a minimum circumscribed axis aligned rectangle, which serves as the basis for the subsequent proportional division of the feature detection area and the stitching writing area.

[0032] The standard area is obtained by removing the top 35% of the outer region. By taking the outer region and removing the top 35% as the standard area, a larger proportion of distant blurred areas is eliminated compared to the area stitching mode. This is because strip stitching usually corresponds to a longer continuous navigation distance, and distant ice surfaces are less reliable due to projection stretching and blur accumulation.

[0033] The feature detection area is defined as 30%–56% vertically and 24%–76% horizontally of the standard region. This region is narrowed vertically to the upper middle section, effectively avoiding the near-ship disturbance area at the bottom (0–30% vertically) and the unstable area at the top (56%–100% vertically). At the same time, it is further narrowed horizontally compared to the area stitching mode (from 10%–90% to 24%–76%) to avoid greater projection deformation and boundary jitter caused by heading fine-tuning in the edge areas of strip stitching, ensuring that the extracted feature points have the strongest texture stability and inter-frame consistency.

[0034] The standard region, comprising 80%–95% vertically and 38%–62% horizontally, is designated as the stitching writing area. This area, located in the lower-rear part of the image and narrower horizontally (retaining only the central 24% bandwidth), perfectly aligns with the core requirement of strip stitching—that is, writing only the core strip with the most reliable projection relationship, the most concentrated new coverage information, and the least affected by edge deformation into the global canvas each time. This results in a narrow strip mosaic extending continuously along the course of flight as the image progresses frame by frame. Overall, this partitioning method significantly improves the robustness of feature matching and the accuracy of global pose estimation by reducing the spatial range of feature detection and pixel writing, while sacrificing the coverage width of a single frame. It effectively suppresses cumulative deformation and lateral misalignment caused by small-angle course drift in strip stitching mode, ensuring the continuity and geometric consistency of the ice rink mosaic image under long-distance flight conditions.

[0035] In the above manner, the present invention obtains feature detection region and stitching writing region in the same top-view ice rink image by quantitative division, so that different regions in the same frame image can respectively undertake the two functions of inter-frame registration and result writing, thereby providing a stable foundation for subsequent local geometric transformation estimation, global pose accumulation and continuous ice rink stitching.

[0036] Optionally, the feature point extraction adopts a scale-invariant feature extraction algorithm to obtain candidate feature points; the feature detection region is divided into multiple grid units, and the candidate feature points are sorted in descending order according to the feature response intensity, and a preset number of high response feature points are retained in each grid unit according to the feature response intensity.

[0037] Specifically, it includes the following steps: 4. Matching stage: Construction of feature detection region and splicing writing region: After completing the feature detection region segmentation, this invention performs feature extraction and inter-frame matching within the feature detection regions of each frame of the top-down ice rink image to establish a correspondence between consecutive images, providing input for subsequent local registration transformation estimation. Specifically, in the current embodiment, this invention employs a scale-invariant feature extraction algorithm to extract and describe feature points within the feature detection region. In the current implementation, the feature extractor is preferably SIFT, and the maximum number of feature points can be set to 12000, contrastThreshold=0.01, and edgeThreshold=12. After obtaining candidate feature points, to avoid excessive concentration of feature points in local high-texture areas, this invention further performs grid-based equalization filtering. The scale-invariant feature extraction algorithm is more suitable for stably finding corresponding points that can be used for stitching in ice rink BEV images with repetitive textures, local deformations, scale / rotation changes, and some low-texture areas.

[0038] In one implementation, the feature detection region can be divided into multiple 96×96 pixel grid units, and candidate feature points are sorted in descending order according to feature response intensity. For each grid unit, only the top 24 high-response feature points are retained. Retaining too few feature points in each grid unit may lead to insufficient effective features in local areas, reducing the stability of inter-frame matching; retaining too many may cause excessive feature points to cluster in high-texture areas, weakening the balance of feature distribution across the entire image and increasing the risk of mismatches due to duplicate textures. Selecting 24 is appropriate. Through this balanced selection, feature points can be more evenly distributed in space, thereby reducing the dominant influence of densely textured local areas on subsequent inter-frame registration results.

[0039] This invention employs the Scale Invariant Feature Transform (SIFT) algorithm to extract and match features from sea ice images. SIFT features have strong adaptability to scale, rotation, and illumination, and high local descriptive discrimination, which can improve the feature matching stability in low-texture and repetitive texture sea ice images, reduce mismatches and registration errors, thereby improving the accuracy and reliability of continuous sea ice field image stitching.

[0040] After feature point extraction and equalization filtering are completed, this invention matches the feature descriptions between the current frame and the previous received frame. In the current embodiment, inter-frame matching includes the following steps:

[0041] A nearest neighbor matching method based on L2 distance can be adopted, and a k=2 nearest neighbor search is performed on each feature point in the current frame. For the nearest neighbor candidate and the second nearest neighbor candidate, a matching ratio criterion is used to remove ambiguous matches. In the current implementation, when using SIFT features, the matching ratio threshold can be set to 0.75, that is, only matching point pairs that satisfy "nearest neighbor distance < 0.75 × second nearest neighbor distance" are retained. For the selected matching point pairs, they can be sorted in ascending order of matching distance to serve as candidate inputs for subsequent local registration transformation estimation. The matching logic is as follows: for each feature point in the current frame, find two points in the previous frame that are "most similar to it". If the first one is significantly more similar than the second one, it means that it is likely that it is a correct match, so it is retained. If the two are about the same, it means that it is difficult to distinguish which one it corresponds to, and this kind of matching is unreliable, so it is discarded.

[0042] To improve matching stability in continuous splicing, this invention, in addition to using adjacent frame matching between the current frame and the previous accepted frame, can also introduce a keyframe-assisted matching mechanism. In the current implementation, when the frame extraction interval is the default value, the keyframe interval can be 6 frames, the minimum keyframe interval can be 3 frames, and the maximum keyframe interval can be 18 frames. For the current frame, it is preferentially matched with the previous accepted frame; simultaneously, it can also be additionally matched with historical keyframes that meet the interval conditions, and the results are used as candidate matching results.

[0043] For candidate matching results, this invention further filters them based on matching quality. In the current implementation, if a candidate result has fewer than 60 feature points, fewer than 24 original matching point pairs, fewer than 12 inliers, or an inlier ratio lower than 0.15, the candidate matching result is deemed invalid. For valid candidate results, comparisons can be made based on the number of inliers, the inlier ratio, and candidate priority, and the candidate matching result with the highest quality is selected for subsequent processing; when multiple candidate results are of similar quality, the matching result with the previously received frame is preferred. Among the candidate matching results, those with more than or equal to 12 inliers are considered valid candidate results; if there are multiple valid candidate results, the one with the most inliers is selected as the result. Features: regions where two images have the same or similar areas, such as ice-water boundaries, ice block edges, areas with abrupt changes in brightness, sea ice areas with obvious local textures, and intersections of edge lines.

[0044] like Figure 6 As shown, through the above processing, the present invention can establish a stable and reliable matching point pair relationship between continuous top-view ice rink images, providing a foundation for subsequent local registration transformation estimation, global pose accumulation and continuous ice rink stitching.

[0045] In one implementation, when the current frame fails to obtain a valid matching result that meets the conditions, the predicted motion of the current frame can be estimated based on the historical motion of the most recent frames and used to maintain short-term continuous stitching. In the current implementation, the historical motion statistics window can be the most recent 5 frames, and the number of consecutive predictions can be limited to no more than 8 frames; 5 frames are used to predict the motion of the current frame for stitching. 8 frames are used to predict using historical motion; if more than 8 frames are exceeded, they are skipped. If this limit is exceeded or the prediction result is still unreliable, the current frame is skipped and not written into the final stitching result. Furthermore, after obtaining the global pose of the current frame, the stitching writing area can be further shrunk based on the spatial distribution of the matched intrapoints in the current frame to form the actual writing area of ​​the current frame; if the area of ​​the actual writing area is too small, it is considered that the current frame does not contribute enough to stable stitching and can be omitted from writing. In the current implementation, this area threshold can be no less than 350 pixels and no less than 1.5% of the original stitching writing area area.

[0046] Optionally, the global pose of the frame in the global stitching coordinate system is obtained frame by frame based on the local registration transformation, including the following steps: Set the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, and set its global pose as the unit transformation matrix. For each subsequent frame, the global pose of the previous received frame is obtained by left-multiplying the local registration transformation of the frame relative to the previous received frame.

[0047] 5. Assembly Phase: Global pose accumulation and continuous ice rink splicing After completing the matching of adjacent frames and obtaining the local registration transformation of the current frame relative to the previous received frame, the present invention further performs global pose accumulation on the continuous images and thereby completes the construction of a continuous ice field mosaic map along the navigation direction.

[0048] Specifically, this invention sets the first received top-down image of the ice rink as the anchor frame of the global stitching coordinate system, and sets its global pose as a unit transformation, i.e. .in, This represents the global pose of the first frame in the global stitching coordinate system. Let represent the unit transformation matrix. For each subsequent frame, if its local registration transformation relative to the previously received frame is denoted as ... The global pose of this frame in the global stitching coordinate system is then obtained frame by frame by the following formula: .in, Indicates the first Global pose of the frame in the global stitching coordinate system Indicates the first The global pose of each accepted frame in the global stitching coordinate system. Indicates the first The local registration transformation of a frame relative to the previously received frame. Indicates the frame number, symbol " " indicates the composite operation of the transformation matrix. In the above way, the "position change of the current frame relative to the previous frame" can be continuously accumulated into the "global position of the current frame in the whole stitched image".

[0049] After obtaining the global pose of each received frame, this invention calculates the global boundary range of all frames based on the projection position of the local area of ​​each frame's top-down ice rink image in the global coordinate system, and constructs a global stitching canvas accordingly. Specifically, the local boundaries of each frame can be transformed by the corresponding global pose, and their minimum and maximum horizontal and vertical values ​​can be statistically analyzed to determine the width and height of the global stitching canvas. To avoid negative coordinates after transformation, this invention further constructs a global coordinate offset transformation:

[0050] .

[0051] in, This represents the global coordinate offset transformation matrix. This represents the minimum value in the horizontal coordinate of all accepted frames after global pose transformation. This represents the minimum value in the vertical coordinate system of all accepted frames after global pose transformation. Through this offset transformation, global coordinates that might have fallen into the negative coordinate range can be shifted to the positive coordinate range of the splicing canvas.

[0052] And the actual write transformation for each frame is written as: .

[0053] in, Indicates the first Frames are used to write the actual transformation matrix of the global mosaic canvas. Indicates the first The global pose transformation matrix of the frame in the global stitching coordinate system, denoted by " "" indicates the composition operation of the transformation matrix. That is, first by Determine the first The position of the frame in the global stitching coordinate system, and then... A uniform translation is performed on the overall coordinates to obtain the actual transformation used when the frame is finally written into the splicing canvas.

[0054] This allows all frames to be mapped onto the same global splicing canvas.

[0055] Subsequently, according to the global pose and actual writing area of ​​each frame, the present invention maps and writes the effective pixels in the corresponding top-down ice rink image frame by frame to the global stitching canvas. When overlapping areas exist, the present invention does not simply overwrite, but constructs a pixel quality weight map based on the writing area and selectively retains the overlapping pixels. In the current implementation, local weights can be generated based on the actual writing area, and a corresponding global quality map can be formed after mapping to the global canvas; wherein, the feathering radius of the local weights can be 48 pixels, and the quality scaling factor can be determined according to the matching inlier ratio, for example, taking... For unoccupied positions on the canvas, the current frame pixels are written directly; for overlapping positions with existing content, replacement is only performed if the quality value of the corresponding pixel in the current frame is more than 0.015 higher than the original quality value.

[0056] After all accepted frames have been written, this invention can generate a stitching result mask based on the final effective pixel range, perform lightweight edge smoothing at the junctions of different frames, and finally crop the effective area of ​​the global stitching canvas to obtain an ice rink stitching image that extends continuously along the ship's sailing direction.

[0057] Through the above processing, this invention can not only obtain the stable position of each frame in the global stitching image, but also accumulate continuously acquired local ice rink images into the same canvas according to a unified coordinate relationship, thereby forming a continuous ice rink image covering a longer navigation path, such as... Figure 7 As shown.

[0058] Furthermore, this invention performs spatiotemporal registration of the side thickness observation results with the aforementioned global ice field long map. Side thickness observations can be obtained when sea ice breaks up, overturns, or ice sections are exposed. For the first... Each effective side thickness observation is recorded, including the observation time, thickness value, and observation location, and these are designated as thickness observation points, such as... Figure 8 As shown.

[0059] In one implementation, when the upper and lower boundaries of the overturned sea ice are visible in the side-view image, the pixel distance is converted into the actual ice thickness based on the calibration scaling factor of the side-view camera. The thickness can be expressed as:

[0060] .

[0061] in, For the first One thickness observation value, This is the scaling factor from the pixel distance in the hull-side image to the actual length. and These represent the upper and lower boundary points of the visible ice cross section, respectively.

[0062] Since the relative position between the hull-side observation device and the ship is fixed, and the aforementioned steps have already obtained the ship's motion relationship in the global ice field map through image registration and global pose accumulation, the hull-side thickness observation can be mapped to the corresponding position in the global ice field map based on the thickness observation time. Let the global pose corresponding to the thickness observation time be... The fixed offset of the side observation point relative to the forward stitching camera reference point is: Then the position of this thickness observation in the global ice rink coordinate system is:

[0063] .

[0064] in, Indicates the first The thickness observation is mapped to the planar position after the global ice rink map. This indicates the position of the thickness observation point in the side observation coordinate system or the hull coordinate system. Thus, discrete side thickness observations are converted into thickness anchor points in the global ice field map.

[0065] Furthermore, the thickness anchor points are associated with ice regions or ice block objects in the global ice rink map. Specifically, the ice region to which the thickness anchor point belongs can be determined based on the location of the thickness anchor point, the acquisition time, and the location and corresponding time of each ice region in the global ice rink map. If a thickness anchor point is located within an ice region, or if the distance between it and the ice region is less than a preset spatial threshold and the time difference between them is less than a preset time threshold, then the thickness anchor point is matched to that ice region.

[0066] After completing thickness anchor point matching, based on the assumption of local ice thickness continuity, a thickness attribute is assigned to the corresponding ice region. If an ice region matches only one thickness anchor point, the thickness value of that anchor point is directly used as the thickness attribute of the region; if an ice region matches multiple thickness anchor points, a weighted average method is used to determine the representative thickness of the region.

[0067] .

[0068] in, Indicates the first The thickness properties of each ice region This represents the set of thickness anchor points that match the ice region. Indicates the first The weight of each thickness anchor point is determined based on the confidence level of the thickness observation, the spatial distance between the thickness anchor point and the ice region, and the time interval.

[0069] For adjacent ice areas where no thickness anchor point is directly matched, this invention can propagate thickness within a limited range. Specifically, if the target area and the already thickened area belong to the same continuous ice block, the same local ice area, or the same adjacent area of ​​the same flight path, and there is no crossing of boiling water, cracks, obvious ice edges, or invalid areas between them, then the thickness attribute of the already thickened area can be propagated to the target area. During the propagation process, the thickness confidence can be reduced based on spatial distance, time interval, and regional continuity to reflect the uncertainty of the propagated thickness.

[0070] By employing the aforementioned methods of hull side thickness observation, thickness anchor point generation, spatiotemporal registration, region matching, and local thickness propagation, this invention integrates intermittent and discrete hull side thickness observations into a global ice field long map reconstructed from shipborne forward images. This results in a final ice field map that includes not only the planar distribution of the ice surface and the ship's navigation trajectory, but also the thickness attributes and thickness confidence levels corresponding to specific ice areas or ice block objects. Ultimately, each valid ice area object can include at least its geometric boundary, area, center location, thickness attributes, and thickness confidence level, thereby forming a continuous ice field representation with thickness attributes suitable for ice condition analysis, ship-ice interaction analysis, and polar vessel operation assistance.

[0071] The above inventions are merely a few specific embodiments of the present invention. However, the embodiments of the present invention are not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. A method for stitching together continuous ice rinks based on shipborne local view images, characterized in that, Includes the following steps: Acquire multiple frames of local ice field time-series images continuously collected during navigation in the ice field, generate an overhead ice field image based on the local ice field time-series images, remove invalid areas, and obtain the usable area; The available area is divided into a feature detection area and a stitching writing area. The feature detection area is a region with clear and stable ice surface texture, which has not been disturbed by ship waves and avoids distant and blurry areas. The stitching writing area is a region with clear imaging, reliable projection relationship and effective representation of the ice field coverage of the current frame. Both the feature detection area and the stitching writing area exclude distant projection unstable areas, the bottommost near-ship disturbance area, ship hull occlusion area and invalid areas on the left and right sides. Feature point extraction and inter-frame matching are performed within the feature detection area of ​​each frame of the top-down ice rink image to establish the matching point pair relationship between the current frame and the previous received frame, and the local registration transformation of the current frame relative to the previous received frame is determined based on the matching result. Using the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, for each subsequent frame, the global pose of that frame in the global stitching coordinate system is obtained frame by frame based on its local registration transformation relative to the previous accepted frame. A global stitching canvas is constructed based on the projection position of each frame of the overhead ice rink image in the global coordinate system. Then, according to the global pose of each frame and the stitching writing area, the effective pixels in the corresponding overhead ice rink image are mapped frame by frame and written into the global stitching canvas to obtain a stitched ice rink image that extends continuously along the ship's sailing direction.

2. The continuous ice rink stitching method based on shipborne local view images as described in claim 1, characterized in that, The process of dividing the feature detection region and the splicing and writing region according to the available area includes the following steps: For the area splicing mode, the minimum and maximum values ​​of all non-zero pixels in the available area in the x / y direction are taken to form the outer area; Remove the top 18% of the outer region to obtain the standard region; The area of ​​18%~64% vertically and 10%~90% horizontally of the standard region is taken as the feature detection region. The standard area is defined as 44% to 98% vertically and 10% to 90% horizontally as the splicing and writing area.

3. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The process of dividing the feature detection region and the splicing and writing region according to the available area includes the following steps: For the strip stitching mode, the minimum and maximum values ​​of all non-zero pixels in the available area in the x / y direction are taken to form the outer region; Remove the top 35% of the outer region to obtain the standard region; The standard region is defined as a range of 30% to 56% vertically and 24% to 76% horizontally as the feature detection region. The area of ​​80%~95% vertically and 38%~62% horizontally of the standard area is taken as the splicing writing area.

4. The continuous ice rink stitching method based on shipborne local view images as described in claim 1, characterized in that, The feature point extraction employs a scale-invariant feature extraction algorithm to obtain candidate feature points. The feature detection region is divided into multiple grid cells, and the candidate feature points are sorted in descending order according to their feature response intensity. A preset number of high-response feature points are retained in each grid cell according to their feature response intensity.

5. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The inter-frame matching includes the following steps: For each feature point within the feature detection region of the current frame, perform a k=2 nearest neighbor search based on L2 distance within the feature detection region of the previous received frame to obtain the nearest neighbor candidate feature point and the second nearest neighbor candidate feature point; Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance. If the nearest neighbor distance is less than the product of the preset matching ratio threshold and the second nearest neighbor distance, then retain the matching point pair; otherwise, discard the matching point pair. The selected matching point pairs are sorted in ascending order of matching distance to serve as the candidate matching point pair set for subsequent local registration transformation estimation.

6. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 5, characterized in that, The inter-frame matching also includes the following steps: The current frame is matched with historical keyframes that meet the interval conditions to obtain a set of additional matching point pairs. The matching results of adjacent frames and the additional matching results are used as candidate matching results, and the effectiveness of the candidate matching results is filtered based on the number of feature points, the number of original matching point pairs, the number of inliers, or the proportion of inliers. The candidate matching result with the best quality is selected to determine the local registration transformation. When multiple candidate results are of similar quality, the matching result between the current frame and the previous accepted frame is preferred.

7. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The method further includes: When the current frame fails to obtain a valid matching result that meets the conditions, the predicted motion of the current frame is estimated based on the historical motion of the most recent frames, and the predicted global pose of the current frame is determined based on the predicted motion. The current frame is then written into the global stitching canvas to maintain short-term continuous stitching. If the number of consecutive stitching attempts using predicted motion exceeds the preset limit for consecutive prediction attempts, or if the prediction result is unreliable, then the current frame is skipped and not written into the final stitching result.

8. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The step of accumulating the global pose of a frame in the global stitching coordinate system frame by frame based on local registration transformation includes the following steps: Set the first accepted top-down image of the ice rink as the anchor frame of the global stitching coordinate system, and set its global pose as the unit transformation matrix. For each subsequent frame, the global pose of the previous received frame is obtained by left-multiplying the local registration transformation of the frame relative to the previous received frame.

9. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The process of constructing a global stitching canvas based on the projection positions of each frame of the overhead ice rink image in the global coordinate system includes the following steps: The local boundaries of the top-view ice rink images of each received frame are projected onto the global coordinate system after corresponding global pose transformation. The minimum and maximum values ​​of all projected boundaries in the horizontal and vertical directions are counted to determine the width and height of the global stitching canvas. Construct a global coordinate offset transformation matrix based on the minimum values ​​of all accepted frames in the horizontal and vertical coordinates after global pose transformation; Based on the global pose of each frame and the offset transformation matrix, the actual transformation matrix for each frame to be written into the global stitching canvas is determined, and all frames are uniformly mapped to the same global stitching canvas.

10. The method for stitching together continuous ice rinks based on shipborne local view images as described in claim 1, characterized in that, The process of mapping and writing the effective pixels in the corresponding overhead view of the ice rink frame by frame to the global stitching canvas includes the following steps: Based on the stitching writing area of ​​each frame and its mapping position in the global stitching canvas, a global quality map corresponding to each frame is generated. For the pixel position mapped to the global mosaic canvas in the current frame, determine whether that position is already occupied by a pixel in an existing frame: If the position is not yet occupied, the pixel value of the corresponding position in the current frame is directly written to the canvas; If the position is already occupied, compare the quality value of the corresponding pixel in the current frame with the quality value of the existing pixel in the canvas. Only if the quality value of the current frame pixel is higher than the quality value of the existing pixel and the difference exceeds the preset replacement threshold, the current frame pixel will be replaced and written to the canvas. Otherwise, the original pixel in the canvas will be retained.

Images