Shipborne panoramic video stitching method based on multiple distributed cameras

Abstract

The invention discloses a ship-borne panoramic video stitching method based on multiple distributed cameras, and relates to the technical field of ship intelligent navigation and computer machine vision, a plurality of network cameras are respectively deployed on a bow, a stern, a left board and a right board of a ship body, multiple paths of video streams are collected in real time through an RTSP protocol, and decoding and frame synchronous caching are carried out; dark channel defogging, image sharpening, OTSU binaryzation, Canny edge detection and edge refining processing are sequentially carried out on each path of video stream. A distributed ordinary network camera architecture is adopted, deployment is performed on the bow, the stern, the left side and the right side of the ship body, the existing installation space and conventional network equipment of the ship are fully utilized, a special panoramic camera does not need to be customized, unified access of video streams is achieved through an RTSP protocol, software splicing is completed through an edge computing server, and the whole process is simplified. Hardware purchase and transformation cost is remarkably reduced, a distributed structure supports modular extension and maintenance, overall operation of the system is not affected by faults of a single camera, and the system has high cost performance and maintainability.

Description

Technical Field

[0001] This invention relates to the fields of ship intelligent navigation and computer machine vision technology, specifically to a shipborne panoramic video stitching method based on distributed multi-camera systems. Background Technology

[0002] With the continuous development of ship intelligent technology, panoramic vision systems have been widely used in applications such as maritime surveillance, collision avoidance and early warning, and unmanned monitoring. By providing a 360° field of view without blind spots, panoramic vision systems enable ships to better control their surrounding environment, thereby effectively improving navigation safety. Traditional shipborne panoramic vision systems mainly rely on spherical or disc-shaped panoramic cameras. Each camera device achieves image stitching through the cooperation of multiple lenses, which can provide a relatively clear panoramic canvas. Due to its integrated design, it has the advantages of compact structure and easy installation, and is widely used in the monitoring systems of some high-end ships and unmanned ships.

[0003] However, existing 360° vision systems still have many technical shortcomings in practical applications: spherical / disc-type panoramic cameras require customized lenses and special chips, resulting in high costs and overall system expenses. Because such systems rely on integrated design, a failure in one lens direction can affect the normal operation of the entire system, leading to difficult maintenance and long downtime. Furthermore, the masts and hull structures of most ships are unsuitable for installing spherical cameras, limiting installation options. Existing stitching technologies often perform poorly in complex sea conditions and low-contrast backgrounds. Traditional image stitching methods, such as feature-point-based algorithms (e.g., SIFT, SURF, ORB), are prone to failure when the sea surface lacks obvious textures and feature points, resulting in large stitching errors and affecting system stability and reliability. Therefore, this paper proposes a distributed multi-camera shipborne panoramic video stitching method to address these problems. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a shipborne panoramic video stitching method based on distributed multi-camera systems, comprising the following steps: S1. Multiple network cameras are deployed at the bow, stern, port and starboard sides of the ship to collect multiple video streams in real time via the RTSP protocol, and to decode and cache the frames synchronously. S2. Perform dark channel dehazing, image sharpening, OTSU binarization, Canny edge detection and edge thinning on each video stream in sequence to enhance the features of the sea-sky antenna structure. S3. Based on probabilistic Hough transform and RANSAC fitting, the sea-line antenna is extracted from the edge map, and the temporal consistency and geometric alignment of the video streams from each viewpoint are achieved through frame synchronization and camera parameter calibration, so as to stably output the angle and position of the sea-line antenna. S4. Using the bow view of the ship as the reference for the panoramic canvas, the y-axis translation correction is performed according to the sea-line height of each view to achieve geometric consistency correction of multiple images. Combined with feature point matching, horizontal stitching is achieved to construct a 360° circular panoramic image. S5. A multi-band fusion algorithm is used to construct Gaussian and Laplacian pyramids, and combined with a linear weight mask to fuse images in the frequency domain in layers, eliminating stitching seams and obtaining a seamlessly fused panoramic image. S6. The stitched panoramic image is encoded into a standard streaming media format in real time and pushed to a network streaming media server via the RTSP protocol for access by multiple terminals.

[0005] Preferably, S1 specifically includes: A set of standard network cameras is deployed at the bow, stern, port side, and starboard side of the hull to form a distributed visual acquisition array. Each network camera broadcasts its acquired H.264 encoded video stream within the local area network via the RTSP protocol, making full use of the ship's existing installation space and significantly reducing hardware deployment costs. The edge computing server deployed on the ship uses the GStreamer framework to concurrently pull multiple RTSP video streams. GStreamer handles RTSP streaming and decoding, while OpenCV processes the raw video frames. Image frames are cached in a synchronous memory queue, effectively improving the real-time performance and stability of data reading and processing. GPS timing is used to align multiple image frames in the cache, and the intrinsic and extrinsic parameters of each network camera are pre-calibrated to establish a mapping relationship between the camera's logical index and its physical location, providing a precise spatiotemporal consistency basis for subsequent image stitching.

[0006] Preferably, S2 specifically includes: For each video stream, an image dehazing algorithm based on dark channel prior is executed sequentially to restore the contrast attenuation caused by sea fog or backlight, improve the overall image clarity and color fidelity, and effectively enhance the visual perception capability in low visibility environments. A predefined high-pass sharpening convolution kernel is used to perform spatial domain sharpening on the dehazed image to enhance its edge and contour information. Then, the image is converted to grayscale and binarized by applying the OTSU adaptive thresholding algorithm to obtain a binarized image, which significantly improves the recognizability of the sea-line structure. The Canny operator is applied to the binarized image for fine edge extraction, and a morphological skeletonization algorithm is further used to refine the extracted edges, generating a single-pixel-width edge map with clear structure and accurate topology, providing high-quality input for subsequent line detection.

[0007] Preferably, in step S3, the process of extracting the sea-line area from the edge map includes: By applying the probabilistic Hough transform to the edge map, all candidate line segments that meet the minimum length threshold are detected, and the parameters of each line segment in the image coordinate system are calculated. The results are then integrated to obtain a set of candidate line segments, which effectively improves the efficiency and completeness of line detection. A random sampling consensus algorithm is used to robustly fit the candidate line segment set, filter out noisy line segments, determine a main sea-sky line representing the sea-sky boundary, and output its angle parameter θ, which significantly enhances the accuracy of sea-sky line extraction. A historical sea-line detection buffer with a multi-frame sliding window is maintained for each network camera. By comparing the center height of the sea-line with that of the current frame and the historical frames, abnormal jumps caused by instantaneous interference are filtered out, and a stable sea-line position and angle are output to ensure the continuous stability of the detection results in dynamic environments.

[0008] Preferably, in step S3, the process of ensuring temporal consistency and geometric alignment of the video streams from different viewpoints includes: Based on the stable position and angle of the sea surface line, combined with the pre-calibrated camera extrinsic parameters, the geometric relationship of the sea surface line in the unified world coordinate system under the view of each network camera is calculated, thereby establishing an accurate spatial correspondence and providing a reliable basis for splicing. Using the sea-line as a globally consistent structural feature provides a stable and reliable geometric constraint benchmark for stitching and aligning multi-view images, replacing unstable local feature points in low-texture sea surface scenes, effectively improving stitching accuracy and environmental adaptability. The geometric consistency of the sea-line at various viewpoints is continuously monitored. If the sea-line at a certain viewpoint is lost or deviates too much, the anomaly handling mechanism is triggered. The historical sea-line cache is used for interpolation to cover the loss, maintain the overall output stability, and ensure that the system can still output a stable panoramic image when there are some anomalies.

[0009] Preferably, in step S4, the process of performing y-axis translation correction based on the sea-line height from each viewpoint includes: The viewpoint of the bow camera is selected as the reference frame for the panoramic canvas. The reference height of its horizon line in the canvas is calculated to provide a unified vertical reference for multi-view alignment and effectively avoid visual jumps. Calculate the center height of the sea horizon in each of the remaining viewpoints, and calculate the vertical translation required for each image accordingly. Through precise height difference compensation, ensure that the sea horizons in each viewpoint are on the same horizontal line after stitching. Based on the vertical translation amount, an affine transformation matrix is ​​constructed for each image to be stitched together, and the image is translated and corrected in the y-axis direction to achieve precise alignment of all viewpoint images at the horizon height, significantly improving the geometric consistency and visual comfort of the panoramic image.

[0010] Preferably, in step S4, the process of constructing a 360° panoramic view includes: Based on the completion of the sea-line height alignment, local feature point detection and matching are performed in the overlapping area of ​​the image to correct minor horizontal misalignment and effectively improve the splicing geometric accuracy. Following the spatial logical sequence from the port side, through the stern, to the starboard side, the geometrically corrected multi-channel images are sequentially stitched onto a circular canvas centered on the bow viewpoint, forming a continuous and natural panoramic view. By combining the height alignment with the horizontal feature matching, a 360° circular panoramic image that is consistent in the vertical direction and coherent in the horizontal direction is formed, which significantly enhances visual consistency and structural integrity.

[0011] Preferably, in step S5, the process of constructing the Gaussian and Laplace pyramids includes: For each image to be fused in the 360° circular panoramic image, a Gaussian pyramid is constructed. Gaussian smoothing and downsampling are performed iteratively to obtain a set of images with decreasing resolution, thereby effectively suppressing high-frequency noise interference while preserving the main structure of the image. Based on the Gaussian pyramid, a corresponding Laplacian pyramid is constructed by upsampling each layer of the image and subtracting it from the previous Gaussian image, thus decomposing the image into different frequency bands. Each layer of the Laplacian pyramid carries the details and texture information of the image at different spatial scales, achieving effective separation of image details and background structure, and providing a processing basis for subsequent frequency band fusion.

[0012] Preferably, in step S5, the process of eliminating stitching seams by combining linear weighted masks to perform layered image fusion in the frequency domain includes: A linearly gradient weight mask is defined for the overlapping regions of the images. The weight mask has a weight of 0 at the left boundary of the overlapping region and a weight of 1 at the right boundary, with a linear transition in between, which ensures a smooth transition of the fusion weights in space. On each layer of the Laplacian pyramid, the weight mask is used to perform a weighted average on the corresponding layers of the two images to be fused to obtain the fusion result of that layer, which can effectively avoid seams and artifacts. Starting from the top layer, the merged Laplacian pyramid is upsampled layer by layer and superimposed with the fusion result of the next layer to reconstruct a seamless panoramic image, achieving a panoramic effect with natural transition in overlapping areas.

[0013] Preferably, S6 specifically includes: The stitched and merged panoramic images are re-encoded into H.264 format through the GStreamer pipeline and pushed to the embedded streaming media server via the RTSP protocol to ensure the real-time performance and smoothness of remote monitoring. The streaming media server publishes panoramic video streams to the outside world, supports real-time retrieval and playback by clients using the standard RTSP protocol, and provides a visual display interface with status information overlay locally, offering flexible multi-terminal access capabilities and intuitive system status awareness. The system incorporates anomaly detection and intelligent rollback mechanisms. When camera disconnection, stitching failure, or encoding error is detected, it automatically activates buffered frame rollback or stream switching strategies to switch to a backup image source or maintain the previous valid frame. This ensures the continuity and stability of the video stream, guarantees the continuity of the output video stream and service availability, and significantly enhances the system's robustness and service continuity under complex sea conditions.

[0014] This invention provides a shipborne panoramic video stitching method based on distributed multi-camera systems. It offers the following advantages: (I) The shipborne panoramic video stitching method based on distributed multi-cameras adopts a distributed ordinary network camera architecture, which is deployed at the bow, stern and port and starboard sides of the ship. It makes full use of the existing installation space and conventional network equipment on the ship, without the need for customized panoramic cameras. The video stream is uniformly accessed through the RTSP protocol, and the software stitching is completed by the edge computing server, which significantly reduces the cost of hardware procurement and modification. The distributed structure supports modular expansion and maintenance. The failure of a single camera does not affect the overall operation of the system, and it has high cost performance and easy maintenance.

[0015] (II) This shipborne panoramic video stitching method based on distributed multi-cameras integrates a series of preprocessing steps such as dark channel defogging, image sharpening, adaptive binarization and edge refinement for harsh environments such as low visibility, fog and backlight at sea. It restores image details through dark channel prior and atmospheric scattering model, and enhances the sea-line structure features by combining Canny edge detection and morphological processing, effectively improving the usability and robustness of images in low contrast and low texture scenes.

[0016] (III) This shipborne panoramic video stitching method based on distributed multi-camera introduces the sea-line as the geometric alignment basis for multi-view image stitching. The main sea-line is extracted by probability Hough transform and RANSAC fitting, and the detection results are stabilized by combining multi-frame sliding window. The sea-line has global consistency and structural stability, effectively replacing the shortcomings of traditional feature point matching methods in marine scenes, and ensuring the vertical consistency and angular continuity of the stitching process.

[0017] (iv) The shipborne panoramic video stitching method based on distributed multi-cameras adopts a multi-band fusion algorithm based on Gaussian pyramid and Laplacian pyramid, and combines linear gradient weight mask to process the image in frequency domain layer by layer. The image is decomposed into frequency band components at different scales, and the low-frequency structural information and high-frequency details are fused respectively, effectively eliminating stitching seams and brightness differences, and outputting a visually coherent and color-uniform seamless panoramic image. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the workflow of a shipborne panoramic video stitching method based on distributed multi-camera according to the present invention. Figure 2 This is a schematic diagram of the method flow of a shipborne panoramic video stitching method based on distributed multi-camera according to the present invention; Figure 3 This is a hardware system diagram of a shipborne panoramic video stitching method based on distributed multi-cameras according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1, please refer to Figure 1 , Figure 2 This invention provides a technical solution: a shipborne panoramic video stitching method based on distributed multi-camera systems, comprising the following steps: S1. Multiple network cameras are deployed at the bow, stern, port, and starboard sides of the ship to acquire multiple video streams in real time via the RTSP protocol, and perform decoding and frame synchronization caching. A set of standard network cameras is deployed at the bow, stern, port, and starboard sides of the ship to form a distributed visual acquisition array. Each network camera broadcasts its acquired H.264 encoded video stream within the local area network via the RTSP protocol, making full use of the existing installation space on the ship and significantly reducing hardware deployment costs. The edge computing server deployed on the ship pulls multiple RTSP video streams concurrently through the GStreamer framework. GStreamer completes RTSP streaming and decoding, while OpenCV completes the processing of the original video frames. The image frames are cached in a synchronous memory queue, effectively improving the real-time performance and stability of data reading and processing. GPS timing is used to achieve time alignment for the multiple image frames in the cache, and the intrinsic and extrinsic parameters of each network camera are pre-calibrated to establish a mapping relationship between the camera's logical index and physical location, providing a precise spatiotemporal consistency basis for subsequent image stitching. The specific work involves deploying a set of standard network cameras at four observation positions on the bow, stern, port, and starboard sides of the ship, forming a distributed visual acquisition array covering a 360-degree field of view around the ship. Each network camera in this distributed visual acquisition array acts as an independent video source, continuously broadcasting its real-time video stream data within the ship's local area network via the RTSP protocol. The video stream is compressed using H.264 encoding to obtain H.264 encoded video streams, effectively reducing network bandwidth consumption and ensuring stable transmission of high frame rate video data. The distributed architecture fully utilizes the ship's existing installation space and conventional network equipment, and the entire acquisition process is managed by a unified configuration file. The configuration includes the network address, authentication information, and physical location number of each camera, achieving systematic integration and initialization of hardware resources. An edge computing server deployed locally on the ship serves as the core processing unit for the aggregation and preprocessing of multiple video streams. The edge computing server integrates the GStreamer multimedia framework to build an efficient data processing pipeline for processing multiple RTSP video streams. The system performs concurrent fetching and efficient parsing. Simultaneously, GStreamer is used to decode each received H.264 encoded video stream in real time, and OpenCV restores it to processable raw image frame data. The decoded raw image frame data is then temporarily stored in an optimized synchronous memory queue. Combined with GPS timing, frame synchronization is performed on image frame data from different network cameras, ensuring that multiple images used within the same processing cycle correspond in time, laying the foundation for cross-view joint analysis. Each network camera in the distributed visual acquisition array is pre-calibrated. The calibration process uses a checkerboard calibration board to accurately measure and calculate the internal and external parameters of each camera. Internal parameters describe the inherent properties of the network camera, including focal length, principal point coordinates, and lens distortion. External parameters define the spatial attitude and position of each network camera relative to the ship's coordinate system. Based on the pre-calibration data, a unique logical index is assigned to each network camera, and a precise mapping relationship is established between it and its actual physical installation position on the ship, ensuring consistency between the virtual panoramic view and the real-world observation orientation. S2. For each video stream, perform dark channel dehazing, image sharpening, OTSU binarization, Canny edge detection, and edge refinement sequentially to enhance the features of the sea-line structure. For each video stream, execute the image dehazing algorithm based on dark channel prior to restore the contrast attenuation caused by sea fog or backlight, improve the overall image clarity and color fidelity, and effectively enhance the visual perception ability in low visibility environments. Use a predefined high-pass sharpening convolution kernel to perform spatial domain sharpening on the dehazed image to enhance its edge and contour information. Then, convert the image to grayscale and apply the OTSU adaptive thresholding algorithm for binarization segmentation to obtain a binarized image, which significantly improves the recognizability of the sea-line structure. Apply the Canny operator to the binarized image for fine edge extraction, and further use the morphological skeletonization algorithm to refine the extracted edges, generating a single-pixel-width edge map with clear structure and accurate topology, providing high-quality input for subsequent line detection. The specific work involves: for each video stream acquired by the distributed visual acquisition array, sequentially executing an image dehazing algorithm based on dark channel priors. By analyzing the minimum pixel value (dark channel) in a local region of the image, the fog concentration distribution is estimated, and then the radiance of the fog-free image is inferred. During processing, dark channel calculations are performed on video frames, and the global atmospheric light value and transmittance map are estimated by combining an atmospheric scattering model. Guided filtering is used to optimize transmittance and reduce block artifacts. Then, inverse transform is used to recover the fog-free image, effectively eliminating contrast attenuation caused by fog or backlighting at sea, improving the dynamic range and color fidelity of the image. This is particularly suitable for ship-surrounding observation scenarios in low-visibility environments. Specifically: calculating the dark channel... For each pixel in the image, the minimum value among its RGB color channels is taken in its local neighborhood, and then the minimum value in its neighborhood is taken again, forming the so-called "dark channel image". ; Estimate atmospheric light A; select the region with the highest brightness value from the dark channel image (usually the first 0.1% of pixels), and extract the RGB color value of the corresponding region in the original image as the atmospheric light intensity estimate for the entire image; Calculate transmittance By utilizing the relationship between the dark channel and atmospheric light, the transmittance of each pixel in the scene can be estimated, i.e., the degree to which the image is obscured by fog. ; in, The dark channel represents the position of the image. Dark channel value at the location; For the image at position The intensity value of the color channel c; Indicates The local neighborhood (window) centered on the center; This indicates taking the minimum value among the R, G, and B channels; This indicates that the minimum value is taken among all pixels in the neighborhood; A represents atmospheric light, which indicates the intensity of atmospheric light in the entire image; Transmittance represents the proportion of light reflected from objects in a scene that is not scattered by fog as it reaches the camera; This is a fog density control parameter, typically ranging from (0.9, 0.98), with a default setting of 0.95 to balance defogging intensity and image naturalness. Restore dehazed image The image was restored using an atmospheric scattering model, yielding a more realistic haze-free image. ; in, It is the lower limit value to prevent the transmittance from being 0, and is usually set to 0.1; Spatial domain sharpening is performed on the dehazed image. A predefined high-pass sharpening convolution kernel (3×3 Laplacian kernel) is used to enhance edge and contour information. The convolution operation compensates for the details and textures that may be lost during the dehazing process by highlighting high-frequency components, making the image edges clearer. The image is then converted to grayscale and binarized using the OTSU adaptive thresholding algorithm to obtain a binarized image. At the same time, the optimal threshold is automatically determined by maximizing the inter-class variance, which can divide the image into foreground and background without manual intervention, effectively separating target objects (such as ships and obstacles) from the background sea area. Fine edge extraction is performed on binarized images using the Canny operator. Double thresholding and non-maximum suppression (NMS) preserve edge pixels with significant gradient changes, suppressing noise interference. A morphological skeletonization algorithm is then used to refine coarse edges, generating a single-pixel-width topological structure. Simultaneously, iterative erosion gradually removes redundant edge pixels while maintaining connectivity and centrality, ensuring the geometric accuracy of the edge map. This results in a clear, topologically accurate single-pixel-width edge map that can be directly used for tasks such as target contour analysis, distance measurement, or 3D reconstruction. Specifically, image sharpening performs edge enhancement operations on the input image, using a high-pass sharpening convolution kernel by default. ; Image sharpening aims to amplify the obvious edge contours of the dehazed image, making them more prominent and easier to separate in subsequent processing steps. OTSU binarization follows, after the sharpened image is converted to grayscale, and then enters the thresholding stage. The OTSU adaptive thresholding algorithm is introduced, performing bimodal histogram analysis on the pixel intensity of the entire image to automatically calculate the segmentation threshold that maximizes the inter-class variance. This does not rely on manually set thresholds; instead, it adaptively selects the optimal threshold based on the image brightness distribution, dividing the image into foreground (e.g., sea surface, contours) and background regions, thus forming a relatively stable binary image. Edge detection, after obtaining the binary image, uses the Canny operator to accurately extract image edges. The edge map generated in this step retains the most representative edges in the image. The representative structural contour, especially the long straight edge features of the sea-line region, meets the requirements of subsequent line detection and fitting. After edge extraction, in order to further simplify the edge shape, remove boundary width redundancy, and retain the geometric topology, the Python function skeletonize is used to process the Canny output to generate a regular edge-refined image with consistent shape and topology. Through a series of processes such as sharpening, binarization, edge extraction, and edge refinement, a high-quality and structurally clear sea-line edge map is finally obtained, which lays the foundation for subsequent Hough line detection and RANSAC fitting. It has significant robustness and versatility for sea surface application scenarios in the open ocean, backlight, or drastically changing weather. S3. Based on probabilistic Hough transform and RANSAC fitting, the sea-sky line is extracted from the edge map. Through frame synchronization and camera parameter calibration, the temporal consistency and geometric alignment of the video streams from different perspectives are achieved, and the angle and position of the sea-sky line are stably output. Probabilistic Hough transform is applied to the edge map to detect all candidate line segments that meet the minimum length threshold. The parameters of each line segment in the image coordinate system are calculated and integrated to obtain a set of candidate line segments, which effectively improves the efficiency and completeness of line detection. A random sampling consensus algorithm is used to robustly fit the set of candidate line segments, filter out noisy line segments, determine a main sea-sky line representing the sea-sky boundary, and output its angle parameter θ, which significantly enhances the accuracy of sea-sky line extraction. A multi-frame sliding window historical sea-sky line detection cache is maintained for each network camera. By comparing the center height of the sea-sky line in the current frame with that in the historical frames, abnormal jumps caused by instantaneous interference are filtered out, and a stable sea-sky line position and angle are output, ensuring the continuous stability of the detection results in dynamic environments. The specific work involves: applying the probabilistic Hough transform to extract straight line features from the preprocessed edge image; identifying all candidate straight line segments in the image that meet the minimum length threshold by randomly sampling edge pixels and accumulating votes, thus avoiding the global computational redundancy of the traditional Hough transform; during the detection process, setting the minimum line segment length to 1 / 10 of the image height and limiting the maximum gap (allowing discontinuities in the line segment) to balance detection integrity and noise resistance; for each candidate line segment, calculating its parameters in the image coordinate system, including the start point coordinates, end point coordinates, center point coordinates, and tilt angle, forming a set of candidate straight line segments; the generation of the candidate straight line segment set is achieved using the OpenCV HoughLinesP function, combined with parallel processing to accelerate the line detection of multi-scale edges and ensure real-time performance; extracting edge line segments that meet the length requirements from the edge image; and using the probabilistic Hough transform to detect the main line. ; The above formula is the polar coordinate equation of a line used for line detection in the probabilistic Hough transform, where: The perpendicular distance from the line to the origin; The angle between the normal to the line and the x-axis indicates the direction of the line's inclination. Here are the coordinates of a point in the image coordinate system, where x is the x-coordinate (column index) of the pixel and y is the y-coordinate (row index) of the pixel. Angle The cosine and sine values ​​| - used to convert rectangular coordinates Mapping to polar coordinate parameter space ; Robust fitting is achieved using the Random Sample Consensus Algorithm (RANSAC). An initial straight line model is fitted by iteratively selecting a minimum subset of lines, and the number of inliers (line segments with an error less than a threshold relative to the model) is counted. The model with the highest proportion of inliers is selected as the optimal solution. The main sea-sky antenna representing the sea-sky boundary is determined from the candidate line segment set. Considering the characteristics of the maritime scene, an inlier distance threshold of 1% of the image width is set, and the tilt angle range of the fitted line is limited (within ±15°) to exclude interference lines from ship hulls and waves. After fitting, the angle parameter θ (the angle relative to the horizontal axis) of the main sea-sky antenna is output with an accuracy of 0.1°, effectively filtering out noisy line segments and ensuring detection stability in low visibility or complex backgrounds. Each network camera is equipped with a historical detection buffer (taking a 5-frame sliding window), which compares the current frame with... The center height of the sea-line antenna in historical frames (the average y-coordinate of the midpoint of the line segment) is calculated, and a height change rate is set with a threshold (±5 pixels / frame) to filter out abnormal jump frames. This eliminates sea-line antenna detection jumps caused by transient interference (such as birds or waves). If the difference in the center height of the line segment is greater than the jump threshold (15 pixels), it is ignored; otherwise, the current optimal sea-line antenna and angle are updated and output. Simultaneously, considering the time consistency constraint of the main sea-line antenna angle parameter θ (angle change rate < 1° / frame), jitter is further suppressed, and the smoothed sea-line antenna position (center point coordinates) and angle θ are output. This provides continuous visual reference for ship attitude estimation and obstacle avoidance tasks, improving the system's robustness in dynamic environments. RANSAC is used to fit the main line for the set of all center points, and the sea-line antenna angle θ is finally returned as the basis for stitching alignment. ; ; In the formula: s is the slope of the line, and b is the intercept; the above formula is the equation of a straight line in a rectangular coordinate system, which complements the polar coordinate form of the Hough transform and is used to describe or transform the straight line model. In addition, the process of ensuring temporal consistency and geometric alignment of video streams from different perspectives includes: based on the stable position and angle of the sea-line, combined with pre-calibrated camera extrinsic parameters, calculating the geometric relationship of the sea-line in a unified world coordinate system from the perspective of each network camera, thereby establishing a precise spatial correspondence and providing a reliable basis for stitching. The sea-line is used as a globally consistent structural feature to provide a stable and reliable geometric constraint benchmark for stitching alignment of multi-view images, replacing unstable local feature points in low-texture sea surface scenes, effectively improving the accuracy and environmental adaptability of stitching. The geometric consistency of the sea-line is continuously monitored. If the sea-line of a certain perspective is lost or deviates too much, an anomaly handling mechanism is triggered, and historical sea-line cache interpolation is used to cover it, maintaining the overall output stability and ensuring that the system can still output panoramic images stably when there are some anomalies. The specific work involves: based on the stable detection of the sea-line position and angle parameters, combined with the pre-calibrated extrinsic parameters of each network camera, mapping the sea-line from the image coordinate system to a unified world coordinate system through inverse perspective projection transformation. Specifically: based on the center point coordinates and angle of the sea-line in the image, constructing its linear equation in the camera coordinate system, and using the camera extrinsic parameters to transform this line to the world coordinate system to obtain its spatial representation. Then, through the spatial line intersection or parallelism constraints of the sea-line from multiple views, determining the relative geometric relationship in the global coordinate system, including the intersection point position, spacing, or parallel distance. Simultaneously, synchronizing the timestamps of each camera and compensating for minor drifts in extrinsic parameters caused by mechanical vibration or temperature changes to ensure long-term consistency of geometric relationships; introducing the sea-line as a structural feature into the multi-view image stitching process, replacing the traditional matching method based on local feature points (such as SIFT, ORB), by minimizing the reprojection error of the sea-line in the world coordinate system from each view, optimizing... The image stitching transformation matrix (homography matrix or affine transformation) is optimized to achieve seamless image fusion, which is especially suitable for low-texture sea surface scenes. This avoids matching failures or mismatches caused by missing local features. At the same time, the angle parameters of the sea surface line further constrain the global view consistency after stitching, preventing geometric distortion caused by excessive view difference and ensuring the geometric rationality of the stitching result in the horizontal and vertical directions. The geometric consistency of the sea surface line at each view point is continuously monitored to maintain the stability of multi-view stitching. Specifically, the position deviation and angle difference of the sea surface line at each view point in the world coordinate system are calculated in real time. If the position of the sea surface line at a certain view point deviates from the historical average by more than a threshold (5% of the image width) or the angle change rate exceeds 1° / frame due to occlusion, detection failure, or external parameter error, it is judged as abnormal. After the abnormality handling mechanism is triggered, the historical sea surface line cache is used for interpolation to overwrite, and the view point is marked as a state to be repaired. After the sea surface line is re-detected and stabilized, its participation in stitching is restored by recalibrating the external parameters. S4. Using the bow view of the ship as the reference for the panoramic canvas, the y-axis translation correction is performed according to the height of the sea-line in each view to achieve geometric consistency correction of multiple images. Combined with feature point matching, horizontal stitching is achieved to construct a 360° circular panoramic image. The view of the bow camera is selected as the reference reference system of the panoramic canvas, and its sea-line in the canvas is calculated to provide a unified vertical reference for multi-view alignment, effectively avoiding visual jumps. The center height of the sea-line in the remaining view images is calculated respectively, and the vertical translation amount required for each image is calculated accordingly. Through precise height difference compensation, it is ensured that the sea-line in each view is on the same horizontal line after stitching. Based on the vertical translation amount, an affine transformation matrix is ​​constructed for each image to be stitched, and the image is translated and corrected in the y-axis direction to achieve precise alignment of all view images on the height of the sea-line, significantly improving the geometric consistency and visual comfort of the panoramic image. The specific work involves: selecting the bow camera's viewpoint as the reference frame; extracting the sea-line from the bow image using a stability detection algorithm; calculating its center point coordinates and angle; mapping the sea-line from the image coordinate system to the world coordinate system using the camera's intrinsic and extrinsic parameters to obtain its spatial straight line equation; further, reprojecting this line onto the reference canvas (the globally stitched virtual coordinate system) using inverse perspective projection transformation to determine its reference height on the canvas. This reference height serves as the absolute reference for vertical alignment, ensuring that the sea-line from all viewpoints is on the same horizontal line after stitching, avoiding visual jumps or geometric distortions caused by differences in viewpoints; for the other viewpoint cameras, extracting the center point coordinates and angle of the sea-line in their images, and mapping them to the world coordinate system using the same extrinsic parameter calibration process to obtain the spatial straight line equation, and then projecting the line... The image is projected onto a reference canvas, and its height difference with the reference sea-line height is calculated. A vertical translation is applied to each viewpoint image to eliminate height differences and ensure alignment of its sea-line height with the reference height. Based on the calculated vertical translation, an affine transformation matrix is ​​constructed for each viewpoint image. While maintaining the horizontal direction of the image, only the y-axis is translated for correction. The image is transformed using the affine transformation matrix to achieve precise alignment of the sea-line height while preserving the original proportions and horizontal structure of the image. The corrected image is further optimized through reprojection error to ensure that the alignment accuracy of the sea-line height in the world coordinate system is <1 pixel, making all viewpoint images consistent in the vertical direction. Specifically, to ensure geometric consistency between multi-view images, a method based on sea-line height alignment is adopted. Y-axis translation correction is performed before image stitching for each image. The average height H can be obtained by calculating the center height h of the sea-line antenna. An affine transformation is then performed on each image to align the sea-line antenna along the y-axis. ; ; In the formula: Let be the affine transformation matrix for the r-th image, a 2×3 matrix used to perform translation transformations on the image; 1, 0, 0 is the first row of the transformation matrix, keeping the x-coordinate unchanged and not performing horizontal scaling or cropping. The second row of the transformation matrix keeps the y-coordinate scaling factor at 1. It is the translation in the y direction; H is the reference sea-line height, the target height that all image sea-line heights should be aligned with; Let be the current sea-line height of the r-th image, and let y-coordinate of the center point of the sea-line in the image be obtained through the sea-line detection algorithm. This represents the vertical translation amount. Positive values ​​shift the image downwards, negative values ​​shift the image upwards, and the objective is to make... ; Let r be the original coordinate vector of the r-th image, expressed as homogeneous coordinates. , which contains the coordinates of all pixels in the image, where T is the transpose; Let r be the coordinate vector after the image is transformed; this avoids misalignment issues caused by slight differences in installation height and posture between different cameras. Furthermore, the process of constructing a 360° circular panorama includes: after completing the alignment of the horizon, local feature point detection and matching are performed in the overlapping area of ​​the images to correct minor horizontal misalignments and effectively improve the geometric accuracy of the stitching. Following the spatial logical order from the port side, through the stern, to the starboard side, the geometrically corrected multi-path images are sequentially stitched onto a circular canvas centered on the bow viewpoint to form a continuous and natural panoramic view. By combining height alignment with horizontal feature matching, a 360° circular panoramic image that is consistent in the vertical direction and coherent in the horizontal direction is formed, significantly enhancing visual consistency and structural integrity. The specific work involves: after completing the vertical alignment of the sea-line, further correcting minor horizontal misalignments in the overlapping areas of the images caused by differences in viewing angles or extrinsic parameter errors. This is achieved by extracting local feature points (such as SIFT, SURF, or ORB) within the overlapping areas of adjacent images, matching them based on feature descriptors, constructing feature point correspondences, using the RANSAC algorithm to remove mismatched points, calculating the affine transformation matrix, and fine-tuning only the horizontal direction (x-axis) to ensure pixel-level alignment of adjacent images at the horizontal seams. It should be noted that this must be combined with the vertical alignment constraint of the sea-line to avoid introducing new vertical deviations during horizontal correction, achieving dual geometric consistency of the images in both the horizontal and vertical directions. Following the spatial order from the port side, through the stern, to the starboard side, the geometrically corrected multi-path images are sequentially stitched onto a circular canvas centered on the bow viewpoint. The circular canvas uses a cylindrical or projected model to simulate a 360° panoramic view, with the bow viewpoint serving as the origin of the reference coordinate system. Other viewpoints are transformed to this coordinate system using camera extrinsic parameters. During stitching... The system determines the projection area of ​​each viewpoint in the circular canvas based on their relative positions, and dynamically adjusts the stitching order based on the feature matching results of overlapping areas to avoid structural breaks caused by viewpoint jumps. This ensures that all images form a continuous visual flow on the circular canvas, with the sea-line height remaining consistent in the vertical direction and seamless horizontal misalignment. By combining vertical sea-line alignment with horizontal feature matching, a global optimization framework is constructed to generate a 360° circular panoramic image. The vertically aligned sea-line height is used as a reference to constrain the projection height of all viewpoints in the vertical direction. The horizontal feature matching results are used to optimize the relative positions of adjacent images. By minimizing reprojection errors, the camera extrinsic parameters and image transformation parameters are jointly adjusted to ensure that the panoramic image achieves sub-pixel alignment accuracy in both the horizontal and vertical directions. The output 360° circular panoramic image must meet the following requirements: no visual jumps in the vertical sea-line height, coherent scene structure in the horizontal direction, and global geometric deformation of less than 1 pixel. It is suitable for real-time monitoring and virtual roaming applications in low-texture sea surface scenes. S5. A multi-band fusion algorithm is used to construct Gaussian and Laplacian pyramids, and combined with a linear weight mask to fuse images in the frequency domain in layers, eliminating stitching seams and obtaining a seamlessly fused panoramic image. S6. The stitched panoramic image is encoded into a standard streaming media format in real time and pushed to a network streaming media server via the RTSP protocol for access by multiple terminals. At the same time, it supports local display, RTSP network streaming and abnormal rollback mechanism to ensure the continuity and stability of video stream under complex sea conditions.

[0021] Example 2, as Figures 1 to 3As shown, based on Embodiment 1, the present invention provides a technical solution: In S5, the process of constructing Gaussian and Laplacian pyramids includes: constructing a Gaussian pyramid for each image to be fused in the 360° circular panoramic image, performing Gaussian smoothing and downsampling iteratively to obtain a set of images with decreasing resolution, thereby effectively suppressing high-frequency noise interference while preserving the main structure of the image; based on the Gaussian pyramid, constructing a corresponding Laplacian pyramid by upsampling each layer of the image and subtracting it from the previous layer of the Gaussian image, decomposing the image into different frequency bands, wherein each layer of the Laplacian pyramid carries the details and texture information of the image at different spatial scales, realizing the effective separation of image details and background structure, and providing a processing basis for subsequent frequency band fusion; The specific work involves: performing Gaussian pyramid layering on each image to be fused within the 360° panoramic image; iteratively applying a Gaussian smoothing filter to decompose the original image into multiple scales. Specifically, at each level, a Gaussian kernel is used to perform convolution operations on the current image to eliminate high-frequency noise and retain key structural information; the smoothed image is then downsampled (typically by a factor of 2) to generate an image sequence with decreasing resolution, progressing layer by layer until a preset minimum resolution level is reached. The construction of the Gaussian pyramid achieves spatial scale decomposition of the image, and the smoothing operation suppresses detail interference between different levels. Based on the completed Gaussian pyramid... Further, a Laplacian pyramid is constructed to extract multi-band detail information from the image. The core operations include: upsampling (2x interpolation) each level of the Gaussian pyramid (except the bottom level) to restore its resolution to the size of the previous level; then, performing pixel-by-pixel interpolation between the upsampled result and the Gaussian image of the previous level to obtain a Laplacian layer containing high-frequency details. Each layer of the Laplacian pyramid carries image details at a specific spatial scale, including high-frequency components of edges and textures. The bottom layer retains an approximate structure of the original image, achieving independent representation of image frequency bands and ensuring the separation of detail and structure. In addition, a Gaussian pyramid is constructed for each image using Gaussian filtering. ; In the formula: I is the original input image, and the panoramic sub-image that needs to be fused and geometrically corrected; This is the 0th layer of the Gaussian pyramid, which is the original input image I itself, and is the highest resolution layer in the pyramid. This represents the o-th level of the Gaussian pyramid, and is the image of the o-th level in the pyramid. The larger the value of o, the lower the resolution of the image. This is the (o-1)th level of the Gaussian pyramid, the current level. The layer above it has a higher resolution than high; The downsampling function is a process involving Gaussian blurring and resizing (typically by halving both width and height). First, Gaussian filtering is performed to eliminate high-frequency noise and aliasing, then downsampling is performed, starting from the original input image I. ,right Perform downsampling (blurring and scaling down) to obtain a first layer with lower resolution. ,right Perform the same operation to obtain the second layer. This process is repeated to generate a series of images. This process continues until a preset number of layers are reached, forming a Gaussian pyramid that captures multi-scale information of the image from details (bottom layer) to overall appearance (top layer). Then construct the Pyramid of Laplace: ; In the formula: This is the o-th layer of the Laplace pyramid, and the image of this layer contains... and The differences between upsampled images, i.e., the details lost at this scale; This is the (o+1)th level of the Gaussian pyramid. The next layer is an image with even lower resolution; - is an upsampling function that enlarges the image size (usually by doubling the length and width) and uses an interpolation algorithm (bilinear or bicubic interpolation) to fill in the new pixels; - is a pixel-by-pixel subtraction function that subtracts the intensity values ​​of corresponding pixels in two images, highlighting the differences between the two images. Repeat the above process to build a 3-5 layer pyramid, automatically adjusting according to image resolution and stitching bandwidth, so that large-scale brightness gradients and local detail textures in the image can be processed separately at different levels; Furthermore, by combining linear weight masks to perform layered image fusion in the frequency domain, the process of eliminating stitching seams includes: defining a linearly gradient weight mask for the overlapping regions of the images. This weight mask has a weight of 0 at the left boundary and a weight of 1 at the right boundary of the overlapping region, with a linear transition in between, ensuring a smooth transition of the fusion weights in space. On each layer of the Laplacian pyramid, the weight mask is used to perform a weighted average on the corresponding layers of the two images to be fused, obtaining the fusion result of that layer, which can effectively avoid seams and artifacts. Starting from the top layer, the fused Laplacian pyramid is upsampled layer by layer and superimposed with the fusion result of the next layer to reconstruct a seamlessly fused panoramic image, achieving a panoramic effect with a natural transition in the overlapping regions. The specific work involves defining a linearly gradient weight mask to smoothly transition overlapping image regions. This weight mask has a weight of 0 at the left boundary and 1 at the right boundary of the overlapping region. The weight in the middle region is continuously adjusted from 0 to 1 using linear interpolation. Specifically, the horizontal coordinate range of the overlapping region is determined, and a normalized weight is calculated for the horizontal coordinate of each pixel within the region. A single-channel grayscale mask with the same size as the overlapping region is generated. This weight mask spatially defines the fusion ratio of the two images. The left boundary completely retains information from the first image, the right boundary completely uses information from the second image, and the middle region achieves a natural transition through linear weighting. It should be noted that this weight mask is aligned with the size of each layer of the Laplacian pyramid to ensure the accuracy of layered fusion. Based on the generated linearly gradient weight mask, weighted fusion is performed on each layer of the Laplacian pyramid of the two images. Each layer of the Laplacian pyramid contains high-frequency detail information (edges, textures) at a specific spatial scale. In practice, each Laplacian image layer is multiplied by the weight mask and its complement to obtain the weighted result, which is then multiplied. The resulting Laplacian pyramid is fused to ensure that the detailed information of each layer is mixed linearly in the overlapping areas, avoiding seams or artifacts caused by direct stitching, while preserving the original details in non-overlapping areas. The layered fusion strategy leverages the advantages of multi-scale decomposition to decouple high-frequency information from low-frequency structures during the fusion process. The fused Laplacian pyramid generates the final panoramic image through a reverse reconstruction process. Reconstruction begins at the top layer (lowest resolution layer), which is directly used as the initial reconstruction result. Each layer of the fused Laplacian image is upsampled by a factor of 2 to match the resolution of the next layer (higher resolution). The upsampled result is then added to the fused image of the next layer, gradually restoring high-frequency details. This top-down progression, with each layer superimposing low-frequency structures with high-frequency details, ultimately results in a seamlessly fused panoramic image at the bottom layer (original resolution). The reconstructed image exhibits a natural transition in overlapping areas due to layered linear fusion, while non-overlapping areas retain the original image information completely, achieving a balance between global structural consistency and lossless local detail. Specifically, to avoid obvious seams from direct stacking, a spatial weight mask is used. The generation mechanism controls the weighting ratio of each pixel in the stitched image pair during the fusion process, and selects a linear gradient function as a mask. ; In the formula: is the weight mask function, a function defined on the horizontal coordinate x, with an output value between [0,1], representing the weight of image 1 in the fusion; x is the horizontal coordinate in the image, the column index (horizontal coordinate) of the pixel in the image. The left boundary of the overlapping region is the starting x-coordinate where the weight begins to increase from 0; The right boundary of the overlapping region is defined by a weight that increases to 1 at the end of the x-coordinate; 0 indicates that image 2 is fully utilized, on the left side of the overlapping region ( A mask value of 0 means that the area uses all the pixels of image 2; 1 means that image 1 is used entirely, on the right side of the overlapping area ( A mask value of 1 means that the area uses all the pixels of image 1. This is a linear transition region, between the left and right boundaries ( The weights increase linearly from 0 to 1 to achieve a smooth blending of the two images; A fusion operation is performed on the Laplacian pyramid image of each layer. The fused pyramid images are then upsampled layer by layer from low resolution to high resolution and weighted and stacked to reconstruct the final seamlessly fused panoramic image. ; ; In the formula: The fused Laplacian pyramid at layer o is the result of weighted mixing of the detailed information of the two images at layer o. For the o-th layer of the Laplacian pyramid in image 1, in coordinates High-frequency detail values ​​at the location; For the o-th layer of the Laplacian pyramid in image 2, in coordinates High-frequency detail values ​​at the location; To extend to two-dimensional weighted masks, in practical applications, one-dimensional... It will expand into a two-dimensional mask of the same size as the image layer. ;1- The complementary weights for image 2 are such that the sum of the two weights is 1, which ensures the smoothness of the fusion. The final seamless fused image is the output of the entire multi-band fusion process; o is the layer index of the pyramid, from the highest layer (coarsest, o=0) to the lowest layer (finest, o=M); M is the total number of layers in the pyramid; The upsampling function is used to sample the fused Laplacian layers. Scale it up to the same size as the next layer (higher resolution); S6 specifically includes: re-encoding the stitched and merged panoramic image into H.264 format through the GStreamer pipeline, and pushing it to the embedded streaming media server via the RTSP protocol to ensure the real-time and smoothness of remote monitoring. The streaming media server publishes the panoramic video stream to the outside world, supporting clients with standard RTSP protocol to pull and play it in real time. At the same time, it provides a visual display interface with status information overlay locally, providing flexible multi-terminal access capabilities and intuitive system status awareness. It is designed with an anomaly detection and intelligent rollback mechanism. When a camera disconnection, stitching failure, or encoding error is detected, it automatically enables the cache frame rollback or stream switching strategy, switching to a backup image source or maintaining the previous valid frame to ensure the continuity and stability of the video stream, guarantee the continuity and service availability of the output video stream, and significantly enhance the robustness and service continuity of the system under complex sea conditions. The specific tasks are as follows: The stitched and merged panoramic images are encoded and transmitted through the GStreamer pipeline. Specifically, the original image data is converted to H.264 format using GStreamer's videoconvert and x264enc plugins. Compression efficiency and bandwidth usage are optimized by adjusting the bitrate, frame rate, and GOP structure. The encoded video stream is then pushed to the embedded streaming media server (RTSPSimpleServer or GStreamer) through the rtspclientsink plugin. The streaming media server is configured with multi-session management, supporting command interactions such as DESCRIBE, SETUP, and PLAY of the standard RTSP protocol (RTSP / 1.0). Simultaneously, the streaming media server enables TCP transmission mode to enhance network penetration and dynamically updates streaming media parameters through SDP files, ensuring that clients can pull and decode the panoramic video stream in real time, achieving low-latency remote monitoring and visualization applications. While publishing video streams externally, the streaming media server provides a local visualization interface with status information. Using the GStreamer tee plugin, the raw video stream before encoding is branched to the local rendering pipeline, and real-time status information, including camera connection status, stitching and fusion progress, encoding frame rate, network bandwidth usage, and abnormal alarms, is overlaid using the cairooverlay or gtkpaintable plugins. The visualization interface adopts a layered design, with a panoramic video preview window at the bottom and a semi-transparent status bar at the top. Dynamic refresh ensures information timeliness and supports logging and backtracking functions. Yes, it can write key events (stream interruption, rollback switching) into the database. In addition, the visual interface is decoupled from the streaming media server and runs on an independent thread to avoid blocking the video stream processing link. It integrates multi-level anomaly detection and intelligent rollback strategy to ensure the continuity of video stream. It monitors the camera connection status through a heartbeat detection mechanism. If no data is received within a timeout, a stream interruption alarm is triggered. In the stitching and fusion stage, the processing result is judged by verifying the matching degree of image feature points. If it fails, it is marked as an abnormal frame. Then, in the encoding stage, the generation interval of H.264 keyframes (I-frames) is monitored. If they are not generated within a timeout, they are judged as encoding errors. When an anomaly is detected, the cached frame rollback strategy is automatically activated. The most recent valid frame is read from the preset circular buffer for filling, or the system is switched to a backup image source (another camera or a static background image). At the same time, the dynamic_pad and valve plugins of GStreamer are used to realize dynamic switching of the stream path to ensure seamless connection of the rollback process. The current rollback status and reason are displayed in real time through the status overlay interface, which improves the robustness and maintainability of the system. Multiple cameras (camera 1, camera 2... camera n) collect on-site video data and push the video stream to an edge computing server via RTSP (Real-Time Streaming Protocol). After receiving the video stream, the edge computing server decodes the video, converting the raw video data into a processable format. Subsequently, it performs sea-line detection to identify and extract key information or specific objects from the video. The video data is further processed by splicing and fusion, integrating the video streams from different cameras into a unified view. The encoded video data is then pushed to the streaming media server again via the RTSP protocol. The streaming media server distributes the encoded video stream to different terminal devices, such as the driver's cab control console, shore-based monitoring systems, and other terminals. After receiving the video stream, each terminal device performs real-time monitoring and necessary operational control, realizing remote monitoring and management.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A shipborne panoramic video stitching method based on distributed multi-camera systems, characterized in that, Includes the following steps: S1. Multiple network cameras are deployed at the bow, stern, port and starboard sides of the ship to collect multiple video streams in real time via the RTSP protocol, and to decode and cache the frames synchronously. S2. Perform dark channel dehazing, image sharpening, OTSU binarization, Canny edge detection and edge thinning on each video stream in sequence. S3. Extract the sea-line antenna from the edge map based on probabilistic Hough transform and RANSAC fitting, and output the sea-line antenna angle and position stably through frame synchronization and camera parameter calibration. S4. Using the bow view of the ship as the reference for the panoramic canvas, the y-axis translation is corrected according to the horizon height of each view, and horizontal stitching is achieved by combining feature point matching to construct a 360° circular panoramic image. S5. A multi-band fusion algorithm is used to construct Gaussian and Laplacian pyramids, and combined with a linear weight mask to fuse images in the frequency domain in layers, eliminating stitching seams and obtaining a seamlessly fused panoramic image. S6. The stitched panoramic image is encoded into a standard streaming media format in real time and pushed to a network streaming media server via the RTSP protocol for access by multiple terminals.

2. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: S1 specifically includes: A set of standard network cameras is deployed at the bow, stern, port side, and starboard side of the hull to form a distributed visual acquisition array. Each network camera broadcasts its acquired H.264 encoded video stream within the local area network via the RTSP protocol. The edge computing server deployed on the ship concurrently pulls multiple RTSP video streams through the GStreamer framework. GStreamer performs RTSP streaming and decoding, while OpenCV processes the raw video frames and caches the image frames in a synchronous memory queue. GPS timing is used to align multiple image frames in the cache, and the intrinsic and extrinsic parameters of each network camera are pre-calibrated to establish a mapping relationship between the camera's logical index and its physical location.

3. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: S2 specifically includes: For each video stream, an image dehazing algorithm based on dark channel prior is executed sequentially to restore the contrast attenuation caused by sea fog or backlighting; A predefined high-pass sharpening convolution kernel is used to perform spatial domain sharpening on the dehazed image to enhance its edge and contour information. Then, the image is converted to grayscale and binarized by applying the OTSU adaptive thresholding algorithm to obtain a binarized image. The Canny operator is applied to the binarized image for fine edge extraction, and a morphological skeletonization algorithm is further used to refine the extracted edges, generating an edge map with clear structure and accurate topology with a single pixel width.

4. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: In step S3, the process of extracting the sea-line area from the edge map includes: Apply the probabilistic Hough transform to the edge map to detect all candidate line segments that meet the minimum length threshold, calculate the parameters of each line segment in the image coordinate system, and integrate them to obtain a set of candidate line segments. A robust fitting of the candidate line segment set is performed using a random sampling consensus algorithm to filter out noisy line segments, determine a main sea-sky line representing the sea-sky boundary, and output its angle parameters. A historical sea-line detection buffer with a multi-frame sliding window is maintained for each network camera. By comparing the center height of the sea-line with that of the current frame and the historical frames, abnormal jumps caused by instantaneous interference are filtered out, and a stable sea-line position and angle are output.

5. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 4, characterized in that: In step S3, the process of ensuring temporal consistency and geometric alignment of video streams from different viewpoints includes: Based on the stable position and angle of the sea surface line, and combined with the pre-calibrated camera extrinsic parameters, the geometric relationship of the sea surface line in the unified world coordinate system from the perspective of each network camera is calculated. The sea-line is used as a globally consistent structural feature to provide a geometric constraint benchmark for stitching and aligning multi-view images, replacing unstable local feature points in low-texture sea surface scenes. The geometric consistency of the sea-line at various viewpoints is continuously monitored. If the sea-line at a certain viewpoint is lost or deviates too much, the anomaly handling mechanism is triggered, and the historical sea-line cache is used for interpolation to overwrite it.

6. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: In step S4, the process of y-axis translation correction based on the sea-line height from each viewpoint includes: The viewpoint of the bow camera was selected as the reference frame for the panoramic canvas, and the reference height of its horizon line in the canvas was calculated. Calculate the center height of the sea horizon in each of the remaining viewpoints, and then calculate the required vertical translation for each image. Based on the vertical translation amount, an affine transformation matrix is ​​constructed for each image to be stitched together, and the image is translated and corrected in the y-axis direction to achieve precise alignment of all viewpoint images at the sea-line height.

7. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 6, characterized in that: In step S4, the process of constructing a 360° panoramic view includes: Based on the completion of the sea-line height alignment, local feature point detection and matching are performed in the overlapping area of ​​the image to correct minor horizontal misalignment; Following the spatial logical sequence from the port side, through the stern, to the starboard side, the geometrically corrected multi-channel images are sequentially stitched onto a circular canvas centered on the bow viewpoint. By combining the height alignment with the horizontal feature matching, a 360° circular panoramic image that is consistent in the vertical direction and coherent in the horizontal direction is formed.

8. The shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: In S5, the process of constructing the Gaussian and Laplace pyramids includes: For each image to be fused in the 360° circular panoramic image, a Gaussian pyramid is constructed, and Gaussian smoothing and downsampling are performed iteratively to obtain a set of images with decreasing resolution. Based on the Gaussian pyramid, a corresponding Laplacian pyramid is constructed by upsampling each layer of the image and subtracting it from the previous Gaussian image, thus decomposing the image into different frequency bands. Each layer of the Laplacian pyramid carries the details and texture information of the image at different spatial scales.

9. A shipborne panoramic video stitching method based on distributed multi-cameras according to claim 8, characterized in that: In step S5, the process of eliminating stitching seams by combining linear weighted masks to perform layered image fusion in the frequency domain includes: Define a linearly gradient weight mask for the overlapping regions of the images. The weight mask has a weight of 0 at the left boundary of the overlapping region and a weight of 1 at the right boundary, with a linear transition in between. On each layer of the Laplacian pyramid, the weight mask is used to perform a weighted average on the corresponding layers of the two images to be fused, and the fusion result of that layer is obtained. Starting from the top layer, the merged Laplacian pyramid is upsampled layer by layer and superimposed with the fusion result of the next layer to reconstruct a seamlessly merged panoramic image.

10. A shipborne panoramic video stitching method based on distributed multi-cameras according to claim 1, characterized in that: S6 specifically includes: The stitched and merged panoramic images are re-encoded into H.264 format through the GStreamer pipeline and pushed to the embedded streaming media server via the RTSP protocol; The streaming media server publishes the panoramic video stream to the outside world, supports clients using the standard RTSP protocol to pull and play it in real time, and provides a visual display interface with status information overlay locally. The design incorporates an anomaly detection and intelligent rollback mechanism. When a camera disconnection, stitching failure, or encoding error is detected, the system automatically activates a buffer frame rollback or stream switching strategy to switch to a backup image source or maintain the previous valid frame.

Images