Ship spherical global image splicing method and system based on image processing

Abstract

The invention discloses a ship spherical global image splicing method and system based on image processing, and the method comprises the following steps: obtaining multi-camera images which are distributed at the top, the bottom and the annular direction, and executing camera calibration and distortion correction; performing hemispherical projection and cylindrical projection by using a hybrid projection link; feature point extraction and matching are carried out, and an initial registration relation between adjacent view fields is established; executing APAP gridding local registration of projection perception, and generating local homography by applying an anisotropic weight kernel consistent with the region projection direction to obtain a partition alignment result set; carrying out weighted fusion and stitching suppression on the overlapped region to generate a panoramic spliced image; and converting the panoramic spliced image into an interactive three-dimensional panoramic view. According to the method, mixed projection and projection perception APAP algorithms are fused, ship spherical surface global high-precision splicing and three-dimensional immersion display are achieved, and the method has the advantages of being small in distortion, high in precision and high in visibility.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a ship spherical global image stitching method and system based on image processing. BACKGROUND

[0002] In the ship navigation scene, in order to realize the all-around observation of the ship domain, the common method is to perform multi-camera image stitching based on a single projection. The cylindrical projection is convenient for the video plane expansion of the ring camera and is suitable for vehicle ring view, but when facing the spherical global coverage, the distortion of the upper and lower hemisphere regions is significant. Although the spherical projection can be compatible with all-around information, it is difficult to obtain ideal results on the conventional plane display terminal due to the non-uniform arrangement of pixels. Limited by such single projection path, the existing method cannot balance between accuracy and display adaptation, especially in the control of geometric deformation of the top and bottom view of the ship body.

[0003] In addition, the traditional stitching relies on global homography or unified polynomial distortion model, which is difficult to deal with local misplacement caused by mixed near / far view and parallax. Although the common local registration can alleviate the problem, it is not combined with the directional weighting of projection geometry, which is easy to produce gaps and edge breaks in the polar region and equatorial zone. The multi-camera calibration, time alignment and overlap zone management lack integrated organization for the "top / bottom / ring" layout, the processing of luminosity consistency and stitching path is not fine enough in the fusion stage, and there is also a lack of quality evaluation superposition tightly coupled with three-dimensional interactive display, which is difficult to form an engineering closed loop considering seamless stitching and immersive observation.

[0004] Therefore, how to provide a ship spherical global image stitching method and system based on image processing is a problem that those skilled in the art need to solve. SUMMARY

[0005] One object of the present application is to provide a ship spherical global image stitching method and system based on image processing. The present application combines hybrid projection and projection perception APAP algorithm to realize high-precision stitching and three-dimensional immersive display of ship spherical global image, and has the advantages of small distortion, high precision and strong visibility.

[0006] According to the ship spherical global image stitching method based on image processing, the method comprises the following steps: Obtain the top, bottom and ring distributed multi-camera images, perform camera calibration and distortion correction, divide into upper and lower hemisphere regions and middle ring belt region to form a partitioned input set; Apply a hybrid projection link to the partitioned input set, perform a hemisphere projection on the upper and lower hemisphere regions, and perform a cylindrical projection on the middle ring belt region to obtain a partitioned projection result set; Feature point extraction and matching are performed on the partition projection result set to establish an initial registration relationship between adjacent view fields and generate an initial registration result; Based on the initial registration result, projection-aware APAP meshing local registration is performed to generate local homographies by applying anisotropic weight kernels consistent with the regional projection direction to the upper and lower hemispherical regions and the middle annular region to obtain a partition alignment result set; The partition alignment result set is subjected to weighted fusion and seam suppression in the overlapping region to generate a panoramic mosaic image; The panoramic mosaic image is converted into an equirectangular projection scale and attached to a virtual sphere to obtain an interactive three-dimensional panoramic view.

[0007] Optionally, the generation of the partition input set specifically includes: The top, bottom, and circumferentially distributed multi-camera images are acquired, and the time stamp and trigger sequence of each camera are recorded to establish a time alignment index table, forming a raw multi-camera image set to be calibrated; Camera calibration is performed on the raw multi-camera image set to be calibrated, multi-view image frames are collected using a planar calibration board, a spatial mapping relationship from the world coordinate to the camera coordinate is described by a rotation matrix and a translation vector, and an imaging projection relationship from the camera coordinate to the pixel coordinate is described by an intrinsic matrix, and a camera calibration result containing intrinsic parameters, extrinsic parameters, and re-projection error evaluation is output; Distortion correction parameter estimation is performed based on the camera calibration result to establish a functional relationship between radial distortion and tangential distortion, and a parameter set for distortion correction is obtained; According to the parameter set for distortion correction, distortion correction is performed on the top, bottom, and circumferentially distributed multi-camera images frame by frame, and a distortion correction image set is output; According to the camera installation pose and the field of view coverage relationship, the intersection area of each camera view cone and the spherical reference surface is calculated to determine the zoning boundaries of the upper hemispherical region, the lower hemispherical region, and the middle annular region, and a spatial division rule is formed; The spatial division rule requires that the union of the upper hemispherical region, the lower hemispherical region, and the middle annular region covers the entire spherical surface, and the upper hemispherical region and the middle annular region do not overlap, and the lower hemispherical region and the middle annular region do not overlap; According to the spatial division rule and the zoning boundaries, the distortion correction image set is subjected to regional segmentation to generate image subsets divided according to the upper and lower hemispherical regions and the middle annular region, and a mapping relationship between the image subsets and the camera calibration result is established; Quality screening is performed on the image subsets to remove motion blur frames and overexposed frames to form valid frame sequences organized according to the upper and lower hemispherical regions and the middle annular region; The valid frame sequences organized according to the upper and lower hemispherical regions and the middle annular region are associated and packaged with the camera calibration result and the time alignment index table to output the partition input set.

[0008] Optionally, the generating of the set of partition projection results specifically comprises: According to the set of partition inputs, for each pixel of each frame, a unit spherical direction vector is calculated according to the camera intrinsic and extrinsic parameters, the pixel coordinates are mapped to a normalized imaging plane through the inverse of the intrinsic matrix, and are converted to the camera reference direction through the inverse rotation transformation of the extrinsic parameters, and are normalized according to the Euclidean norm, to obtain a set of unit spherical directions and a corresponding region mask; The longitude and latitude of the set of unit spherical directions are calculated according to the region mask, and a set of spherical parameters is output; The set of spherical parameters located in the upper hemisphere region and the lower hemisphere region are executed hemispherical projection to generate an upper hemispherical projection plane coordinate field and resample to obtain an upper hemispherical projection atlas and a lower hemispherical projection atlas; The set of spherical parameters located in the middle annulus region are executed cylindrical projection to generate a middle annulus projection plane coordinate field and resample to obtain a middle annulus projection atlas; The middle annulus projection plane coordinate field is executed angle unfolding and wrapping normalization to generate a set of unfolded longitudes through the polar seamless mapping; The upper hemispherical projection atlas, the lower hemispherical projection atlas and the middle annulus projection atlas are spatially registered and boundary index aligned according to the region mask, combined into a set of partition projection results, and an overlap strip mapping relationship table is established for the overlapping strips of adjacent view fields.

[0009] Optionally, the generating of the initial registration result specifically comprises: The upper hemispherical projection atlas, the lower hemispherical projection atlas and the middle annulus projection atlas are read from the set of partition projection results, combined with the set of region projection metadata and the overlap strip mapping relationship table, and a list of adjacent view field pairs is generated according to the principle of grouping two adjacent view fields in the same region, and an overlap strip clipping window and a boundary index are established for each pair of adjacent view fields; Feature point extraction and descriptor generation are performed in the corresponding overlap strip clipping window for each projection image in the list of adjacent view field pairs, feature point extraction and scale invariant descriptor encoding are performed using the scale invariant feature transform, and a set of feature points containing key point positions, scales and principal directions and a corresponding descriptor matrix are output; Nearest neighbor and second nearest neighbor matching are performed in the overlap strip range for each pair of projection images in the list of adjacent view field pairs, the Euclidean distance between each descriptor and its nearest neighbor descriptor is calculated as a first distance, and the Euclidean distance between the second nearest neighbor descriptor is calculated as a second distance, and the matching pair with a ratio of the first distance to the second distance not greater than a preset threshold is taken as a candidate matching pair; The geometric consistency screening is performed on the candidate matching pairs, a homography is estimated on the projection plane by using a random sample consensus method, such that the homogeneous pixel coordinates in one projection image are equal to the homogeneous pixel coordinates in another projection image in a proportional sense after a homography transformation, and geometric outliers not satisfying the geometric consistency screening are removed to obtain a geometric consistent matching set; An optimization problem with a minimum re-projection error square sum as a target is established on the geometric consistent matching set to obtain an initial homography matrix, and a corresponding inlier set index, an inlier proportion, and an overlap band boundary index are recorded to form an initial homography estimation result for adjacent view fields; The geometric consistent matching set, the initial homography matrix, the inlier proportion, the overlap band boundary index, and the feature point to pixel reverse search index are summarized for each pair of adjacent view fields, and are organized into an initial registration result according to an upper hemisphere region, a lower hemisphere region, and a middle annular band region.

[0010] Optionally, the generation of the partition alignment result set specifically includes: The adjacent view field pairing list, the geometric consistent matching set, and the initial homography matrix are read from the initial registration result, and the upper hemisphere projection image set, the lower hemisphere projection image set, and the middle annular band projection image set, and the region projection metadata set are read from the partition projection result set, and an overlap band sub-block set to be locally registered is generated according to the overlap band boundary index of adjacent cameras in the same region; Grid division is performed on each overlap band to be locally registered on a corresponding projection plane to obtain an APAP grid vertex and a grid cell index, and the initial homography matrix is taken as an initial value of each grid cell to form a grid initialization result taking the initial homography matrix as an a priori; An anisotropic weight kernel direction field consistent with the region projection direction is constructed for the upper hemisphere region, the lower hemisphere region, and the middle annular band region respectively, the main weight direction is set along the longitude and latitude directions for the upper hemisphere region and the lower hemisphere region, the main weight direction is set along the cylindrical expansion direction for the middle annular band region, and a projection-aware anisotropic weight kernel parameter set is output; At each grid vertex, a vertex weighting coefficient is calculated based on the anisotropic weight kernel parameter set according to the projection plane distance and the included angle of the main weight direction to the matching points in the geometric consistent matching set, and all the weighting coefficients at the same vertex are normalized to obtain a vertex weighting distribution; Weighted least squares estimation is performed on the geometric consistent matching set in each grid cell with the vertex weighting distribution as a weight, a local homography of the grid cell is updated according to the grid initialization result, and a corresponding relationship between the grid cell and the local homography is recorded; The local homographies of adjacent grid cells are continuously constrained and interpolated according to the shared boundary, a weight interpolation function taking the distance to the boundary as an independent variable is used to smoothly fuse the local homographies to generate a local homography interpolation field; According to the local homographic interpolation field, the dependent view in the adjacent view is resampled to the coordinate system of the main view, and a sub-block registration result after local alignment is output. The sub-block registration result is associated with the overlap strip boundary index and the inverse search index of the feature points to the pixels, and a sub-block level partition alignment intermediate result is obtained; The boundary splicing and gap detection are performed on all sub-block level partition alignment intermediate results of the same pair of adjacent views. If a local under-constrained area is found, the vertex weighting distribution and the local homography are recalculated, so that the sub-block level partition alignment intermediate result meets the preset continuity criterion. The view level partition alignment results of all adjacent views in the same area are regionally summarized to form a partition alignment result set.

[0011] Optionally, the generation of the panoramic stitched image specifically includes: According to the partition alignment result set, the overlap region candidate set is generated according to the overlap strip boundary index of the main view and the dependent view in the same area in combination with the region projection metadata set and the overlap strip mapping relationship table, and a pixel-level index is established for each overlap region. The photometric consistency parameter is estimated in the overlap region candidate set, the exposure and color gain alignment of the dependent view is performed, and a photometric alignment image set is output. Based on the photometric alignment image set, a stitching energy graph is constructed, the intensity difference term, the gradient difference term and the edge continuity term are calculated for each overlap region, and are linearly weighted, and a stitching energy graph is output. The stitching path optimization is performed on the stitching energy graph, the stitching path set is generated in each overlap region by using the energy minimum path search, and the stitching strip mask and the distance field of the path to the pixel are generated. According to the stitching strip mask and the distance field, a fusion weight field is constructed, the exponential function form distance decay value of the main view and the dependent view to the stitching path is calculated respectively, the fusion weight is obtained after weighted normalization of the two, and the occlusion priority corresponding to the overlap strip boundary index is superimposed, and a fusion weight field is output. A multi-scale pyramid is constructed for each overlap region, the Laplace component is weighted and fused according to the fusion weight field at each scale, and the Gaussian component is fused at the highest layer, the layer-by-layer reconstruction is completed, and a sub-block level fusion result set is output. The stitching suppression and artifact detection are performed on the sub-block level fusion result set, the fusion intensity residual and the gradient residual are calculated and compared with the preset threshold, if the threshold is exceeded, the fusion weight field and the stitching path are recalculated in the corresponding overlap strip range, and a view level fusion result set is output. The intensity residual is equal to the sum of the absolute values of the intensity difference between the fused pixel and the pixels of the main view and the dependent view, and the gradient residual is equal to the sum of the absolute values of the gradient difference between the fused pixel and the pixels of the main view and the dependent view. Perform regional summary and boundary splicing on all the field-level fusion result sets in the same region, respectively obtain upper hemisphere regional fusion result, lower hemisphere regional fusion result and middle ring zone regional fusion result, perform end splicing at the regional boundary according to the seamless transition rule, output the panoramic mosaic picture, and save the set of stitching path, fusion weight field and residual statistics.

[0012] Optionally, the generation of the three-dimensional panoramic view specifically comprises: Read the panoramic mosaic picture and its corresponding set of stitching paths, fusion weight field and residual statistics, generate the panoramic mosaic picture metadata containing luminosity, geometry and stitching information; Establish an equirectangular projection scale reference, expand the panoramic mosaic picture according to a preset width-height ratio, define a linear mapping relationship between longitude and latitude, convert the spherical coordinates into equirectangular plane coordinates, perform resampling and periodic wraparound, and obtain an equirectangular projection picture; Construct a vertex coordinate grid of a virtual sphere, generate a three-dimensional vertex set according to the longitude and latitude distribution of the equirectangular projection picture, and attach the equirectangular projection picture to the surface of the virtual sphere to establish a mapping relationship between pixels and textures, and output the virtual sphere with textures; Establish an interactive view angle control interface, define the heading angle and the pitch angle as the main interactive inputs, calculate the corresponding view rotation transformation and apply it to the virtual sphere, and realize the interactive browsing and attitude switching of the three-dimensional panoramic view; In the interactive rendering process, generate a visible continuity evaluation overlay layer according to the residual statistics and the fusion weight field, display it in a semi-transparent form, and use it to display the continuity state of the panoramic mosaic region in real time, and output the interactive three-dimensional panoramic view.

[0013] The ship spherical global image splicing system based on image processing according to the embodiment of the application comprises: The partition input generation module is configured to acquire top, bottom and ring-distributed multi-camera images, perform camera calibration and distortion correction, and divide the upper hemisphere, lower hemisphere and middle ring zone regions according to the camera installation pose; The hybrid projection module is configured to perform hemispherical projection and cylindrical projection, complete longitude and latitude coordinate mapping and regional resampling; The feature matching module is configured to extract feature points and establish a matching relationship between adjacent fields, calculate an initial homography matrix according to geometric consistency, and form an initial registration result; The projection-aware registration module is configured to perform projection-aware APAP grid-based local registration, apply an anisotropic weight kernel consistent with the projection direction to different regions, and generate a set of partition alignment results; The weighted fusion module is configured to perform luminosity alignment, energy minimum path search and multi-scale pyramid weighted fusion in the overlapping region of the set of partition alignment results, and output the panoramic mosaic picture. The panoramic mapping module is used to convert the panoramic stitched image into an equidistant rectangular projection scale and attach it to a virtual sphere, establish texture mapping and interactive viewpoint control, and realize the generation of a 3D panoramic view.

[0014] The beneficial effects of this invention are: This invention introduces a hybrid projection link combining hemispherical and cylindrical projections into the global spherical stitching of a ship, and integrates a projection-aware APAP gridded local registration algorithm. This enables images from different regions to achieve geometric alignment and continuous stitching within the optimal projection space, significantly reducing distortion in the upper and lower hemispheres and misalignment in the central ring region, achieving a high-precision, low-distortion global stitching effect. Compared to traditional single projection or global homography models, this invention can adaptively adjust the weighted kernel distribution according to the projection direction of each region, resulting in smoother stitching transitions and more natural seam edges, ensuring stable and consistent visual results even under complex ship motion and varying lighting conditions.

[0015] Furthermore, in the fusion output stage, this invention converts the panoramic stitched image into an equidistant rectangular projection image with a 2:1 scale and maps it onto a virtual sphere, achieving a 3D rotatable and interactive panoramic display. This allows crew members to intuitively observe the ship's surrounding environment from multiple perspectives. This 3D spherical view conforms to the human eye's perception of spatial scenes, facilitating rapid identification of surrounding obstacles and route information, significantly enhancing the immersive experience and safety of monitoring. The system operates in a closed-loop manner throughout the entire process, from image acquisition, projection transformation, registration and fusion to 3D visualization. The modules are tightly coupled, and the processing links are efficient and reliable, meeting the comprehensive requirements of real-time performance, accuracy, and visibility during ship navigation. It possesses excellent engineering adaptability and promotional value. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0017] Fig. 1 This is a flowchart of a method for stitching together a ship's spherical global image based on image processing, as proposed in this invention. Fig. 2 This is a schematic diagram illustrating the generation of the partition alignment result set of a ship spherical global image stitching method based on image processing proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer toFigs. 1-2 A method for stitching together a spherical global image of a ship based on image processing includes the following steps: Acquire multi-camera images distributed at the top, bottom, and circumference, perform camera calibration and distortion correction, and divide the input into partitioned regions according to the upper and lower hemisphere regions and the middle ring region; Apply a hybrid projection link to the partitioned input set, perform hemispherical projection on the upper and lower hemispheres, and cylindrical projection on the middle ring region to obtain the partitioned projection result set; Feature points are extracted and matched on the partitioned projection result set to establish the initial registration relationship between adjacent fields of view and generate the initial registration result. Based on the initial registration results, APAP meshed local registration with projection sensing is performed. Anisotropic weight kernels consistent with the region projection direction are applied to the upper and lower hemisphere regions and the middle ring region to generate local homography, resulting in a set of partition alignment results. The partition alignment result set is weighted and fused in the overlapping area and stitching suppression is applied to generate a panoramic stitched image. The panoramic image is converted into an equidistant rectangular projection scale and attached to a virtual sphere to obtain an interactive 3D panoramic view.

[0020] In this embodiment, the generation of the partition input set specifically includes: Acquire multi-camera images distributed at the top, bottom, and circumferential directions, record the timestamp and trigger sequence of each camera, establish a time alignment index table, and form a set of original multi-camera images to be calibrated; Camera calibration is performed on the original multi-camera image set to be calibrated. Multi-view image frames are acquired using a planar calibration board. The spatial mapping relationship from world coordinates to camera coordinates is described by rotation matrix and translation vector. The imaging projection relationship from camera coordinates to pixel coordinates is described by intrinsic parameter matrix. The output is a camera calibration result including intrinsic parameters, extrinsic parameters and reprojection error evaluation. Based on the camera calibration results, distortion correction parameters are estimated, a functional relationship between radial distortion and tangential distortion is established, and a set of distortion correction parameters is obtained. Based on the set of distortion correction parameters, distortion correction is performed frame by frame on multi-camera images distributed in the top, bottom, and circumferential directions, and a set of distortion-corrected images is output. Based on the relationship between the camera mounting pose and the field of view coverage, the intersection area between the camera's viewing cone and the spherical reference surface is calculated, and the boundaries of the upper hemisphere region, lower hemisphere region, and middle ring zone region are determined to form spatial division rules. The spatial division rule requires that the upper hemisphere region, the lower hemisphere region, and the middle ring region be combined to cover the entire sphere, and that the upper hemisphere region and the middle ring region do not overlap, and the lower hemisphere region and the middle ring region do not overlap. Based on spatial division rules and zoning boundaries, the distortion-corrected image set is divided into regions to generate image subsets divided according to the upper and lower hemisphere regions and the middle ring region, and a mapping relationship between the image subsets and the camera calibration results is established. A quality screening is performed on a subset of images to remove motion-blurred and overexposed frames, forming an effective frame sequence organized by the upper and lower hemisphere regions and the middle ring region; The effective frame sequences organized by upper and lower hemisphere regions and middle ring region are associated and packaged with camera calibration results and time alignment index table to output a partitioned input set.

[0021] In this embodiment, the generation of the partition projection result set specifically includes: Based on the partitioned input set, the unit spherical orientation vector is calculated for each pixel of each frame according to the camera intrinsic and extrinsic parameters. The pixel coordinates are mapped to the normalized imaging plane through the inverse mapping of the intrinsic parameter matrix, and transformed to the camera reference direction through the inverse rotation transformation of the extrinsic parameters. The vector is then normalized according to its Euclidean norm to obtain the unit spherical orientation set and the corresponding region mask. ; in, Represents the direction vector of the unit sphere. Represents the rotation matrix. This represents the intrinsic parameter matrix, where u and v represent the x and y coordinates of pixels in the image coordinate plane. Denotes the Euclidean norm; Calculate the longitude and latitude of the unit spherical orientation set according to the regional mask, and output the spherical parameter set; Perform hemispherical projection on the spherical parameter set located in the upper and lower hemisphere regions to generate the upper hemisphere projection plane coordinate field and resample it to obtain the upper hemisphere projection map set and the lower hemisphere projection map set; Perform cylindrical projection on the spherical parameter set located in the middle ring zone region to generate the middle ring zone projection plane coordinate field and resample it to obtain the middle ring zone projection atlas. Angle expansion and wrap-around normalization are performed on the projection plane coordinate field of the central ring zone, and the expanded longitude set is generated through seamless polar mapping. The upper hemisphere projection atlas, lower hemisphere projection atlas, and middle ring zone projection atlas are spatially registered and aligned with boundary indices using regional masks. They are then combined with the middle ring zone projection atlas to form a set of partitioned projection results. An overlapping zone mapping table is also established for overlapping zones of adjacent fields of view.

[0022] In this embodiment, the generation of the initial registration result specifically includes: Read the upper hemisphere projection atlas, lower hemisphere projection atlas and middle ring zone projection atlas from the partitioned projection result set. Combine the regional projection metadata set and the overlapping zone mapping table. Generate a pairing list of adjacent fields of view according to the principle of pairing adjacent camera fields of view in the same region. Establish an overlapping zone clipping window and boundary index for each pair of adjacent fields of view. For each projection map in the adjacent field-of-view pairing list, feature point extraction and descriptor generation are performed within its corresponding overlap band clipping window. Scale-invariant feature transformation is used for feature point extraction and scale-invariant descriptor encoding. The output is a set of feature points containing key point positions, scales and main directions, as well as the corresponding descriptor matrix. For each pair of projection maps in the adjacent field-of-view pairing list, perform nearest neighbor and second nearest neighbor matching within the overlapping band. Calculate the Euclidean distance between each descriptor and its nearest neighbor descriptor as the first distance, and calculate the Euclidean distance between each descriptor and its second nearest neighbor descriptor as the second distance. Select matching pairs whose ratio of the first distance to the second distance is not greater than a preset threshold as candidate matching pairs. The candidate matching pairs are subjected to geometric consistency screening. The homography relationship is estimated on the projection plane using the random sampling consistency method, so that the homography coordinates of the homography in one projection map are equal to the homography coordinates of the homography in another projection map in a proportional sense after homography transformation. Geometric out-of-points that do not meet the geometric consistency screening are eliminated to obtain the geometrically consistent matching set. An optimization problem is established on the geometrically consistent matching set with the objective of minimizing the sum of squared reprojection errors. The initial homography matrix is ​​obtained, and the corresponding inlier set index, inlier ratio and overlap zone boundary index are recorded to form the initial homography estimation result for adjacent fields of view. For each pair of adjacent fields of view, a geometrically consistent matching set, initial homography matrix, inlier ratio, overlap zone boundary index, and feature point to pixel lookup index are compiled, and the initial registration results are organized into upper hemisphere region, lower hemisphere region, and middle ring zone region.

[0023] In this embodiment, the generation of the partition alignment result set specifically includes: Read the adjacent field of view pairing list, geometrically consistent matching set and initial homography matrix from the initial registration result. At the same time, read the upper hemisphere projection atlas, lower hemisphere projection atlas and middle ring zone projection atlas and regional projection metadata set from the partitioned projection result set. Generate the overlapping zone sub-block set to be locally registered based on the overlapping zone boundary index of adjacent camera fields of view in the same region. For each overlapping band to be locally registered, meshing is performed on the corresponding projection plane to obtain APAP mesh vertices and mesh cell indices. The initial homography matrix is ​​used as the initial value of each mesh cell to form a mesh initialization result with the initial homography matrix as a priori. For the upper hemisphere region, lower hemisphere region and middle ring zone region, respectively construct an anisotropic weight kernel direction field consistent with the regional projection direction. The sovereign weight direction is set along the latitude and longitude direction in the upper hemisphere region and the sovereign weight direction is set along the cylindrical unfolding direction in the middle ring zone region. Output the anisotropic weight kernel parameter set of projection perception. At each grid vertex, based on the anisotropic weight kernel parameter set, the vertex weighting coefficient is calculated according to the angle between the distance of its projection plane to the matching point in the geometrically consistent matching set and the sovereign weight direction. All weighting coefficients at the same vertex are then normalized to obtain the vertex weighting distribution. ; in, Represents the vertex weighting coefficient. Indicates the distance in the projection plane along the sovereign weight direction. This represents the distance to the projection plane in a direction orthogonal to the sovereign direction. Indicates the sovereign weight direction scale parameter. This represents the orthogonal direction scale parameter. ; Using the vertex weighted distribution as the weight, a weighted least squares estimation is performed on the geometrically consistent matching set within each grid cell. The local homography of the grid cell is updated based on the grid initialization results, and the correspondence between the grid cell and the local homography is recorded. For the local homography of adjacent grid cells, continuous constraint interpolation is performed according to the shared boundary. A weighted interpolation function with the distance to the boundary as the independent variable is used to smoothly fuse the local homography and generate a local homography interpolation field. Based on the local homography interpolation field, the subordinate fields of view in the adjacent fields of view are resampled to the main field of view coordinate system, and the locally aligned sub-block registration results are output. The sub-block registration results are then associated and stored with the overlapping zone boundary index and the feature point to pixel reverse lookup index to obtain the sub-block level partition alignment intermediate results. Boundary stitching and gap detection are performed on all sub-block level partition alignment intermediate results of the same pair of adjacent fields of view. If a local under-constrained region is found, the vertex weighted distribution and local homography are recalculated to ensure that the sub-block level partition alignment intermediate results meet the preset continuity criteria. The field-level partition alignment results of all adjacent fields of view in the same region are summarized to form a partition alignment result set.

[0024] In this embodiment, the generation of the panoramic mosaic image specifically includes: Based on the partition alignment result set, combined with the regional projection metadata set and the overlapping zone mapping table, a candidate set of overlapping regions is generated according to the overlapping zone boundary index of the main field of view and the subordinate field of view in the same region, and a pixel-level index is established for each overlapping region. Estimate the photometric consistency parameter within the candidate set of overlapping regions, perform exposure and color gain alignment on the subordinate field of view, and output a photometric alignment atlas. A stitching energy map is constructed based on the photometric alignment atlas. For each overlapping region, the intensity difference term, gradient difference term, and edge continuity term are calculated and linearly weighted to output the stitching energy map. Perform stitch path optimization on the stitch energy map, use minimum energy path search to generate a set of stitch paths in each overlapping region, and generate a stitch band mask and a path-to-pixel distance field; A fusion weight field is constructed based on the stitching mask and the distance field. The distance attenuation value in the form of an exponential function to the stitching path is calculated for the main field of view and the subordinate field of view respectively. After weighted normalization of the two, the final fusion weight is obtained. At the same time, the occlusion priority corresponding to the boundary index of the overlapping band is superimposed, and the fusion weight field is output. For each overlapping region, a multi-scale pyramid is constructed. The Laplacian component is weighted and fused at each scale according to the fusion weight field, and the Gaussian component is fused at the highest level to complete the layer-by-layer reconstruction and output the sub-block level fusion result set. Perform stitch suppression and artifact detection on the sub-block level fusion result set, calculate the intensity residual and gradient residual after fusion and compare them with the preset threshold. If the threshold is exceeded, recalculate the fusion weight field and stitching path within the corresponding overlap zone and output the field-of-view level fusion result set. The intensity residual is equal to the sum of the absolute values ​​of the intensity differences between the fused pixel and the pixels in the main field of view and the subordinate field of view, and the gradient residual is equal to the sum of the absolute values ​​of the gradient differences between the fused pixel and the pixels in the main field of view and the subordinate field of view. For all field-level fusion results within the same region, perform region aggregation and boundary stitching to obtain the fusion results for the upper hemisphere region, the lower hemisphere region, and the middle ring region, respectively. At the region boundaries, perform end stitching according to the seamless transition rule to output the panoramic stitching map, and save the stitching path set, fusion weight field, and residual statistics.

[0025] In this embodiment, the generation of the three-dimensional panoramic view specifically includes: Read the panoramic mosaic image and its corresponding set of stitching paths, fuse the weight field and residual statistics, and generate panoramic mosaic image metadata containing photometric, geometric and stitching information; Establish an equidistant rectangular projection scale benchmark, unfold the panoramic mosaic image according to the preset width and height ratio, define the linear mapping relationship between longitude and latitude, convert the spherical coordinates into equidistant rectangular plane coordinates, perform resampling and periodic wrapping, and obtain the equidistant rectangular projection image. Construct a vertex coordinate grid for the virtual sphere, generate a 3D vertex set based on the latitude and longitude distribution of the isometric rectangular projection map, attach the isometric rectangular projection map to the surface of the virtual sphere, establish a pixel-to-texture mapping relationship, and output a textured virtual sphere. An interactive view control interface is established, with the heading angle and pitch angle defined as the main interactive inputs. The corresponding view rotation transformation is calculated and applied to the virtual sphere to realize interactive browsing and attitude switching of the three-dimensional panoramic view. During the interactive rendering process, a visual continuity evaluation overlay layer is generated based on residual statistics and fusion weight field. This overlay layer is then displayed in a semi-transparent form to showcase the continuity status of the panoramic stitching area in real time and output an interactive 3D panoramic view.

[0026] A ship spherical full-domain image stitching system based on image processing includes: The partitioned input generation module is used to acquire multi-camera images distributed in the top, bottom, and circumferential directions, perform camera calibration and distortion correction, and divide the upper hemisphere, lower hemisphere, and middle ring zone regions according to the camera mounting pose. The hybrid projection module is used to perform hemispherical projection and cylindrical projection, and to complete latitude and longitude coordinate mapping and region resampling; The feature matching module is used to extract feature points and establish matching relationships between adjacent fields of view. It also calculates the initial homography matrix based on geometric consistency to form the initial registration result. The projection-aware registration module is used to perform APAP meshed local registration of projection sensing, apply anisotropic weight kernels consistent with the projection direction to different regions, and generate a set of partition alignment results. The weighted fusion module is used to perform photometric alignment, minimum energy path search and multi-scale pyramid weighted fusion in the overlapping areas of the partition alignment result set, and output a panoramic mosaic image. The panoramic mapping module is used to convert the panoramic stitched image into an equidistant rectangular projection scale and attach it to a virtual sphere, establish texture mapping and interactive viewpoint control, and realize the generation of a 3D panoramic view.

[0027] Example 1: To verify the feasibility of this invention in practice, it was applied to the navigation monitoring scenario of a large ocean-going vessel. During its navigation from nearshore to deep sea, this vessel needs to achieve real-time monitoring of the entire spherical environment surrounding the hull to assist the navigator in course judgment and obstacle avoidance. Previous panoramic stitching methods were generally based on a single projection model. While cylindrical projection is convenient for planar display, it suffers from severe geometric distortion in the top and bottom areas of the vessel, resulting in obvious image stitching seams and spatial distortion. Spherical projection, although covering the entire field of view, suffers from uneven pixel distribution and severe image stretching in edge areas, which is detrimental to real-time display and interaction. This invention addresses these problems and was deployed and verified throughout the entire navigation monitoring system of this vessel.

[0028] In practical applications, multiple high-definition industrial cameras are deployed on the top, bottom, and circumferential sides of the ship's hull, acquiring multi-view image data through time synchronization and a unified triggering mechanism. The system first performs precise camera calibration and distortion correction on all camera images to ensure geometric consistency among the multi-source fields of view. Then, based on the camera installation pose, the images are divided into three regions: the upper hemisphere, the lower hemisphere, and the central ring zone, corresponding to the sky, water surface, and circumferential environment, respectively. For the geometric characteristics of different regions, the system employs a hybrid projection method combining hemispherical and cylindrical projections to generate a set of projection results on a spherical reference model, thereby reducing distortion while maintaining image coherence. The projected images undergo feature point extraction and matching, and the initial homography relationship between adjacent fields of view is estimated using a random sampling consensus algorithm, providing a foundation for subsequent fine-grained registration.

[0029] During the registration stage, the system executes the projection-aware APAP meshed local registration algorithm, applying anisotropic weight kernels according to the projection directions of the upper hemisphere, lower hemisphere, and central ring, ensuring a smooth transition of local deformations in the principal direction. This significantly improves the error accumulation problem in surface stitching using traditional global homography models. The generated partition alignment results undergo further photometric alignment, minimum energy path search, and multi-scale weighted fusion in the overlapping areas to ensure the continuity of brightness, color, and gradient features, effectively eliminating visible boundaries and artifacts at the seams. The final output panoramic stitched image is converted into a 2:1 scale equidistant rectangular projection map and attached to a virtual sphere, enabling a 3D interactive display of the panoramic image.

[0030] In the ship's bridge, the operator can freely rotate the viewing angle through a 3D visual interface to intuitively view the environment above, below, and around the hull. The system can overlay a semi-transparent evaluation layer containing photometric and stitching residual information in real time to reflect the continuity of the stitched image, providing a reliable basis for safety monitoring. After multiple consecutive voyages, the hybrid projection and projection sensing registration link of this invention maintained stable stitching accuracy and visual consistency under complex sea conditions. The stitched area transitions naturally, geometric errors are controllable, and the 3D panoramic view accurately reflects the spatial environment surrounding the ship. Application results show that this invention significantly improves the immersiveness and safety of ship monitoring, providing clear and stable visual environmental information under conditions of night navigation, fog, and multi-directional obstacles, providing a highly reliable image foundation for ship navigation auxiliary decision-making.

[0031] Table 1. Performance comparison between image processing-based ship spherical full-domain image stitching method and traditional methods.

[0032] As can be seen from the comparison results in Table 1, the image processing-based ship spherical global image stitching method proposed in this invention outperforms existing cylindrical projection, spherical projection, and improved global homography stitching methods in all core indicators. Regarding the average stitching error, this invention introduces a projection-aware APAP gridded local registration strategy, enabling continuous constraints on nonlinear transformations of images in different regions within the local grid. This reduces the stitching error to 1.26 pixels, improving accuracy by approximately 63% compared to the traditional cylindrical projection method, significantly enhancing the accuracy of geometric alignment. In terms of geometric distortion rate, thanks to the hybrid projection link using hemispherical projection for the upper and lower hemispheres and cylindrical projection for the central ring, this invention effectively suppresses edge stretching and parallax distortion, keeping the overall distortion rate at 3.7% and significantly improving visual realism.

[0033] In the seam visibility and photometric consistency scoring, this invention achieves a smooth transition of brightness and color across the field of view through a combination of photometric alignment, energy minimum path search, and multi-scale pyramid weighted fusion. The seam score reaches 9.1, and the photometric consistency score reaches 9.3. The image fusion is natural with no obvious transition traces, far exceeding the subjective quality level of traditional methods. At the same time, because this invention uses anisotropic weight kernels to guide the local registration stage, the APAP mesh converges faster and has higher resampling efficiency in the main projection direction. Therefore, while ensuring high accuracy, the stitching time is shortened to 2.4 seconds / frame, which is about 30% more efficient than the spherical projection method, meeting the real-time monitoring needs of ships during navigation.

[0034] Furthermore, in the 3D visualization stage, this invention maps the panoramic stitched image to a 2:1 scale equidistant rectangular projection and attaches it to a virtual sphere, achieving a high frame rate interactive display with a 3D visual smoothness of 30fps, significantly higher than traditional solutions. This result demonstrates that the hybrid projection link not only improves stitching quality but also ensures real-time rendering performance, enabling operators to observe the entire ship environment in 3D space with a smooth interactive experience. In summary, this invention, through the collaborative design of projection adaptation, geometric optimization, and fusion enhancement, successfully achieves breakthroughs in accuracy, stability, and visibility, providing a highly reliable and immersive visual solution for panoramic ship monitoring.

[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A ship spherical global image stitching method based on image processing, characterized in that, The method comprises the following steps: Obtaining top, bottom and annularly distributed multi-camera images, performing camera calibration and distortion correction, dividing into upper and lower hemispherical regions and middle annular regions to form a partitioned input set; Applying a hybrid projection link to the partitioned input set, performing hemispherical projection on the upper and lower hemispherical regions and cylindrical projection on the middle annular regions to obtain a partitioned projection result set; Performing feature point extraction and matching on the partitioned projection result set, establishing an initial registration relationship between adjacent fields of view, and generating an initial registration result; Based on the initial registration result, performing projection-aware APAP grid-based local registration, applying anisotropic weight kernels consistent with the projection direction of the upper and lower hemispherical regions and the middle annular regions to generate local homographies, and obtaining a partitioned alignment result set; Performing weighted fusion and seam suppression on the partitioned alignment result set in the overlapping region to generate a panoramic mosaic image; Converting the panoramic mosaic image into an equirectangular projection and attaching it to a virtual sphere to obtain an interactive three-dimensional panoramic view.

2. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the partitioned input set specifically comprises: Obtaining top, bottom and annularly distributed multi-camera images, recording the time stamp and trigger sequence of each camera respectively, establishing a time alignment index table, and forming a set of original multi-camera images to be calibrated; Performing camera calibration on the set of original multi-camera images to be calibrated, collecting multi-view image frames using a planar calibration board, describing the spatial mapping relationship from the world coordinate to the camera coordinate through a rotation matrix and a translation vector, and describing the imaging projection relationship from the camera coordinate to the pixel coordinate through an intrinsic matrix, and outputting the camera calibration result containing intrinsic parameters, extrinsic parameters and re-projection error evaluation; Based on the camera calibration result, performing distortion correction parameter estimation, establishing the functional relationship between radial distortion and tangential distortion, and obtaining a parameter set for distortion correction; Performing distortion correction on the top, bottom and annularly distributed multi-camera images frame by frame based on the parameter set for distortion correction, and outputting a set of distortion corrected images; According to the camera installation pose and the field of view coverage relationship, calculating the intersection region of each camera's view cone and the spherical reference surface, determining the zoning boundaries of the upper hemispherical region, the lower hemispherical region and the middle annular region, and forming a spatial division rule; The spatial division rule requires that the union of the upper hemispherical region, the lower hemispherical region and the middle annular region covers the entire spherical surface, and the upper hemispherical region and the middle annular region do not overlap, and the lower hemispherical region and the middle annular region do not overlap; According to the spatial division rule and the zoning boundaries, the set of distortion corrected images is regionally divided to generate image subsets divided according to the upper and lower hemispherical regions and the middle annular region, and a mapping relationship from the image subsets to the camera calibration result is established; Performing quality screening on the image subsets to remove motion blur frames and overexposed frames, and forming valid frame sequences organized according to the upper and lower hemispherical regions and the middle annular region; Associating and packaging the valid frame sequences organized according to the upper and lower hemispherical regions and the middle annular region with the camera calibration result and the time alignment index table to output the partitioned input set.

3. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the partitioned projection result set specifically comprises: According to the partition input set, a unit spherical direction vector of each pixel of each frame is calculated according to camera internal and external parameters, pixel coordinates are mapped to a normalized imaging plane through an inverse internal parameter matrix, and are converted to a camera reference direction through an inverse rotation transformation of the external parameters, and are normalized according to the Euclidean norm, to obtain a unit spherical direction set and a corresponding region mask; Longitudes and latitudes of the unit spherical direction set are calculated according to the region mask, and a spherical parameter set is output; A hemispherical projection is performed on the spherical parameter set located in the upper hemisphere region and the lower hemisphere region, to generate an upper hemispherical projection plane coordinate field, and according to this, an upper hemispherical projection atlas and a lower hemispherical projection atlas are resampled; A cylindrical projection is performed on the spherical parameter set located in the middle annular zone, to generate a middle annular zone projection plane coordinate field, and according to this, a middle annular zone projection atlas is resampled; Angle unfolding and winding normalization are performed on the middle annular zone projection plane coordinate field, to generate an unfolded longitude set through polar seamless mapping; The upper hemispherical projection atlas, the lower hemispherical projection atlas and the middle annular zone projection atlas are spatially registered and boundary index aligned according to the region mask, are combined into a partition projection result set, and an overlap zone mapping relationship table is established for the overlapping zones of adjacent view fields.

4. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the initial registration result specifically includes: The upper hemispherical projection atlas, the lower hemispherical projection atlas and the middle annular zone projection atlas are read from the partition projection result set, the region projection metadata set and the overlap zone mapping relationship table are combined, a neighboring view field pairing list is generated according to the principle that two view fields in the same region are paired, and an overlap zone clipping window and a boundary index are established for each pair of neighboring view fields; Feature point extraction and descriptor generation are performed on each projection image in the neighboring view field pairing list in the corresponding overlap zone clipping window, feature point extraction and scale invariant descriptor encoding are performed using a scale invariant feature transform, and a feature point set containing key point positions, scales and main directions and a corresponding descriptor matrix are output; Nearest neighbor and second nearest neighbor matching are performed on each pair of projection images in the neighboring view field pairing list within the overlap zone range, the Euclidean distance between each descriptor and its nearest neighbor descriptor is calculated as a first distance, the Euclidean distance between the second nearest neighbor descriptor is calculated as a second distance, and the matching pairs with a ratio of the first distance to the second distance not greater than a preset threshold are taken as candidate matching pairs; Geometric consistency screening is performed on the candidate matching pairs, a homography relationship is estimated on the projection plane using a random sample consensus method, the homogeneous pixel coordinates in one projection image are equal in a proportional sense to the homogeneous pixel coordinates in another projection image after homography transformation, and geometric outliers that do not satisfy the geometric consistency screening are removed to obtain a geometric consistent matching set; An optimization problem with a minimum re-projection error square sum as an objective is established on the geometric consistent matching set, an initial homography matrix is obtained, and corresponding inlier set indexes, inlier proportions and overlap zone boundary indexes are recorded, to form an initial homography estimation result facing the neighboring view fields. The geometric consistent matching set, the initial homography matrix, the inlier ratio, the overlapping strip boundary index and the feature point to pixel reverse index of each pair of adjacent view fields are summarized, and the initial registration result is organized into an upper hemisphere region, a lower hemisphere region and a middle ring band region.

5. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the partition alignment result set specifically includes: The adjacent view field pairing list, the geometric consistent matching set and the initial homography matrix are read from the initial registration result, and the upper hemisphere projection graph set, the lower hemisphere projection graph set and the middle ring band projection graph set and the region projection metadata set are read from the partition projection result set, the overlapping strip sub-block set to be locally registered is generated according to the overlapping strip boundary index of the adjacent camera view fields in the same region, and the sub-block level partition alignment intermediate result is obtained. The APAP grid vertex and the grid cell index are obtained by performing meshing division on each overlapping strip to be locally registered on the corresponding projection plane, and the initial homography matrix is taken as the initial value of each grid cell to form the grid initialization result taking the initial homography matrix as the priori. An anisotropic weight kernel direction field consistent with the region projection direction is constructed for the upper hemisphere region, the lower hemisphere region and the middle ring band region respectively, the main weight direction is set along the longitude and latitude direction for the upper hemisphere region and the lower hemisphere region, the main weight direction is set along the cylindrical development direction for the middle ring band region, and the projection-aware anisotropic weight kernel parameter set is output. At each grid vertex, the vertex weighting coefficient is calculated based on the anisotropic weight kernel parameter set according to the projection plane distance and the included angle of the main weight direction of the matching point in the geometric consistent matching set, and the vertex weighting distribution is obtained by normalizing all the weighting coefficients at the same vertex. The weighted least square estimation is performed on the geometric consistent matching set in each grid cell by taking the vertex weighting distribution as the weight, the local homography of the grid cell is updated according to the grid initialization result, and the correspondence between the grid cell and the local homography is recorded. The local homographies of adjacent grid cells are continuously constrained and interpolated according to the shared boundary, the local homographies are smoothly fused by using the weight interpolation function taking the distance to the boundary as the independent variable to generate the local homography interpolation field. The dependent view fields in the adjacent view fields are resampled into the master view field coordinate system according to the local homography interpolation field, the sub-block registration result after local alignment is output, and the sub-block registration result is stored in association with the overlapping strip boundary index and the feature point to pixel reverse index to obtain the sub-block level partition alignment intermediate result. The boundary splicing and gap detection are performed on all the sub-block level partition alignment intermediate results of the same pair of adjacent view fields, the vertex weighting distribution and the local homography are recalculated if a locally under-constrained region is found, the sub-block level partition alignment intermediate result meets the preset continuity criterion, and the view field level partition alignment result of all the adjacent view fields in the same region is regionally summarized to form the partition alignment result set.

6. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the ring panorama image specifically includes: According to the partition alignment result set, the region projection metadata set and the overlapping strip mapping relationship table, the overlapping region candidate set is generated according to the overlapping strip boundary index of the master view field and the dependent view field in the same region, and the pixel level index is established for each overlapping region. Estimating photometric consistency parameters within the set of overlap region candidates, performing exposure and color gain alignment on the dependent view, and outputting a set of photometric alignment maps; Constructing a stitching energy map based on the set of photometric alignment maps, calculating intensity difference, gradient difference and edge continuity terms at each overlap region and linearly weighting them, and outputting the stitching energy map; Performing stitching path optimization on the stitching energy map, generating a set of stitching paths in each overlap region using energy-minimizing path search, and generating a stitching band mask and a distance field from the paths to the pixels; Constructing a blending weight field according to the stitching band mask and the distance field, calculating the exponential function form of distance attenuation values to the stitching paths for the main view and the dependent view respectively, and obtaining the final blending weight after weighting and normalization, while superimposing the occlusion priority corresponding to the overlap band boundary index, and outputting the blending weight field; Constructing a multi-scale pyramid for each overlap region, performing weighted fusion of Laplacian components according to the blending weight field at each scale, and fusing Gaussian components at the highest layer, completing layer-by-layer reconstruction, and outputting a set of sub-block level fusion results; Performing stitching suppression and artifact detection on the set of sub-block level fusion results, calculating the intensity residual and the gradient residual after fusion and comparing them with the preset threshold, and if the threshold is exceeded, recalculating the blending weight field and the stitching path in the corresponding overlap band range, and outputting a set of view level fusion results; Wherein the intensity residual is equal to the sum of the absolute values of the intensity difference between the fused pixel and the main view and the dependent view pixel, and the gradient residual is equal to the sum of the absolute values of the gradient difference between the fused pixel and the main view and the dependent view; Performing regional aggregation and boundary splicing on all view level fusion result sets in the same region, respectively obtaining the upper hemisphere region fusion result, the lower hemisphere region fusion result and the middle annular band region fusion result, and performing end splicing at the regional boundary according to the seamless transition rule, outputting the panoramic stitching image, and saving the stitching path set, the blending weight field and the residual statistics.

7. The ship spherical global image stitching method based on image processing according to claim 1, characterized in that, The generation of the three-dimensional panoramic view specifically includes: Reading the panoramic stitching image and its corresponding stitching path set, blending weight field and residual statistics, and generating a panoramic stitching image metadata containing photometric, geometric and stitching information; Establishing an equirectangular projection scale reference, unfolding the panoramic stitching image according to the preset width-height ratio, defining the linear mapping relationship of longitude and latitude, converting the spherical coordinates to equirectangular plane coordinates, performing resampling and periodic wrapping, and obtaining an equirectangular projection image; Constructing a vertex coordinate grid of a virtual sphere, generating a three-dimensional vertex set according to the longitude and latitude distribution of the equirectangular projection image, and attaching the equirectangular projection image to the surface of the virtual sphere to establish a mapping relationship between pixels and textures, and outputting a virtual sphere with textures; Establishing an interactive view angle control interface, defining the heading angle and the pitch angle as the main interactive inputs, calculating the corresponding view rotation transformation and applying it to the virtual sphere to realize interactive browsing and pose switching of the three-dimensional panoramic view; In the process of interactive rendering, generating a visible continuity evaluation overlay layer according to the residual statistics and the blending weight field, and superimposing and displaying it in a translucent form to display the continuity state of the panoramic stitching region in real time, and outputting an interactive three-dimensional panoramic view.

8. A ship spherical global image stitching system based on image processing, executing the face living body detection method based on machine learning in any one of claims 1 to 7, characterized in that, It includes: The partition input generation module is configured to acquire top, bottom and annularly distributed multi-camera images, perform camera calibration and distortion correction, and divide the upper hemisphere, the lower hemisphere and the middle annular zone according to the camera installation pose; The hybrid projection module is configured to perform hemispherical projection and cylindrical projection, complete the longitude and latitude coordinate mapping and the area resampling; The feature matching module is configured to extract feature points and establish a matching relationship between adjacent fields of view, filter and calculate an initial homography matrix according to geometric consistency, and form an initial registration result; The projection-aware registration module is configured to perform projection-aware APAP meshing local registration, apply an anisotropic weight kernel consistent with the projection direction to different zones, and generate a set of partition alignment results; The weighted fusion module is configured to perform photometric alignment, energy minimum path search and multi-scale pyramid weighted fusion in the overlapping area of the set of partition alignment results, and output a panoramic stitching image; The panoramic mapping module is configured to convert the panoramic stitching image into an equirectangular projection scale and attach it to a virtual sphere, establish texture mapping and interactive view control, and realize three-dimensional panoramic view generation.

Images