Large Field of View Situation Mosaic System and Method Based on Multi-Device Joint Imaging

Abstract

The present invention belongs to the technical field of image processing, and particularly relates to a large field-of-view situation stitching system and method based on multi-device joint imaging. It includes an image acquisition module that real-time acquires image data of the camera subsystems of multiple measurement stations; a communication module that conducts real-time communication with multiple measurement stations and real-time acquires data information of multiple measurement stations; an image processing module that real-time constructs a situation image based on the image data and data information of multiple measurement stations; an image display module that real-time displays the image data and situation image of each measurement station; an image output module that converts the situation image into SD-SDI format or HD-SDI format through an image output card, and segmentally outputs the converted situation image to the display system of the command and control center to achieve real-time monitoring of each target. The present invention can realize the joint imaging of observation data of multiple devices, thereby providing a comprehensive and real-time situation monitoring map.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a large field-of-view situation stitching system and method based on multi-device joint imaging. Background Art

[0002] Due to factors such as the observation range and deployment, a single imaging device often can only cover a limited observation range and usually focuses on tracking a single target. This limitation results in the command center being unable to form a clear and comprehensive multi-dimensional perception description of the target situation. There are some 3D situation simulation software on the market that can construct a virtual environment, but the simulated targets and backgrounds are all preset virtual elements, lacking real-time data support and unable to directly monitor the real environment situation. These simulation software have obvious deficiencies in providing real-time and intuitive situation perception, resulting in missing key details and affecting the overall application efficiency. Existing fusion stitching technologies include: 1) Multi-source data fusion technology uses algorithms to integrate data from different sensors to provide more comprehensive situation perception, but this technology focuses on data-level fusion rather than image-level stitching. 2) Image stitching technology mainly relies on feature points in the image, similar regions in the image, or adjacent overlapping fields of view. However, in a complex environment, a single imaging device is limited by the observation range and deployment, can only cover a limited area and track a single target, and there is a lack of overlapping fields of view between multiple measurement stations, making it difficult for current image stitching solutions to construct a large field-of-view situation map. Although there are some 3D situation simulation software on the market that can construct a virtual battlefield environment, the simulated targets and backgrounds are all preset virtual elements, lacking real-time battlefield data support and unable to directly monitor the real battlefield situation. These simulation software have obvious deficiencies in providing real-time and intuitive battlefield situation perception. Summary of the Invention

[0003] In view of this, the present invention aims to provide a large field-of-view situation stitching system and method based on multi-device joint imaging to solve the obvious deficiencies of existing simulation software in providing real-time and intuitive situation perception, etc. The present invention can realize the joint imaging of multi-device observation data, thereby providing a comprehensive and real-time situation monitoring map.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] A large field-of-view situation stitching system based on multi-device joint imaging, comprising: an image acquisition module for real-time acquisition of image data of the camera subsystems of multiple measurement stations;

[0006] A communication module for real-time communication with multiple measurement stations and real-time acquisition of data information of multiple measurement stations;

[0007] An image processing module that constructs a situation image in real time based on the image data and data information of multiple measurement stations;

[0008] An image display module that displays the image data and situation image of each measurement station in real time;

[0009] An image output module that converts the situation image into SD-SDI format or HD-SDI format through an image output card and outputs the converted situation image in segments to the display system of the command center to achieve real-time monitoring of each target.

[0010] Furthermore, the image processing module includes:

[0011] A target position repositioning sub-module that performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations to obtain the pixel repositioning information of each target from the perspective of the observation station, enabling spatial alignment of each target from the perspective of the observation station;

[0012] A target attitude remapping sub-module that realizes cross-station target attitude remapping based on the real-time image data of multiple measurement stations and the pixel repositioning information of each target, simulating the attitude of each target from the perspective of the observation station;

[0013] A target extraction and segmentation sub-module that extracts and segments each target after cross-station target attitude remapping based on a target segmentation algorithm;

[0014] A background image generation sub-module that reconstructs the missing background image data from the perspective of the observation station using a large field of view situation background image reconstruction method based on interpolation to obtain a reconstructed background image;

[0015] An image stitching sub-module that seamlessly stitches the pixel repositioning information of each target, each target output by the target extraction and segmentation sub-module, and the reconstructed background image using an image seamless stitching method to obtain a situation image.

[0016] Furthermore, the measurement station includes a camera subsystem, an image tracking subsystem, and a main control subsystem. The data information includes the target miss distance and gate information output by the image tracking subsystem, the station address information, encoder data, and ballistic data output by the main control subsystem, and the target images collected by the camera subsystem.

[0017] A large field of view situation stitching method based on multi-device joint imaging, which is realized by using a large field of view situation stitching system based on multi-device joint imaging, and specifically includes the following steps:

[0018] S1: An image acquisition module collects the image data of the camera subsystems of multiple measurement stations in real time;

[0019] S2: The communication module communicates with multiple measurement stations in real time and collects the data information of multiple measurement stations in real time. The data information includes gate information.

[0020] S3: The image processing module constructs a situation image in real time based on the image data and data information of multiple measurement stations.

[0021] S4: The image display module displays the image data and situation image of each measurement station in real time.

[0022] S5: The image output module converts the situation image into SD-SDI format or HD-SDI format through an image output card, and outputs the converted situation image in segments to the display system of the command and control center to realize the real-time monitoring of each target.

[0023] Further, step S3 specifically includes the following steps:

[0024] S31: The target position repositioning sub-module performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations, and obtains the pixel repositioning information of each target from the perspective of the observation station, so that each target is spatially aligned from the perspective of the observation station.

[0025] S32: The target attitude remapping sub-module realizes cross-station target attitude remapping based on the real-time image data of multiple measurement stations and the pixel repositioning information of each target, and simulates the attitude of each target from the perspective of the observation station.

[0026] S33: The target extraction and segmentation sub-module extracts and segments each target after cross-station target attitude remapping based on the target segmentation algorithm.

[0027] S34: The background image generation sub-module reconstructs the missing background image data from the perspective of the observation station by using the large field-of-view situation background image reconstruction method based on interpolation to obtain the reconstructed background image.

[0028] S35: The image stitching sub-module performs seamless stitching on each target, the pixel repositioning information of each target and the reconstructed background image output by the target extraction and segmentation sub-module based on the image seamless stitching method to obtain the situation image.

[0029] Further, step S31 specifically includes the following steps:

[0030] S311: Under the same ellipsoid datum, the geodetic coordinates of the observation station and each measurement station are converted into geocentric coordinates through the following formula to obtain the geocentric coordinates of the observation station and the geocentric coordinates of each measurement station:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] wherein, is the geodetic coordinate system of the observation station or measurement station to be converted, B is the latitude, L is the longitude, H is the height, N is the radius of curvature of the circle at the position of the observation station or measurement station to be converted, is the semi-major axis of the ellipse corresponding to the geodetic coordinate system, b is the semi-minor axis of the ellipse corresponding to the geodetic coordinate system, is the first eccentricity;

[0037] S312: Establish a first world coordinate system with the camera subsystem of the observation station as the origin, perform coordinate transformation on the geocentric coordinates of each measurement station, and obtain the world coordinates of each measurement station in the first world coordinate system:

[0038] ;

[0039] wherein, is the world coordinate of the nth measurement station in the first world coordinate system;

[0040] S313: Calibrate the central pixel coordinates of the mth target captured by the nth measurement station through the camera internal parameters of the camera subsystem of the nth measurement station to obtain the camera coordinates of the mth target in the camera coordinate system :

[0041] ;

[0042] wherein, is the central pixel coordinate of the mth target, is the principal point coordinate, is the focal length of the camera subsystem of the current measurement station, is the x-axis pixel size of the camera subsystem of the current measurement station, is the y-axis pixel size of the camera subsystem of the current measurement station, is the depth information of the mth target observed by the current measurement station;

[0043] S314: According to the azimuth and elevation angles of the mth target observed by the nth measurement station, convert the mth target in the camera coordinate system from the camera coordinate system to the second world coordinate system to obtain the world coordinates of the mth target in the second world coordinate system, and the second world coordinate system takes the current measurement station as the origin;

[0044] S315: Calculate the rotation and translation matrix from the nth measurement station to the observation station, and calculate the world coordinates of the mth target captured by the nth measurement station in the first world coordinate system according to the rotation and translation matrix. :

[0045] ;

[0046] ;

[0047] where R is the rotation matrix of the current measurement station relative to the observation station, and T is the translation vector of the current measurement station relative to the observation station. is the coordinate of the mth target captured by the nth measurement station in the second world coordinate system, and M is the rotation and translation matrix of the current measurement station.

[0048] S316: Based on the external and internal camera parameters of the observation station, map the coordinates of the mth target captured by the nth measurement station in the first world coordinate system back to the pixel domain to obtain the coordinates of the mth target captured by the nth measurement station in the image coordinate system, so that the mth target is imaged in the extended field of view of the observation station.

[0049] S317: Repeat steps S313 - S316 to image the targets captured by each measurement station in the extended field of view of the observation station, realize the coordinate mapping of all targets across the observation stations, and obtain the pixel repositioning information of all targets from the perspective of the observation station.

[0050] Furthermore, in step S316, during the process of imaging the mth target in the extended field of view of the observation station, the extended pixel size in the extended field of view is:

[0051] If , then the extended pixels in the x - direction are ; is the multi - extended pixel of the x - axis boundary, and is a constant, and the width of the extended image resolution after repositioning is , otherwise, keep the pixel size in the x - direction unchanged;

[0052] If , then the extended pixels in the y - direction are ; is the multi - extended pixel of the y - axis boundary, and is a constant, and the height of the extended image resolution after repositioning is , otherwise, keep the pixel size in the y - direction unchanged;

[0053] The miss distance of the repositioned target is , ;

[0054] The principal point coordinates of the camera subsystem of the observation station in the pixel coordinate system.

[0055] Furthermore, step S32 specifically includes the following steps:

[0056] S321: Use the SIFT algorithm, SURF algorithm, or ORB algorithm to detect feature points in the target image measured by the nth measurement station, and obtain at least four non-collinear feature points;

[0057] S322: Perform pixel relocalization on all the feature points obtained in step S321 to obtain the coordinates of the relocalized feature points;

[0058] S323: Repeat steps D321 - S322 to obtain the coordinates of the relocalized feature points corresponding to each measurement station;

[0059] S324: Match the coordinates of the feature points of each measurement station with the coordinates of the corresponding relocalized feature points to obtain the perspective transformation matrix T h :

[0060] ;

[0061] where ~ are the elements of the perspective transformation matrix;

[0062] S325: Apply the perspective transformation matrix to transform the target images of each relocalized measurement station, simulate the poses of each target from the perspective of the observation station, and obtain the transformed reprojection images.

[0063] Furthermore, step S33 specifically includes the following steps:

[0064] S331: Receive the gate information output by the image tracking subsystems of each measurement station, and sequentially calculate the remapping positions of each gate according to the perspective transformation matrix:

[0065] ;

[0066] ;

[0067] where is the current gate position, is the remapping position of the current gate;

[0068] [[ID=SS6]]S332: Calculate the minimum bounding rectangle according to the remapping positions of all the gates, and crop each reprojection image according to the minimum bounding rectangle;

[0069] S333: Input the cropped reprojection image into the YOLOv8n model in sequence to extract and segment the targets, and correspondingly output the segmentation masks;

[0070] S334: Perform edge expansion operations on each segmentation mask to expand both the rows and columns of each segmentation mask by m pixels, and multiply each expanded segmentation mask with the reprojection image in sequence to obtain the sum of each target and the target image transition region corresponding to each target.

[0071] Further, step S34 specifically includes the following steps:

[0072] S341: Analyze and calculate the envelope of the flight trajectories of each target from the perspective of the observation station to determine the maximum expanded field-of-view resolution ;

[0073] S342: Use the target segmentation algorithm to separate each target from the background region, and adopt the bilinear interpolation method to perform pixel-level filling on the background image after extracting the targets, and reconstruct the target-free background image of the observation station;

[0074] S343: Expand the target-free background image using the bilinear interpolation method, and the expansion size is to obtain the reconstructed background image.

[0075] Further, based on the image seamless stitching method, perform seamless stitching on each target, the pixel repositioning information of each target, and the reconstructed background image output by the target extraction and segmentation sub-module to obtain the situation image:

[0076] ;

[0077] ;

[0078] Among them, is the reconstructed background image, is the target image, is the target image transition region, is the fused situation image, is the weight gradient factor, is the left boundary coordinate of the overlapping region in the th row, is the right boundary coordinate of the overlapping region in the th row, is the number of overlapping pixels in the th row, that is , x is the row pixel coordinate, and y is the column pixel coordinate.

[0079] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0080] (1) The large field of view situation stitching system and method based on multi-device joint imaging described in this invention creatively proposes a cross-station target repositioning method based on 3D modeling. This technology aims to solve the problem of multi-target repositioning mapping from the perspective of a certain observation station. With the observation station as the origin, 3D modeling is carried out. On the premise of 3D modeling, pixel repositioning of targets across observation stations is achieved through a multi-coordinate transformation mechanism and field of view expansion.

[0081] (2) The large field of view situation stitching system and method based on multi-device joint imaging described in this invention creatively proposes a cross-station target attitude re-mapping method based on perspective transformation. Based on the cross-station target repositioning technology and feature extraction technology based on 3D modeling, re-mapping of the target image attitude across measurement stations is realized, and then the target imaging effects from different perspectives are simulated.

[0082] (3) The large field of view situation stitching system and method based on multi-device joint imaging described in this invention creatively proposes a large field of view situation background image reconstruction method based on interpolation. Combining ballistic data information, the envelope of the target flight trajectory is analyzed and calculated to determine the maximum extended field of view range; using the existing background pixel information, new pixel values are generated through interpolation technology to perform pixel-level reconstruction on the missing field of view areas, realizing virtual field of view widening.

[0083] (4) Compared with multi-source data fusion technology, the large field of view situation stitching system and method based on multi-device joint imaging described in this invention not only integrates multi-dimensional measurement data from multiple observation stations, but also fuses image data, and performs joint imaging through multi-source data fusion, so as to comprehensively monitor and accurately describe the entire environmental situation in an intuitive and convenient manner; compared with traditional image stitching technology, this invention abandons the dependence on feature points, similar regions or adjacent overlapping fields of view, effectively overcomes the limitation of the field of view range of a single measurement station, and realizes joint imaging from cross-station perspectives; compared with existing 3D situation simulation software on the market, this invention combines real background and target image data as well as real-time measurement data to directly monitor the actual environmental situation, significantly improving the observation and command efficiency of the command center. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The drawings constituting a part of this invention are used to provide a further understanding of this invention. The schematic embodiments of this invention and their descriptions are used to explain this invention and do not constitute an improper limitation to this invention. In the drawings:

[0085] Figure 1 It is a schematic structural diagram of the large field of view situation stitching system based on multi-device joint imaging described in the embodiment of this invention;

[0086] Figure 2Schematic flowchart of the large field-of-view situation mosaicking method based on multi-device joint imaging according to the embodiments of the present invention

[0087] Figure 3 Schematic flowchart of the processing of the target position repositioning sub-module according to the embodiments of the present invention

[0088] Figure 4 Schematic diagram of the extended pixels for cross-station target repositioning according to the embodiments of the present invention

[0089] Figure 5 Schematic flowchart of the processing of the target attitude remapping sub-module according to the embodiments of the present invention

[0090] Figure 6 Schematic flowchart of the processing of the target extraction and segmentation sub-module according to the embodiments of the present invention

[0091] Figure 7 Schematic flowchart of the processing of the background image generation sub-module according to the embodiments of the present invention

[0092] Figure 8 Schematic flowchart of the processing of the image mosaicking sub-module according to the embodiments of the present invention

[0093] Explanation of reference numerals:

[0094] 1. Image acquisition module; 2. Communication module; 3. Image processing module; 4. Image display module; 5. Image output module; 31. Target position repositioning sub-module; 32. Target attitude remapping sub-module; 33. Target extraction and segmentation sub-module; 34. Background image generation sub-module; 35. Image mosaicking sub-module; 6. Measurement station; 61. Camera subsystem; 62. Image tracking subsystem; 63. Main control subsystem; 7. Command center. Detailed implementation manners

[0095] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0096] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0097] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0098] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", "coupling" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific circumstances.

[0099] The present invention will be described in detail below with reference to the drawings and in combination with embodiments.

[0100] As Figure 1 shown, the present invention provides a large field-of-view situation stitching system based on multi-device joint imaging, including: an image acquisition module 1 that real-time acquires the image data of the camera subsystems 61 of multiple measurement stations 6; a communication module 2 that communicates with multiple measurement stations 6 in real time and real-time acquires the data information of multiple measurement stations 6; an image processing module 3 that constructs a situation image in real time based on the image data and data information of multiple measurement stations 6; an image display module 4 that real-time displays the image data and situation image of each measurement station 6; and an image output module 5 that converts the situation image into SD-SDI format or HD-SDI format through an image output card and outputs the converted situation image in segments to the display system of the command and control center to achieve real-time monitoring of each target.

[0101] Based on the real-time processing of the above-mentioned modules, the present invention performs real-time stitching on the data acquired by each measurement station 6 to obtain a large-field-of-view situation map. First, the image acquisition module 1 acquires the image data of each measurement station 6 in real time and synchronously receives data such as the station address information, encoder data, and target miss distance of each measurement station 6. The target position repositioning sub-module 31 performs three-dimensional modeling with the observation station (selecting one of the measurement stations 6 as the observation station according to user requirements) as the origin of the world coordinate system based on the input data such as station address information, encoder data, and target miss distance, and uses multi-coordinate transformation technology to spatially align the target images to determine the pixel repositioning information of each target from the perspective of the observation station; the target attitude remapping sub-module 32 calculates the perspective transformation matrix through the input image data according to the target position repositioning sub-module 31 and feature extraction technology, and then realizes the remapping of the cross-station target image attitude; subsequently, the target extraction and segmentation sub-module 33 precisely extracts and segments the targets from the remapped image to obtain each target; the background image generation sub-module 34 analyzes and calculates the envelope of the flight trajectories of each target based on the ballistic data to determine the extended field-of-view range, and then performs super-resolution reconstruction of the specified size on the original input image data by means of interpolation; the image stitching sub-module 35 seamlessly stitches and renders the extracted and segmented targets and the super-resolution reconstructed background image in combination with the repositioning positions of multiple targets from the perspective of the observation station to ensure the continuity and naturalness of the image, and then generates a comprehensive, coherent, and high-resolution large-field-of-view situation map. Finally, the situation map is converted into SD-SDI format or HD-SDI format through an image output card and output in segments to the display system of the command and control center so that the commander can monitor the situation of each target in real time.

[0102] In some examples, the image processing module 3 includes: a target position repositioning sub-module 31 that performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations 6 to obtain the pixel repositioning information of multiple targets from the perspective of the observation station, so as to spatially align each target from the perspective of the observation station; a target attitude remapping sub-module 32 that realizes cross-station target attitude remapping based on the real-time image data of multiple measurement stations 6 and the pixel repositioning information of each target, and simulates the attitudes of each target from the perspective of the observation station; a target extraction and segmentation sub-module 33 that extracts and segments each target after cross-station target attitude remapping based on a target segmentation algorithm; a background image generation sub-module 34 that reconstructs the missing background image data from the perspective of the observation station by means of a large-field-of-view situation background image reconstruction method based on interpolation to obtain a reconstructed background image; and an image stitching sub-module 35 that seamlessly stitches the pixel repositioning information of each target, each target output by the target extraction and segmentation sub-module 33, and the reconstructed background image by using an image seamless stitching method to obtain a situation image.

[0103] In some embodiments, the measurement station 6 includes a camera subsystem 61, an image tracking subsystem 62, and a main control subsystem 63. The data information includes the target miss distance and gate information output by the image tracking subsystem 62, as well as the station location information, encoder data, ballistic data output by the main control subsystem 63, and the target images collected by the camera subsystem.

[0104] It should be noted that the present invention proposes a cross-station target relocalization technology based on three-dimensional modeling. First, three-dimensional modeling of the observation station is performed, and pixel relocalization of the targets of multiple measurement stations 6 at a specific perspective is achieved through a multi-coordinate transformation mechanism and a field-of-view expansion method, ensuring the accurate alignment of the target images observed by different measurement stations 6 in space; a cross-station target attitude remapping technology based on perspective transformation is proposed, and the remapping of the target image attitude across stations is realized by using the relocalization information and feature extraction method, and then the target attitude under the perspective of the observation station is simulated; a target segmentation algorithm based on the region of interest is used to achieve the precise extraction and segmentation of the targets observed by each measurement station 6; in order to make up for the limitation of the observation range of a single measurement station 6, a large field-of-view situation background image reconstruction method based on interpolation is adopted to achieve pixel-level reconstruction of the missing field-of-view area; an image seamless stitching method is used to seamlessly stitch and render the extracted and segmented target images and the reconstructed background images in combination with the position information and attitude information of multiple targets at a specific perspective, ensuring the continuity and naturalness of the images, and then generating a comprehensive, coherent, and high-resolution situation image.

[0105] Furthermore, the image processing module 3 is the core execution entity of the present invention, responsible for the all-round integration and presentation of the situation image. The target position relocalization sub-module 31: integrates the cross-station target relocalization method based on three-dimensional modeling, accurately analyzes the pixel relocalization information of multiple targets, and ensures the accurate alignment of the target images at different observation points in space; the target attitude remapping sub-module 32: integrates the cross-station target attitude remapping method based on perspective transformation to simulate the target imaging effects at different perspectives. The target extraction and segmentation sub-module 33: integrates the YOLOv8n (or other) target segmentation algorithm based on the region of interest to ensure the accuracy and integrity of target extraction and segmentation. The background image generation sub-module 34: integrates the large field-of-view situation background image reconstruction method based on interpolation to reconstruct the missing background image data to make up for the limitation of the observation range of a single measurement station 6. The image stitching sub-module 35: integrates the image seamless stitching method based on fade-in and fade-out (or other) to seamlessly stitch and render the extracted and segmented target images and the high-resolution reconstructed background images in combination with the position and attitude information of multiple targets at a specific perspective (the perspective of the observation station), ensuring the continuity and naturalness of the situation image.

[0106] Furthermore, the image display module 4: can simultaneously display the image data of multiple measurement stations 6 and the stitched and rendered large field-of-view situation image.

[0107] As Figure 2 shown, a large field of view situation mosaicking method based on multi-device joint imaging proposed by the present invention is realized by using a large field of view situation mosaicking system based on multi-device joint imaging, and specifically includes the following steps: S1: The image acquisition module 1 collects the image data of the camera subsystems 61 of multiple measurement stations 6 in real time; S2: The communication module 2 communicates with multiple measurement stations 6 in real time and collects the data information of multiple measurement stations 6 in real time. The data information includes gate information; S3: The image processing module 3 constructs a situation image in real time based on the image data and data information of multiple measurement stations 6; S4: The image display module 4 displays the image data and situation image of each measurement station 6 in real time; S5: The image output module 5 converts the situation image into SD-SDI format or HD-SDI format through an image output card, and outputs the converted situation image in segments to the display system of the command and control center to realize the real-time monitoring of each target.

[0108] It should be noted that step S1 realizes the pixel re-coordinate mapping of cross-station targets through multi-coordinate transformation and field of view expansion under three-dimensional modeling, so as to obtain the pixel positions of each target from the perspective of the observation station. The parameters required for this process include the site information of multiple measurement stations 6, camera internal parameters (focal length, principal point coordinates, pixel size), encoder information, and target miss distance, etc.

[0109] In some instances, step S3 specifically includes the following steps: S31: The target position repositioning sub-module 31 performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations 6 to obtain the pixel repositioning information of multiple targets from the perspective of the observation station, so that each target is spatially aligned from the perspective of the observation station; S32: The target attitude remapping sub-module 32 realizes the cross-station target attitude remapping based on the real-time image data of multiple measurement stations 6 and the pixel repositioning information of each target, and simulates the attitudes of each target from the perspective of the observation station; S33: The target extraction and segmentation sub-module 33 extracts and segments each target after cross-station target attitude remapping based on the target segmentation algorithm; S34: The background image generation sub-module 34 reconstructs the missing background image data from the perspective of the observation station by using the large field of view situation background image reconstruction method based on interpolation to obtain the reconstructed background image; S35: The image mosaicking sub-module 35 performs seamless mosaicking on each target, the pixel repositioning information of each target, and the reconstructed background image output by the target extraction and segmentation sub-module 33 by using the image seamless mosaicking method to obtain the situation image.

[0110] Establish a world coordinate system with the camera subsystem of the observation station as the origin. According to the site information of the observation station and the measurement stations 6n (N = 1, 2, 3,...), generally, the site coordinates are in the geodetic coordinate system (latitude (B), longitude (L), and height (H)), and convert the geodetic coordinate systems of the observation station and the measurement stations 6n into the world coordinate system.

[0111] In some embodiments, as Figure 3 shown, step S31 specifically includes the following steps: S311: Under the same ellipsoid datum, convert the geodetic coordinates of the observation station and each measurement station 6 into geocentric coordinates through the following formula, and obtain the geocentric coordinates of the observation station and each measurement station 6 as:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] Among them, is the geodetic coordinate system of the observation station or measurement station to be converted, B is the latitude, L is the longitude, H is the height, N is the radius of curvature of the circle at the position of the observation station or measurement station to be converted, is the semi-major axis of the ellipse corresponding to the geodetic coordinate system, b is the semi-minor axis of the ellipse corresponding to the geodetic coordinate system, is the first eccentricity.

[0118] S312: Establish a first world coordinate system with the camera subsystem of the observation station as the origin. The world coordinate system of the observation station is , perform coordinate conversion on the geocentric coordinates of each measurement station, and obtain the world coordinates of each measurement station 6 in the first world coordinate system:

[0119] ;

[0120] Among them, is the world coordinate of the nth measurement station 6 in the first world coordinate system.

[0121] S313: Calibrate the central pixel coordinates of the mth target captured by the nth measurement station 6 through the camera internal parameters of the camera subsystem of the nth measurement station 6, and obtain the camera coordinates of the mth target in the camera coordinate system (currently, the camera coordinate system is established based on the camera subsystem of the nth measurement station 6) :

[0122] ;

[0123] Among them, is the central pixel coordinate of the m-th target, is the principal point coordinate, is the focal length of the camera subsystem of the current measurement station, is the pixel size of the x-axis of the camera subsystem of the current measurement station, is the pixel size of the y-axis of the camera subsystem of the current measurement station, is the depth information of the m-th target observed by the current measurement station;.

[0124] S314: According to the azimuth and elevation angles of the m-th target observed by the n-th measurement station 6, calculate the external parameter matrix of the camera subsystem of the current measurement station 6, and convert the m-th target in the camera coordinate system to the second world coordinate system to obtain the coordinates of the m-th target in the second world coordinate system , and the second world coordinate system takes the current measurement station 6 as the origin.

[0125] Assume that the camera subsystem of the measurement station 6n (equivalent to the n-th measurement station 6) is taken as the origin. When the elevation angle and azimuth angle of the measurement station 6n are , the camera coordinate system coincides with the world coordinate system . When the elevation angle of the measurement station 6n is , and the azimuth angle is , then the camera coordinate system rotates around the axis in the world coordinate system by , and rotates around the axis in the world coordinate system by , that is:

[0126] ;

[0127] Among them, is the coordinate of the m-th target in the second world coordinate system.

[0128] Adopt the coordinate matching technology to calculate the rotation and translation matrix from the measurement station 6 to the observation station, so that the coordinates of the observation station and the measurement station 6 can be accurately docked to ensure the consistency of the coordinates; let the rotation matrix of the measurement station 6 relative to the observation station be R, and the translation vector be T, then the rotation and translation matrix M is expressed as:

[0129] ;

[0130] Among them, R is a 3x3 rotation matrix, and the rotation matrix R is obtained by calculating the orientation difference between two coordinate systems according to the layout direction of the measurement station 6. Generally, the layout principle of each measurement station 6 follows the northeast celestial coordinate system. Taking the observation station as the coordinate origin, the geodetic coordinates of the observation station that the measurement station 6n needs to use in the ENU coordinate system with the observation station as the coordinate origin are , and the rotation matrix is:

[0131] ;

[0132] T is a 3x1 translation vector, and the translation vector T of the measurement station 6n relative to the observation station is:

[0133] .

[0134] S315: Calculate the rotation and translation matrix from the nth measurement station to the observation station, and calculate the coordinates of the mth target captured by the nth measurement station 6 in the first world coordinate system according to each rotation and translation matrix :

[0135] ;

[0136] ;

[0137] Among them, R is the rotation matrix of the current measurement station 6 relative to the observation station, T is the translation vector of the current measurement station 6 relative to the observation station, is the coordinates of the mth target captured by the nth measurement station 6 in the second world coordinate system, and M is the rotation and translation matrix of the current measurement station 6.

[0138] S316: Based on the external and internal camera parameters of the observation station, map the coordinates of the mth target captured by the nth measurement station 6 in the first world coordinate system back to the pixel domain to ensure that the target is accurately imaged in the extended field of view of the observation station, and obtain the coordinates of the mth target captured by the nth measurement station 6 in the image coordinate system, so that the mth target is imaged in the extended field of view of the observation station.

[0139] S317: Repeat steps S313~S316 to make each target captured by each measurement station 6 be imaged in the extended field of view of the observation station, realize the coordinate mapping of all targets across the observation stations, and achieve the pixel repositioning of multiple targets at a specific viewing angle.

[0140] When the pitch angle of the observation station is , and the azimuth angle is , the camera coordinates of the mth target captured by the nth measurement station 6 at the observation station ( , , ) are:

[0141] ;

[0142] Further transform the image coordinate system to:

[0143] ;

[0144] Wherein, in the camera subsystem of the observation station, f is the focal length, is the coordinate of the image coordinate system, which is the physical size. Since pixel expansion is to be performed, and it is known that dx and dy are the pixel sizes of the x-axis and y-axis respectively, the judgment condition for the expanded pixel size is as follows, and Zc is the depth information of the target from the camera subsystem.

[0145] In some embodiments, as Figure 4 shown, in step S316, during the process of imaging the mth target in the expanded field of view of the observation station, the expanded pixel size in the expanded field of view is:

[0146] If , then the expanded pixels in the x direction are ; is the multi-expanded pixel at the x-axis boundary, and is a constant, and the width of the expanded image resolution after repositioning is , otherwise keep the pixel size in the x direction unchanged;

[0147] If , then the expanded pixels in the y direction are ; is the multi-expanded pixel at the y-axis boundary, and is a constant, and the height of the expanded image resolution after repositioning is , otherwise keep the pixel size in the y direction unchanged;

[0148] The miss distance of the repositioned target is , ; is the principal point coordinate of the camera subsystem of the observation station in the pixel coordinate system.

[0149] Based on the above cross-station target repositioning technology and feature extraction technology of multi-coordinate transformation, realize the re-mapping of the cross-station target image attitude, and then simulate the target imaging effects under different viewing angles (the viewing angles of the observation stations specified by the user).

[0150] In some embodiments, as Figure 5 shown, step S32 specifically includes the following steps:

[0151] S321: Use the SIFT algorithm, SURF algorithm, or ORB algorithm to detect feature points in the target image of the nth measurement station 6, and obtain at least four stable and non-collinear feature points.

[0152] It should be noted that the pixel coordinates of at least four feature points are respectively , and ensure that they can provide sufficient geometric constraints.

[0153] S322: Perform pixel relocalization on all the feature points obtained in step S321 to obtain the coordinates of the relocalized feature points;

[0154] It should be noted that the coordinates of the relocalized feature points are respectively .

[0155] S323: Repeat steps D321~S322 to obtain the coordinates of the relocalized feature points corresponding to each measurement station.

[0156] S324: Match the coordinates of the feature points of each measurement station 6 with the coordinates of the corresponding relocalized feature points, and calculate the perspective transformation matrix T:

[0157] ;

[0158] where ~ are the elements of the perspective transformation matrix;

[0159] S325: Apply the perspective transformation matrix to transform the target images of each relocalized measurement station 6, simulate the poses of each target from the perspective of the observation station, and obtain the transformed reprojection images.

[0160] The reprojection images correspond one-to-one with the target images, and the difference is that: the reprojection images are obtained by simulating the pose transformation of the targets in the target images.

[0161] Using advanced deep learning algorithms, intelligently identify and extract targets from the target pose reprojection images, and at the same time perform precise segmentation. It provides clear target contours for the subsequent fusion of large field-of-view situation images, ensuring the accuracy of target information in the stitched situation images. Aiming at the problems of slow segmentation speed and low accuracy of most algorithms, based on prior knowledge, the present invention adopts a YOLOv8n target segmentation algorithm for the region of interest. The YOLOv8n model, as the main technical means for target extraction and segmentation, has the advantages of being lightweight, high-precision, having good real-time performance, and being easy to train.

[0162] In some embodiments, as Figure 6 shown, step S33 specifically includes the following steps:

[0163] S331: Receive the gate information output by the image tracking subsystem of each measurement station, and calculate the remapping positions of each gate in sequence according to the perspective transformation matrix:

[0164] ;

[0165] ;

[0166] wherein, is the current gate position, is the remapping position of the current gate.

[0167] It should be noted that the gate information is obtained from the target after attitude transformation.

[0168] S332: Calculate the minimum bounding rectangle according to the remapping positions of all gates, and crop the reprojection images according to each pair of the minimum bounding matrices.

[0169] S333: Input the cropped reprojection images into the YOLOv8n model in sequence for target extraction and segmentation, and correspondingly output segmentation masks; this can effectively avoid the influence of other background information on the algorithm accuracy.

[0170] S334: Perform edge expansion operations on each segmentation mask to expand the rows and columns of each segmentation mask by m pixels simultaneously, multiply each segmentation mask expanded by m pixels with the reprojection image in sequence, and obtain the sum of each target and the target image transition region corresponding to each target. This operation aims to increase the buffer area between the target and the background, so as to achieve a smoother transition effect in the subsequent image stitching and fusion process.

[0171] Adopt an image reconstruction technique based on interpolation algorithm, utilize the existing pixel information, estimate and generate new pixel values through the interpolation algorithm, perform pixel-level reconstruction on the missing field of view area, and achieve virtual field of view broadening. For the background super-resolution reconstruction problem, there are currently many solutions, including image interpolation, super-resolution reconstruction technology, and deep learning-based image generation technology, etc. Considering that the image expansion of a single measurement station 6 only involves the background image, and mostly the sky background with unclear image details, considering the real-time limitation of the algorithm, after comprehensive evaluation, the present invention selects an image interpolation method with relatively low computational complexity.

[0172] In some embodiments, as Figure 7 shown, step S34 specifically includes the following steps:

[0173] S341: Analyze and calculate the envelope of the flight trajectories of each target from the perspective of the observation station, and determine the maximum extended field of view resolution ;

[0174] To ensure the display coherence and consistency of the situation image at a fixed focal length, it is necessary to accurately define the extended image field of view. From the perspective of the observation station, by combining ballistic data and multi-source intelligence information, analyze and calculate the envelope of the target flight trajectory, and then determine the maximum field of view range of the extension to achieve comprehensive monitoring and situation awareness.

[0175] It is known that the ballistic coordinates of target m relative to measurement station 6 are , and based on the rotation and translation matrix in the cross-station target repositioning technology of three-dimensional modeling, calculate the ballistic coordinates of target m relative to the observation station as . Among the k missile trajectory points, find the maximum coordinate in the X direction , and record the corresponding ballistic space point as the envelope point ; find the minimum coordinate on the X axis , and record the corresponding ballistic space point as the envelope point ; among the k missile trajectory points, find the maximum coordinate in the Y direction , and record the corresponding ballistic space point as the envelope point ; the minimum coordinate in the Y direction , and record the corresponding ballistic space point as the envelope point ; record that the camera subsystem of the observation station points to the envelope point . At this time, the azimuth and pitch of the observation station are:

[0176] ;

[0177] ;

[0178] Under this condition, calculate the envelope point The repositioning target miss distance in the X direction , then the maximum extended resolution in the X direction is .

[0179] Similarly, record that the camera subsystem of the observation station points to the envelope point . At this time, calculate the azimuth and pitch of the observation station, and under this condition, calculate the envelope point The repositioning target miss distance in the Y direction , then the maximum extended resolution in the Y direction is . Finally, determine the maximum field of view resolution of the extension as .

[0180] S342: Use the target segmentation algorithm to separate each target from the background area, and use the bilinear interpolation method to perform pixel-level filling on the background image after extracting the target, and reconstruct the target-free background image of the observation station.

[0181] S343: Expand the background image without targets using the bilinear interpolation method, with the expansion size being , to obtain the reconstructed background image.

[0182] As Figure 8 shown, based on the reconstructed background image, the multi-target segmentation map, and the repositioning information of the multi-targets, complete fusion and stitching of the 6-point image data from different measurement stations are achieved, and then a complete and natural situation image is constructed. To improve the operation efficiency and avoid obvious stitching gaps, the present invention adopts the fade-in and fade-out image fusion method.

[0183] In some embodiments, in step S35, seamless stitching is performed on each target output by the target extraction and segmentation sub-module, the pixel repositioning information of each target, and the reconstructed background image based on the image seamless stitching method to obtain the situation image:

[0184] ;

[0185] ;

[0186] wherein, is the reconstructed background image, is the target image, is the target image transition area, is the fused situation image, is the weight gradual change factor, is the left boundary coordinate of the overlapping area of the th row, is the right boundary coordinate of the overlapping area of the th row, is the number of overlapping pixels in the th row, that is, , x is the row pixel coordinate, and y is the column pixel coordinate.

[0187] It should be understood that various forms of the process shown above can be used, and steps can be reordered, added, or deleted. For example, the steps recorded in the present invention disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitations are imposed herein.

[0188] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A large field of view situation stitching system based on multi-device joint imaging, characterized in that: It includes: An image acquisition module that acquires the image data of the camera subsystems of multiple measurement stations in real time; A communication module that communicates with multiple measurement stations in real time and acquires the data information of multiple measurement stations in real time; An image processing module that constructs a situation image in real time based on the image data and data information of multiple measurement stations; An image display module that displays the image data of each measurement station and the situation image in real time; An image output module that converts the situation image into SD-SDI format or HD-SDI format through an image output card and outputs the converted situation image in segments to the display system of the command and control center to achieve real-time monitoring of each target; The image processing module includes: A target position repositioning sub-module that performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations, obtains the pixel repositioning information of each target from the perspective of the observation station, and aligns each target spatially from the perspective of the observation station; A target attitude remapping sub-module that realizes cross-station target attitude remapping based on the real-time image data of multiple measurement stations and the pixel repositioning information of each target, and simulates the attitudes of each target from the perspective of the observation station; A target extraction and segmentation sub-module that extracts and segments each target after cross-station target attitude remapping based on the target segmentation algorithm; A background image generation sub-module that reconstructs the missing background image data from the perspective of the observation station by using a large field-of-view situation background image reconstruction method based on interpolation to obtain a reconstructed background image; An image stitching sub-module that seamlessly stitches the pixel repositioning information of each target, each target output by the target extraction and segmentation sub-module, and the reconstructed background image by using an image seamless stitching method to obtain a situation image.

2. The large field-of-view situation stitching system based on multi-device joint imaging according to claim 1, wherein: The measurement station includes a camera subsystem, an image tracking subsystem, and a main control subsystem. The data information includes the target miss distance and gate information output by the image tracking subsystem, the station address information, encoder data, and ballistic data output by the main control subsystem, and the target images collected by the camera subsystem.

3. A large - field - of - view situation stitching method based on multi - device joint imaging, which is realized by using the large - field - of - view situation stitching system based on multi - device joint imaging described in any one of claims 1 to 2, and is characterized in that: Specifically, it includes the following steps: S1: The image acquisition module acquires the image data of the camera subsystems of multiple measurement stations in real time; S2: The communication module communicates with multiple measurement stations in real time and acquires the data information of multiple measurement stations in real time. The data information includes gate information; S3: The image processing module constructs a situation image in real time based on the image data and data information of multiple measurement stations; S31: The target position repositioning sub-module performs three-dimensional modeling with the observation station as the origin of the world coordinate system based on the real-time data information of multiple measurement stations, obtains the pixel repositioning information of each target from the perspective of the observation station, and aligns each target spatially from the perspective of the observation station; S32: The target attitude remapping sub-module realizes cross-station target attitude remapping based on the real-time image data of multiple measurement stations and the pixel repositioning information of each target, and simulates the attitudes of each target from the perspective of the observation station; S33: The target extraction and segmentation sub-module extracts and segments each target after cross-station target attitude remapping based on the target segmentation algorithm; S34: The background image generation sub-module reconstructs the missing background image data from the perspective of the observation station based on the large field-of-view situation background image reconstruction method using interpolation, and obtains the reconstructed background image; S35: The image stitching sub-module seamlessly stitches each target, the pixel repositioning information of each target, and the reconstructed background image output by the target extraction and segmentation sub-module based on the image seamless stitching method to obtain the situation image; S4: The image display module displays the image data and the situation image of each measurement station in real time; S5: The image output module converts the situation image into SD-SDI format or HD-SDI format through an image output card, and outputs the converted situation image in segments to the display system of the command and control center to realize real-time monitoring of each target.

4. The large field-of-view situation stitching method based on multi-device joint imaging according to claim 3, characterized in that: Step S31 specifically includes the following steps: S311: Under the same ellipsoid datum, convert the geodetic coordinates of the observation station and each measurement station into geocentric coordinates through the following formula to obtain the geocentric coordinates of the observation station and the geocentric coordinates of each measurement station: ； ； ； ； ； Wherein, is the geodetic coordinate system of the observation station or measurement station to be converted, B is the latitude, L is the longitude, H is the altitude, and N is the radius of curvature of the meridian at the position of the observation station or measurement station to be converted, is the semi-major axis of the ellipse corresponding to the geodetic coordinate system, and b is the semi-minor axis of the ellipse corresponding to the geodetic coordinate system, is the first eccentricity; S312: Establish a first world coordinate system with the camera subsystem of the observation station as the origin, and perform coordinate transformation on the geocentric coordinates of each measurement station to obtain the world coordinates of each measurement station in the first world coordinate system: ； Among them, is the world coordinate of the nth measurement station in the first world coordinate system; S313: Calibrate the central pixel coordinates of the m-th target captured by the n-th measurement station using the camera internal parameters of the camera subsystem of the n-th measurement station to obtain the camera coordinates of the m-th target in the camera coordinate system : ； wherein, is the central pixel coordinate of the m-th target, is the principal point coordinate, is the focal length of the camera subsystem of the current measurement station, is the x-axis pixel size of the camera subsystem of the current measurement station, is the y-axis pixel size of the camera subsystem of the current measurement station, is the depth information of the m-th target observed by the current measurement station; S314: According to the azimuth and elevation angles of the m-th target observed by the n-th measurement station, convert the m-th target in the camera coordinate system to the second world coordinate system to obtain the world coordinates of the m-th target in the second world coordinate system, where the second world coordinate system takes the current measurement station as the origin; S315: Calculate the rotation and translation matrix from the nth measurement station to the observation station, and calculate the world coordinates of the mth target captured by the nth measurement station in the first world coordinate system according to the rotation and translation matrix : ； ； where R is the rotation matrix of the current measurement station relative to the observation station, and T is the translation vector of the current measurement station relative to the observation station. is the coordinate of the m-th target captured by the n-th measurement station in the second world coordinate system, and M is the rotation and translation matrix of the current measurement station. S316: Based on the external camera parameters and internal camera parameters of the observation station, map the coordinates of the m-th target captured by the n-th measurement station in the first world coordinate system back to the pixel domain to obtain the coordinates of the m-th target captured by the n-th measurement station in the image coordinate system, so that the m-th target is imaged in the extended field of view of the observation station; S317: Repeat steps S313 - S316 to make each target captured by each measurement station be imaged in the extended field of view of the observation station, realize the coordinate mapping of each target across the observation stations, and obtain the pixel repositioning information of each target from the perspective of the observation station.

5. The large field of view situation stitching method based on multi-device joint imaging according to claim 4, wherein: In step S316, during the process of imaging the m-th target in the extended field of view of the observation station, the size of the extended pixels in the extended field of view is: If , the pixels extended in the x direction are ; is the multi-extended pixel of the x-axis boundary, and is a constant. The width of the extended image resolution after relocation is , otherwise, keep the pixel size in the x direction unchanged; If , the pixels extended in the y direction are ; is the number of pixels extended for the y-axis boundary, and is a constant. After relocation, the height of the extended image resolution is , otherwise, the pixel size in the y direction remains unchanged; The miss distance of the repositioned target is , ; It is the principal point coordinate of the camera subsystem of the observation station in the pixel coordinate system.

6. The large field of view situation stitching method based on multi-device joint imaging according to claim 3, wherein: Step S32 specifically includes the following steps: S321: Use the SIFT algorithm, SURF algorithm, or ORB algorithm to detect feature points on the target image measured by the n-th measurement station, and obtain at least four non-collinear feature points; S322: Perform pixel repositioning on all the feature points obtained in step S321 to obtain the coordinates of the repositioned feature points; S323: Repeat steps D321 - S322 to obtain the coordinates of the repositioned feature points corresponding to each measurement station; S324: Match the coordinates of each feature point of each measurement station with the coordinates of each corresponding repositioned feature point to obtain the perspective transformation matrix T h : ； Among them, ~ are the elements of the perspective transformation matrix; S325: Apply the perspective transformation matrix to transform the target images of each measurement station after repositioning to simulate the postures of each target from the perspective of the observation station, and obtain the transformed reprojection image.

7. The method for stitching large field-of-view situation based on multi-device joint imaging according to claim 3, characterized in that: Step S33 specifically includes the following steps: S331: Receive the gate information output by the image tracking sub-systems of each measurement station, and sequentially calculate the remapped positions of each gate according to the perspective transformation matrix: ； ； Among them, is the current gate position, is the remapping position of the current gate; S332: Calculate the minimum bounding rectangle based on the remapped positions of all gates, and crop each reprojection image according to the minimum bounding rectangle; S333: Sequentially input the cropped reprojection images into the YOLOv8n model for target extraction and segmentation, and correspondingly output segmentation masks; S334: Perform edge expansion operations on each segmentation mask, expand the rows and columns of each segmentation mask by m pixels simultaneously, and sequentially multiply each expanded segmentation mask by the reprojection image to obtain the sum of each target and the target image transition region corresponding to each target.

8. The method for stitching large field-of-view situation based on multi-device joint imaging according to claim 3, characterized in that: Step S34 specifically includes the following steps: S341: Analyze and calculate the envelope of the flight trajectories of each target from the perspective of the observation station to determine the maximum field of view resolution of the expansion ; S342: Use the target segmentation algorithm to separate each target from the background region, and perform pixel-level filling on the background image after target extraction using the bilinear interpolation method to reconstruct the target-free background image of the observation station; S343: Expand the background image without targets using the bilinear interpolation method, with the expansion size being , to obtain the reconstructed background image.

9. The large field of view situation stitching method based on multi-device joint imaging according to claim 8, characterized in that: Perform seamless stitching on each target, the pixel repositioning information of each target, and the reconstructed background image output by the target extraction and segmentation sub-module based on the image seamless stitching method to obtain the situation image: ； ； Among them, is the reconstructed background image, is the target image, is the target image transition area, is the situation image, is the weight gradient factor, is the left boundary coordinate of the overlapping area of the th row, is the right boundary coordinate of the overlapping area of the th row, is the number of overlapping pixels in the th row, that is, , where x is the row pixel coordinate and y is the column pixel coordinate.

Images