Image transmission method and satellite communication system
By dividing satellite images into image sub-blocks and matching preset templates, hash encoding and stitching instructions are sent, the problem of insufficient structure multiplexing in satellite remote sensing image transmission is solved, and efficient data transmission and high-quality image restoration are achieved.
Patent Information
- Application Number
- CN202510781177.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-08-08
- Estimated Expiration
- 2045-06-12
AI Technical Summary
The existing satellite remote sensing image transmission technology lacks a coding mechanism for structure multiplexing, and it is impossible to achieve cross-frame transmission multiplexing of image structures, and it is difficult to take into account both compression and restoration accuracy when splicing fails.
Divide the satellite image into multiple image sub-blocks, and prioritize the image template or image sub-table combination in the preset template set. If the match is successful, only hash encoding and stitching instructions will be sent. If the splicing cannot be stitched, the original image data will be sent. The receiver reconstructs the image sub-block based on hash encoding and stitching instructions, and can fuse the original data to complete the image.
It realizes efficient identification and multiplexing of repeated structure areas in satellite images, significantly reducing the amount of data transmission, ensuring image restoration quality, and reducing redundant data occupancy of link bandwidth.
Smart Images

Figure CN120302016B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image transmission, and in particular to an image transmission method and a satellite communication system. Background Art
[0002] Satellite remote sensing image transmission technology typically uses full-image compression encoding for transmission. However, due to the massive data volumes of high-resolution images and the limitations of satellite communication links due to bandwidth, latency, and channel availability, traditional image compression methods face bottlenecks in terms of real-time performance and resource utilization efficiency. To improve transmission efficiency, some technologies have attempted to introduce regional encoding methods based on image content redundancy, but these still rely on frame-by-frame image compression, making it difficult to fully utilize the large amount of structural repetitive information in images. This is especially true in Earth observation scenarios where scenes are stable over long periods of time and image structures are highly periodic.
[0003] For example, the Chinese patent with authorization announcement number CN105357472B provides a method for real-time transmission of video images of a remote sensing satellite system, wherein the remote sensing satellite system comprises: a satellite camera device, mounted on the satellite, for acquiring video images of target objects in real time; a ground receiving device, disposed on the ground, for receiving the video images in real time; and a data transmission device, mounted on the satellite, for transmitting the video images to the ground receiving device. This method solves the problems of dynamic target detection, identification and tracking, which are difficult to achieve with existing satellites, as well as the rapid application of satellites. It allows users to quickly obtain video information of the target area, which can greatly improve the satellite's ability to quickly apply in response to emergencies.
[0004] The above patents all have the problems raised by this background technology: lack of coding mechanism for structural reuse, inability to achieve cross-frame transmission multiplexing of image structure, and difficulty in balancing compression and restoration accuracy when splicing fails. To solve the above problems, this application designs an image transmission method and satellite communication system. Summary of the Invention
[0005] The present invention addresses the shortcomings of existing technologies and provides an image transmission method and satellite communication system. These methods, applied in satellite communication systems, include a transmitter and a receiver. The transmitter divides a satellite image into multiple image sub-blocks and prioritizes matching an image template or combination of image sub-templates from a preset template set. If a match is successful, only a hash code and splicing instructions are transmitted; if splicing is not possible, the original image data is transmitted. The receiver reconstructs the image sub-blocks based on the hash code and splicing instructions and can integrate the original data to complete the image.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] An image transmission method, applied to a transmitting end, wherein the transmitting end is configured in a satellite communication system, comprises:
[0008] Dividing the acquired satellite image into a plurality of image sub-blocks, wherein the size of the image sub-blocks is consistent with the size of each image template in the preset template set, and each image template includes a plurality of image sub-templates;
[0009] Perform a traversal operation to match the current image block with any image template in the preset template set. Based on the judgment engine, determine a hash code that matches the corresponding image template and a splicing instruction for splicing the corresponding image template. The hash code is generated based on the unique structural feature information of the matched image template or image sub-template in the preset template set. The splicing instruction is used to indicate the relative splicing order and spatial position relationship of the image sub-template in the image sub-block.
[0010] A data frame is constructed according to the hash code and the splicing instruction, and the data frame is sent to a receiving end via a satellite link.
[0011] The determining the relationship between the image sub-block and the image template includes:
[0012] Determine whether the structural features of the current image sub-block match those of any image template in the preset template set;
[0013] If there is a match, get the hash code of the corresponding image template;
[0014] If there is no match, if a splicing path exists, a splicing path formed by combining multiple image sub-templates and matching the structural features of the current image sub-block is determined from the preset image sub-template splicing tree;
[0015] Get the hash code of the image sub-template involved in stitching and generate the corresponding stitching instructions.
[0016] Determining a splicing path formed by combining multiple image sub-templates and matching the structural features of the current image sub-block from a preset image sub-template splicing tree includes:
[0017] Calculating structural clues of the current image sub-block, wherein the structural clues include edge main directions, corner point distribution patterns and local contour features;
[0018] According to the structural clues, a plurality of nodes matching the structural clues are marked in the image sub-template splicing tree as candidate nodes;
[0019] Taking the candidate node as a starting point, performing a fixed-length combined path construction along the image sub-template stitching tree, and generating a stitching image structure according to the combined path;
[0020] The similarity of the structural features between the spliced image structure and the current image sub-block is calculated, and the combined path corresponding to the spliced image structure with the greatest structural feature similarity is used as the splicing path.
[0021] Performing a fixed-length combined path construction along the image sub-template stitching tree, including:
[0022] Divide the current image sub-block into a plurality of structured splicing blocks with ordered positions, and determine covered blocks and blocks to be covered based on the positions of the candidate nodes in the current image sub-block, wherein the size of the structured splicing blocks is consistent with the size of the image sub-template;
[0023] The main extension direction of the covered block is determined according to the relative position of the candidate node, and the next block to be covered is determined according to the extension main direction. The structural clues of the next block to be covered are obtained, and a child node is selected from the child nodes of the candidate node as a new candidate node according to the structural clues. The block to be covered is marked as a covered block, and the main extension direction of the covered block is repeatedly determined until a fixed length is reached.
[0024] If there is no new candidate node among the child nodes of the candidate node, the method further includes:
[0025] Reselect a candidate node as a new root node and perform an iterative operation on the new root node, wherein the root node represents the starting point of the combined path, and the iterative operation includes:
[0026] Determine the main extension direction of the covered block based on the relative position of the root node, determine the next block to be covered based on the extension main direction, obtain the structural clue of the next block to be covered, select a child node from the child nodes of the root node as a new candidate node based on the structural clue, and mark the block to be covered as a covered block. Repeat the determination of the main extension direction of the covered block until a fixed length is reached;
[0027] If the new root node cannot reach the fixed length, a new candidate node is selected as the current new root node, and the iterative operation is performed on the current new root node until all candidate nodes are traversed.
[0028] If there is no splicing path, the image data of the current image sub-block is encoded as the original data, and a data frame is constructed and sent to the receiving end.
[0029] An image transmission method is applied to a receiving end, wherein the receiving end is configured in a satellite communication system, and the method comprises:
[0030] receiving a data frame transmitted via a satellite link, the data frame including a hash code of an image template, a splicing instruction, or image data;
[0031] Performing a restoration operation on the hash code information contained in the data frame according to a locally preset template set to obtain an image sub-block;
[0032] Restore the corresponding image sub-blocks according to the splicing instruction;
[0033] A satellite reconstructed image is generated according to the image sub-blocks, the restored image sub-blocks and the image data.
[0034] The stitching instruction is used to indicate the relative stitching order and spatial position of multiple image sub-templates in the image sub-block. The stitching instruction controls the combination of multiple image sub-templates according to a preset topological relationship to restore the image structure of a single image sub-block.
[0035] Upon receiving the transmitted satellite image, the method further comprises:
[0036] performing image comparison processing on the satellite reconstructed image and the satellite image;
[0037] Identifying, based on the image comparison results, regions with missing structures or regions with splicing artifacts in the satellite reconstructed image;
[0038] Performing image restoration processing on the structure missing area or the splicing artifact area;
[0039] According to the spatial distribution characteristics of the structure missing area or the splicing artifact area, the tree nodes where the image template hash matching fails or the splicing path is abnormal in the preset image sub-template splicing tree are located;
[0040] The tree nodes are updated, and the update results are synchronized at the sending end.
[0041] A satellite communication system, comprising: a transmitting end and a receiving end, wherein:
[0042] The transmitting end is configured to: divide the acquired satellite image into a plurality of image sub-blocks, wherein the size of the image sub-blocks is consistent with the size of each image template in a preset template set, and each image template includes a plurality of image sub-templates; perform a traversal operation to match the current image sub-block with any image template in the preset template set, and set a judgment engine to judge the relationship between the image sub-block and the image template, determine a hash code that matches the corresponding image template and a splicing instruction for splicing the corresponding image template, wherein the hash code is generated based on unique structural feature information of the matched image template or image sub-template in the preset template set, and the splicing instruction is used to indicate the relative splicing order and spatial position relationship of the image sub-templates in the image sub-block; construct a data frame based on the hash code and the splicing instruction, and transmit the data frame to the receiving end via a satellite link;
[0043] The receiving end is configured to receive a data frame transmitted via a satellite link, the data frame including a hash code of an image template, a splicing instruction, or image data; perform a restoration operation on the hash code information included in the data frame according to a locally preset template set to obtain an image sub-block; restore the corresponding image sub-block according to the splicing instruction; and generate a satellite reconstructed image based on the image sub-block, the restored image sub-block, and the image data.
[0044] Compared with the prior art, the present invention has the following beneficial effects:
[0045] By introducing an image sub-template splicing tree structure and an encoding mechanism based on structural feature matching, this invention achieves efficient identification and reuse of repetitive structural regions in satellite imagery. This significantly reduces data transmission while ensuring image restoration quality. Compared to traditional whole-image compression methods, this invention transmits only image regions already in the historical template library in hash-coded form, preventing redundant data from occupying link bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0047] Figure 1 This is a schematic diagram of an exemplary application scenario of an embodiment of the present application;
[0048] Figure 2 This is a flow chart of an image transmission method according to an embodiment of the present application;
[0049] Figure 3 A schematic diagram of a flow chart for determining the relationship between an image sub-block and an image template according to an embodiment of the present application;
[0050] Figure 4 This is a structural diagram of the image sub-template splicing tree according to an embodiment of the present application;
[0051] Figure 5 A flowchart of a method for determining a splicing path according to an embodiment of the present application is provided;
[0052] Figure 6 This is a schematic diagram of image sub-block splitting according to an embodiment of the present application;
[0053] Figure 7 This is a schematic diagram of the block expansion principle of an embodiment of the present application;
[0054] Figure 8 This is a schematic diagram of the splicing principle of some embodiments of this application;
[0055] Figure 9 This is a flowchart of another image transmission method according to an embodiment of the present application. DETAILED DESCRIPTION
[0056] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0057] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It will be understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0058] This application is mainly aimed at high-frequency, low-variability satellite imaging scenarios, and its application scenarios include but are not limited to:
[0059] Ocean surface observation, dynamic monitoring of desert areas, farmland inspection in fixed areas, port and terminal management, and remote sensing of specific urban landforms.
[0060] The selection of application scenarios is based on the common characteristics of the region. The characteristics that the selected application scenarios must have include:
[0061] The observation area is relatively fixed, that is, the coordinates of the ground area repeatedly photographed by the satellite remain basically unchanged;
[0062] The image structure features are stable, such as coastlines, farmland boundaries, and urban road distribution, which have strong temporal consistency and topological regularity;
[0063] Background changes are slow or predictable, such as seasonal vegetation changes and tidal changes, which can be modeled through long-term statistics.
[0064] See also Figure 1 , which is a schematic diagram of an exemplary application scenario provided in an embodiment of the present application.
[0065] like Figure 1 As shown in the figure, the application scenario consists of a meteorological satellite, a transmitter, and a receiver. The three interact with each other through a transmission network.
[0066] Figure 1 A meteorological satellite is shown as an image source, which is equipped with an image sensor and an image preprocessing module for collecting large-scale surface images on orbit and performing preliminary image enhancement, denoising, and region division.
[0067] Figure 1The transmitter is located on the satellite platform or in a relay module, responsible for image encoding and compression. After image preprocessing, the transmitter performs image sub-block division and template compression decisions, minimizing data transmission. In certain cases, the transmitter can also be located in the data transfer link between the satellite and the ground station, providing relay-level processing and cache optimization.
[0068] Figure 1 The receiving end is shown. This is located in a ground-based image reception center or remote sensing application system and is responsible for receiving, restoring, and reconstructing complete satellite images. This image data can ultimately be used for downstream tasks such as weather forecast model input, geological environment analysis, agricultural monitoring systems, and marine situational awareness. Figure 1 It shows that the receiving end includes a data frame receiving module, a satellite image restoration module and a splicing path optimization module, and the sending end includes a data frame generation module, a judgment engine and a satellite image segmentation module.
[0069] In one example, the transmission network includes a Ka-band high-throughput communication link between the satellite and the ground station, and a fiber optic backbone network between the ground station and the data center, which supports high-speed and stable transmission of image data and meets the needs of multi-source, multi-channel concurrent data reception.
[0070] In another example, to improve system robustness, the receiving end can also be set up between multiple ground sites, and data redundant reception and verification can be achieved through a distributed network architecture, thereby improving the success rate of image restoration in scenarios such as environmental occlusion and link interruption.
[0071] Next, an image transmission method provided in an embodiment of the present application is introduced with reference to the accompanying drawings.
[0072] See also Figure 2 , which is a flow chart of an image transmission method provided in an embodiment of the present application. Figure 2 The method shown can be applied to a transmitting end of a satellite communication system, wherein the satellite communication system further comprises a receiving end configured to receive data frames and restore satellite images based on the data frames. Figure 2 The method shown includes the following steps S1-S3, and the specific steps are as follows:
[0073] S1: Divide the acquired satellite image into multiple image sub-blocks;
[0074] The size of the image sub-block is consistent with the size of each image template in the preset template set, and each image template includes multiple image sub-templates;
[0075] In this embodiment, the satellite imagery originates from fixed observation areas, including but not limited to remote sensing target areas with long-term stable structures, such as farmland, coastal zones, and desert fringes. By spatially partitioning the satellite imagery into multiple, uniformly sized image sub-blocks, this ensures a uniform baseline for subsequent matching operations. Because the captured scenes are fixed and their structural patterns are stable, a template set can be pre-built for efficient recognition and reconstruction, reducing overall data processing complexity and improving matching success rates.
[0076] S2: Match the current image sub-block with any image template in the preset template set, and set a judgment engine to judge the relationship between the image sub-block and the image template, determine the hash code matching the corresponding image template and the splicing instruction for splicing the corresponding image template;
[0077] The hash code is generated based on the unique structural feature information of the matched image template or image sub-template in the preset template set, and the splicing instruction is used to represent the relative splicing order and spatial position relationship of the image sub-template in the image sub-block;
[0078] In this embodiment, the judgment engine quickly compares the current image subblock to see if it fully matches any full-image template. If not, it further searches the stitching tree structure to see if a combination of multiple image sub-templates can restore the current image subblock. This structure prioritizes template matching and supplements stitching judgment, thereby improving the success rate of compressed transmission and reducing the transmission ratio of the original image.
[0079] In this embodiment, hash coding adopts a structural signature generation algorithm to ensure the uniqueness and reliability of template recognition, and the splicing instruction adopts a construction method based on template position index and relative position matrix, taking into account both transmission efficiency and splicing restoration accuracy.
[0080] S3: constructing a data frame according to the hash code and the splicing instruction, and sending the data frame to a receiving end via a satellite link;
[0081] In this embodiment, the data frame encapsulation includes a data category identification field, a template hash value, and splicing structure instructions or the original image content field. Ultimately, the data frame is transmitted to a ground receiver via a satellite link. This link can utilize Ka-band or laser communication channels, offering high bandwidth and strong anti-interference properties, ensuring real-time and efficient downlink of remote sensing image data.
[0082] Although research on satellite image compression and transmission has made certain progress in existing technologies, such as reducing data volume through region of interest coding, adaptive resolution compression, or image feature extraction compression, in actual satellite communication scenarios, especially during real-time transmission, there are problems such as limited bandwidth, short-term channel availability, or frequent interruptions. On the one hand, most existing compression methods rely on overall image structure analysis, which is complex to process and computationally intensive, making them unsuitable for satellite-side deployment. On the other hand, although image compression reduces data volume, its compression granularity and structural reuse capabilities are still relatively weak in remote sensing areas where repeated structures frequently appear, resulting in a large number of redundant images that still need to be transmitted, causing a waste of channel resources.
[0083] In this embodiment, after dividing a satellite image into sub-blocks amenable to structured analysis, the system prioritizes whether the sub-block can be restored using an existing image template or combination of image sub-templates. Once a match is found, only the hash code and stitching instructions need to be sent, eliminating the need to transmit the image itself. This breaks through the limitations of traditional image compression, which relies on content weight reduction, and instead approaches it from a structural reuse perspective, establishing a new paradigm for image space reconstruction. The use of a stitching tree allows for predefined and efficient search of topological stitching paths between multiple image sub-templates, achieving high restoration accuracy with low computational overhead. This approach is particularly suitable for resource-constrained environments such as satellite platforms.
[0084] It should be noted that the splicing path judgment process used in this embodiment is not a simple image similarity matching, but rather a process of extracting structural clues and mapping spatial splicing blocks. Taking an actual scenario as an example, when a satellite repeatedly monitors a port area, its shoreline, berths, roads, cargo yards, and other structures remain basically unchanged. A certain image sub-block may not be directly matched with the template due to slight differences in lighting or viewing angle. However, through the splicing tree structure, if it is found that its image content is highly consistent with several sub-templates in the historical splicing path, it is only necessary to splice a pre-set number of sub-templates to restore the original sub-block. At this time, only the hash code of the template and the splicing instruction are transmitted to complete the communication, greatly alleviating the link pressure.
[0085] See also Figure 3 , this figure is a schematic diagram of the process of determining the relationship between an image sub-block and an image template provided in an embodiment of the present application. Figure 3 The method shown can be applied to S2 of the above method, and the specific steps are as follows:
[0086] S2.1: Determine whether the structural features of the current image sub-block match those of any image template in the preset template set;
[0087] Specifically, the fundamental purpose of this determination is to prioritize whether the current image subblock already appears in the historical template set through lightweight structural feature analysis before transmission, thereby minimizing the amount of raw image data contained in the data frame. In satellite communications, limited by physical resource constraints such as communication window duration and available bandwidth, the efficiency of image data compression and matching directly determines the integrity and real-time performance of data transmission. Therefore, structural feature matching can quickly screen for the presence of reusable templates without decoding the image content.
[0088] It should be noted that in this embodiment, structural features and structural clues are two clearly distinguished concepts in this application, with non-overlapping functions and different stages of action. They are used in two stages of the image sub-block processing flow: overall structure matching and stitching path construction. Structural features are used to measure the overall image representation information used to measure the structural composition and topological geometric properties of the entire image sub-block. They include but are not limited to describing the macroscopic structural outline, shape distribution, key geometric point relationships, and edge configuration patterns of the image sub-block. They are characterized by indivisibility, closure, and integrity. The goal is to discover whether the image sub-block has appeared as a whole in the historical template library.
[0089] In this embodiment, the calculation process of the structural features includes but is not limited to the following:
[0090] First, edge detection is performed on the current image sub-block. The Canny operator is used to extract the main edge contours. To avoid texture interference, Gaussian blur is performed to remove high-frequency noise. Edge lines with high continuity and high contour closure are retained to form a preliminary contour map.
[0091] Perform edge direction extraction on the preliminary contour image, count the gradient direction distribution of all contour pixels, construct an edge direction histogram and calculate the directional principal component (main direction angle) to obtain the structural symmetry and directional consistency information of the image.
[0092] Perform corner detection (Harris or Shi-Tomasi) on the preliminary contour map, establish a corner connection map for the corner points through Delaunay triangulation or K-neighbor connection, and extract parameters such as connection length statistics, average connectivity, and corner distribution heat map as feature dimensions.
[0093] The geometric form of the preliminary contour image is characterized by shape moments (Hu moments, Zernike moments), the geometric ratio of the internal area of the closed contour to the external boundary is calculated, and the structural geometric features regarding compactness, complexity and structural repetition are extracted.
[0094] The image sub-blocks are divided into 3×3 or 4×4 grids. Local indicators such as edge strength, main direction, and corner point distribution density are repeatedly extracted in each grid and spliced into overall features to enhance the sensitivity of feature expression to positional relationships.
[0095] After vector concatenation of the structural parameters (gradient, corner points, topology, shape) of the aforementioned multiple modes, dimensionality reduction is performed using methods such as PCA and ICA to obtain fixed-length, stable structural features; the structural features are saved in the form of vectors, which are used as input for subsequent structural template matching judgment.
[0096] Furthermore, the matching process of this embodiment includes three stages: template feature index construction, initial screening and comparison, and structure mapping verification, as follows:
[0097] First, a feature index library is pre-built on the sending end based on the structural features of all image templates in the template collection. This index library uses an approximate nearest neighbor retrieval structure based on vector space distribution. In one example, all template vectors are grouped using locality-sensitive hashing to form a hash bucket structure, where the structural features within the buckets share similar main directions and topological configurations. In another example, a KD tree is used to construct a multidimensional feature partitioning space, supporting fast range queries. Each template structural feature vector is labeled with a unique template ID and mapped to its image hash code.
[0098] Secondly, when performing a matching judgment for the current image sub-block, its structural features are first used as input, and an approximate nearest neighbor initial screening operation is performed in the feature index library to obtain a set of candidate templates. The number of candidates can be set by those skilled in the art. Within this candidate set, a similarity measure between the structural features is further calculated. In this embodiment, the Euclidean distance is not used. Instead, a multi-core feature similarity fusion method is used, that is, similarity measurements are performed on the edge principal direction sub-vector, the corner map topology sub-vector, and the regional compactness sub-vector respectively. The comprehensive structural feature similarity score is then calculated according to a weighted fusion rule. The specific weights can be set by those skilled in the art through extensive experiments.
[0099] Preferably, this embodiment also includes structure mapping verification, which performs a pixel-level structure alignment operation between the current sub-block and the candidate template. Instead of directly comparing pixels, the two edge maps are reprojected into a normalized grid coordinate system, and the spatial overlap of their edge areas, the direction difference of the main contour lines, and the consistency rate of the corner mapping are calculated, which are used as the structure mapping verification score.
[0100] Finally, based on the combined structural feature similarity score and the structural mapping verification score, the total score is mapped to the match confidence level using an S-shaped confidence function. If the confidence level exceeds a set dynamic judgment threshold, which is adaptively adjusted based on the historical false positive rate and regional structural complexity, the image sub-block is considered to have successfully matched the structural features of the corresponding template in the template library. At this point, the hash code corresponding to the template is output for subsequent data frame construction.
[0101] S2.2: If there is a match, obtain the hash code of the corresponding image template;
[0102] S2.3: If there is no match, if a splicing path exists, determine a splicing path formed by combining multiple image sub-templates and matching the structural features of the current image sub-block from a preset image sub-template splicing tree;
[0103] Specifically, if structural feature matching fails, directly returning to the original image transmission path will still incur a high communication burden. Therefore, in this embodiment, driven by structural clues, possible splicing combination paths are dynamically constructed in the image sub-template splicing tree, attempting to approximate the current image sub-block structure by combining multiple image sub-templates. If the splicing path is successfully constructed, there is no need to transmit the original image; only the splicing structure and hash code information are sent.
[0104] In this embodiment, in order to make the path construction controllable and guideable, structural clue information is first extracted from the current image sub-block. Structural clues are different from structural features. They are used to describe the incomplete structural properties of the unmatched area in the current image sub-block, including but not limited to the main direction of the fracture edge, the directionality of the corner point connection missing area, and the boundary normal information of the structural fracture in the local contour.
[0105] To further extract structural clues, the image sub-blocks are first divided into a 3×3 position-aware grid, and edge structure distribution heatmaps and corner sparsity maps are annotated. Edge extension direction fitting is performed on uncovered areas to extract the main fracture direction vector. A tension map is then calculated for the blank structural areas near this direction. This evaluates the geometric extension trend from the current structural boundary to the blank area (e.g., whether it favors straight-line splicing, L-shaped bend splicing, or curve-fitting splicing), forming a structural clue map. Each structural clue consists of a starting point region, a direction vector, and a structural description of the target splicing segment (e.g., if L-shaped corner padding or straight-line edge padding is required in the upper left corner).
[0106] Furthermore, after obtaining structural clues, they are used as the starting point and direction guide input for the stitching path, initiating path expansion in the stitching tree. The stitching tree is a pre-defined directed graph structure, with all image sub-templates as tree nodes. Edges connecting nodes indicate the presence of stitchable boundaries between templates. Connection attributes include information such as the stitching direction and the matching level of boundary features.
[0107] It should be noted that the maximum depth of path construction is equal to the upper limit of the number of stitching. Since the size of the image sub-block is constant and the size of the sub-template is constant and the same, under preset conditions, it is possible to stitch a fixed number of sub-templates to generate a stitched image with the same size as the image sub-block.
[0108] S2.4: Obtain the hash code of the image sub-template involved in the stitching and generate the corresponding stitching instruction;
[0109] Take the image sub-template stitching tree as an example, see Figure 4 , Figure 4 This paper demonstrates an example of constructing an image sub-template stitching tree, where each node corresponds to an image sub-template, and directed edges between nodes indicate that the node can be spatially stitched with its underlying nodes. The stitching relationship not only includes stitching direction information but also further describes the feature consistency of the stitching boundary, such as edge alignment, corner closure, and texture distribution coherence.
[0110] Figure 4 In the image sub-template stitching tree, each node represents an image sub-template that can be used for structure restoration. The root node is the starting fragment of any image sub-template, and it has multiple child nodes, such as Figure 4 Subnodes 1 through 8 in the diagram represent the next template candidate that can be spliced based on the current node. Nodes are connected by directed edges, each of which is labeled with a corresponding splicing direction, such as "up," "down," "left," "right," "upper left," "upper right," "lower left," or "lower right," to guide the spatial expansion of the splicing path.
[0111] It should be noted that although the figure shows the sub-node structure in eight typical directions, not every node in the actual splicing tree has complete eight-directional branches. Whether there is a sub-node in a certain direction depends on whether the image sub-template represented by the node supports the splicing logic in that direction in terms of boundary structure, texture features, and spatial compatibility. Some nodes may only have valid expansion paths in 1 to 2 directions, forming a non-fully balanced tree structure. In addition, the use of ellipsis and leaf nodes 1 to leaf nodes N in the figure to represent lower-level nodes and branches is only for reference, and not all paths and nodes are expanded. In the actual structure, the depth and width of the tree are determined by the image partition granularity and the number of templates, and are dynamically scalable.
[0112] Furthermore, Figure 4The leaf node shown in represents the termination template fragment of a certain splicing path. Its corresponding splicing depth has met the image sub-block structure coverage requirements, or it is terminated due to no valid direction expansion. Multiple leaf nodes may belong to the same parent node, or they may be independently derived from different paths. The splicing tree supports structural reuse and path overlap. Figure 4 Under the organizational mode, the direction guidance, spatial recursion and efficient search of the structure splicing path can be realized, providing a path planning basis for the judgment engine of the sending end.
[0113] In this application, the image sub-template stitching tree theoretically includes the stitching combination of all sub-templates. To ensure the convergence and uniqueness of the stitching path construction and the reproducibility of the stitching structure, the image sub-template stitching tree must meet the following conditions:
[0114] The root node of the tree can be any structural sub-template, which is used to support the splicing requirements of any structural starting point and meet the flexibility requirements of local image reconstruction;
[0115] The child nodes of each node are arranged in the order of splicing direction, such as storing them according to the priority of up, down, left, right, upper left, lower right, etc., to ensure the predictability and directional consistency of path traversal, and facilitate direction-guided expansion driven by structural clues;
[0116] Only spatially continuous and structurally compatible connections are allowed between nodes. Specific examples include but are not limited to: the edges connecting two nodes are adjacent in space, the boundary direction angle difference does not exceed the set tolerance, the boundary length ratio difference does not exceed the set ratio threshold, and the corner-connected graph has an isomorphic subgraph relationship;
[0117] Duplicate nodes are not allowed in each splicing path to avoid redundant transmission problems caused by self-overlapping structures and cyclic paths, and to maintain the semantic integrity and decodability of the path structure.
[0118] After each stitching path is formed, it should meet the structural closure condition, that is, the boundary of the image area after stitching should be a closed contour, and no isolated edges or internal breakpoints should appear.
[0119] The maximum depth of the tree structure can be set according to the granularity of the image sub-block division. The number of sub-nodes in each layer is limited to a fixed window to prevent the exponential growth of the path space. It should be noted that since the sub-paths of the splicing path may also serve as the final splicing path in actual applications, the specific depth can be set by those skilled in the art and is not specifically limited in this application.
[0120] Each node is accompanied by spatial position information and its offset relative to the upper node. This information will be used for spatial mapping of splicing instructions to ensure the consistency of the structural restoration logic of the spliced image sub-blocks.
[0121] In an example, the process of constructing the image sub-template splicing tree is as follows:
[0122] First, the structural encoding information of all image sub-templates in the template set is obtained. This structural encoding includes, but is not limited to, edge contour orientation, corner topology, local shape descriptors, and spatial position information. In the initial stage, each image sub-template is treated as an independent root node candidate, forming multiple construction starting points. This ensures that any image structure can serve as the starting point of the stitching path, supporting subsequent stitching requirements with adaptive local starting points.
[0123] Secondly, for each root node, we enumerate eight standard spatial splicing directions for its structural boundaries: up, down, left, right, upper left, upper right, lower left, and lower right. For each direction, we search the template collection to see if there is a target sub-template that meets the splicing criteria. The specific splicing criteria have been described above and will not be repeated here.
[0124] Third, when the above conditions are met, a directed edge is established, and the target template is added to the splicing tree structure as a child of the current node. To maintain the tree structure's traversal order and splicing consistency, the child nodes of each node are stored in order according to a preset splicing direction. In this embodiment, the directions can be sorted as up, down, left, right, upper left, upper right, lower left, and lower right.
[0125] Fourthly, in order to control the scale of the splicing tree and limit its expansion width and depth, the tree construction process also includes recursive termination conditions: the splicing depth does not exceed the set maximum number of splicing fragments, the same node cannot appear repeatedly in the same path, and if there are holes in the boundaries of the structural area formed by splicing or the corner connection is broken, pruning is immediately carried out.
[0126] See also Figure 5 , which is a flow chart of a method for determining a splicing path provided in an embodiment of the present application. Figure 5 The method shown can be applied to S2.3 of the above method. The specific steps are as follows:
[0127] S2.3.1: Calculate structural clues of the current image sub-block, wherein the structural clues include edge main directions, corner point distribution patterns, and local contour features;
[0128] The structural cues and calculation methods discussed in this application have been previously introduced and will not be repeated here. It is important to note that structural cues are not abstract features representing the overall structure of an image sub-block, but rather guidance information specifically used to complete local structure during the stitching path construction phase. These cues possess directional, position-dependent, and morphologically compensated guidance properties. Their core purpose is to limit the selection range for the starting point and the range of expansion directions for the stitching path, thereby reducing the combinational complexity of the stitching path construction and ensuring that the stitching process occurs within a controllable, reversible, and spatially continuous path space.
[0129] S2.3.2: Marking, in the image sub-template stitching tree, a plurality of nodes matching the structural clues as candidate nodes based on the structural clues;
[0130] Specifically, the role of structural clues is to transform the stitching path construction from blind tree structure traversal to target-guided path starting point positioning. The purpose of this step is to use the structural clues extracted in the previous stage to accurately select a group of image sub-template nodes that may constitute the stitching starting point in the image sub-template stitching tree as candidate nodes for subsequent path expansion.
[0131] In this embodiment, the candidate node screening mechanism includes two stages. The first stage is direction matching. Based on the main edge direction and corner tension direction in the structural clues, templates with similar main edge directions and nodes with a close corner clustering pattern are screened from all sub-template nodes in the splicing tree. The second stage is position estimation screening. Combined with the spatial position of the defective area in the current image sub-block, nodes with relative spatial offset directions in the structural position in the splicing tree are preferentially selected. It is easy to understand that the defective area does not refer to the area in the image sub-block that is defective, but refers to the area in the image sub-block that is significantly different from other image templates, that is, the area that has changed and has no corresponding template.
[0132] Further, the screening mechanism can refer to the following:
[0133] First, the structural clue includes the principal direction vector of the edge break region (the normal direction of the principal break edge). Each image sub-template node in the splicing tree, when constructed, already includes a set of structural direction description vectors, indicating the presence of open or splicable edges in a certain direction within the template. During screening, the angle between the structural clue direction vector and the structural direction vector of each node in the tree is calculated, using the structural clue direction vector as a reference. Nodes with orientation differences exceeding a set tolerance are filtered out, and the relative splicing direction indexes of nodes with similar orientations are marked.
[0134] Secondly, the structural clues also include the tension direction and density distribution of corner points within the incomplete region, indicating new corner features that may need to be connected in that region. Each node in the splicing tree also stores corner topology information, including corner heatmaps, corner map connectivity, and boundary opening patterns. By encoding the structural clue corner hotspots in a standard grid and comparing them with the node corner distribution for similarity, nodes with completely different corner distributions are removed, retaining structural templates with potential connectivity.
[0135] Third, structural clues also include the spatial location of the missing region within the image sub-block. Each node in the splicing tree is bound to its offset vector in the direction of its parent node during construction. This phase quickly matches the clue's spatial position information with the node's offset path (for example, matching the parent node's center to determine whether it can be spliced to the target location), eliminating nodes with obvious spatial conflicts.
[0136] Fourthly, nodes filtered through the above steps are marked as candidate nodes and sorted by their comprehensive similarity scores. The comprehensive similarity score is composed of a direction matching score, a corner hotspot similarity score, and a spatial pose overlap score. These scores are normalized and then calculated using a weighted fusion strategy. Each candidate node is assigned a confidence score, which is used to prioritize expansion during subsequent path construction.
[0137] S2.3.3: Starting from the candidate node, construct a fixed-length combined path along the image sub-template stitching tree, and generate a stitched image structure according to the combined path;
[0138] Specifically, after selecting multiple candidate nodes, to restore the image sub-block, this step sequentially uses these candidate nodes as starting points, extending the stitching path in the stitching tree in the direction suggested by the structural clues with a finite number of steps, and constructing a set of combined stitching image structures based on the path. The goal of path construction is to complete the combination of no more than the maximum number of stitching segments while satisfying structural continuity, spatial coverage, and stitching consistency, thus forming a candidate stitching solution.
[0139] In this embodiment, path construction adheres to the principles of structure-driven, spatial expansion, and directional consistency. At each expansion step, an expansion vector is calculated based on the spatial position of the current node and the direction of the structural clue. The next hop node is searched only in that direction. The next hop node is then checked to see if it meets certain conditions, including but not limited to: matching the splicing boundary (within a side length error threshold), splicing direction coherence (direction difference less than a set angle), and structural closure potential (whether closure is possible after splicing). All nodes in the path must be non-duplicate, non-spatially overlapping, and spatially continuous. Once the path reaches a specified length, expansion ceases, forming a single splicing path.
[0140] Furthermore, the constructed path corresponds to a set of image sub-template hash ID sequences and their spatial displacement matrices, and virtual stitching is immediately performed locally (no pixel-level stitching, only contour map reconstruction) to form a candidate stitched image structure for structural feature matching with the original image sub-blocks.
[0141] S2.3.4: Calculate the similarity of the structural features between the stitched image structure and the current image sub-block, and use the combined path corresponding to the stitched image structure with the greatest structural feature similarity as the stitching path;
[0142] Specifically, after generating multiple stitching paths and their corresponding image structures, a structural similarity calculation is performed to determine which path is closest to the target image sub-block and optimize it. This step not only aims to find the optimal reconstruction path but also provides accurate stitching instructions and template ID input for data frame construction, ensuring the correct restoration of the image sub-block.
[0143] In this embodiment, structural feature vectors are first extracted for the stitched image structure and the current image sub-block (using the same feature extraction logic as in S2.1). Sub-similarity metrics are then calculated for multiple sub-vector dimensions (edge principal direction, corner map, shape moment, and boundary closure). Finally, a weighted fusion strategy is used to generate a structural similarity score. This score reflects the degree of similarity between the stitched image structure and the original image structure in terms of geometric contours, topological relationships, and spatial density distribution.
[0144] It is important to note that in the aforementioned path construction step, the generation of each stitching path is strictly driven by the structural cues of the image sub-block. These cues explicitly define the main edge direction, corner distribution trend, and spatial defect location of the unmatched regions in the current image sub-block. The stitching path generation algorithm uses directional consistency constraints, corner isomorphism graph verification, and spatial stitching closure as extended judgment criteria. In other words, the prerequisite for successful path construction is that its local structural features complement the defective regions of the target sub-block and that the connections are reasonable. Furthermore, spatial occupancy control and contour closure judgment mechanisms are implemented during the stitching path construction process to ensure that the stitched image structure not only covers the missing structural regions in the target image sub-block, but also maintains continuous stitching boundaries, connected corners, and forms closed loops or effective connections at the contour level. This process essentially completes a structural screening and rough alignment. Therefore, in this step, there is no need to re-judge similarity. Instead, a further optimization operation is performed by comparing the structural feature vectors among the candidate paths constructed through the structural consistency constraints.
[0145] Taking image sub-block splitting as an example, you can refer to Figure 6 To understand, Figure 6 This is a schematic diagram of image sub-block splitting according to an embodiment of the present application. Figure 6 This figure illustrates the spatial division of an image sub-block during the path stitching construction process. In this embodiment, to improve readability and comprehensibility, the image sub-block is divided into a 3×3 grid of structural stitching blocks, where the size of each block matches the size of the image sub-template. It is easy to understand that in this case, nine image sub-templates are required for stitching. The divided blocks are numbered and identified in a two-dimensional spatial order, dynamically recording the current stitching path coverage.
[0146] Figure 6 The image shows a patch in the lower right corner of the current image sub-block, which has been covered by a candidate node and is labeled "covered." The remaining eight patches are in the "pending patch" state. In the initial path construction phase, the patching process starts with the covered patch and, based on the node's relative position within the image sub-block, derives the main patching extension direction guided by structural clues. Based on this direction, matching nodes are selected from the child nodes of the patching tree for the next patching step.
[0147] It should be noted that in actual applications, since satellite images are not necessarily squares in the strict sense, image sub-blocks cannot necessarily be cut into squares. Therefore, the specific settings need to be based on the actual situation. In addition, you also need to consider whether splicing is possible when splicing. This application assumes that satellite images can be split into multiple squares of the same area, and the same applies to image sub-blocks. Therefore, when splicing, you do not need to consider whether the image sub-template meets the length and width requirements of splicing.
[0148] Taking the extension block as an example, please refer to Figure 7 To understand, Figure 7 This is a schematic diagram of the block expansion principle of the embodiment of this application. Figure 7 The principle and steps of path expansion based on the currently covered block and the direction guided by the structural clues in the stitching path construction process are shown. Specifically, in the current image sub-block, the area covered by the stitching template is marked as "covered block", and the remaining area is "block to be covered". Figure 7 In the case shown, the current stitching path starts from the template at the lower right corner of the image sub-block, and its structural clue analysis shows that the main direction of expansion is "upward" (e.g. Figure 7 Therefore, the upper block to be covered is selected as the next expansion target of the current path.
[0149] In this embodiment, the calculation of the extended main direction is not determined by a preset rule, but is derived through spatial analysis based on the spatial distribution state of the area to be covered in the image sub-block, the directional information of the structural clues and the spatial center offset vector of the covered block.
[0150] Specifically, we first perform position grid division on the image sub-blocks, number each spliced block and build a position matrix. For example, we number the nine blocks as to , corresponding to its row and column position in the image sub-block. For all blocks covered by the current stitching path, calculate the centroid coordinates of these blocks; at the same time, calculate the collective centroid of the remaining "blocks to be covered".
[0151] Furthermore, a spatial direction vector is calculated based on the centroid coordinates and the set centroid. This spatial direction vector represents the direction from the center of the covered structure toward the center of the area to be expanded. This direction reflects the centroidal trend of the natural completion of the image structure in space and can serve as a candidate basis for the main direction of expansion.
[0152] Furthermore, the spatial direction vector is cross-validated with directional information extracted from structural cues within the currently covered area. Structural cues include the main strike vector of the local fracture edge, reflecting the geometric extension trend of the structural fracture in the current image. The angle between the spatial direction vector and the directional information is calculated. If the angle is less than a set directional consistency threshold, the two are considered to be converging. In this case, the direction indicated by the spatial direction vector is preferentially used as the path extension direction. Otherwise, it indicates a conflict between the geometric direction of the image structure defect and the spatial region expansion direction. In this case, the fusion vector of the spatial direction vector and the directional information is used as the primary reference vector, and the expansion target is repositioned in this direction.
[0153] Figure 7 It shows that it is determined whether there is a template that can be spliced in the block to be covered, and its structural clues are input into the image sub-template splicing tree to perform sub-node matching. Figure 7 As shown, the current node in the splicing tree has multiple child nodes (child node 1, child node 2), each of which is arranged in an orderly fashion based on a specific splicing direction and a structural template as an index. In this embodiment, the child nodes of the splicing tree are encoded and arranged according to a preset directional priority (e.g., up, down, left, right, upper left, upper right, lower left, lower right). Therefore, when performing splicing expansion, the target child node in the corresponding direction can be directly located for a preferred match based on the primary expansion direction contained in the structural clue of the block to be covered.
[0154] Furthermore, within the prioritized child node, operations such as directional consistency determination, edge structure overlap comparison, and corner point connectivity verification are sequentially performed to determine whether it meets the splicing requirements. If the child node structure does not meet the requirements, a neighbor diffusion strategy is further implemented within the splicing direction neighborhood of the node, that is, a progressive matching attempt is made towards child nodes in adjacent directions. For example, if the main direction is "up" and the node directly above does not meet the conditions, the child nodes in the "upper left" and "upper right" directions are tried, with the diffusion range within the controllable angle threshold.
[0155] Figure 7 Taking child node 2 as an example, if this child node meets the splicing requirements of the current block to be covered in terms of structural orientation, boundary topology, and spatial occupancy, it can be directly added to the path queue as the next hop node of the splicing path, and the "covered" status of the image sub-block will be updated. At the same time, the offset attribute of this node in the splicing tree will be used to record the splicing instructions, ensuring the correct spatial position of the subsequent image restoration.
[0156] It's easy to understand that if the current stitching is successful—that is, the block to be covered is covered by an image sub-template that meets the structural requirements—then the child node corresponding to that template is marked as "covered" and immediately becomes the latest path end node in the stitching path, participating in the next round of path expansion. At this point, the new node becomes the current expansion node, and the main direction of the next expansion is dynamically adjusted based on its spatial position within the image sub-block and the position and direction of the uncovered area as indicated by the structural clues.
[0157] In this embodiment, the method for determining the main extension direction has been described above. The entire path extension process ends with the complete coverage of the image sub-blocks, and the current path is marked as a candidate splicing path.
[0158] Furthermore, after each round of expansion, the "covered blocks" status map within the image sub-blocks is updated in real time, while the "blocks to be covered" queue and the remaining structural clues are also updated synchronously. This ensures that the splicing path expansion not only maintains directional continuity and structural rationality, but also spatial integrity and splicing closure, ensuring that the resulting splicing path is decodable and structurally coherent for image restoration purposes.
[0159] Obviously, there is a common situation where no child nodes of a candidate node meet the expansion requirements and can serve as a new candidate node. This situation usually occurs in the middle of the splicing path extension. The specific reasons include: the splicing direction of the current node is inconsistent with the structural clue, all its connected child nodes are spatially covered, or the child nodes structurally do not match the constraints of the current block to be covered, such as the boundary direction, corner connectivity, and contour continuity. Such situations will cause the path extension process to reach a dead end, and it will be impossible to complete the structural loop or complete regional coverage.
[0160] In this embodiment, in order to avoid meaningless path depth expansion and resource waste, when it is detected that the tail node of the current path cannot be further expanded in the splicing tree, the path is immediately determined to be invalid and invalidated, that is, the current path construction chain is abandoned, and the next candidate node that has not been used is selected from the candidate node set, and the splicing path construction process is restarted.
[0161] Specifically, the image sub-template splicing tree has a tree-like structural characteristic characterized by nonlinearity, branch redundancy, and strong path mutual exclusivity. During the splicing tree construction process, all template nodes are directedly connected according to structural boundary compatibility, spatial splicing direction, and structural closure potential. Each candidate node can be regarded as an independent starting branch in the tree, and its path expansion does not depend on the state of other node paths. Therefore, even if a path is currently interrupted during splicing, it will not affect the path feasibility of other candidate nodes. Each candidate node can independently start a splicing construction chain. All construction paths do not overlap and are state-isolated, with a natural retry mechanism and path independence.
[0162] The specific contents of candidate node reconstruction are as follows:
[0163] A candidate node is reselected as a new root node, and an iterative operation is performed on the new root node. The iterative operation includes:
[0164] Determine the main expansion direction of the covered block based on the relative position of the root node, determine the next block to be covered based on the main expansion direction, obtain the structural clues of the next block to be covered, select a child node from the child nodes of the root node as a new candidate node based on the structural clues, and mark the block to be covered as a covered block until a fixed length is reached;
[0165] If the new root node cannot reach the fixed length, a new candidate node is selected as the current new root node, and the iterative operation is performed on the current new root node until all candidate nodes are traversed.
[0166] It's important to note that the preceding description describes the case where a stitching path exists. However, in practice, it's inevitable that after traversing all candidate nodes, no valid stitching path that satisfies the structural closure condition may be found. This typically occurs when the image sub-block structure exhibits strong local deformation, the target region structure has no approximate representation in the template library, or all feasible paths in the stitching tree have been pruned due to spatial overlap or structural discontinuity.
[0167] In this embodiment, the image data of the current image sub-block is encoded as original data, and a data frame is constructed and sent to the receiving end.
[0168] As a preferred solution, although the above solution is simple, there is still a problem of not achieving optimal efficiency in the process of transmitting image sub-blocks. In order to improve the overall image transmission compression efficiency, a hybrid encoding strategy of partial splicing and original completion can be used as a preferred solution.
[0169] Specifically, when the system detects that there is no complete stitching path, it does not directly send the entire image sub-block as the original image. Instead, it traces back the aforementioned path construction history and selects the path with the longest completed stitching segment and the highest structural coverage among all the stitching paths that failed to be constructed before as the valid partial path, retaining its hash coding information and stitching instructions; the remaining area in the image sub-block not covered by this partial path is encoded and packaged using the original image data to jointly construct a mixed data frame and send it to the receiving end.
[0170] It's understandable that during the stitching path construction process, the spatial coverage map and corresponding node sequence of each path as it expands can be maintained in real time. This allows for the selection of paths that aren't completely closed, as long as their partial structure matches the current image sub-block, to serve as a valid sub-region compression structure. The path scoring mechanism prioritizes metrics related to structural similarity and spatial coverage, ensuring that the selected path is not only structurally sound but also covers the majority of the image sub-block as much as possible.
[0171] For reference Figure 8 To understand, Figure 8 This is a schematic diagram of the splicing principle of some embodiments of this application. Figure 8 It shows that the current image sub-block is divided into several stitching blocks, where the dark gray blocks represent the image areas that have been successfully covered by the stitching path, and the light gray blocks represent the areas that cannot be structurally covered by any image sub-template after all stitching paths are traversed, that is, "uncoverable blocks."
[0172] Figure 8 It shows that among all the failed stitching paths, the path with the largest stitching length and the best spatial coverage is selected as the "partial stitching path", and the hash codes of all image sub-templates involved in the path and their spatial position relationships in the image sub-blocks are extracted, encapsulated as a stitching instruction field and sent to the receiving end; at the same time, for the remaining uncovered areas in the image sub-blocks (i.e. Figure 8 The pixel data of the corresponding area is directly intercepted from the original image and attached to the data frame in the form of a compressed image block for transmission.
[0173] It should be noted that in order to ensure accurate restoration of data at the receiving end, the data frame can be divided into three parts: image sub-template hash code, splicing instructions and original image patch data using clear fields, and the data frame header can be marked as the image sub-block using partial splicing completion mode. When restoring, the receiving end first restores the image sub-template combination to the covered area according to the splicing instructions, and then superimposes the original image patch to the remaining blank area according to spatial mapping, and finally constructs a complete image sub-block to ensure that the restored structure is consistent with the original. Figure 1 To.
[0174] Through the above-mentioned optimal mechanism, this embodiment can still achieve maximum compression and data reuse in the splicing failure scenario, preventing the entire image sub-block from falling into the inefficient original transmission path, improving the overall compression ratio of the system and reducing data frame bandwidth occupancy.
[0175] See also Figure 9 , this figure is a flow chart of another image transmission method provided in an embodiment of the present application. Figure 9 The method shown can be applied to a receiving end of a satellite communication system, which also includes a transmitting end. For a description of the transmitting end, reference can be made to the description above and will not be repeated here. Figure 9 The method shown includes the following A1-A4, and the specific steps are as follows:
[0176] A1: receiving a data frame transmitted via a satellite link, wherein the data frame includes a hash code of an image template, a splicing instruction, or image data;
[0177] In this embodiment, the receiving end receives data frames from the satellite transmitter in real time via a pre-set high-speed demodulator and data link layer protocol components. Each data frame employs a multi-segment structure, including a template identification segment (i.e., a hash code of the image sub-template), a structure splicing segment (i.e., a sequence of splicing instructions used to construct image sub-blocks), and an image patch segment (i.e., a segment of raw image data when some splicing fails). The data frame structure clearly identifies the encoding type and corresponding region index, ensuring that subsequent processing logic can distinguish between the hash template and the image data, allowing for targeted decoding operations.
[0178] A2: performing a restoration operation on the hash code information contained in the data frame according to a locally preset template set to obtain an image sub-block;
[0179] In this embodiment, the receiving end preloads a set of image sub-templates identical to those used by the sending end. Each template is stored in a local search database using a hash value as the index key. After receiving the hash code, a fast hash match and template call are performed to complete the local restoration of the image sub-template. To improve search speed, a B+ tree structure or hash map is used for index acceleration. Offset vectors and boundary constraint information are also stored to ensure that the restored template maintains directional and positional consistency during the subsequent stitching process.
[0180] A3: Restore the corresponding image sub-blocks according to the stitching instructions;
[0181] In this embodiment, the receiving end sequentially loads hash templates according to the instructions and geometrically combines the image segments within the image sub-block space based on the offsets and stitching order. To ensure stitching accuracy, boundary alignment constraints and connection consistency detection mechanisms are introduced. These mechanisms verify the continuity of pixel intensity gradients, contour orientations, and connection points at the stitching boundaries. If errors exceed the tolerance, the instructions are rolled back and region-level repair is performed.
[0182] A4: Generate a satellite reconstructed image based on the image sub-blocks, the restored image sub-blocks, and the image data;
[0183] In this embodiment, the receiver performs structural fusion on the full image canvas based on the image stitching results of the current frame (including the restored stitching blocks and the received image patch data). A tile index map and spatial layout matrix are used to manage the position and coverage of each image sub-block. Border feathering, brightness balancing, and contour fusion are performed on the seams between the original image patches and the stitched areas, ensuring that the resulting satellite reconstructed image meets usable standards in terms of structural integrity, brightness consistency, and visual coherence.
[0184] As an example, in this embodiment, the satellite image may be transmitted during the channel idle time. Therefore, when the receiving end subsequently receives the transmitted satellite image, the method further includes:
[0185] A5: performing image comparison processing on the satellite reconstructed image and the satellite image;
[0186] Specifically, the purpose of this step is to introduce the original satellite image as a posterior reference to determine whether the satellite reconstructed image generated based on stitching restoration has quality problems such as structural loss, local stitching misalignment, or image artifacts.
[0187] In this embodiment, the satellite link scheduling strategy schedules additional delayed transmission of the original satellite image (or its high-fidelity sampling area) during idle periods outside of peak data transmission periods. Upon receiving this image, the receiver compares it pixel by pixel with the locally reconstructed image. This comparison includes, but is not limited to, composite evaluation methods such as the Multi-Scale Structural Similarity Index (MS-SSIM), Gradient Difference, and Structural Consistency Error Heatmap to comprehensively assess structural fidelity and restoration consistency.
[0188] A6: Identify, based on the image comparison results, regions with missing structures or regions with splicing artifacts in the satellite reconstructed image;
[0189] Specifically, this step is used to extract structural anomalies in the image reconstruction results that were not successfully restored or were introduced due to abnormal splicing paths. The identification results will be used as input for subsequent repair and splicing path attribution analysis.
[0190] In this embodiment, the receiving end uses the aforementioned difference heatmap to compare the original and reconstructed images, automatically segmenting regions with discontinuous edges, damaged textures, and abrupt seams, and generating a structural loss mask. This mask not only records the spatial distribution of the abnormal regions but also assigns a structural distortion type label (such as misaligned splicing, template breakage, and incomplete contour) to each region, allowing for tracing the cause when mapping back to the splicing path.
[0191] A7: Performing image restoration processing on the structure missing area or the splicing artifact area;
[0192] Specifically, to ensure that the final output image has practical value, the identified structural abnormal areas need to be repaired to make the overall image continuous and readable.
[0193] In this embodiment, structure migration and filling are performed based on the structural orientation field of the abnormal area and the surrounding area. Semi-automatic filling is performed using known structural templates in the sample library or structural edges in the context space. For edge fractures, curvature-driven edge extension is used; for texture anomalies, texture block splicing and fusion are performed.
[0194] A8: Based on the spatial distribution characteristics of the structure missing area or the splicing artifact area, locate the tree nodes in the preset image sub-template splicing tree where the image template hash matching fails or the splicing path is abnormal;
[0195] Specifically, this step aims to reversely infer the specific nodes that cause the problem in the stitching path based on the location and type of the structural defect area recorded during the image restoration process, thereby achieving tree-level attribution of structural defects.
[0196] In this embodiment, when restoring each image sub-block, the receiving end creates a mapping table between stitching instructions and stitching tree paths, recording the spatial projection position of each node within the image sub-block. Once a structural defect area is identified, the system can reverse-search the projection table to quickly locate the stitching template or segment causing the structural break or texture misalignment. If a node repeatedly causes stitching artifacts in multiple image sub-blocks, it is preliminarily identified as a node with abnormal quality in the stitching tree and placed in a pool for update.
[0197] A9: Update the tree node and synchronize the update result at the sending end;
[0198] Specifically, after identifying a template node with a structural matching deviation, its connection relationship, structural template content or indexing strategy in the stitching tree needs to be dynamically updated to prevent the node from being incorrectly referenced in subsequent image transmissions, while ensuring that the stitching tree has adaptive capabilities.
[0199] In this embodiment, the receiving end packages the hash ID of the node to be updated, a description of the structural deviation, and a replacement suggestion (such as replacing it with an adjacent structural template ID or deleting the path) into a structural quality feedback packet and sends it to the sending end via a feedback link (such as a reverse control channel). After receiving this feedback, the sending end replaces or disconnects the corresponding node in the template set and splicing tree, and simultaneously refreshes the instruction encoding logic to prevent the node from being selected again in subsequent image encoding.
[0200] As an example, an embodiment of the present application includes a satellite communication system, the system including: a transmitting end and a receiving end, wherein:
[0201] The transmitting end is configured to: divide the acquired satellite image into a plurality of image sub-blocks, wherein the size of the image sub-blocks is consistent with the size of each image template in a preset template set, and each image template includes a plurality of image sub-templates; perform a traversal operation to match the current image sub-block with any image template in the preset template set, and set a judgment engine to judge the relationship between the image sub-block and the image template, determine a hash code that matches the corresponding image template and a splicing instruction for splicing the corresponding image template, wherein the hash code is generated based on unique structural feature information of the matched image template or image sub-template in the preset template set, and the splicing instruction is used to indicate the relative splicing order and spatial position relationship of the image sub-templates in the image sub-block; construct a data frame based on the hash code and the splicing instruction, and transmit the data frame to the receiving end via a satellite link;
[0202] The receiving end is configured to receive a data frame transmitted via a satellite link, the data frame including a hash code of an image template, a splicing instruction, or image data; perform a restoration operation on the hash code information included in the data frame according to a locally preset template set to obtain an image sub-block; restore the corresponding image sub-block according to the splicing instruction; and generate a satellite reconstructed image based on the image sub-block, the restored image sub-block, and the image data.
[0203] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. An image transmission method, applied to a transmitting end of a satellite communication system, wherein the satellite communication system further comprises a receiving end for receiving data frames from the transmitting end, wherein the transmitting end and the receiving end are configured with the same preset template set, wherein: The method comprises: Dividing the acquired satellite image into a plurality of image sub-blocks, wherein the size of the image sub-blocks is consistent with the size of each image template in the preset template set, and each image template includes a plurality of image sub-templates; Performing a traversal operation to match the current image sub-block with any image template in a preset template set, and setting a judgment engine to judge the relationship between the image sub-block and the image template, determining a hash code that matches the corresponding image template and a splicing instruction for splicing the corresponding image template, wherein the hash code is generated based on unique structural feature information of the matched image template or image sub-template in the preset template set, and the splicing instruction is used to indicate the relative splicing order and spatial position relationship of the image sub-templates in the image sub-block; constructing a data frame according to the hash code and the splicing instruction, and sending the data frame to a receiving end via a satellite link; The determining the relationship between the image sub-block and the image template includes: Determine whether the structural features of the current image sub-block match those of any image template in the preset template set; If there is a match, get the hash code of the corresponding image template; If there is no match, if a splicing path exists, a splicing path formed by combining multiple image sub-templates and matching the structural features of the current image sub-block is determined from the preset image sub-template splicing tree, including: Calculating structural clues of the current image sub-block, wherein the structural clues include edge main directions, corner point distribution patterns and local contour features; According to the structural clues, a plurality of nodes matching the structural clues are marked in the image sub-template splicing tree as candidate nodes; Taking the candidate node as a starting point, performing a fixed-length combined path construction along the image sub-template stitching tree, and generating a stitched image structure according to the combined path, including: Divide the current image sub-block into multiple structured splicing blocks with orderly positions; Determine a covered block and a block to be covered according to the position of the candidate node in the current image sub-block, wherein the size of the structure splicing block is consistent with the size of the image sub-template; Determine the main extension direction of the covered block based on the relative position of the candidate node, determine the next block to be covered based on the extension main direction, obtain the structural clue of the next block to be covered, select a child node from the child nodes of the candidate node as a new candidate node based on the structural clue, and mark the block to be covered as a covered block. Repeat the determination of the main extension direction of the covered block until a fixed length is reached; Calculating the similarity of structural features between the stitched image structure and the current image sub-block, and taking the combined path corresponding to the stitched image structure with the greatest structural feature similarity as the stitching path; Get the hash code of the image sub-template involved in stitching and generate the corresponding stitching instructions.
2. The image transmission method according to claim 1, characterized in that: If there is no new candidate node among the child nodes of the candidate node: Reselect a candidate node as a new root node and perform an iterative operation on the new root node, wherein the root node represents the starting point of the combined path, and the iterative operation includes: Determine the main extension direction of the covered block based on the relative position of the root node, determine the next block to be covered based on the extension main direction, obtain the structural clue of the next block to be covered, select a child node from the child nodes of the root node as a new candidate node based on the structural clue, and mark the block to be covered as a covered block. Repeat the determination of the main extension direction of the covered block until a fixed length is reached; If the new root node cannot reach the fixed length, a new candidate node is selected as the current new root node, and the iterative operation is performed on the current new root node until all candidate nodes are traversed.
3. The image transmission method according to claim 1, wherein: If there is no splicing path, the image data of the current image sub-block is encoded as the original data, and a data frame is constructed and sent to the receiving end.
4. An image transmission method, applied to a receiving end of a satellite communication system, the satellite communication system also including a transmitting end, the transmitting end and the receiving end being configured with the same preset template set, the transmitting end dividing a satellite image into a plurality of image sub-blocks, matching image templates or combinations of image sub-templates in the preset template set, and constructing corresponding data frames based on the matching results, characterized in that: The method comprises: receiving a data frame transmitted via a satellite link, the data frame including a hash code of an image template, a splicing instruction, or image data; Performing a restoration operation on the hash code information contained in the data frame according to a locally preset template set to obtain an image sub-block; Restore the corresponding image sub-blocks according to the splicing instruction; generating a satellite reconstructed image according to the image sub-blocks, the restored image sub-blocks and the image data; performing image comparison processing on the satellite reconstructed image and the satellite image; Identifying, based on the image comparison results, regions with missing structures or regions with splicing artifacts in the satellite reconstructed image; Performing image restoration processing on the structure missing area or the splicing artifact area; According to the spatial distribution characteristics of the structure missing area or the splicing artifact area, the tree nodes where the image template hash matching fails or the splicing path is abnormal in the preset image sub-template splicing tree are located; The tree nodes are updated, and the update results are synchronized at the sending end.
5. The image transmission method according to claim 4, characterized in that: The stitching instruction is used to indicate the relative stitching order and spatial position of multiple image sub-templates in the image sub-block. The stitching instruction controls the combination of multiple image sub-templates according to a preset topological relationship to restore the image structure of a single image sub-block.
6. A satellite communication system for implementing an image transmission method according to any one of claims 1 to 5, characterized in that: The system includes: a receiving end and a transmitting end, wherein: The transmitting end is configured to: divide the acquired satellite image into a plurality of image sub-blocks, wherein the size of the image sub-blocks is consistent with the size of each image template in a preset template set, and each image template includes a plurality of image sub-templates; perform a traversal operation to match the current image sub-block with any image template in the preset template set, and set a judgment engine to judge the relationship between the image sub-block and the image template, determine a hash code that matches the corresponding image template and a splicing instruction for splicing the corresponding image template, wherein the hash code is generated based on unique structural feature information of the matched image template or image sub-template in the preset template set, and the splicing instruction is used to indicate the relative splicing order and spatial position relationship of the image sub-templates in the image sub-block; construct a data frame based on the hash code and the splicing instruction, and transmit the data frame to the receiving end via a satellite link; The receiving end is configured to receive a data frame transmitted via a satellite link, the data frame including a hash code of an image template, a splicing instruction, or image data; perform a restoration operation on the hash code information included in the data frame according to a locally preset template set to obtain an image sub-block; restore the corresponding image sub-block according to the splicing instruction; and generate a satellite reconstructed image based on the image sub-block, the restored image sub-block, and the image data.
Citation Information
Patent Citations
Real-time Transmission Method of Video Image in Remote Sensing Satellite System
CN105357472B
Electric heavy truck self-adaptive positioning method and system based on multi-sensor fusion
CN119828191A
Large-scale satellite stereoscopic image data processing method, medium and system
CN120088413A