Fragment global reassembly method, system, computer device and storage medium

Abstract

The application provides a fragment global recombination method, system, computer device and storage medium. The method comprises the following steps: acquiring a plurality of fragments and constructing triad information of each fragment; judging the positional relationship of any two fragments according to the triad information of the two fragments; placing the plurality of fragments in a fragment splicing grid one by one; in each fragment placing link, for any first fragment in a plurality of first fragments which have not been placed, calculating a selection score based on the effective area, global connectivity and dynamic front connection strength of the first fragment; taking the first fragment with the highest selection score as the current to-be-placed fragment; determining target grid coordinates according to the grid coordinates of a plurality of second fragments which have been placed and the positional relationship between the current to-be-placed fragment and the plurality of second fragments; and placing the current to-be-placed fragment in the fragment splicing grid according to the target grid coordinates. The application can significantly improve the low global recombination accuracy of fragments.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method, system, computer device, and storage medium for global fragment reassembly. Background Technology

[0002] Ancient books, manuscripts, and archives, among other paper-based cultural heritage, are prone to paper embrittlement, edge wear, and text loss during long-term preservation, ultimately existing as multiple scattered fragments. Existing fragment reconstruction schemes typically estimate the contact compatibility between fragments using contour shapes, boundary textures, or local visual features, and then employ greedy or combinatorial optimization methods to gradually place the fragments. These methods can achieve some success in scenarios where the torn edges are well-preserved and the textures are clear. However, after long-term corrosion, wear, and staining, the geometric complementarity and semantic continuity of ancient book fragments are significantly weakened. Single local evidence is prone to mismatches under severe degradation conditions, resulting in low accuracy in global fragment reconstruction. Summary of the Invention

[0003] In view of this, this application proposes a method, system, computer device and storage medium for global reconstruction of fragments, in order to solve the problem that existing methods for reconstructing ancient paper fragments rely on local visual evidence such as contours and boundaries, which are prone to mismatches when fragments are corroded, worn and degraded, resulting in a decrease in geometric complementarity and semantic continuity, thus causing low accuracy in global reconstruction of fragments.

[0004] A first aspect of this application proposes a global fragment reassembly method, the method comprising: Multiple fragments from the same page are obtained, and triple information for each fragment is constructed; the triple information includes a fragment image, an edge map corresponding to the fragment image, and a text sequence; the edge map is used to represent the boundary and contour of the fragment; For any two fragments among the plurality of fragments, the positional relationship between the two fragments is determined based on the triplet information of the two fragments; the positional relationship includes whether they are adjacent and their adjacent directions; The multiple fragments are placed one by one into the fragment splicing grid; wherein, each fragment placement step includes the following sub-steps: Step S1: For any one of the currently unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontal connection strength. Global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to all fragments other than the first fragment. Dynamic frontal connection strength refers to the average adjacency affinity of the first fragment with the currently placed second fragments. Adjacency affinity refers to the probability that the first fragment will form an adjacency relationship with a corresponding fragment. Step S2: Select the first fragment with the highest score from the plurality of first fragments as the current fragment to be placed; Step S3: Determine the target grid coordinates of the currently placed fragment based on the grid coordinates of the several second fragments already placed and the positional relationship between the fragment to be placed and the several second fragments; Step S4: Place the currently placed fragment into the fragment splicing grid according to the target grid coordinates.

[0005] This application's embodiments construct a triplet of information including fragment images, edge maps, and text sequences, enabling the three types of evidence—paper appearance, tear geometry, and text semantics—to complement each other, significantly reducing the impact of severe edge degradation on a single feature. By determining the positional relationship between any two fragments and jointly outputting the adjacency category and the relative direction of the principal axis, contradictions caused by separate modeling are reduced, and directly executable relational constraints are provided for the global layout. By calculating selection scores based on effective area, global connectivity, and dynamic frontier connectivity strength, relation-aware processing sequence planning is achieved, prioritizing fragments with more complete information and more reliable connections to the current local layout, effectively reducing error propagation caused by early misplacement. By determining the target grid coordinates based on the grid coordinates and positional relationships of the placed fragments, local relation prediction is converted into an integer grid layout without coordinate conflicts, ensuring coordinate legality and layout compactness, thereby significantly improving the global reconstruction accuracy of severely degraded ancient book fragments.

[0006] In this embodiment of the application, determining the positional relationship between the two fragments based on the triplet information of the two fragments includes: Construct joint multimodal input information containing triplet information of the two fragments, wherein the joint multimodal input information includes fragment images, edge maps and text sequences of the two fragments; The joint multimodal input information is input into a task-adapted visual language model, and the positional relationship between the two fragments is output. The positional relationship includes the adjacency category and the relative direction of the main axis. The adjacency category is adjacent or non-adjacent, and the relative direction of the main axis is up, down, left or right.

[0007] This application's embodiments construct joint multimodal input information containing two fragmented triplet information, simultaneously inputting the original image, edge map, and text sequence into a visual language model, enabling the model to synthesize three types of complementary evidence in a single inference. By jointly outputting the adjacency category and the relative direction of the principal axis through a task-adapted visual language model, the contradictions caused by the separate processing of adjacency judgment and direction judgment are avoided, improving the reliability of relationship judgment. By limiting the positional relationship to the structured output of adjacency category (adjacent / non-adjacent) and relative direction of the principal axis (up / down / left / right), the system can stably parse and directly call the data, reducing parsing errors caused by free text output, thereby providing high-quality structured relationship constraints for subsequent global coordinate recovery.

[0008] In this embodiment of the application, the selection score of the first fragment is calculated based on its effective area, global connectivity, and dynamic frontier connectivity strength, including:

[0009] in, Indicates the first The first fragment in the second fragment placement stage The choice of score, For the first fragment The effective area, For the first fragment Global connectivity, For the first fragment Dynamic frontal connection strength, To be with the first fragment The first weighting coefficient corresponding to the effective area, To be with the first fragment The second weight coefficient corresponding to global connectivity. To be with the first fragment The third weighting coefficient corresponding to the dynamic frontier connection strength.

[0010] In this embodiment of the application, determining the target grid coordinates of the currently placed fragment based on the grid coordinates of the plurality of second fragments already placed and the positional relationship between the currently placed fragment and the plurality of second fragments includes: The grid coordinates of the currently placed second fragments, and the positional relationship between the currently placed fragment and the several second fragments, are used as structured information and input into the language model candidate anchor points generated by the task-adapted language model. From the positional relationships between the current fragment to be placed and the plurality of second fragments, a first positional relationship with a confidence level higher than a preset confidence threshold is selected, and a deterministic relationship anchor point is generated based on the first positional relationship; The candidate anchor points of the language model, the anchor points of the deterministic relations, and the unoccupied grid coordinates in the neighborhood of the candidate anchor points of the language model and the anchor points of the deterministic relations are combined into a set of candidate coordinates; For any candidate coordinate in the candidate coordinate set, calculate the unified score of the candidate coordinate. The unified score includes the degree of consistency of the candidate coordinate's adjacency prediction, the degree of consistency of the candidate coordinate's direction prediction, the cost of the layout bounding rectangle expansion caused by the candidate coordinate, and the distance of the candidate coordinate from the candidate anchor point of the language model or the deterministic relation anchor point. The candidate coordinates with the highest uniform score are selected from the candidate coordinate set and used as the target grid coordinates for the current fragment to be placed.

[0011] This application's embodiments generate language model candidate anchor points by introducing a task-adapted language model, and provide position suggestions using the language model's prior knowledge of the layout rules of ancient book pages; deterministic relationship anchor points are generated by screening high-confidence positional relationships, directly transforming highly reliable local relationships into executable spatial coordinates; by combining language model candidate anchor points, deterministic relationship anchor points, and their neighborhood unoccupied grid coordinates into a candidate coordinate set, collisions or overexpansion caused by relying entirely on a single coordinate generation method are avoided; by using a unified scoring function to comprehensively consider the degree of adjacency prediction consistency, direction prediction consistency, layout bounding rectangle expansion cost, and anchor point deviation distance, while ensuring non-overlapping coordinates, relationship consistency and layout compactness are also taken into account, thereby stably transforming local relationships into a conflict-free page-level integer grid layout.

[0012] In this embodiment of the application, the adjacency prediction consistency and direction prediction consistency of the candidate coordinates are calculated in the following manner: The second fragment that is already placed and adjacent to the candidate coordinates is identified as the reference fragment; Obtain the second positional relationship between the currently placed fragment and the reference fragment; The degree of adjacency prediction consistency of the candidate coordinates relative to the reference fragment is determined based on the adjacency category in the second positional relationship, and the degree of orientation prediction consistency of the candidate coordinates relative to the reference fragment is determined based on the principal axis relative direction in the second positional relationship.

[0013] In this embodiment of the application, the cost of expanding the layout bounding rectangle due to the candidate coordinates is calculated in the following way: Determine the maximum, minimum, maximum, and minimum values ​​of the grid coordinates of the currently placed second fragments, and calculate the first area of ​​the current layout's circumscribed rectangle based on these values. Based on the candidate coordinates and the grid coordinates of the currently placed second fragments, determine the new maximum value of the horizontal coordinate, the new minimum value of the horizontal coordinate, the new maximum value of the vertical coordinate, and the new minimum value of the vertical coordinate, and calculate the second area of ​​the outer rectangle of the new layout based on the new maximum value of the horizontal coordinate, the new minimum value of the horizontal coordinate, the new maximum value of the vertical coordinate, and the new minimum value of the vertical coordinate. The cost of expanding the layout bounding rectangle caused by the candidate coordinates is determined based on the difference between the second area and the first area.

[0014] In this embodiment of the application, the candidate anchor points of the language model, the deterministic relation anchor points, and the unoccupied grid coordinates within the neighborhood of the candidate anchor points of the language model and the deterministic relation anchor points are combined into a candidate coordinate set, including: For any anchor point among the candidate anchor points of the language model and the anchor points of the deterministic relation, a neighborhood expansion is performed on the anchor point with an expansion radius of 1; Expand the grid around the current layout's outer rectangle by a preset number; The expanded unoccupied grid coordinates are combined with the language model candidate anchors and the deterministic relation anchors to form a candidate coordinate set, wherein the number of candidate coordinates is capped.

[0015] An embodiment of the second aspect of this application provides a global fragment reassembly system, comprising: The triplet information construction module is used to acquire multiple fragments on the same page and construct triplet information for each fragment; the triplet information includes fragment images, edge maps corresponding to the fragment images, and text sequences; the edge maps are used to represent the boundaries and contours of the fragments; The positional relationship determination module is used to determine the positional relationship between any two fragments among the plurality of fragments based on the triplet information of the two fragments; the positional relationship includes whether they are adjacent and their adjacent directions; A fragment placement module is used to place the plurality of fragments one by one into a fragment splicing grid. Each fragment placement step includes the following sub-steps: For any one of the currently unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontal connection strength; global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to all fragments except the first fragment; dynamic frontal connection strength refers to the average adjacency affinity of the first fragment with the currently placed second fragments; adjacency affinity refers to the probability that the first fragment and a corresponding fragment will form an adjacency relationship; select the first fragment with the highest selection score from the plurality of first fragments as the current fragment to be placed; determine the target grid coordinates of the current fragment to be placed based on the grid coordinates of the currently placed second fragments and the positional relationship between the current fragment to be placed and the second fragments; place the current fragment to be placed into the fragment splicing grid according to the target grid coordinates.

[0016] An embodiment of the third aspect of this application provides a computer device including a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the fragment global reassembly method described in the first aspect.

[0017] An embodiment of the fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the fragment global reassembly method described in the first aspect above. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a global fragment reassembly method provided in an embodiment of this application is shown; Figure 2 A schematic diagram of the adjacency prediction and direction prediction process provided in an embodiment of this application is shown; Figure 3 A flowchart illustrating another fragment global reassembly method provided in an embodiment of this application is shown; Figure 4A flowchart illustrating the process of determining the target grid coordinates of a fragment to be placed, according to an embodiment of this application, is shown. Figure 5 This invention provides a schematic diagram of the structure of a global fragment reassembly system according to an embodiment of the present application. Figure 6 This illustration shows a schematic diagram of the structure of a computer device according to an embodiment of this application; Figure 7 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0019] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0020] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0021] The following describes the technical background and shortcomings of the prior art involved in the embodiments of this application: Existing fragment joining schemes typically estimate the contact compatibility between fragments using contour shapes, boundary textures, or local visual features, and then employ greedy, graph search, or combinatorial optimization methods to gradually place the fragments. These schemes can achieve some success in scenes with intact edges and clear tear textures.

[0022] However, after long-term corrosion, wear, stains, and content loss, ancient book fragments suffer from weakened geometric complementarity and semantic continuity, making mismatches prone to occur from single local evidence. Furthermore, traditional methods often separate adjacency and direction judgments, leading to contradictions between the two. Moreover, correct pairwise local relationships do not necessarily result in a valid full-page layout; there is a lack of an effective mechanism to stably convert local relationships into a coordinate-conflict-free integer grid layout, easily causing coordinate overlap, layout drift, and error propagation.

[0023] In view of this, embodiments of this application provide a fragment global reconstruction method. By coordinating relationship-aware sequence planning, multimodal relationship reasoning, and constrained global layout, it unifies local fragment relationship identification and page-level global coordinate recovery into the same process, thereby improving the reliability of relationship judgment and the consistency of overall layout under severe degradation conditions. The specific technical solutions of embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0024] This application provides a method for global fragment reassembly. For example... Figure 1 As shown, the method includes the following steps: Step 101: Obtain multiple fragments from the same page and construct the triplet information for each fragment.

[0025] Specifically, obtain fragment images of multiple fragments on the same page (e.g.) , Indicates the first A fragmented image, After representing the total number of fragment images, each fragment is assigned a unique number and preprocessed, including background transparency or paper area extraction, size normalization, orientation normalization, and effective area cropping. Preferably, the model input image is scaled or filled to approximately 512×512 pixels while maintaining the aspect ratio and the integrity of the fragment's main body.

[0026] More specifically, the specific steps for constructing the triplet information for each fragment include: For the first i Three types of complementary inputs are generated from each fragment: the first type is the fragment... i Fragment images of individual pieces The first category is used to preserve text, paper texture, ink marks, typography, and overall appearance; the second category is... i Edge map of a fragment This is used to highlight tear boundaries, hole boundaries, and fragment outlines. The edge map can be generated by paper mask boundaries, grayscale gradients, Canny operators, Sobel operators, or learned edge detectors; the third type is... i A fragmented sequence of text This is used to provide clues about surviving characters, semantic continuity, and the reading order of ancient texts, and can be obtained through OCR technology (such as PaddleOCR or other text recognition models). For fragments with low OCR confidence or no text, It can be an empty sequence, and the fragment image and edge map are mainly used for judgment.

[0027] After preprocessing, the triplet information for each fragment is as follows: =( , , ), and calculate the effective area of ​​the fragments for subsequent sequence planning.

[0028] Step 102: For any two fragments among the plurality of fragments, determine the positional relationship between the two fragments based on the triplet information of the two fragments.

[0029] In some specific embodiments, step S102 includes steps S1021-S1022: Step S1021: Construct joint multimodal input information containing triplet information of the two fragments.

[0030] Specifically, for any two fragments among multiple fragments on the same page. Constructing multimodal input information ,in, They represent fragments. Fragmented images, edge maps, and text sequences They represent fragments. Fragmented images, edge maps, and text sequences.

[0031] Step S1022: Input the joint multimodal input information into the task-adapted visual language model and output the positional relationship between the two fragments.

[0032] Specifically, such as Figure 2 As shown, multimodal input information Input a task-adapted visual language model Jointly output adjacency prediction and direction prediction:

[0033] in, To input multimodal information Input visual language model In the output, the prediction results, For adjacency prediction (i.e., adjacency category), including fragments With fragments Adjacent or non-adjacent, specifically, two fragments are defined as adjacent if and only if the corresponding mesh cells share an edge; they are not adjacent only if they are in contact at corners. For directional prediction, it represents fragments. Relative fragments The main displacement axis directions include up, down, left, and right.

[0034] In some specific embodiments, a set of positional relationships can be formed based on the positional relationship between any two fragments among a plurality of fragments. This is for use in subsequent global coordinate recovery.

[0035] In some specific embodiments, the visual language model The model output uses a constrained JSON format, including fields such as "adjacency category" and "relative direction of the main axis." The adjacency category only allows "adjacent / non-adjacent," and the relative direction of the main axis only allows "up / down / left / right." By using fixed fields and a limited set of enumerated values, parsing errors caused by free text output are reduced.

[0036] In some specific embodiments, the visual language model The model backbone is Qwen3-VL-8B-Instruct, and LoRA is used for supervised fine-tuning; the LoRA rank is 16, and the scaling factor is... The learning rate is 32, and the dropout rate is 0.05. Training is performed using AdamW with an initial learning rate of 1×10⁻⁶. 4 The training process employs a weight decay of 0.1, a cosine learning rate scheduling, a warm-up ratio of 0.03, 3 epochs of training, an effective batch size of 8, bf16 precision, an inference temperature of 0, and greedy decoding. The above training and efficient parameter fine-tuning can be implemented using the SWIFT lightweight fine-tuning infrastructure or similar frameworks. It should be noted that the above model backbone and hyperparameters are merely examples and do not constitute a limitation of the present invention; in other embodiments, other visual language models, multi-tower encoders, cross-modal Transformers, or graph neural networks can also be used, and task adaptation methods such as LoRA, Adapter, cue-based fine-tuning, partial parameter fine-tuning, or full parameter fine-tuning can be employed.

[0037] In some specific embodiments, for and Perform bidirectional consistency checks: adjacency categories should be the same, and the vertical or horizontal directions should be opposite to each other; for inconsistent relationships, the decision can be made based on model confidence, adjacency priority, or subsequent global score.

[0038] This application embodiment utilizes the structured reasoning of the joint multimodal input and task-adapted visual language model described above. The text and paper appearance information provided by the original fragment image, the fracture geometry information provided by the edge map, and the semantic continuity information provided by the OCR text sequence can complement each other in a single reasoning process, enabling the model to maintain high reliability of relation judgment even under severe edge degradation conditions. At the same time, the joint output mechanism of adjacency category and relative direction of principal axis avoids the contradictions caused by the separate processing of adjacency judgment and direction judgment, providing directly executable structured relation constraints for global layout.

[0039] Step S103: Place the multiple fragments one by one into the fragment splicing grid.

[0040] Among them, such as Figure 3 As shown, each fragment placement step includes the following sub-steps: Step S1: For any one of the several unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontier connectivity strength.

[0041] Specifically, the global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to the fragments other than the first fragment among the plurality of fragments; the dynamic frontier connectivity strength refers to the average adjacency affinity of the first fragment with several currently placed second fragments; adjacency affinity refers to the probability that the first fragment and the corresponding fragment form an adjacency relationship.

[0042] In some specific embodiments, neighbor affinity It can be pre-calculated using a lightweight relational programming model.

[0043] Specifically, the lightweight planning model uses ResNet-18 to extract fragment features, employs a two-layer TransformerEncoder to aggregate fragment information within a page, and predicts adjacency affinity for planning using a symmetric pairwise multilayer perceptron. The above network structure is only a preferred implementation and can be replaced by other visual encoders and graph neural networks.

[0044] In some specific embodiments, the selection score of the first fragment is calculated based on its effective area, global connectivity, and dynamic frontier connectivity strength, including:

[0045] in, Indicates the first The first fragment in the second fragment placement stage The choice of score, For the first fragment The effective area, For the first fragment Global connectivity, For the first fragment Dynamic frontal connection strength, To be with the first fragment The first weighting coefficient corresponding to the effective area, To be with the first fragment The second weight coefficient corresponding to global connectivity. To be with the first fragment The third weighting coefficient corresponding to the dynamic frontier connectivity strength , Fixed or learnable weights. Area term. Prefers to select larger fragments with more visual context; global connectivity terms Preferential selection of fragments with richer relationships with other fragments; dynamic frontier items It tends to prioritize fragments that are reliably connected to the current local layout.

[0046] Step S2: Select the first fragment with the highest score from the plurality of first fragments as the current fragment to be placed.

[0047] Specifically, the first fragment with the highest selection score is selected, as shown in the following formula:

[0048] in, Indicates the first In the next fragment placement phase, select the first fragment with the highest score.

[0049] Step S3: Determine the target grid coordinates of the currently placed fragment based on the grid coordinates of the several second fragments that have already been placed, and the positional relationship between the currently placed fragment and the several second fragments.

[0050] In some specific embodiments, such as Figure 4 As shown, step S3 includes steps S31-S35: Step S31: The grid coordinates of the currently placed second fragments and the positional relationship between the currently placed fragment and the second fragments are used as structured information and input into the language model candidate anchor points generated by the task-adapted language model.

[0051] In a preferred embodiment, the language model is a Qwen3-4B language model fine-tuned under LoRA (Low-Rank Adaptation) supervision. This model is responsible for providing positional priors based on the current local layout state and the predicted relationships between fragments. Specifically, the system encodes the current layout state as input in structured text or JSON format. The structured input includes at least the following information: The unique identifier of the currently placed second fragment and its grid coordinates (e.g., fragment A is located at (0,0), fragment B is located at (1,0)); The positional relationship between the currently placed fragment (denoted as fragment X) and each of the already placed second fragments (e.g., fragment X is adjacent to fragment A and located below fragment A; fragment X is not adjacent to fragment B).

[0052] After receiving the structured input, the task-adapted language model, based on its prior knowledge of the layout patterns of ancient book pages (such as pages typically arranged in rows and columns, and adjacent fragments having semantic continuity), infers and outputs one or more candidate coordinates. The output also uses a structured format (such as JSON) to facilitate system parsing.

[0053] The system parses the above output, extracts the coordinate values, and obtains a set of candidate anchor points for the language model, for example: {(0,1), (0,2), (-1,1)}.

[0054] It should be noted that the candidate anchor points provided by the language model are only location priors and do not guarantee that the location is unoccupied or satisfies all relation constraints. Even if the anchor points given by the language model are not completely accurate, subsequent neighborhood search and unified scoring mechanisms can still find more suitable legitimate coordinates near the anchor point.

[0055] Step S32: From the positional relationships between the current fragment to be placed and the plurality of second fragments, select the first positional relationship with a confidence level higher than a preset confidence threshold, and generate a deterministic relationship anchor point based on the first positional relationship.

[0056] Specifically, the positional relationships are output by the visual language model in step 102, and each positional relationship is accompanied by a confidence score (e.g., 0.95). A preset confidence threshold (e.g., 0.9) is set, and only positional relationships with a confidence score greater than or equal to this threshold are identified as high-confidence relationships.

[0057] In some specific embodiments, for any highly reliable first positional relationship, if its adjacency category is "adjacent" and contains a clear relative direction of the principal axis, the system directly calculates the deterministic relationship anchor point using unit displacement based on the known grid coordinates of the placed second fragment. The unit displacement rule is as follows: Specifically, if the predicted fragment to be placed is located to the right and adjacent to the already placed fragment, and the coordinates of the already placed fragment are... Then a deterministic relationship anchor point is generated. In the image coordinate system The axis arrow points to the right. The axis arrow points downwards; If the prediction is located on the left and adjacent, a deterministic relationship anchor point is generated. ; If the prediction is below and adjacent, a deterministic relationship anchor point is generated. ; If the prediction is above and adjacent, a deterministic relationship anchor point is generated. .

[0058] Example: Fragment A is located at (0,0). The predicted positional relationship between fragment X and A is "adjacent, downward direction", with a confidence level of 0.95 (higher than the threshold of 0.9). Therefore, the anchor point for generating a deterministic relationship is (0, 0+1) = (0,1).

[0059] If the adjacency category of a highly reliable relation is "non-adjacent" or the direction is unclear, a deterministic relation anchor point cannot be generated. In this case, only candidate anchor points from the language model are relied upon.

[0060] Step S33: Combine the candidate anchor points of the language model, the deterministic relationship anchor points, and the unoccupied grid coordinates in the neighborhood of the candidate anchor points of the language model and the deterministic relationship anchor points into a candidate coordinate set.

[0061] In some specific embodiments, the candidate coordinate set is constructed as follows: Step a1: For any anchor point among the candidate anchor points of the language model and the anchor points of the deterministic relationship, perform neighborhood expansion on the anchor point.

[0062] Specifically, the neighborhood expansion includes Manhattan domain expansion, with an expansion radius of 1, meaning for each anchor point... Generate its four adjacent coordinates (up, down, left, right): , , , The Manhattan neighborhood does not include the diagonal direction.

[0063] Step a2: Expand the grid around the current layout bounding rectangle by a preset number of grids.

[0064] Specifically, for example, if the circumscribed rectangle formed by the currently placed fragments is Allowing expansion by 2 grid cells extends the valid search range to [the specified range]. .

[0065] Step a3: Combine the expanded unoccupied grid coordinates with the language model candidate anchor points and the deterministic relationship anchor points to form a candidate coordinate set, wherein the number of candidate coordinates is capped.

[0066] Specifically, check if the expanded coordinates described above are already occupied by a second fragment that has been placed. If a grid coordinate is already occupied (e.g., fragment A already exists at (0,0)), then that coordinate is excluded.

[0067] More specifically, the filtered unoccupied grid coordinates are merged with the original language model candidate anchor points and deterministic relation anchor points, and after deduplication, a set of candidate coordinates is formed. To control computational load, there is an upper limit to the number of candidate coordinates (e.g., 8). If the number of candidates after merging and deduplication exceeds the upper limit, the coordinates closest to each anchor point in Manhattan distance are retained first.

[0068] To illustrate steps a1-a3 above, a specific example is provided: Assume the candidate anchor points for the language model are {(0,1),(0,2)}, and the anchor point for the deterministic relation is {(0,1)}. Expanding (0,1) yields (0,0)[occupied], (0,2)[included], (1,1), and (-1,1); expanding (0,2) yields (0,1)[included], (0,3), (1,2), and (-1,2). After deduplication, excluding the occupied (0,0), and limiting to the valid range, the candidate coordinate set is, for example, {(0,1), (0,2), (1,1), (-1,1), (0,3), (1,2), (-1,2)}.

[0069] Step S34: For any candidate coordinate in the candidate coordinate set, calculate the unified score of the candidate coordinate.

[0070] In this step, for each candidate coordinate in the candidate coordinate set... Calculate its uniform score. The uniform score is calculated using the following formula:

[0071] in, Indicates the first t Candidate coordinates in the secondary fragment placement stage A unified scoring system Indicate candidate coordinates The degree of consistency of adjacency prediction Indicate candidate coordinates Consistency of directional predictions Indicate candidate coordinates The resulting cost of expanding the circumscribed rectangle of the layout, Indicate candidate coordinates The distance from the candidate anchor point of the language model or the anchor point of the deterministic relation. This indicates the weight of each item.

[0072] Specifically, the calculation methods for each item are as follows: (1) Adjacency prediction consistency : Candidate coordinates As the current placement location of the fragment to be placed, check whether this location is actually adjacent to each of the already placed second fragments (i.e., whether they share mesh edges). If the actual adjacency matches the adjacency category in the predicted positional relationship in step 102 (e.g., predicted as adjacent and actually sharing an edge, or predicted as non-adjacent and actually not sharing an edge), then... Take a higher value (e.g., 1.0); if inconsistent (e.g., predicted as non-adjacent but actually sharing an edge), take a lower value (e.g., 0).

[0073] (2) Degree of consistency in directional prediction For actually adjacent placed fragments, calculate candidate coordinates. The actual orientation relative to the placed fragment (determined by comparing the absolute values ​​of horizontal and vertical displacements). If the actual orientation matches the predicted principal axis orientation, then... Take the higher value (e.g., 1.0); if partially consistent (e.g., predicted to be below but actually to the lower left and not directly adjacent), take the middle value (e.g., 0.3); if completely inconsistent, take 0.

[0074] (3) Cost of expanding the circumscribed rectangle Calculate the first area of ​​the bounding rectangle formed by the currently placed fragments. Assume the fragment to be placed is positioned at the candidate coordinates. Then, the second area of ​​the new circumscribed rectangle is calculated. The expansion cost is the difference between the second area and the first area. The larger the difference, the looser the layout, and the higher the cost.

[0075] (4) Anchor point deviation distance Calculate candidate coordinates The Manhattan distance to the nearest language model candidate anchor or deterministic relation anchor. A larger distance indicates a greater deviation from the prior position. The larger.

[0076] Example calculation: Assuming weights =1.0, =1.0, =0.5, =0.3, for candidate coordinates (0,1): It is actually adjacent to A(0,0) and its direction is downward, consistent with the prediction: =1.0, =1.0; After placement, the circumscribed rectangle changes from 2×1=2 to 2×2=4. =2; The distance from the deterministic anchor point (0,1) is 0. =0; Total score: 1.0 × 1.0 + 1.0 × 1.0 0.5×2 0.3 × 0 = 1.0.

[0077] Step S35: Select the candidate coordinates with the highest unified score from the candidate coordinate set and use them as the target grid coordinates of the current fragment to be placed.

[0078] In this step, firstly, grid cells that are already occupied in the candidate coordinate set are excluded. Then, the candidate coordinates with the highest uniform score are selected from the remaining candidate coordinates and used as the target grid coordinates for the current fragment to be placed.

[0079] If multiple candidate coordinates have the same score, the system will make a deterministic selection based on the following rules: First priority: Select candidate coordinates that result in a smaller layout bounding rectangle; Second priority: Select candidate coordinates that are closer to the anchor point; Third priority: Select candidate coordinates with a high degree of adjacency consistency; Fourth priority: Select according to a fixed coordinate order (e.g., first compare the horizontal coordinates, with the smaller one taking precedence; if the horizontal coordinates are the same, then compare the vertical coordinates, with the smaller one taking precedence).

[0080] After determining the target grid coordinates, place the current fragment to be placed at those coordinates, update the local layout, and proceed to the next fragment placement stage or perform local refinement.

[0081] Step S4: Place the currently placed fragment into the fragment splicing grid according to the target grid coordinates.

[0082] The following example illustrates steps S1-S4 above: Suppose a page contains 10 fragments; When placing the first fragment into the fragment splicing grid, the fragment with the highest score is selected from the above 10 fragments as the current fragment to be placed and placed in the fragment splicing grid (since it is the first fragment to be placed, it is not necessary to determine the specific placement position of the fragment). When placing the 7th fragment into the fragment patching grid, first select the highest-scoring first fragment from the 4 currently unplaced first fragments as the current fragment to be placed; then, based on the grid coordinates of the 6 currently placed second fragments, and the positional relationship between the current fragment to be placed and these 6 second fragments (this positional relationship can be derived from the aforementioned set of positional relationships),... (Obtain from the target grid) to determine the target grid coordinates of the fragment to be placed; finally, place the fragment to be placed in the fragment splicing grid according to the target grid coordinates.

[0083] In some specific embodiments, after a placement is completed, local refinement can be performed on the newly placed fragments or a small number of already placed fragments within a limited range. Adjustments are only accepted if the uniform score improves after the move and no coordinate overlap occurs, to avoid excessive computation caused by unbounded global search. The final layout is obtained after all fragments have been processed.

[0084] In some specific embodiments, after multiple fragments are placed onto the fragment stitching grid, the method further includes: To eliminate the equivalent solution caused by the overall translation, the final layout is translated to the minimum. Coordinates and minimum All coordinates are 0, or the coordinates of the specified reference fragment are normalized to 0. The output should include at least: fragment number and grid coordinate mapping, predicted adjacency graph, predicted directional relationships, page bounding rectangle size, reassembled preview image arranged by coordinates, and relationship constraint satisfaction rate.

[0085] Further consistency checks can be performed: the coordinates of any two fragments must not be the same; highly reliable fragment pairs predicted to be adjacent should preferentially satisfy the shared edge condition; opposite direction relationships should satisfy the same coordinate sign; the layout's circumscribed rectangle should not be unnecessarily expanded; the overall score before and after local optimization should not decrease. If an anomaly is detected, the previous layout can be rolled back, the candidate neighborhood expanded, or low-reliability relationships replaced before a new search.

[0086] In applications requiring pixel-level stitching, the grid coordinates output by this invention can be used as the initial structure for subsequent fine-grained boundary registration, affine transformation, or non-rigid registration, significantly reducing the fine-grained search range. The core output of this invention is a page-level relative structure, and perfect pixel boundary matching is not a necessary condition.

[0087] The embodiments of this application have the following technical effects: First, by using a three-modal joint input of fragmented images, edge maps, and text sequences, the paper appearance, tear geometry, and text semantics can complement each other, reducing the impact of severe edge degradation on single features and improving the reliability of relationship judgment.

[0088] Second, by jointly outputting the adjacency category and the relative direction of the main axis for ordered fragment pairs, a single structured reasoning can output two relational dimensions simultaneously, reducing the contradictions caused by separate modeling and providing directly executable relational constraints for global layout.

[0089] Third, by using relation-aware sequence planning, the processing order is dynamically determined by comprehensively considering the normalized area of ​​fragments, global connectivity, and dynamic frontier connectivity strength. Fragments with more complete information and more reliable connections to the current local layout are processed first, reducing the propagation of errors caused by early misplacement.

[0090] Fourth, by combining candidate anchor points of the language model with deterministic relation anchor points, and expanding to generate a set of candidate coordinates in the neighborhood, and with the help of a unified scoring function that includes adjacency consistency, direction consistency, layout bounding rectangle expansion penalty and anchor point deviation penalty, relation prediction is transformed into an integer grid layout without coordinate conflicts, ensuring coordinate validity and layout compactness.

[0091] Corresponding to the above implementation of the global fragmentation reassembly method, this application also provides a global fragmentation reassembly system for executing the global fragmentation reassembly method described in the above embodiments. For example... Figure 5 As shown, the global fragment reassembly system includes: The triplet information construction module is used to acquire multiple fragments on the same page and construct triplet information for each fragment; the triplet information includes fragment images, edge maps corresponding to the fragment images, and text sequences; the edge maps are used to represent the boundaries and contours of the fragments; The positional relationship determination module is used to determine the positional relationship between any two fragments among the plurality of fragments based on the triplet information of the two fragments; the positional relationship includes whether they are adjacent and their adjacent directions; A fragment placement module is used to place the plurality of fragments one by one into a fragment splicing grid. Each fragment placement step includes the following sub-steps: For any one of the currently unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontal connection strength; global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to all fragments except the first fragment; dynamic frontal connection strength refers to the average adjacency affinity of the first fragment with the currently placed second fragments; adjacency affinity refers to the probability that the first fragment and a corresponding fragment will form an adjacency relationship; select the first fragment with the highest selection score from the plurality of first fragments as the current fragment to be placed; determine the target grid coordinates of the current fragment to be placed based on the grid coordinates of the currently placed second fragments and the positional relationship between the current fragment to be placed and the second fragments; place the current fragment to be placed into the fragment splicing grid according to the target grid coordinates.

[0092] Optionally, the positional relationship determination module is further configured to construct joint multimodal input information containing triple information of the two fragments, wherein the joint multimodal input information includes fragment images, edge maps, and text sequences of the two fragments; input the joint multimodal input information to a task-adapted visual language model, and output the positional relationship of the two fragments; wherein the positional relationship includes adjacency category and relative direction of principal axis; wherein the adjacency category is adjacent or non-adjacent, and the relative direction of principal axis is up, down, left, or right.

[0093] Optionally, the fragment placement module is further configured to input the grid coordinates of the currently placed plurality of second fragments, and the positional relationship between the current fragment to be placed and the plurality of second fragments as structured information, into the language model candidate anchor point generated by the task-adapted language model; filter out a first positional relationship with a confidence level higher than a preset confidence threshold from the positional relationship between the current fragment to be placed and the plurality of second fragments, and generate a deterministic relationship anchor point based on the first positional relationship; combine the language model candidate anchor point, the deterministic relationship anchor point, and the unoccupied grid coordinates in the neighborhood of the language model candidate anchor point and the deterministic relationship anchor point into a candidate coordinate set; calculate a unified score for any candidate coordinate in the candidate coordinate set, the unified score including the adjacency prediction consistency of the candidate coordinate, the direction prediction consistency of the candidate coordinate, the layout bounding rectangle expansion cost caused by the candidate coordinate, and the distance of the candidate coordinate from the language model candidate anchor point or the deterministic relationship anchor point; and select the candidate coordinate with the highest unified score from the candidate coordinate set as the target grid coordinate of the current fragment to be placed.

[0094] Optionally, the fragment placement module is further configured to calculate the adjacency prediction consistency and direction prediction consistency of the candidate coordinates by: determining a second fragment already placed adjacent to the candidate coordinates as a reference fragment; obtaining a second positional relationship between the current fragment to be placed and the reference fragment; determining the adjacency prediction consistency of the candidate coordinates relative to the reference fragment based on the adjacency category in the second positional relationship; and determining the direction prediction consistency of the candidate coordinates relative to the reference fragment based on the principal axis relative direction in the second positional relationship.

[0095] Optionally, the fragment placement module is further configured to calculate the layout bounding rectangle expansion cost caused by the candidate coordinates in the following manner: determining the maximum, minimum, maximum, and minimum values ​​of the grid coordinates of the currently placed second fragments, and calculating the first area of ​​the current layout bounding rectangle based on the maximum, minimum, maximum, and minimum values ​​of the grid coordinates; determining new maximum, minimum, maximum, and minimum values ​​of the grid coordinates based on the candidate coordinates and the grid coordinates of the currently placed second fragments, and calculating the second area of ​​the new layout bounding rectangle based on the new maximum, minimum, maximum, and minimum values ​​of the grid coordinates; and determining the layout bounding rectangle expansion cost caused by the candidate coordinates based on the difference between the second area and the first area.

[0096] Optionally, the fragment placement module is further configured to perform neighborhood expansion on any anchor point among the language model candidate anchor points and the deterministic relationship anchor points, with an expansion radius of 1; expand a preset number of grids around the current layout bounding rectangle; and combine the expanded unoccupied grid coordinates with the language model candidate anchor points and the deterministic relationship anchor points to form a candidate coordinate set, wherein the number of candidate coordinates has an upper limit.

[0097] The fragment global reassembly system and the fragment global reassembly method provided in the above embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0098] This application also provides a computer device for performing the above-described global fragmentation reassembly method. Please refer to... Figure 6 This illustrates a schematic diagram of a computer device provided by some embodiments of this application. For example... Figure 6 As shown, the computer device 6 includes: a processor 600, a memory 601, a bus 602, and a communication interface 603. The processor 600, the communication interface 603, and the memory 601 are connected via the bus 602. The memory 601 stores a computer program that can run on the processor 600. When the processor 600 runs the computer program, it executes the fragment global reassembly method provided in any of the foregoing embodiments of this application.

[0099] The memory 601 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0100] Bus 602 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 601 is used to store programs. After receiving an execution instruction, processor 600 executes the program. The global fragmentation reassembly method disclosed in any of the foregoing embodiments can be applied to processor 600, or implemented by processor 600.

[0101] The processor 600 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 600 or by instructions in software form. The processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 601. Processor 600 reads the information in memory 601 and, in conjunction with its hardware, completes the steps of the above method.

[0102] The computer device provided in this application embodiment and the fragment global reassembly method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0103] This application also provides a computer-readable storage medium corresponding to the fragment global reassembly method provided in the foregoing embodiments. Please refer to... Figure 7 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the fragment global reassembly method provided in any of the foregoing embodiments.

[0104] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0105] The computer-readable storage medium provided in the above embodiments of this application and the fragment global reassembly method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0106] It should be noted that: Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0107] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0108] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0109] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for global fragment reassembly, characterized in that, The method includes: Multiple fragments from the same page are obtained, and triple information for each fragment is constructed; the triple information includes a fragment image, an edge map corresponding to the fragment image, and a text sequence; the edge map is used to represent the boundary and contour of the fragment; For any two fragments among the plurality of fragments, the positional relationship between the two fragments is determined based on the triplet information of the two fragments; the positional relationship includes whether they are adjacent and their adjacent directions; The multiple fragments are placed one by one into the fragment splicing grid; wherein, each fragment placement step includes the following sub-steps: Step S1: For any one of the currently unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontal connection strength. Global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to all fragments other than the first fragment. Dynamic frontal connection strength refers to the average adjacency affinity of the first fragment with the currently placed second fragments. Adjacency affinity refers to the probability that the first fragment will form an adjacency relationship with a corresponding fragment. Step S2: Select the first fragment with the highest score from the plurality of first fragments as the current fragment to be placed; Step S3: Determine the target grid coordinates of the currently placed fragment based on the grid coordinates of the several second fragments already placed and the positional relationship between the fragment to be placed and the several second fragments; Step S4: Place the currently placed fragment into the fragment splicing grid according to the target grid coordinates.

2. The method according to claim 1, characterized in that, Determining the positional relationship between the two fragments based on the triplet information of the two fragments includes: Construct joint multimodal input information containing triplet information of the two fragments, wherein the joint multimodal input information includes fragment images, edge maps and text sequences of the two fragments; The joint multimodal input information is input into a task-adapted visual language model, and the positional relationship between the two fragments is output. The positional relationship includes the adjacency category and the relative direction of the main axis. The adjacency category is adjacent or non-adjacent, and the relative direction of the main axis is up, down, left or right.

3. The method according to claim 1 or 2, characterized in that, Based on the effective area, global connectivity, and dynamic frontier connectivity strength of the first fragment, the selection score of the first fragment is calculated, including: in, Indicates the first The first fragment in the second fragment placement stage The choice of score, For the first fragment The effective area, For the first fragment Global connectivity, For the first fragment Dynamic frontal connection strength, To be with the first fragment The first weighting coefficient corresponding to the effective area, To be with the first fragment The second weight coefficient corresponding to global connectivity. To be with the first fragment The third weighting coefficient corresponding to the dynamic frontier connection strength.

4. The method according to claim 1, characterized in that, Based on the grid coordinates of the currently placed second fragments and the positional relationship between the currently placed fragment and the second fragments, the target grid coordinates of the currently placed fragment are determined, including: The grid coordinates of the currently placed second fragments, and the positional relationship between the currently placed fragment and the several second fragments, are used as structured information and input into the language model candidate anchor points generated by the task-adapted language model. From the positional relationships between the current fragment to be placed and the plurality of second fragments, a first positional relationship with a confidence level higher than a preset confidence threshold is selected, and a deterministic relationship anchor point is generated based on the first positional relationship; The candidate anchor points of the language model, the anchor points of the deterministic relations, and the unoccupied grid coordinates in the neighborhood of the candidate anchor points of the language model and the anchor points of the deterministic relations are combined into a set of candidate coordinates; For any candidate coordinate in the candidate coordinate set, calculate the unified score of the candidate coordinate. The unified score includes the degree of consistency of the candidate coordinate's adjacency prediction, the degree of consistency of the candidate coordinate's direction prediction, the cost of the layout bounding rectangle expansion caused by the candidate coordinate, and the distance of the candidate coordinate from the candidate anchor point of the language model or the deterministic relation anchor point. The candidate coordinates with the highest uniform score are selected from the candidate coordinate set and used as the target grid coordinates for the current fragment to be placed.

5. The method according to claim 4, characterized in that, The consistency of the candidate coordinates in terms of adjacency prediction and direction prediction is calculated in the following way: The second fragment that is already placed and adjacent to the candidate coordinates is identified as the reference fragment; Obtain the second positional relationship between the currently placed fragment and the reference fragment; The degree of adjacency prediction consistency of the candidate coordinates relative to the reference fragment is determined based on the adjacency category in the second positional relationship, and the degree of orientation prediction consistency of the candidate coordinates relative to the reference fragment is determined based on the principal axis relative direction in the second positional relationship.

6. The method according to claim 4, characterized in that, The cost of expanding the layout bounding rectangle due to the candidate coordinates is calculated in the following way: Determine the maximum, minimum, maximum, and minimum values ​​of the grid coordinates of the currently placed second fragments, and calculate the first area of ​​the current layout's circumscribed rectangle based on these values. Based on the candidate coordinates and the grid coordinates of the currently placed second fragments, determine the new maximum value of the horizontal coordinate, the new minimum value of the horizontal coordinate, the new maximum value of the vertical coordinate, and the new minimum value of the vertical coordinate, and calculate the second area of ​​the outer rectangle of the new layout based on the new maximum value of the horizontal coordinate, the new minimum value of the horizontal coordinate, the new maximum value of the vertical coordinate, and the new minimum value of the vertical coordinate. The cost of expanding the layout bounding rectangle caused by the candidate coordinates is determined based on the difference between the second area and the first area.

7. The method according to claim 4, characterized in that, The candidate coordinate set is formed by combining the language model candidate anchor points, the deterministic relation anchor points, and the unoccupied grid coordinates within the neighborhood of the language model candidate anchor points and the deterministic relation anchor points, including: For any anchor point among the candidate anchor points of the language model and the anchor points of the deterministic relation, a neighborhood expansion is performed on the anchor point with an expansion radius of 1; Expand the grid around the current layout's outer rectangle by a preset number; The expanded unoccupied grid coordinates are combined with the language model candidate anchors and the deterministic relation anchors to form a candidate coordinate set, wherein the number of candidate coordinates is capped.

8. A global fragment reassembly system, characterized in that, The system includes: The triplet information construction module is used to acquire multiple fragments on the same page and construct triplet information for each fragment; the triplet information includes fragment images, edge maps corresponding to the fragment images, and text sequences; the edge maps are used to represent the boundaries and contours of the fragments; The positional relationship determination module is used to determine the positional relationship between any two fragments among the plurality of fragments based on the triplet information of the two fragments; the positional relationship includes whether they are adjacent and their adjacent directions; A fragment placement module is used to place the plurality of fragments one by one into a fragment splicing grid. Each fragment placement step includes the following sub-steps: For any one of the currently unplaced first fragments, calculate the selection score of the first fragment based on its effective area, global connectivity, and dynamic frontal connection strength; global connectivity refers to the average adjacency affinity of the first fragment with all other fragments, where all other fragments refer to all fragments except the first fragment; dynamic frontal connection strength refers to the average adjacency affinity of the first fragment with the currently placed second fragments; adjacency affinity refers to the probability that the first fragment and a corresponding fragment will form an adjacency relationship; select the first fragment with the highest selection score from the plurality of first fragments as the current fragment to be placed; determine the target grid coordinates of the current fragment to be placed based on the grid coordinates of the currently placed second fragments and the positional relationship between the current fragment to be placed and the second fragments; place the current fragment to be placed into the fragment splicing grid according to the target grid coordinates.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the fragment global reassembly method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the fragment global reassembly method according to any one of claims 1 to 7.

Images