A method for intelligent punctuation compression and line layout for East Asian text typesetting

Abstract

This invention discloses an intelligent punctuation compression and line layout method for East Asian text typesetting. East Asian full-width punctuation marks are categorized into six types based on their typesetting function: start punctuation, end punctuation, sentence-end punctuation, sentence-in-sentence punctuation, exclamation / interrogative punctuation, and center punctuation. Each type defines an independent compressibility and compression direction. The categorized punctuation marks are modeled as composite typesetting elements carrying both width flexibility parameters and line break cost parameters. The line break cost and remaining compression capacity are correlated within the same element, and the composite typesetting element is incorporated into the cost optimization process of the line layout algorithm. The selection of line break points and punctuation spacing allocation are jointly determined in the same optimization calculation. A power-law cost function and hierarchical spacing allocation priority (punctuation compression takes precedence over word spacing adjustment, which in turn takes precedence over character spacing adjustment) are employed, and rules are implemented through hard constraints. Vertical layout mode, vertical-center-horizontal unit modeling, and cross-language configurable rules are supported. This invention solves the problem of suboptimal global typesetting quality caused by the decoupling of punctuation compression and line break algorithms in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of digital document typesetting engine technology, specifically to an intelligent punctuation mark width compression method for East Asian scripts (including but not limited to Chinese, Japanese, Korean, and mixed typeset scenarios), and a corresponding line layout algorithm. The method classifies punctuation marks according to their typesetting functions, establishes a context-aware compression rule system, and incorporates punctuation compression into the cost function of a globally optimal line-breaking algorithm, achieving adaptive adjustment of punctuation width and unified optimization of justified alignment. Background Technology

[0002] East Asian scripts (Chinese, Japanese, and Korean, hereinafter collectively referred to as CJK) use full-width punctuation marks in typesetting. The design width of a full-width punctuation mark is the width of one Chinese character, but its actual strokes only occupy about half the character width, with the remaining space being the inherent white space of the punctuation mark. When consecutive punctuation marks are adjacent, visually excessive white gaps are created between them, severely affecting the compactness and aesthetics of the typesetting. This problem does not exist in pure Latin typesetting; therefore, mainstream typesetting engines designed for Latin do not have East Asian punctuation compression capabilities by default.

[0003] The existing technology has the following drawbacks:

[0004] Defect D1 – The punctuation classification is crude and lacks a unified classification system guided by typesetting functions. The existing solution only classifies punctuation according to character encoding or a simple blacklist, without semantic classification based on the functional role of punctuation in typesetting, resulting in the inability to formulate differentiated compression strategies for punctuation with different functions.

[0005] Defect D2 – Punctuation compression and line break algorithms are decoupled, preventing global optimization. In existing solutions, punctuation compression is typically performed as a post-processing step after line breaks, failing to utilize the flexibility of punctuation compression to optimize line break point selection, resulting in suboptimal typesetting quality. While W3C CLREQ (Chinese Typesetting Requirements) and JLREQ (Japanese Typesetting Requirements) provide descriptions of punctuation classification and line adjustment priority, both documents explicitly state that they "do not propose specific processing techniques," remaining merely descriptive documents without defining algorithm implementations. The CSS text-spacing-trim property (W3C CSS Text Level 4, 2024-2025 draft) provides a static post-processing switch after line breaks, not affecting line break point selection, fundamentally different from the joint optimization scheme of this invention.

[0006] Defect D3 – Lack of a hierarchical cost model, failing to express compression priorities. Existing solutions typically use a uniform allocation method to process all adjustable elements, lacking a hierarchical cost function to express the priority relationship between different adjustment methods.

[0007] Defect D4 – Lack of consistency across languages. Existing solutions either only target a single language or use uniform, simple rules that ignore language differences, lacking a unified punctuation classification framework to support differentiated processing across multiple languages.

[0008] Defect D5 – Incomplete handling of punctuation in mixed vertical and horizontal layouts. Existing technologies typically limit the handling of vertical punctuation to character rotation, failing to incorporate the compression direction transformation of vertical punctuation, modeling of the width of vertical and horizontal units, and calculation of combined character sizes into a unified typesetting model.

[0009] Defect D6 – Lack of cross-platform typesetting consistency. Punctuation processing behavior is inconsistent across different operating systems and browsers, potentially resulting in different punctuation compression and line break positions for the same text.

[0010] The Knuth-Plass line break algorithm (US4956809A, expired in 2009) is currently the best Latin line break algorithm, using a Box-Glue-Penalty node model. In the original scheme, Box (content) and Penalty (line break cost) are separate node types. Latin punctuation has no compressible properties, making it unsuitable for the coupled modeling requirements of East Asian full-width punctuation. Summary of the Invention

[0011] Technical issues.

[0012] The technical problem to be solved by this invention is: to establish a punctuation classification system for typesetting functions, to jointly optimize punctuation compression and line breaking algorithms, to construct a hierarchical line layout cost model, to support unified processing of multiple languages, to incorporate mixed vertical and horizontal layouts into the unified model, and to achieve cross-platform deterministic typesetting output.

[0013] Technical solution.

[0014] This invention categorizes East Asian full-width punctuation marks into multiple types based on their typesetting function, including: start punctuation (compressible leading whitespace), end punctuation (compressible trailing whitespace), sentence-end punctuation (compressible trailing whitespace), sentence-middle punctuation (compressible trailing whitespace), exclamatory and interrogative punctuation (compressible trailing whitespace), and centered punctuation (uncompressible on both sides). Each category defines an independent compressibility amount and compression direction. The classification framework is implemented through a configurable character-category mapping table and supports language-based expansion.

[0015] The key feature of this invention is that each categorized full-width punctuation mark is modeled as a composite layout element. This composite layout element simultaneously carries a width flexibility parameter (based on compressibility) and a line break cost parameter. The line break cost is related to the remaining compression capacity within the same element, rather than the separation of Box and Penalty in the original Knuth-Plass scheme. This composite layout element is incorporated into the line layout cost optimization process, and the selection of line break points and the allocation of punctuation spacing are jointly determined in the same optimization calculation.

[0016] The row layout cost function is a power function of the ratio of deviation to elastic capacity. When the power is 3, the cost increases sharply as the elastic capacity tends to be exhausted. Spacing allocation is performed according to hierarchical priority: in the compression scenario, redundant punctuation whitespace is consumed first (cost constant 10), then the inter-word spaces are adjusted (50), then the multilingual text boundary spacing is adjusted (80), and finally the micro-spacing between characters is adjusted (200).

[0017] Prohibition handling is implemented through hard constraints: candidate line break positions that violate line start or line end prohibitions are directly excluded from the candidate set, without soft suppression through increasing penalty costs. Prohibition character sets can be configured independently at the paragraph level by language.

[0018] For the boundary between East Asian characters and Latin characters, an elastic spacing node (with a natural width of one-quarter of the font size) is automatically inserted. This node participates in the unified hierarchical spacing distribution.

[0019] In vertical layout mode, the width flexibility semantics of compound layout elements are converted to vertical height flexibility semantics, and the punctuation compression direction changes from horizontal to vertical; consecutive 2 to 4 digit characters are merged into vertical-middle-horizontal units to participate in line layout.

[0020] Beneficial effects.

[0021] Effect E1: By classifying punctuation marks into fine-grained categories for typesetting functions and assigning independent compression parameters to each type of punctuation mark, more precise punctuation width control is achieved than the coarse-grained switch, thus resolving defect D1.

[0022] Effect E2: By incorporating the compressible space of punctuation into the elastic model of the line break algorithm, the entire paragraph is optimized in a coordinated manner, and the line break quality is better than that of the greedy algorithm, thus solving the defect D2.

[0023] Effect E3: The hierarchical cost function ensures that the typesetting engine adjusts according to the preset priority sequence, and the typesetting result conforms to East Asian typesetting aesthetics, thus solving defect D3.

[0024] Effect E4: A unified punctuation classification system and an extensible character-category mapping table enable the same algorithm to support Chinese, Japanese, Korean and their arbitrary mixed combinations, thus solving defect D4.

[0025] Effect E5: The same Box-Glue-Penalty model handles both horizontal and vertical arrangements simultaneously, with the horizontal units in the vertical center participating in the unified layout as special nodes, thus resolving defect D5.

[0026] Effect E6: The typesetting engine compiles to platform-independent bytecode, producing deterministic and consistent typesetting results on different operating systems with the same input, thus resolving defect D6.

[0027] Performance E7: Real-world testing shows that layout calculations for 1000 characters (approximately 15 lines) take less than 0.1 milliseconds; layouts for 10000 characters (approximately 150 lines) take less than 2 milliseconds; large paragraphs exceeding 500 lines automatically downgrade to a greedy algorithm and issue warnings, balancing quality and responsiveness.

[0028] Attached image description.

[0029] Figure 1 shows the system architecture of the typesetting engine, illustrating the interconnected relationships and external configuration inputs of the five core modules: character classifier, Box-Glue-Penalty generator, line break solver, inline spacing allocator, and glyph renderer.

[0030] Figure 2 is a classification system diagram of punctuation mark typesetting functions, showing that full-width punctuation marks are divided into six categories according to their functional roles, as well as the compression direction attributes of each category.

[0031] Figure 3 is a schematic diagram of Box-Glue-Penalty node modeling, showing the node sequence of the text segment "Artificial Intelligence (AI) Era", including pure Box nodes of CJK characters, Box+Penalty composite nodes of punctuation marks, and Glue nodes at the CJK-Latin boundary.

[0032] Figure 4 shows the complete flowchart of line break and spacing allocation, illustrating the entire process from BGP node sequence input through rule filtering, cost calculation, dynamic programming solution to spacing allocation according to hierarchical priority.

[0033] Figure 5 is a schematic diagram of hierarchical cost priority, showing the priority chain of each adjustment method and the corresponding cost constant in the two scenarios of compression and stretching.

[0034] Figure 6 is a schematic diagram of vertical and horizontal layout, showing the text layout direction, punctuation vertical font, and the processing of vertical and horizontal number units in vertical layout mode.

[0035] Detailed implementation method.

[0036] Implementation Method 1: Character Classification and Punctuation Function Category Definition The character classifier categorizes each character in the Unicode character stream into one of the following categories: CJK Chinese characters (Unicode range 0x3400-0x4DBF, 0x4E00-0x9FFF, etc.), CJK punctuation (subdivided by function into CL_OPEN / CL_CLOSE / CL_PERIOD / CL_COMMA / CL_EXCLAIM / CL_MIDDLE), Latin characters, Arabic numerals, spaces, and others. The parameters for each CJK punctuation category are as follows: Start punctuation (CL_OPEN): natural width 1em, compressible space 0.5em, compression direction is leading whitespace; End punctuation (CL_CLOSE): natural width 1em, compressible space 0.5em, compression direction is trailing whitespace; Sentence-ending punctuation (CL_PERIOD) and sentence-middle punctuation (CL_COMMA) are the same as end punctuation; Centered punctuation (CL_MIDDLE): natural width 1em, compressible space 0, no compression on either side.

[0037] Implementation Method 2: Generation of Box-Glue-Penalty Node Sequences Each typographic element in the paragraph text is generated as one of the following nodes: CJK Chinese character generation: Pure Box node, 1em width, non-compressible and non-stretchable.

[0038] CJK punctuation generates a Box+Penalty composite node: the Box part carries the natural width (1em) and compressibility (0.5em); the Penalty part carries the line break cost (10), indicating that the position allows line break at a lower cost.

[0039] Automatically insert Glue nodes at the boundary between CJK and Latin characters: natural width 0.25em, compressible to 0.4 times the natural width, and stretchable to 0.6 times the natural width.

[0040] The space between Latin words generates a Glue node: natural width 0.33em, compressible size 0.11em, stretchable size 0.5em.

[0041] Implementation Method 3: Row Cost Calculation and Dynamic Programming for Row Disconnection For each candidate line break interval, perform a prohibition check: if the last character of the interval belongs to the line end prohibition set or the first character of the interval belongs to the line beginning prohibition set, the line break point is directly excluded.

[0042] Calculate the total natural width of the candidate lines that pass the rule check, and the deviation diff from the target line width (a positive value indicates a too long line, and a negative value indicates a too short line). Select the elastic capacity in the corresponding direction (compress the capacity when the line is too long, and stretch the capacity when the line is too short), and calculate the line cost according to the cost function with a power of 3.

[0043] Dynamic programming searches for the line breaking scheme that minimizes the total cost of the entire paragraph in the set of all candidate line breaking points.

[0044] Embodiment 4: Inter-line spacing allocation After determining the line breaking points, allocate the spacing to each line according to hierarchical priorities: when the line is too long, in step one, preferentially compress the redundant white space of punctuation (the compression amount for each punctuation does not exceed 0.5em), in step two, compress the space between words (not exceeding 0.11em), and in step three, compress the CJK-Latin boundary spacing (not exceeding 0.4 times the natural width); when the line is too short, in step one, preferentially stretch the space between words (not exceeding 0.5em), in step two, stretch the boundary spacing (not exceeding 0.6 times the natural width), and in step three, slightly stretch the CJK character spacing (not exceeding 0.05em).

[0045] Embodiment 5: Example of the complete typesetting process Input text: In the era of artificial intelligence (AI), PeckOffice defines the "productivity foundation".

[0046] Step 1, character classification: artificial intelligence → CJK×4; (→ CL_OPEN; AI → Latin×2; ) → CL_CLOSE; era → CJK×2;, → CL_COMMA; PeckOffice → Latin×10; (space); defines → CJK×3; "→ CL_OPEN; productivity foundation → CJK×5; "→ CL_CLOSE;. → CL_PERIOD.

[0047] Step 2, BGP node generation: Each CJK character generates a pure Box; each punctuation generates a Box+Penalty composite node; no Glue is inserted at the CJK→CL_OPEN boundary between neng and (; no Glue is inserted at the CL_CLOSE→CJK boundary between ) and shi; Pangu Glue (0.25em) is inserted between, and P; Pangu Glue is inserted between e (the end) and ding; no Glue is inserted between le and " (directly adjacent CJK→CL_OPEN).

[0048] Step 3, line breaking solution: After performing rule filtering according to the line width constraint, find the optimal line breaking points through dynamic programming, and the 0.5em compression space of the punctuation composite node participates in the cost calculation.

[0049] Step 4, Spacing Allocation: Allocate compression or stretching amounts for each line according to hierarchical priority, and output the final position coordinates of each character.

Claims

1. Claim 1. A method for arranging punctuation marks in East Asian text typesetting, characterized in that, Includes the following steps: (a) East Asian full-width punctuation marks are divided into multiple categories according to their functional roles in typesetting, including at least start punctuation, end punctuation and sentence punctuation. Each category is defined with independent compressibility and compression direction. (b) Model each full-width punctuation mark after classification as a composite layout element. The composite layout element carries both a width elasticity parameter and a line break cost parameter based on the compressibility of the punctuation mark. The line break cost and the remaining compression capacity are associated within the same element. The composite layout element is incorporated into the cost optimization process of the line layout algorithm. The selection of the line break point and the allocation of the punctuation mark spacing are jointly determined in the same optimization calculation.

2. Claim 2. A punctuation mark line layout system for East Asian text typesetting, characterized in that, include: The character classification module is configured to divide East Asian full-width punctuation marks into multiple categories according to their typesetting function roles, with each category having an independent definition of compressibility and compression direction. The composite node generation module is configured to model the classified full-width punctuation marks as composite layout elements that simultaneously carry width elasticity parameters and line break cost parameters; The line break solution module is configured to incorporate the composite layout elements into the line layout cost optimization process, and jointly solve the line break point selection and punctuation spacing allocation.

3. Claim 3. The method according to claim 1, characterized in that, The multiple categories mentioned in step (a) include: start punctuation (leading whitespace is compressible), end punctuation (tailing whitespace is compressible), sentence-end punctuation (tailing whitespace is compressible), sentence-middle punctuation (tailing whitespace is compressible), and center punctuation (both sides are not compressible); the classification framework is implemented through a configurable character-category mapping table, which supports expanding the character set in the mapping table by language or region.

4. Claim 4. The method according to claim 1, characterized in that, The joint determination of line break point selection and punctuation spacing allocation in step (b) is achieved through the following hierarchical priority: punctuation redundancy is consumed first according to the compressibility of punctuation type, then word spacing is adjusted, multilingual text boundary spacing is adjusted, and finally micro-spacing between characters is adjusted; the adjustment amount at each level does not exceed its respective preset upper limit.

5. Claim 5. The method according to claim 1, characterized in that, In step (b), the cost function for optimizing row layout cost is a power function of the ratio of row width deviation to available elastic capacity, where the power is not less than 2 and the basic cost coefficient is a positive real number. When the power is 3, the cost increases sharply as the elastic capacity tends to be exhausted, guiding the row break algorithm to prioritize row break schemes with sufficient elastic capacity.

6. Claim 6. The method according to claim 1, characterized in that, It also includes a hard constraint prohibition processing step: at each line break candidate position, check whether the adjacent characters belong to the line start prohibited character set or the line end prohibited character set. Candidate positions that violate the prohibition are directly excluded from the candidate set without using a method that increases the penalty cost to achieve exclusion; the prohibited character set can be configured independently at the paragraph level according to the language.

7. Claim 7. The method according to claim 1, characterized in that, It also includes context-aware compression rules: when two full-width punctuation marks are adjacent, the whitespace on the adjacent sides is completely compressed to zero; when the start punctuation mark appears at the beginning of the line, its leading whitespace is compressed to zero; when the end punctuation mark or the punctuation mark in the middle of the sentence appears at the end of the line, its trailing whitespace is compressed to zero; the above rules are used as lower bound constraints on the amount of compression in the cost optimization of step (b).

8. Claim 8. The method according to claim 1, characterized in that, The generation of the flexible typesetting node sequence in step (b) further includes: automatically inserting flexible spacing nodes between East Asian characters and adjacent Latin letters or Arabic numerals, wherein the natural width of the flexible spacing node is one-quarter of the current font size, which can be compressed to zero and not exceed the natural width; the flexible spacing node is in the adjacent priority level with the space between words in the spacing allocation in step (b).

9. Claim 9. The method according to claim 1, characterized in that, The method also supports vertical writing mode; in vertical mode, the width elastic semantics of the composite typesetting element in step (b) is converted into vertical height elastic semantics, and the punctuation compression direction changes from horizontal to vertical; consecutive 2 to 4 digit characters in vertical text are identified as vertical-middle-horizontal sequences, merged into a single composite typesetting node to participate in line layout, and its vertical height is equal to the height of a full-width character.

10. The method according to claim 1, characterized in that, The method also includes a two-way mapping step between typesetting parameters and office document formats: establishing a corresponding mapping relationship between the configuration parameters of the punctuation classification and compression rules described in step (a) and the East Asian typesetting attributes of the office document format, so as to achieve lossless parsing of East Asian typesetting semantics during import and complete preservation of typesetting settings when exporting after content modification.

11. The system according to claim 11. The system according to claim 2, characterized in that, It also includes a prohibition configuration module, which stores the prohibited character sets at the beginning and end of lines defined by language, and can be independently overridden at the paragraph level; the output of the prohibition configuration module directly drives the candidate position exclusion operation of the line break solution module, without going through the cost function path.

12. The system according to claim 12, characterized in that, The line break solution module has a built-in paragraph size detection mechanism. When the number of nodes in the paragraph to be formatted exceeds a preset threshold, it automatically switches to a greedy line break algorithm. The greedy algorithm and the global optimal algorithm use the same composite typesetting element model and cost function. The line break results of the two modes have the same format and are transparent to the upper-level rendering module.

13. Claim 13. The system according to claim 2, characterized in that, The system is compiled into a platform-independent bytecode format and runs in a virtual machine. The same input text produces deterministic and consistent typesetting results in different operating systems and runtime environments. This determinism is guaranteed by standardized floating-point arithmetic specifications and font metric calculations that are independent of the host environment.

Images