Page generation method and device, equipment, medium and program product

Abstract

The invention provides a page generation method which can be applied to the technical field of artificial intelligence, and relates to application of a large model in the fields of information security and financial science and technology. The page generation method comprises the following steps: processing a report image applied to overprinting by utilizing a first model to obtain a first coordinate set of N report elements in the report image; analyzing the report image based on the first coordinate set by using a second model to obtain M layout elements and a second coordinate set thereof, the second coordinate set being determined in at least one region defined based on the first coordinate set, and N and M being integers greater than or equal to 1; based on the image features of the report image, the M layout elements and the second coordinate set, a target page having a position mapping relation with the report image is generated, and the position mapping relation indicates the mapping relation between the at least one printing position. The invention further provides a page generation device and equipment, a storage medium and a program product.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, to the application of large models in information security and fintech, and more specifically to a page generation method, apparatus, device, medium, and program product. Background Technology

[0002] With the development of information technology, traditional overprinting technology is costly, time-consuming, lacks flexibility and rapid response, and struggles to meet rapidly changing needs. For example, in traditional bank branches, whether it's counter transactions or new transactions under new models, the issue of printing account opening / change application forms is unavoidable. Traditionally, the application process involves developers measuring standard application form parameters with a ruler based on actual conditions before development. However, this method suffers from drawbacks such as accumulated errors in line spacing and margins due to manual measurement, requiring multiple debugging sessions; inconsistencies between regional and national templates necessitate repeated development; and the need to write layout logic line by line for static HTML development. Summary of the Invention

[0003] In view of the above problems, this application provides a page generation method, apparatus, device, medium and program product.

[0004] According to a first aspect of this application, a page generation method is provided, comprising: processing a report image applied for overprinting using a first model to obtain a first coordinate set of N report elements in the report image; parsing the report image based on the first coordinate set using a second model to obtain M layout elements and their second coordinate set, wherein the second coordinate set is determined within at least one region defined based on the first coordinate set, and N and M are integers greater than or equal to 1; and generating a target page having a positional mapping relationship with the report image based on image features of the report image, the M layout elements, and the second coordinate set, wherein the positional mapping relationship indicates a mapping relationship between at least one overprinting position.

[0005] According to an embodiment of this application, generating a target page with a positional mapping relationship to the report image based on the image features of the report image, M layout elements, and a second coordinate set includes: generating multiple candidate page codes based on a first programming language using a third model, based on the image features of the report image, M layout elements, and the second coordinate set, wherein the multiple candidate page codes respectively indicate different local areas of the target page; obtaining target page code in a second programming language based on the multiple candidate page codes, the target page code being used to form the target page, wherein the complexity of the first programming language is less than the complexity of the second programming language.

[0006] According to an embodiment of this application, using a second model to parse a report image based on a first coordinate set to obtain M layout elements and their second coordinate set includes: using a second model to parse a report image based on a first coordinate set to obtain a layer segmentation result; performing edge detection on the target layer based on the layer segmentation result to obtain an edge density distribution heatmap; and obtaining M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap.

[0007] According to an embodiment of this application, obtaining M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap includes: determining local areas where the edge density is less than or equal to a preset value based on the density distribution; using the local areas as cutting points and determining a target cutting path based on a target planning algorithm; and cutting the report image based on the target cutting path to obtain M layout elements and their second coordinate set.

[0008] According to an embodiment of this application, after generating the target page code of the target page, the method further includes: retrieving layout conflict features of the target page based on the target page code, wherein the layout conflict features characterize the interference of page elements in the target page; and updating the target cutting path based on the layout conflict features using a target planning algorithm in response to the layout conflict features meeting preset conditions, wherein the updated target cutting path indicates a cutting path for obtaining M layout elements and their second coordinate set again.

[0009] According to an embodiment of this application, generating multiple candidate page codes based on a first programming language using a third model based on image features of a report image, M layout elements, and a second coordinate set includes: using the third model based on the syntax rules of the first programming language to process image features and text features of the M layout elements and the second coordinate set to generate multiple candidate page codes.

[0010] According to an embodiment of this application, before obtaining the target page code based on the second programming language based on the multiple candidate page codes, the method further includes: matching multiple layout information obtained based on the multiple candidate page codes with a set of candidate templates in a preset template library, wherein the candidate templates are obtained based on local regions in the candidate report images applied to overlay printing; and adjusting the corresponding candidate page code based on the target template in response to at least one of the multiple layout information matching the corresponding target template.

[0011] According to an embodiment of this application, obtaining target page code based on a second programming language from multiple candidate page codes includes: processing image features and multiple candidate page codes using a fourth model based on the syntax rules of a first programming language, generating intermediate page code based on the first programming language, wherein the intermediate page code indicates a candidate page that has a positional mapping relationship with the report image; and converting the intermediate page code into target page code based on the second programming language.

[0012] A second aspect of this application provides a page generation apparatus, comprising: a first acquisition module, configured to process a report image applied for overprinting using a first model to acquire a first coordinate set of N report elements in the report image; a second acquisition module, configured to parse the report image based on the first coordinate set using a second model to acquire M layout elements and their second coordinate set, wherein the second coordinate set is determined within at least one region defined based on the first coordinate set, and N and M are integers greater than or equal to 1; and a generation module, configured to generate a target page having a positional mapping relationship with the report image based on image features of the report image, the M layout elements, and the second coordinate set, wherein the positional mapping relationship indicates a mapping relationship between at least one overprinting position.

[0013] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0015] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0016] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 The illustrations depict application scenarios of page generation methods, apparatus, devices, media, and program products according to embodiments of this application.

[0018] Figure 2 A flowchart illustrating a page generation method according to an embodiment of this application is shown schematically;

[0019] Figure 3This schematically illustrates a flowchart of generating a target page that has a positional mapping relationship with a report image according to an embodiment of this application;

[0020] Figure 4 This schematically illustrates a flowchart of a process according to an embodiment of the present application, in which a second model is used to parse a report image based on a first coordinate set to obtain M layout elements and their second coordinate set.

[0021] Figure 5 This schematically illustrates a flowchart of obtaining M layout elements and their second coordinate set based on the density distribution in an edge density distribution heatmap according to an embodiment of this application;

[0022] Figure 6 This schematically illustrates another flowchart for obtaining M layout elements and their second coordinate set based on the density distribution in an edge density distribution heatmap according to an embodiment of this application;

[0023] Figure 7 This schematically illustrates a structural block diagram of a page generation apparatus according to an embodiment of the present application; and

[0024] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a page generation method according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0030] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0031] The following is an explanation of some of the technical terms used in this application:

[0032] Overlay printing: This generally refers to the process of printing multiple contents or templates on the same sheet of paper. Overlay printing technology is often used in scenarios where it is necessary to combine fixed-format templates (such as forms, invoices, contracts, etc.) with dynamic content (such as data, personal information, etc.) for printing.

[0033] OCR (Optical Character Recognition) is a computer input technology that uses optical input methods such as scanning to convert the text of various tickets, newspapers, books, manuscripts and other printed materials into image information, and then uses character recognition technology to convert the image information into usable text.

[0034] Grab Cut refers to an image segmentation algorithm based on graph cut, used to automatically select and extract regions of interest in an image. It automatically performs fine segmentation using a small amount of user-provided interactive information (such as bounding boxes or doodle marks), and is suitable for fields such as image editing, computer vision, and machine learning.

[0035] DSL (Domain Specific Language): This refers to a computer language focused on a specific application domain. It is typically simpler and easier to understand and use than general-purpose programming languages. In front-end development, DSLs can help developers build complex user interfaces and interaction logic more quickly.

[0036] This application provides a page generation method, comprising: processing a report image applied for overlay printing using a first model to obtain a first coordinate set of N report elements in the report image; parsing the report image based on the first coordinate set using a second model to obtain M layout elements and their second coordinate set, wherein the second coordinate set is determined within at least one region defined based on the first coordinate set, and N and M are integers greater than or equal to 1; and generating a target page with a positional mapping relationship to the report image based on the image features of the report image, the M layout elements, and the second coordinate set, wherein the positional mapping relationship indicates a mapping relationship between at least one overlay printing position. By combining the first and second models into a joint optimization model, such as the GrabCut-OCR joint optimization model, text regions can be automatically detected, avoiding manual measurement errors and improving image segmentation efficiency. By directly generating a standard front-end static page from the report image, adaptability can be improved, and code development efficiency and accuracy can be enhanced.

[0037] Figure 1 The illustrations depict application scenarios of page generation methods, apparatus, devices, media, and program products according to embodiments of this application.

[0038] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0039] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0040] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0041] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0042] It should be noted that the page generation method provided in this application embodiment can generally be executed by server 105. Correspondingly, the page generation apparatus provided in this application embodiment can generally be located in server 105. The page generation method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the page generation apparatus provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0043] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0044] The following will be based on Figure 1 The described scene, through Figures 2-6 The page generation method according to the embodiments of this application will be described in detail.

[0045] Figure 2 A flowchart illustrating a page generation method according to an embodiment of this application is shown schematically.

[0046] like Figure 2 As shown, the page generation method of this embodiment includes operations S210 to S230.

[0047] In operation S210, the first model is used to process the report image applied to the overlay printing to obtain the first coordinate set of N report elements in the report image.

[0048] In the embodiments of this application, the first model can be an algorithm module based on optical character recognition (OCR) or object detection, specifically designed to process report images in pre-printed documents (such as bank account opening application forms shown in Table 1). It can accurately identify the structured coordinate data of multiple report elements in the image and output the coordinate set of these elements, i.e., the first coordinate set. Report elements refer to all visual components in a report image (such as bank account opening application forms, invoices, contracts, etc.) that need to be filled with business data or participate in layout analysis. These elements typically correspond to reserved blank areas or fixed content in the template and are the basic units for dynamic data filling and page generation. Each element can be bound to a specific business field, such as "customer name," "amount," or "signature field," and can be a text input box, table cell, checkbox, barcode, etc. Each element can have a defined coordinate range (x, y, width, height) in the image, and the coordinate set of N report elements is the first coordinate set.

[0049] For example, OCR (Optical Character Recognition) technology is primarily used to extract text region information from banking application templates and use it as initial parameters for subsequent image segmentation (Grab Cut algorithm). OCR technology identifies text regions in the template and outputs the bounding box (Rect parameter, i.e., rectangle coordinates: x, y, width, height) for each text block. For example, the bounding box for the "Customer Name" field in the template is (x=100, y=200, w=150, h=30). All text bounding boxes detected by OCR are merged into an initial rectangular region (rect), which is then used as input to the Grab Cut algorithm. This helps subsequent algorithms prioritize focusing on densely texted areas (foreground) and ignore irrelevant background.

[0050] Table 1 Application Form for Change of Bank Settlement Account by XX Bank

[0051]

[0052] In operation S220, the report image is parsed using the second model based on the first coordinate set to obtain M layout elements and their second coordinate set. The second coordinate set is determined within at least one region defined based on the first coordinate set, where N and M are integers greater than or equal to 1.

[0053] In embodiments of this application, the second model, a post-processing module of the first model, may include GrabCut, which, based on the report element coordinates (first coordinate set) output by the first model, further analyzes the layout structure of the report image (such as table columns, floating areas, and grouped fields), and outputs more refined layout elements and their coordinates, i.e., the second coordinate set. The analysis scope of the second model is limited to further analysis within the field area identified by the first model. For example, if the OCR identifies a large bounding box for the "Account Information Table," the second model only analyzes the row and column structure of the table within that area.

[0054] For example, Grab Cut is a graph-cut-based image segmentation algorithm primarily used for interactive image segmentation. Users can specify foreground and background regions using simple markers (such as rectangles or doodles), and the algorithm automatically performs fine-grained segmentation. Grab Cut segments the field region map identified by the first model into foreground and background through iterative optimization, combining color distribution and spatial information. Grab Cut models the color distributions of the foreground and background separately using a Gaussian Mixture Model (GMM), and then minimizes the energy function using a graph-cut algorithm to achieve segmentation. For example, users mark foreground and background regions with rectangles or doodles. Pixels outside the rectangle are labeled as background, and pixels inside the rectangle are labeled as possible foreground. A Gaussian Mixture Model (GMM) is trained for the foreground and background separately, such as using a 5-Gaussian distribution. Each pixel is assigned to the closest GMM component. A graph is constructed where nodes are pixels and edges represent relationships between pixels (color similarity and spatial proximity). Segmentation is completed by minimizing the energy function (including data terms and smoothing terms). GMM training and graph-cut optimization are repeated until convergence or the maximum number of iterations is reached. The final result is the segmentation of the foreground and background.

[0055] In operation S230, based on the image features of the report image, M layout elements and a second set of coordinates, a target page with a positional mapping relationship to the report image is generated. The positional mapping relationship indicates the mapping relationship between at least one overprint position.

[0056] In the embodiments of this application, target page generation involves mapping the layout and content of a report image (such as a bank account opening application form) to a standard static page (such as HTML / CSS) so that subsequent dynamic data can be accurately filled into the correct positions. Position mapping means that the generated page must be perfectly aligned with the layout of the original report image, allowing for direct printing after dynamic data filling without any misalignment or serial errors.

[0057] According to embodiments of this application, a joint optimization model, such as the GrabCut-OCR joint optimization model, is formed by combining a first model and a second model. This model can automatically detect text regions, avoid errors from manual measurement, and improve image segmentation efficiency. By directly generating standard front-end static pages from report images, adaptability can be improved, as well as code development efficiency and accuracy.

[0058] Figure 3 The flowchart illustrating the generation of a target page that has a positional mapping relationship with a report image according to an embodiment of this application is shown in the illustration.

[0059] like Figure 3 As shown, the generation of the target page that has a positional mapping relationship with the report image in this embodiment includes operations S231 to S232.

[0060] In operation S231, based on the image features of the report image, M layout elements and a second set of coordinates, a third model is used to generate multiple candidate page codes based on a first programming language, wherein the multiple candidate page codes respectively indicate different local areas of the target page.

[0061] In embodiments of this application, the third model can be a cross-modal Transformer architecture that can parse image features of the report image, M layout elements, and a second coordinate set to generate multiple candidate page codes in a first programming language, such as DSL code snippets. Each snippet describes a local area of ​​the target page, such as a table or title bar. Multiple DSL implementations (such as absolute positioning or Flex layout) are generated for the same local area, and conflicts and weights are marked.

[0062] For example, a cross-modal transformer architecture can be designed to jointly train image features and code syntax, thereby generating a DSL for sub-regions. Deep convolutional neural networks can be used to extract image features, which are then fused with DSL syntax rules encoded using a pre-trained language model based on the transformer architecture through cross-modal attention. DSL statements can be dynamically generated using a special neural network architecture such as pointer networks, supporting nested layout descriptions and generating front-end page DSLs.

[0063] In operation S232, target page code in a second programming language is obtained based on multiple candidate page codes. The target page code is used to form the target page. The complexity of the first programming language is less than that of the second programming language.

[0064] In the embodiments of this application, the generation of target page code is a transformation process from highly abstract intermediate code (a first programming language, such as a DSL) to concrete executable code (a second programming language, such as HTML / CSS). The first programming language, such as a DSL, can describe the semantics and structure of the page layout. The second programming language, such as HTML / CSS, can implement specific page rendering and interaction. For example, multiple candidate DSL code fragments (each fragment describing a local area of ​​the target page) are examined for compatibility (e.g., whether the areas overlap or whether data binding conflicts), and the optimal candidate is selected based on confidence weights (e.g., layout rationality and adaptability). The selected DSL fragments are then merged into a complete DSL. The (x, y, w, h) values ​​in the DSL are converted into absolute positioning properties in CSS, and the binding fields of the DSL are converted into data-binding properties in HTML. Corresponding CSS styles are added according to the layout type (e.g., tables, floating areas). The generated HTML page is overlaid with the original report image, and positional errors are verified by matching local features through key points. Pre-defined rules, such as margin offset compensation formulas, are called to dynamically adjust the DSL parameters. The generated HTML code is parsed into an AST, and layout conflicts (e.g., overlapping floating elements) are detected. If the error exceeds the limit, a model-free reinforcement learning algorithm is triggered to optimize the DSL generation strategy (such as adjusting the layout cutting path). Ultimately, a standard static HTML page with minimal error and high efficiency is generated.

[0065] According to embodiments of this application, target page code in a second programming language is obtained based on multiple candidate page codes, thereby improving the accuracy and adaptability of code generation.

[0066] Figure 4 The flowchart illustrates a process according to an embodiment of this application, in which a second model is used to parse a report image based on a first coordinate set to obtain M layout elements and their second coordinate set.

[0067] like Figure 4 As shown, this embodiment utilizes a second model to parse the report image based on a first coordinate set to obtain M layout elements and their second coordinate set, including operations S410 to S430.

[0068] In operation S410, the report image is parsed using the second model based on the first coordinate set to obtain the layer segmentation result.

[0069] In the embodiments of this application, the second model, such as Grab Cut, performs layer segmentation on the first coordinate set (such as text boxes, table lines, etc.) of the N report elements output by the first model, combines the text region (Rect parameter) detected by OCR as a prior constraint, optimizes the segmentation accuracy, generates a mask matrix, and distinguishes the foreground (such as text, tables) from the background (such as blank areas).

[0070] In operation S420, edge detection is performed on the target layer based on the layer segmentation results to obtain a heat map of edge density distribution.

[0071] In embodiments of this application, the Canny algorithm can be used to detect the edges of the target layer, obtain the edge pixel distribution, and generate an edge density distribution heatmap. For example, local density can be calculated using a sliding window (e.g., 16x16 pixels), and potential cutting points are marked when the density is below a threshold.

[0072] In operation S430, based on the density distribution in the edge density distribution heatmap, obtain M layout elements and their second coordinate set.

[0073] In the embodiments of this application, the edge density distribution heatmap can identify densely populated (e.g., table lines) and sparsely populated (e.g., blank fields) areas in the layout. For example, areas with a density ≤ a preset value in the heatmap (e.g., windows with a density < 5%) are located, and a dynamic programming algorithm is used to select the optimal cutting path to minimize layout fragmentation (e.g., avoiding splitting a table into multiple discontinuous areas). The report image is divided into M local areas (e.g., tables, signature bars) according to the cutting path, and the second coordinate set (x, y, w, h) of each area is recorded to obtain M layout elements and their second coordinate sets, which are used for subsequent DSL generation.

[0074] According to the embodiments of this application, layout elements and their coordinates are extracted from the report image through edge density analysis, which improves the layout cutting accuracy and avoids the problem of manually re-measuring cutting parameters across templates (such as Shenzhen branch and national template), which is time-consuming and error-prone, and has high efficiency adaptability.

[0075] Figure 5 The flowchart illustrates a process for obtaining M layout elements and their second coordinate set based on the density distribution in an edge density distribution heatmap according to an embodiment of this application.

[0076] like Figure 5 As shown, this embodiment obtains M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap, including operations S510 to S530.

[0077] In operation S510, local areas with edge densities less than or equal to preset values ​​are determined based on density distribution.

[0078] In the embodiments of this application, a density threshold is set according to business requirements (e.g., areas with a density ≤ 5% are candidate cutting points), the heat map is scanned, and all local areas with a density less than the threshold are marked (e.g., blank spaces between fields, table separators). The output set of candidate cutting points (coordinate set) represents possible layout separation positions.

[0079] When operating S520, the local area is used as the cutting point, and the target cutting path is determined based on the target planning algorithm.

[0080] In the embodiments of this application, local regions are used as cutting points, and the target cutting path is determined based on a goal programming algorithm. For example, in a dynamic programming algorithm, the heatmap is considered as a grid, and the state of each grid cell (such as a 16×16 pixel window) is (i, j, d), representing the cutting cost of the optimal path from the starting point to coordinate (i, j) in the direction d (horizontal / vertical). The cost of the cutting path passing through low-density regions is 0 (ideal cutting point). The cost of passing through high-density regions is +1 (avoiding cutting dense edges such as table lines). Horizontal or vertical cutting paths are used to find a path from the upper left corner to the lower right corner of the image that minimizes the total cost (i.e., passes through low-density regions as much as possible). Initially, the horizontal and vertical path costs of the starting point (0, 0) are both 0. The dp value of each grid is calculated in row or column order, and the optimal path direction is saved. Tracing backward from the endpoint, the path with the minimum cost is selected as the final cutting line.

[0081] In operation S530, the report image is cut based on the target cutting path to obtain M layout elements and their second coordinate set.

[0082] In the embodiments of this application, the report image is divided into multiple sub-regions (such as tables and signature bars) along the cutting path, and the bounding box coordinates (x, y, w, h) of each sub-region are calculated to form a second set of coordinates.

[0083] According to embodiments of this application, determining the optimal cutting path through a target planning algorithm can avoid layout fragmentation (such as tables being incorrectly segmented) that is easily caused by manual measurement or uniform grid cutting, thereby improving cutting efficiency and accuracy.

[0084] Figure 6 The diagram illustrates another flowchart of obtaining M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap according to an embodiment of the present application.

[0085] like Figure 6 As shown, this embodiment obtains M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap, including operations S610 to S620.

[0086] In operation S610, the layout conflict features of the target page are retrieved based on the target page code. The layout conflict features characterize the interference between page elements in the target page.

[0087] In embodiments of this application, layout conflict features may include overlapping floating elements (such as the intersection of a table and a signature bar area), misaligned elements (such as field offsets exceeding the tolerance range), and so on. For example, HTML code can be converted into an AST (Abstract Syntax Tree) to identify all page elements and their style attributes (such as position and margin), and conflict features can be extracted. For example, the bounding boxes (x, y, w, h) of element A and element B may intersect, or the actual position of an element may deviate from its expected coordinates (the second set of coordinates with weight 4) by a preset value (such as 5 pixels).

[0088] In operation S620, in response to the layout conflict feature meeting the preset conditions, the target cutting path is updated based on the layout conflict feature using the target planning algorithm. The updated target cutting path indicates the cutting path to obtain M layout elements and their second coordinate set again.

[0089] In the embodiments of this application, after detecting layout conflicts (such as element overlap or coordinate offset), the cutting path is dynamically adjusted through a target planning algorithm to ensure that the regenerated layout elements are non-overlapping or misaligned and maintain integrity (e.g., tables are not split). Based on the retrieved layout conflict features (such as conflict element coordinates and error types), the cutting path planning algorithm is rerun according to the new parameters output by the model-free reinforcement learning algorithm. For example, the cutting parameters are adjusted (e.g., density threshold from 5% to 3%, increasing path smoothing weight) so that the new cutting path avoids the conflict area, and the resulting M layout elements and their coordinate set are recut.

[0090] According to embodiments of this application, by detecting layout conflicts and optimizing cutting paths, the regenerated layout elements are made to be non-overlapping or misaligned and maintain integrity, which can reduce manual intervention and improve accuracy.

[0091] In the embodiments of this application, generating multiple candidate page codes based on the image features of the report image, M layout elements, and a second coordinate set using a third model includes: using the third model based on the syntax rules of the first programming language to process the image features and the text features of the M layout elements and the second coordinate set to generate multiple candidate page codes.

[0092] The third model can be a cross-modal transformer architecture, including an image encoder that converts image features into vector sequences; a text encoder that converts DSL statements into vector sequences; a cross-modal fusion layer that jointly learns the correlation between image and text features; and a grammatically constrained decoder that generates grammatically correct code based on the fused features. The third model can fuse image and text features through the transformer's multi-head attention mechanism to generate candidate page code that conforms to the syntax of the first programming language (such as DSL), i.e., generate sub-region DSLs. Image features can be high / low density regions in an image identified by an edge density heatmap, or cut layout elements and their coordinates, etc. Text features can be DSL statements, etc. Furthermore, multiple logically related sub-elements (such as fields or headers in a table) can be bundled into a single unit, maintaining their relative positions and hierarchical relationships to form grouped layout sub-region DSLs, or elements can be removed from the regular document flow and suspended at a specified position on the page (such as stamps, signature bars, or tooltips) to form floating layout sub-region DSLs.

[0093] According to embodiments of this application, by jointly training image features and code syntax through a cross-modal Transformer architecture, candidate page code conforming to the syntax of a first programming language (such as DSL) can be generated, which can significantly improve the accuracy of code generation, support code generation for complex layouts, and reduce manual intervention.

[0094] In embodiments of this application, before obtaining target page code based on a second programming language based on multiple candidate page codes, the method further includes: matching multiple layout information obtained based on multiple candidate page codes with a set of candidate templates in a preset template library, wherein the candidate templates are obtained based on local regions in candidate report images applied for overlay printing; and adjusting the corresponding candidate page code based on the target template in response to at least one of the multiple layout information matching the corresponding target template.

[0095] Multiple candidate page codes can be a collection of DSL code snippets. The preset template library refers to the collection of candidate templates, each template corresponding to a layout rule for a local area. Candidate templates refer to local area layout rules extracted from report images of historical successful cases, such as the "signature bar" of a bank account opening form always being located in the lower right corner of the table and offset 20px to the right.

[0096] The system automatically identifies the template type of a region by matching local features using keypoints, and dynamically adjusts the parameters of the DSL code snippet by applying preset rules such as margin offset compensation formulas. For example, key features such as element type, relative position, and size ratio are extracted from the layout information. The current layout is compared with candidate templates in the template library using keypoint matching local features. If the similarity exceeds a threshold (e.g., >90%), it is determined that a target template has been matched. Then, preset rules such as margin offset compensation formulas are applied to dynamically adjust the parameters of the DSL code snippet.

[0097] According to the embodiments of this application, candidate page code is intelligently matched and optimized through a preset template library, so that the generated final code not only conforms to the syntax rules, but also accurately adapts to the actual business needs, thereby improving accuracy and adaptability.

[0098] In embodiments of this application, obtaining target page code based on a second programming language from multiple candidate page codes includes: using a fourth model based on the syntax rules of a first programming language to process image features and multiple candidate page codes, generating intermediate page code based on the first programming language, wherein the intermediate page code indicates a candidate page that has a positional mapping relationship with the report image; and converting the intermediate page code into target page code based on the second programming language.

[0099] The fourth model can be a transformer-based encoder-decoder structure. The encoder fuses DSL code and image features into a multimodal vector. The decoder generates intermediate code that conforms to the DSL syntax and preserves positional mappings. An attention mechanism associates image coordinates with layout attributes in the code, generating optimized and valid DSL code during the decoding phase. This optimized DSL code, i.e., the intermediate page code, perfectly matches the layout of the report image. Based on predefined mapping rules, DSL tags are converted to target language tags, preserving coordinates and styles, generating directly renderable code. The target page code can be directly executable standard code (such as HTML) that can be directly used for browser rendering.

[0100] According to embodiments of this application, converting page code into target page code based on a second programming language using a fourth model can improve code generation efficiency and accuracy, and eliminates the need for repeated manual debugging.

[0101] Based on the above page generation method, this application also provides a page generation apparatus. The following will be combined with... Figure 7 The device is described in detail.

[0102] Figure 7 A schematic block diagram of a page generation apparatus according to an embodiment of this application is shown.

[0103] like Figure 7As shown, the page generation apparatus 700 of this embodiment includes a first acquisition module 710, a second acquisition module 720, and a generation module 730.

[0104] The first acquisition module 710 is used to process the report image applied to the overlay printing using the first model to obtain a first set of coordinates for N report elements in the report image. In one embodiment, the first acquisition module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0105] The second acquisition module 720 is used to parse the report image based on the first coordinate set using the second model to obtain M layout elements and their second coordinate set. The second coordinate set is determined within at least one region defined based on the first coordinate set, where N and M are integers greater than or equal to 1. In one embodiment, the second acquisition module can be used to perform the operation S220 described above, which will not be repeated here.

[0106] The generation module 730 is used to generate a target page that has a positional mapping relationship with the report image based on the image features of the report image, M layout elements, and a second coordinate set. The positional mapping relationship indicates the mapping relationship between at least one overlay position. In one embodiment, the generation module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0107] In an embodiment of this application, the second acquisition module 720 is further configured to use a second model to parse the report image based on the first coordinate set to obtain M layout elements and their second coordinate set. Specifically, this includes: using the second model to parse the report image based on the first coordinate set to obtain layer segmentation results; performing edge detection on the target layer based on the layer segmentation results to obtain an edge density distribution heatmap; and obtaining M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap.

[0108] In the embodiments of this application, the second acquisition module 720 is further configured to acquire M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap, specifically including: determining local areas with edge density less than or equal to a preset value based on the density distribution; using the local areas as cutting points and determining a target cutting path based on a target planning algorithm; and cutting the report image based on the target cutting path to obtain M layout elements and their second coordinate set.

[0109] In embodiments of this application, the generation module 730 is further configured to generate a target page with a positional mapping relationship to the report image based on the image features of the report image, M layout elements, and a second coordinate set. Specifically, this includes: generating multiple candidate page codes based on a first programming language using a third model based on the image features of the report image, M layout elements, and the second coordinate set, wherein the multiple candidate page codes respectively indicate different local areas of the target page; obtaining target page code in a second programming language based on the multiple candidate page codes, the target page code being used to form the target page, wherein the complexity of the first programming language is less than the complexity of the second programming language.

[0110] In embodiments of this application, the generation module 730 is further configured to obtain target page code based on a second programming language based on multiple candidate page codes, specifically including: using a fourth model based on the syntax rules of a first programming language to process image features and multiple candidate page codes, generating intermediate page code based on the first programming language, wherein the intermediate page code indicates a candidate page that has a positional mapping relationship with the report image; and converting the intermediate page code into target page code based on the second programming language.

[0111] In the embodiments of this application, the page generation device 700 further includes an update module. The update module is used to retrieve the layout conflict features of the target page based on the target page code after generating the target page code of the target page. The layout conflict features characterize the interference of page elements in the target page. In response to the layout conflict features meeting preset conditions, the target cutting path is updated based on the layout conflict features using a target planning algorithm. The updated target cutting path indicates the cutting path for obtaining M layout elements and their second coordinate set again.

[0112] In the embodiments of this application, the page generation device 700 further includes a candidate page code generation module. The candidate page code module is used to generate multiple candidate page codes based on the image features of the report image, M layout elements and a second coordinate set using a third model. Specifically, it includes: using the third model based on the syntax rules of the first programming language to process the image features and the text features of the M layout elements and the second coordinate set to generate multiple candidate page codes.

[0113] In embodiments of this application, the page generation apparatus 700 further includes a matching module, which is used to match multiple layout information obtained from multiple candidate page codes with a set of candidate templates in a preset template library before obtaining target page code based on multiple candidate page codes. The candidate templates are obtained based on local areas in candidate report images applied for overlay printing. In response to at least one of the multiple layout information matching the corresponding target template, the corresponding candidate page code is adjusted based on the target template.

[0114] According to embodiments of this application, any plurality of modules among the first acquisition module 710, the second acquisition module 720, and the generation module 730 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the first acquisition module 710, the second acquisition module 720, and the generation module 730 can be at least partially implemented as a hardware circuit, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented by any other reasonable means of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the first acquisition module 710, the second acquisition module 720, and the generation module 730 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0115] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a page generation method according to an embodiment of this application.

[0116] like Figure 8 As shown, an electronic device 800 according to an embodiment of this application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0117] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0118] According to embodiments of this application, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0119] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0120] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0121] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the page generation method provided in the embodiments of this application.

[0122] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0123] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0124] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0125] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0127] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A page generation method, comprising: The first model is used to process the report image applied to the overprinting process to obtain the first set of coordinates of N report elements in the report image; The report image is parsed using a second model based on the first coordinate set to obtain M layout elements and their second coordinate set. The second coordinate set is determined within at least one region defined based on the first coordinate set, where N and M are integers greater than or equal to 1. Based on the image features of the report image, the M layout elements, and the second coordinate set, a target page with a positional mapping relationship to the report image is generated, wherein the positional mapping relationship indicates the mapping relationship between at least one overprint position.

2. The method according to claim 1, characterized in that, The step of generating a target page with a positional mapping relationship to the report image based on the image features of the report image, the M layout elements, and the second coordinate set includes: Based on the image features of the report image, the M layout elements, and the second coordinate set, a third model is used to generate multiple candidate page codes based on the first programming language, wherein the multiple candidate page codes respectively indicate different local areas of the target page; Based on the multiple candidate page codes, a target page code in a second programming language is obtained. The target page code is used to form the target page, wherein the complexity of the first programming language is less than that of the second programming language.

3. The method according to claim 1 or 2, characterized in that, The step of using a second model to parse the report image based on the first coordinate set to obtain M layout elements and their second coordinate set includes: The report image is analyzed using the second model based on the first coordinate set to obtain the layer segmentation result; Based on the layer segmentation results, edge detection is performed on the target layer to obtain an edge density distribution heatmap. Based on the density distribution in the edge density distribution heatmap, obtain the M layout elements and their second coordinate set.

4. The method according to claim 3, characterized in that, The process of obtaining the M layout elements and their second coordinate set based on the density distribution in the edge density distribution heatmap includes: Based on the density distribution, determine local regions where the edge density is less than or equal to a preset value; Using the local region as the cutting point, the target cutting path is determined based on the target planning algorithm; The report image is cut based on the target cutting path to obtain the M layout elements and their second coordinate set.

5. The method according to claim 4, characterized in that, After generating the target page code for the target page, the method further includes: Based on the target page code, the layout conflict features of the target page are retrieved, and the layout conflict features characterize the interference of page elements in the target page; In response to the layout conflict feature meeting the preset conditions, the target cutting path is updated based on the layout conflict feature using the target planning algorithm, wherein the updated target cutting path indicates the cutting path for obtaining the M layout elements and their second coordinate set again.

6. The method according to claim 2, characterized in that, The step of generating multiple candidate page codes based on the image features of the report image, the M layout elements, and the second coordinate set using a third model and a first programming language includes: The third model, based on the syntax rules of the first programming language, processes the image features and the text features of the M layout elements and the second coordinate set to generate the multiple candidate page codes.

7. The method according to claim 2 or 6, characterized in that, Before obtaining the target page code based on the multiple candidate page codes, the method further includes: Multiple layout information obtained based on the multiple candidate page codes are matched with a set of candidate templates in a preset template library, wherein the candidate templates are obtained based on local areas in the candidate report images applied to overlay printing; In response to at least one of the plurality of layout information matching a corresponding target template, the corresponding candidate page code is adjusted based on the target template.

8. The method according to claim 2, characterized in that, The process of obtaining the target page code based on the multiple candidate page codes and the second programming language includes: Using the fourth model based on the syntax rules of the first programming language, the image features and the multiple candidate page codes are processed to generate intermediate page code based on the first programming language. The intermediate page code indicates the candidate page that has the positional mapping relationship with the report image. The intermediate page code is converted into target page code based on a second programming language.

9. A page generation device, characterized in that, The device includes: The first acquisition module is used to process the report image applied to the form printing using the first model, so as to obtain the first coordinate set of N report elements in the report image; The second acquisition module is used to parse the report image based on the first coordinate set using a second model to obtain M layout elements and their second coordinate set, wherein the second coordinate set is determined within at least one region defined based on the first coordinate set, and N and M are integers greater than or equal to 1; and The generation module is used to generate a target page with a positional mapping relationship to the report image based on the image features of the report image, the M layout elements and the second coordinate set, wherein the positional mapping relationship indicates the mapping relationship between at least one overprint position.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images