Form item coordinating method, device and equipment based on position clustering

Abstract

The invention discloses a form item coordinating method, device and equipment based on position clustering, and belongs to the field of data processing. Comprising the steps that all nodes in a target form are obtained, a node list is obtained, the maximum independent structure container node of each node is determined according to a father node of the node, and for each node, space coordinates of a target area where the maximum independent structure container node of the node is located serve as absolute space information of the node; determining an effective area of the target form, and performing area division on the effective area of the target form in the vertical direction to obtain a plurality of longitudinal logic blocks; matching the plurality of longitudinal logic blocks with a plurality of nodes of the node list, and determining a longitudinal structure label; for each longitudinal logic block, carrying out clustering based on left and right boundary coordinates in absolute space information, and generating a transverse structure label of the node; and according to the transverse structure label and the longitudinal structure label of each node, form item coordinate information of the target form is determined.

Description

Technical Field

[0001] This application belongs to the field of data processing, and specifically relates to a method, apparatus and device for coordinate conversion of form items based on location clustering. Background Technology

[0002] While the spatial location information of HTML elements can be extracted relatively easily, in practice, form tags typically correspond to only a small rendering area. Their location information often only covers the plain text itself, making it difficult to reflect the complete form item area actually perceived by the user, consisting of "element + tag + decorative structure (such as borders, icons)". Traditional methods of understanding forms usually rely on the structural binding relationship in the DOM tree, directly associating tags with input elements. However, in modern web design, the spatial and structural relationship between tag text and elements is highly decoupled, often visually combined through CSS style layers. As a result, tags and elements are structurally independent, making it impossible to intuitively establish a connection through the DOM hierarchy.

[0003] Further observation of the form element structure reveals a clear "blocking" characteristic in its visual presentation: a single element often carries two types of information simultaneously: textual information (small space occupation, strong independence) and block information (large space occupation, strong structure). Therefore, relying solely on text nodes to obtain coordinates typically only captures local character regions, ignoring the overall visual semantic context. This bias affects the quality of coordinate-based cue word construction, thereby reducing the effectiveness of large language models in understanding form intent and the accuracy of field filling. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, and device for coordinate representation of form items based on location clustering, which can transform the original HTML form into a structured text representation with clear location semantics.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a method for coordinate representation of form items based on location clustering, the method comprising: Retrieve all nodes in the target form to obtain a node list, where each node in the node list represents a form item in the target form; Based on the parent node of each node in the target form, determine the largest independent structural container node of that node. The largest independent structural container node is a node that contains only the form item corresponding to that node and the label of that form item. For each node, the spatial coordinates of the target region where the node's largest independent structural container node is located are used as the node's absolute spatial information. The absolute spatial information includes the node's upper and lower boundary coordinates and left and right boundary coordinates. Determine the valid area of ​​the target form, and divide the valid area of ​​the target form vertically to obtain multiple vertical logical blocks; The multiple vertical logical blocks are matched with multiple nodes in the node list to determine the vertical structure label of each node; For each vertical logical block, clustering is performed based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structure label of the node; Based on the horizontal and vertical structure labels of each node, the coordinate information of the form items in the target form is determined.

[0006] Optionally, determining the largest independent structural container node for each node based on its parent node in the target form includes: In each parent node, nodes with visible areas are retained, while blank nodes without dimensions and auxiliary nodes for debugging are filtered out. If the current parent node contains only two nodes with visible areas, and the size of the visible area of ​​one of these nodes is smaller than a preset area threshold, continue searching upwards; If the current parent node has multiple sibling nodes with visible areas or presents a complex structural layout, determine the upper boundary of the form item corresponding to that node, stop traversing upwards, and select the current parent node as the largest independent structural container node.

[0007] Optionally, before using the spatial coordinates of the target region where the node's largest independent structural container node is located as the node's absolute spatial information, the following is also included: Obtain the spatial coordinates of the target area where the largest independent structural container node is located, and calculate the absolute spatial information of the spatial coordinates in the target form in combination with the scroll offset; For the spatial coordinates of the target region where each largest independent structural container node is located, determine whether the spatial difference between the target region where the largest independent structural container node is located and the spatial coordinates of the target region where the existing records are located is less than the tolerance threshold. If the spatial coordinates of the target area where the largest independent structural container node is located are less than the tolerance threshold compared with the spatial coordinates of the target area where the largest independent structural container node is located in the existing records, it is determined to be a duplicate area, and the absolute spatial information of the spatial coordinates of the target area where the largest independent structural container node is located in the target form is not recorded.

[0008] Optionally, determining the valid area of ​​the target form includes: For each node in the node list, extract the upper and lower bound coordinates of the input element node in the vertical direction, and determine the minimum upper bound coordinate and maximum lower bound coordinate of the multiple input element nodes contained in the target form. Based on the minimum upper bound coordinates and the maximum lower bound coordinates, the global vertical range of the target form is expanded to construct the effective area of ​​the form region of the target form.

[0009] Optionally, the effective area of ​​the target form is divided vertically to obtain multiple vertical logical blocks, including: Within the vertical direction of the effective area, extract the upper and lower boundary coordinates of all nodes in the node list to construct a set of ordinate points; The upper and lower boundary coordinates in the set of vertical coordinate points are arranged in ascending order to obtain the set of Y-axis coordinate points. The set of Y-axis coordinate points includes the coordinates of multiple center points. The adjacent coordinates of nodes whose continuous difference between the upper and lower boundary coordinates does not exceed a preset number of pixels are merged into one center point. Construct a list of vertical logical blocks, where each vertical logical block represents a vertical spatial region between two adjacent center points.

[0010] Optionally, the step of matching the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node includes: Determine the center point that is closest to the upper and lower boundary coordinates of each node; Based on the nearest center point, determine the vertical logic blocks that overlap with the nearest center point; The vertical structure label of the node is determined based on the vertical logical blocks that overlap with the nearest center point.

[0011] Optionally, the step of clustering each vertical logical block based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structural label of the node includes: Within the horizontal direction of the effective area, extract the left and right boundary coordinates of all nodes in the node list to construct a set of horizontal coordinate points; By using K-means clustering, the set of horizontal coordinate points is structurally summarized to obtain multiple cluster centers; Arrange multiple cluster centers horizontally to construct intervals between multiple adjacent cluster centers, and define each interval as a horizontal logical block; Based on the horizontal logic block, the horizontal structure label of each node is determined.

[0012] Optionally, after determining the coordinates of the form items in the target form, the method further includes: The coordinates of multiple form items contained in the form item coordinate information are arranged in ascending order according to the vertical logical block number to obtain the first sequence; If multiple form items fall within the same vertical logical block in the first sequence, the form items within the same logical block are arranged according to the order of the horizontal structure labels to obtain the second sequence. For the second sequence, the coordinates of all form items belonging to the same vertical logic block are merged and used as input information for the large language model to perform the form filling task.

[0013] Secondly, embodiments of this application provide a form item coordinate system based on location clustering, the system comprising: The node acquisition module is used to acquire all nodes in the target form and obtain a node list, wherein each node in the node list represents a form item in the target form. The node determination module is used to determine the largest independent structural container node of each node based on its parent node in the target form. The largest independent structural container node is a node that contains only the form item corresponding to the node and the label of the form item. The node location information determination module is used to determine the absolute spatial information of each node by taking the spatial coordinates of the target area where the largest independent structural container node of the node is located. The absolute spatial information includes the upper and lower boundary coordinates and the left and right boundary coordinates of the node. The vertical logic block division module is used to determine the valid area of ​​the target form and divide the valid area of ​​the target form into multiple vertical logic blocks in the vertical direction. The vertical structure label determination module is used to match the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node. The horizontal structure label determination module is used to cluster each vertical logical block based on the left and right boundary coordinates in the absolute spatial information of each node, and generate the horizontal structure label of the node. The form item coordinate information determination module is used to determine the form item coordinate information of the target form based on the horizontal and vertical structure labels of each node.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] In this application embodiment, a method, apparatus, and device for coordinate representation of form items based on location clustering are provided. The method includes: obtaining all nodes in a target form to obtain a node list, wherein each node in the node list represents a form item in the target form; determining the maximum independent structural container node of each node based on its parent node in the target form, wherein the maximum independent structural container node is a node that contains only the form item corresponding to the node and the label of the form item; for each node, using the spatial coordinates of the target area where the maximum independent structural container node of the node is located as the absolute spatial information of the node, wherein the absolute spatial information includes the upper and lower boundary coordinates and the left and right boundary coordinates of the node; determining the effective area of ​​the target form and dividing the effective area of ​​the target form vertically to obtain multiple vertical logical blocks; matching the multiple vertical logical blocks with multiple nodes in the node list to determine the vertical structural label of each node; for each vertical logical block, clustering based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structural label of the node; and determining the form item coordinate information of the target form based on the horizontal and vertical structural labels of each node.

[0016] The technical solution of this application enables more accurate extraction and structuring of form item location information. By first extracting the accurate location information of each form item and then structuring this information, a format that is easier for large language models to understand is generated. This allows the large language model to more accurately understand the intent and structure of the target form, thereby improving the accuracy and efficiency of field filling. Finally, the generated structured form item coordinate information can be used for subsequent automated processing tasks such as form filling. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for coordinate representation of form items based on location clustering, provided in an embodiment of this application. Figure 2 This is an example diagram of a target form provided in an embodiment of this application; Figure 3 This is a schematic diagram of the framework of a form item coordinate device based on location clustering provided in an embodiment of this application; Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] The core idea of ​​this application is to start from the element node, search from bottom to top for the largest independent structural container node that contains only the element and its tag, and use the spatial boundary of the largest independent structural container node as the overall representation of the form item in the form, which can then be used for subsequent prompt generation and element rearrangement strategy design.

[0021] Specifically, it includes two main stages: The first stage involves the acquisition and preprocessing of HTML text from online forms. Taking online forms from real web pages as the research object, the target form areas are directly extracted from the HTML source code, focusing on key HTML elements with input functionality (such as text boxes, dropdown lists, and tags). A Chrome browser plugin is developed to automatically extract the text content of the fields and their spatial information on the page (such as x and y coordinates and width and height dimensions), thereby achieving the joint acquisition of semantic information and spatial location of form elements.

[0022] The second stage involves constructing the coordinate and structured representation of form items. Based on the extracted spatial information, this stage maps each form item to a rectangular region on a two-dimensional plane and uses a location clustering algorithm to divide logical blocks. Building upon this, a concise and efficient textual representation method is constructed to preserve the positional information, size ratio, and relative arrangement of form items, thereby generating a coordinate-based form representation capable of expressing both relative and absolute positional information. Through this process, the original HTML form can be transformed into a structured textual representation with explicit positional semantics.

[0023] The following description, in conjunction with the accompanying drawings, details a method for coordinate representation of form items based on location clustering provided in this application, through specific embodiments and application scenarios.

[0024] Figure 1 This is a flowchart illustrating a method for coordinate representation of form items based on location clustering, according to an embodiment of this application. Figure 2 This is an example diagram of a target form provided in an embodiment of this application, see reference. Figure 1 One embodiment of this application provides a method for coordinate mapping of form items based on location clustering, the method including steps S11 to S17: Step S11: Obtain all nodes in the target form to obtain a node list, wherein each node in the node list represents a form item in the target form.

[0025] In this embodiment, the target form refers to an online HTML form in a real webpage. To obtain the target form, it's first necessary to acquire all nodes within it, including text nodes and input element nodes. Each node represents a form item in the target form, i.e., a form element, such as a "username" tag and the corresponding input box. By traversing the entire form's DOM tree, all nodes contained in the target form are collected and stored in a node list for subsequent processing.

[0026] Step S12: Based on the parent node of each node in the target form, determine the largest independent structural container node of that node. The largest independent structural container node is a node that contains only the form item corresponding to that node and the label of that form item.

[0027] In this embodiment, for each node obtained through traversal, its parent node is searched upwards from that node to find the parent node containing the form item corresponding to that node and the label corresponding to that form item. This parent node is then used as the maximum independent structural container node for that node. The maximum independent structural container node is the target container for that node, including not only the node itself but also non-primary elements such as borders, icons, and buttons related to the form item corresponding to the node. By determining the maximum independent structural container node for each node, the spatial boundary of the maximum independent structural container node can be used as the overall representation of the form item corresponding to the node in the target form.

[0028] Step S13: For each node, the spatial coordinates of the target area where the largest independent structural container node of the node is located are used as the absolute spatial information of the node. The absolute spatial information includes the upper and lower boundary coordinates and the left and right boundary coordinates of the node.

[0029] In this embodiment, the spatial coordinates of the largest independent structural container node of each node are extracted, including the coordinates of the top and bottom boundaries and the coordinates of the left and right boundaries. This coordinate information will be used as the absolute spatial information of the node. By combining the scroll offset, we calculate the absolute position of the spatial coordinates within the page where the target form is located, which is used as the absolute spatial information of the node.

[0030] During the above processing steps, the absolute spatial information of each form item in the target form has been obtained. However, this absolute spatial information has two obvious problems: First, it is represented by pixel coordinates in floating-point form, resulting in a large numerical range and highly discrete distribution, with low overlap between the upper and lower boundaries of the form items; second, this low-level, fine-grained absolute spatial information is not suitable for direct input into a large language model, because large language models have weak processing capabilities for numerical relationships, addition and subtraction logic, etc., which can easily lead to misunderstandings or calculation errors.

[0031] Therefore, in order to further achieve high-level modeling and structural coding of the target form area, it is necessary to combine clustering strategies and coordinate interval normalization of form item structured positioning to realize the clear row and column belonging of form items in the target form space.

[0032] The core idea of ​​this process is as follows: First, determine the effective form area of ​​the target form on the webpage by the vertical position range of the node. Then, cluster the nodes based on their horizontal coordinates to automatically divide the column structure of the form. Finally, generate structural tags with both horizontal and vertical axes to realize the abstract encoding of each form item in the spatial structure. For details, please refer to the following steps.

[0033] Step S14: Determine the valid area of ​​the target form and divide the valid area of ​​the target form vertically to obtain multiple vertical logical blocks.

[0034] Since the foundation of target form modeling lies in input element nodes, i.e., input controls, it is necessary to use input element nodes as the core positioning basis. Because the text nodes extracted in the above steps may contain some suggestive or decorative information that does not belong to the form, their information density is low, semantic ambiguity is high, and they may interfere with structural modeling and prompt generation, therefore, at this stage, we choose to discard plain text nodes and only retain input element nodes.

[0035] In this embodiment, for all traversed nodes, their upper and lower boundary coordinates in the vertical direction are extracted, and the effective area of ​​the target form in the webpage is determined based on these node upper and lower boundary coordinates. Then, the area is divided vertically, the upper and lower boundary coordinates of all nodes are extracted, a set of vertical coordinate points is constructed, and adjacent upper and lower boundary coordinates with a continuous difference not exceeding a preset pixel threshold (e.g., 10 pixels) are merged into a center point. Based on these normalized center points, a list of vertical logical blocks is constructed. The list of vertical logical blocks contains multiple vertical logical blocks, and a vertical logical block refers to a vertical interval where a container can be placed.

[0036] Step S15: Match the plurality of vertical logic blocks with the plurality of nodes in the node list to determine the vertical structure label of each node.

[0037] In this embodiment, after dividing the data into multiple vertical logical blocks, the upper and lower boundary information of each node's absolute spatial information is matched one-to-one with the vertical logical blocks, and a corresponding vertical structure label is assigned to each node. Specifically, the upper and lower boundary coordinates of the node can be snapped to the nearest center point first, then all vertical logical blocks that overlap with the center point are found, and based on the vertical logical block, the vertical structure label of the node on the Y-axis is generated (for example, [3:4] indicates that the node falls between the 3rd and 4th vertical logical blocks). To simplify the representation, if the range of the vertical structure label is greater than two vertical logical blocks, only the values ​​of the first and last vertical logical blocks are retained as the vertical structure label to construct a range expression of the form [1:4], thereby structurally representing the position information of the form items in the vertical direction.

[0038] Step S16: For each vertical logical block, cluster the nodes based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structure label of the node.

[0039] In this embodiment, for each vertical logical block, the left and right boundary coordinates of each input element node are extracted to form a set of horizontal coordinate points. Then, the K-means clustering method is used to structurally summarize these horizontal coordinate information to generate cluster centers. The cluster centers are arranged in ascending order according to the size of their horizontal coordinates, and intervals between adjacent cluster centers are constructed. Each interval is regarded as a column and labeled with uppercase letters AZ. Next, each node is assigned its corresponding horizontal logical block, and its structural label on the X-axis is generated (for example, [A:E] indicates that the node covers the horizontal logical block from column A to column E). This structurally represents the positional information of the form items in the horizontal direction, thereby generating complete two-dimensional positional information.

[0040] Step S17: Determine the coordinate information of the form items in the target form based on the horizontal and vertical structure labels of each node.

[0041] In this embodiment, the vertical structure labels of the vertical logic blocks are arranged in ascending order. If multiple nodes are within the same vertical logic block, they are further arranged in ascending order according to the size of the horizontal column labels. Finally, the structure labels of all nodes within each vertical logic block are merged and output, constructing a result in the form of "Name:[A1:B1] Age:[B1:D4] Gender:[E1,E1]". This generates structured form item coordinate information, where each form item has its corresponding vertical and horizontal structure labels. These labels help the large language model better understand and process the data.

[0042] The technical solution described above enables more accurate extraction and structuring of form item location information. By first extracting the accurate location information of each form item and then structuring this information, a format that is easier for large language models to understand is generated. This allows the large language model to more accurately understand the intent and structure of the target form, thereby improving the accuracy and efficiency of field filling. Finally, the generated structured form item coordinate information can be used for subsequent automated processing tasks such as form filling.

[0043] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, step S12, "determining the largest independent structural container node of each node based on its parent node in the target form," specifically includes steps S12-1 to S12-3: Step S12-1: In each parent node, retain nodes with visible areas, and filter out blank nodes without dimensions and auxiliary nodes for debugging.

[0044] In this embodiment, for each node, its parent node is searched upwards layer by layer. Within each parent node, only those nodes with actual visible areas are retained—that is, those nodes that occupy space and are visible on the webpage. Simultaneously, blank nodes without dimensions and auxiliary nodes used for debugging are filtered out, as these nodes are not important in the actual form structure and do not affect the user's perception of the form item area, ensuring that subsequent processed nodes all have actual content and visible areas.

[0045] Step S12-2: If the current parent node contains only two nodes with visible areas and the size of the visible area of ​​one of the nodes is less than the preset area threshold, continue searching upwards.

[0046] In this embodiment, the search continues upwards until a largest independent structural container node containing the current node and its label is found. This is especially true when the current parent node contains only two nodes with visible areas, and the size of the visible area of ​​one of these nodes is smaller than a preset area threshold (e.g., ...). Figure 2 (e.g., a small icon or button, or other non-primary element), continue searching upwards. Since these nodes, smaller than the preset area threshold, are likely decorative elements rather than the main part of the form item, continuing to search upwards ensures that a more complete container node is found that more accurately represents the area of ​​the form item as perceived by the user.

[0047] Step S12-3: If the current parent node has multiple sibling nodes with visible areas or presents a complex structural layout, determine the upper boundary of the form item corresponding to the node, stop traversing upwards, and select the current parent node as the largest independent structural container node.

[0048] In this embodiment, it is checked whether the current parent node has multiple sibling nodes with visible areas or presents a complex structural layout. If it has multiple sibling nodes with visible areas or presents a complex structural layout, it means that the current parent node no longer belongs exclusively to the original node, that is, it no longer has independence.

[0049] If these conditions are met, it is considered that the upper boundary of the form item's structure corresponding to that node has been reached, and the current parent node is the largest independent structural container node containing the current node and its label. Therefore, the upward search stops at this point, and the current parent node is selected as the largest independent structural container node. This ensures that the search scope is not excessively expanded during the search process, thereby avoiding the inclusion of irrelevant elements.

[0050] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate conversion of form items based on location clustering. In this method, before executing step S13, "using the spatial coordinates of the target region where the largest independent structural container node of the node is located as the absolute spatial information of the node", steps S21 to S23 are also included: Step S21: Obtain the spatial coordinates of the target area where the largest independent structural container node is located, and calculate the absolute spatial information of the spatial coordinates in the target form in combination with the scroll offset.

[0051] In this embodiment, to more accurately extract and record the position information of form items, the spatial coordinates of the target area where each largest independent structural container node is located are first obtained. These coordinates include the upper and lower boundary coordinates and the left and right boundary coordinates of the target area. When the full size of the target form exceeds the page, since users may scroll the page while browsing, it is necessary to combine the scroll offset to calculate the absolute position of these spatial coordinates in the target form. It is ensured that the obtained coordinate information is absolute spatial information relative to the entire form, and not just relative to the current page viewpoint.

[0052] Step S22: For the spatial coordinates of the target area where each largest independent structural container node is located, determine whether the spatial difference between the target area where the largest independent structural container node is located and the spatial coordinates of the target area where the existing records are located is less than the tolerance threshold.

[0053] In this embodiment, the spatial coordinates of the target region where each largest independent structural container node is located are checked to determine whether the spatial difference between it and the spatial coordinates of the target regions where other recorded largest independent structural container nodes are located is less than a preset tolerance threshold. The tolerance threshold is a small pixel value, such as 5 pixels. If the spatial coordinate difference between two target regions is less than this tolerance value, the two target regions are considered to overlap spatially.

[0054] Step S23: If the spatial difference between the target area where the largest independent structural container node is located and the spatial coordinates of the target area where the already recorded largest independent structural container node is located is less than the tolerance threshold, it is determined to be a duplicate area, and the absolute spatial information of the spatial coordinates of the target area where the largest independent structural container node is located in the target form is not recorded.

[0055] In this embodiment, when the spatial coordinates of a target region containing a maximum independent structural container node are found to differ from the spatial coordinates of other recorded target regions by less than a tolerance threshold, these two target regions are identified as duplicate regions. For such duplicate regions, their absolute spatial coordinates in the target form will not be recorded again, thereby avoiding the generation of duplicate data, improving data accuracy and processing efficiency, and preventing redundant data from interfering with subsequent processing.

[0056] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, the "determining the effective area of ​​the target form" in step S14 specifically includes steps S14-1-1 and S14-1-2: Step S14-1-1: For each node in the node list, extract the upper and lower bound coordinates of the input element node in the vertical direction, and determine the minimum upper bound coordinate and maximum lower bound coordinate of the multiple input element nodes contained in the target form.

[0057] In this embodiment, for each input element node in the node list, its upper and lower bound coordinates in the vertical direction are first extracted to determine the vertical position range of the form item corresponding to each node on the target form. By traversing all nodes, the minimum upper bound coordinate and maximum lower bound coordinate of these nodes are found. The minimum upper bound coordinate represents the highest upper boundary among all nodes, while the maximum lower bound coordinate represents the lowest lower boundary among all nodes.

[0058] Step S14-1-2: Based on the minimum upper bound coordinates and the maximum lower bound coordinates, the global vertical range of the target form is expanded to construct the effective area of ​​the form region of the target form.

[0059] In this embodiment, the coordinates are expanded upwards by a certain number of pixels (e.g., 200 pixels) based on the minimum upper bound coordinates, and downwards by the same number of pixels based on the maximum lower bound coordinates. This boundary expansion accommodates labels and other possible adjacent structures, ensuring that the effective area of ​​the target form can completely contain all related form items and their decorative structures. In this way, the effective area of ​​the target form is constructed, providing an accurate range for subsequent logical block division and structured processing.

[0060] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, step S14, "dividing the effective area of ​​the target form vertically to obtain multiple vertical logical blocks," specifically includes steps S14-2-1 to S14-2-3: Step S14-2-1: Within the vertical direction of the effective area, extract the upper and lower boundary coordinates of all nodes in the node list to construct a set of ordinate points.

[0061] In this embodiment, the vertical range of the valid area of ​​the target form is first determined. The vertical range is defined by the minimum upper bound coordinates and the maximum lower bound coordinates determined in step S14-1-2. Within this vertical range, the upper and lower bound coordinates of all nodes in the node list are extracted. Using these coordinates, a set of ordinate points is constructed, which contains the upper and lower bound coordinates of all nodes.

[0062] Step S14-2-2: Arrange the upper and lower boundary coordinates in the set of vertical coordinate points in ascending order to obtain the set of Y-axis coordinate points. The set of Y-axis coordinate points includes the coordinates of multiple center points. The adjacent coordinates of nodes whose continuous difference between the upper and lower boundary coordinates does not exceed a preset number of pixels are merged into one center point.

[0063] In this embodiment, the upper and lower boundary coordinates of the set of vertical coordinate points constructed in step S14-2-1 are arranged in ascending order. Then, these coordinates are processed, merging adjacent coordinates with consecutive differences not exceeding a preset number of pixels (e.g., 10 pixels) into a single center point. The preset number of pixels is a tolerance threshold used to handle coordinate points that are visually almost overlapping or very close. In this way, a set of Y-axis coordinate points is obtained, containing the coordinates of multiple center points. This simplifies the coordinate point set, reduces redundant information, and provides clearer reference points for the division of vertical logical blocks.

[0064] Step S14-2-3: Based on the coordinates of the center point in the set of Y-axis coordinate points, construct a list of vertical logic blocks, where each vertical logic block represents the vertical spatial region between two adjacent center points.

[0065] In this embodiment, the area between adjacent center points is defined as a vertical logical block. Each vertical logical block represents a spatial region in the target form in the vertical direction. These spatial regions divide the entire effective area of ​​the form into multiple vertical logical blocks, thereby achieving a structured division of the effective area of ​​the form in the vertical direction.

[0066] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, step S15, "matching the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node," specifically includes steps S15-1 to S15-3: Step S15-1: Determine the center point that is closest to the upper and lower boundary coordinates of each node.

[0067] In this embodiment, it is first necessary to determine the center point closest to the set of Y-axis coordinates of the upper and lower boundary coordinates of each node. These center points are obtained by merging adjacent coordinates in step S14-2-2. The closest center point is found by calculating the distance between the upper and lower boundary coordinates of each node and these center points.

[0068] Step S15-2: Based on the nearest center point, determine the vertical logic blocks that overlap with the nearest center point.

[0069] In this embodiment, it is further determined whether the upper and lower boundary coordinates of each node overlap with the boundary coordinates of a certain vertical logic block. If there is an overlap, it is determined that the node belongs to this vertical logic block.

[0070] Step S15-3: Determine the vertical structure label of the node based on the vertical logical blocks that overlap with the nearest center point.

[0071] In this embodiment, a vertical structure label is assigned to each node based on the vertical logical blocks that overlap with the nearest center point. The vertical structure label indicates the node's position in the vertical direction of the form. For example, if a node overlaps with the 3rd and 4th vertical logical blocks, its vertical structure label can be represented as [3:4]. If the overlapping vertical logical block range is large, only the first and last labels are retained to simplify the representation.

[0072] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, step S16, "for each vertical logical block, clustering is performed based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structure label of the node," specifically includes steps S16-1 to S16-4: Step S16-1: Within the horizontal direction of the effective area, extract the left and right boundary coordinates of all nodes in the node list to construct a set of horizontal coordinate points.

[0073] In this embodiment, the horizontal range of the valid area of ​​the target form is first determined, which is based on the valid area of ​​the form determined in the previous steps. Within this range, the left and right boundary coordinates of all nodes in the node list are extracted; that is, the pixel coordinates of the left and right boundaries of each input element node are extracted. These coordinates include the horizontal position of the left and right boundary coordinates of each node. Using these coordinates, a set of horizontal coordinate points is constructed, which contains the left and right boundary coordinates of all nodes to collect the horizontal position information of all nodes.

[0074] Step S16-2: Using K-means clustering, the set of horizontal coordinate points is structurally summarized to obtain multiple cluster centers.

[0075] In this embodiment, cluster analysis is performed on the constructed set of horizontal coordinate points. Specifically, the K-means clustering algorithm is used to structurally summarize these horizontal coordinate points. K-means clustering is a commonly used clustering algorithm that can divide data points into multiple clusters and find the center point of each cluster. In this process, a suitable number of clusters is selected (e.g., 21). This parameter can be flexibly adjusted according to the complexity of the actual form to cover the multi-column structure requirements of common forms. The clustering algorithm divides the horizontal coordinate points into multiple clusters based on their distribution and calculates the center point of each cluster, i.e., the cluster center. The cluster center represents the key position points of the target form in the horizontal direction, thus enabling the structural summarization of the horizontal positions of form items through cluster analysis.

[0076] Step S16-3: Arrange multiple cluster centers horizontally to construct intervals between multiple adjacent cluster centers, and define each interval as a horizontal logical block.

[0077] In this embodiment, the obtained cluster centers are arranged in ascending order horizontally. Then, intervals between adjacent cluster centers are constructed, with each interval considered as a column and labeled with uppercase letters AZ for subsequent column membership determination in structural positions. Each interval represents a horizontal logical block, which divides the effective area of ​​the entire target form into multiple structured regions horizontally.

[0078] Step S16-4: Determine the horizontal structure label of each node based on the horizontal logic block.

[0079] In this embodiment, each node is assigned a corresponding horizontal logical block interval, all overlapping vertical logical blocks are found, the horizontal span of each node is determined (i.e., the row interval covered by [horizontal left boundary, horizontal right boundary]), and the corresponding horizontal label list is returned. For example, if the left and right boundary coordinates of a node cover the 1st and 5th horizontal logical blocks, its horizontal structure label can be represented as [A:E] (assuming each horizontal logical block is represented by a letter), thus generating a concise and accurate horizontal structure label for the node to more clearly represent the position and range of each form item in the horizontal direction.

[0080] In conjunction with the technical solutions of the above embodiments, an embodiment of this application also provides another method for coordinate representation of form items based on location clustering. In this method, after "determining the coordinate information of the form items of the target form" in step S17, steps S17-1 to S17-3 are further included: Step S17-1: The coordinates of multiple form items contained in the form item coordinate information are arranged in ascending order according to the vertical logical block number to obtain the first sequence.

[0081] In this embodiment, the form item coordinate information includes the vertical structure label and the horizontal structure label for each form item. All form items are sorted in ascending order according to their vertical logical block numbers to obtain the first sequence. The vertical logical block numbers are determined in the previous step S15 and represent the vertical position of the form items. This sorting ensures that the form items are arranged in a top-to-bottom order, conforming to the natural order in which users fill out the form.

[0082] Step S17-2: If multiple form items in the first sequence fall within the same vertical logic block, the form items within the same logic block are arranged according to the order of the horizontal structure labels to obtain the second sequence.

[0083] In this embodiment, if multiple form items fall within the same vertical logical block in the first sequence, these form items need to be further sorted. Specifically, the form items within the same vertical logical block can be arranged according to the order of the horizontal structure labels. The horizontal structure labels are determined in the previous step S16 and represent the horizontal position of the form items. This sorting ensures that the form items within the same vertical logical block are arranged from left to right, conforming to the natural horizontal order.

[0084] Step S17-3: For the second sequence, the coordinates of all form items belonging to the same vertical logic block are merged in turn and used as input information for the large language model to perform the form filling task.

[0085] In this embodiment, the coordinate information of all form items belonging to the same vertical logical block in the second sequence is merged. Specifically, the coordinate information of all form items within the same vertical logical block is merged into a single structured form item coordinate information. This coordinate information serves as the input information for the large language model to perform the form filling task. This method ensures that the input information received by the large language model is structured, ordered, and contains all necessary positional information, thereby improving the accuracy and efficiency of the form filling task.

[0086] The specific sorting rules are as follows: form items are sorted in ascending order according to their vertical structure labels (i.e., according to the order of vertical logical blocks (logical rows); if multiple form items fall within the same vertical logical block, they are further sorted in ascending order according to the order of their horizontal structure labels. The rule for generating the structure string is as follows: treat each vertical logic block as a row, merge the structure tags of all form items within the same vertical logic block, and output the result as follows: Name: [A1:B1] Age: [B1:D4] Gender: [E1,E1].

[0087] for Figure 2 The online form structure in the example generates a coordinate-based form containing coordinate information for multiple form items, as shown below:

[0088] Figure 3 This is a schematic diagram of the framework of a form item coordinateization device based on location clustering according to an embodiment of this application. (Refer to...) Figure 3 One embodiment of this application provides a form item coordinate system based on location clustering, the system comprising: The node acquisition module 11 is used to acquire all nodes in the target form and obtain a node list, wherein each node in the node list represents a form item in the target form. The node determination module 12 is used to determine the largest independent structural container node of each node based on the parent node of each node in the target form. The largest independent structural container node is a node that only contains the form item corresponding to the node and the label of the form item. The node location information determination module 13 is used to determine the spatial coordinates of the target area where the largest independent structural container node of the node is located for each node as the absolute spatial information of the node. The absolute spatial information includes the upper and lower boundary coordinates and the left and right boundary coordinates of the node. The vertical logic block division module 14 is used to determine the valid area of ​​the target form and divide the valid area of ​​the target form in the vertical direction to obtain multiple vertical logic blocks. The vertical structure label determination module 15 is used to match the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node. The horizontal structure label determination module 16 is used to cluster each vertical logical block based on the left and right boundary coordinates in the absolute spatial information of each node, and generate the horizontal structure label of the node. The form item coordinate information determination module 17 is used to determine the form item coordinate information of the target form based on the horizontal and vertical structure labels of each node.

[0089] Optionally, the node determination module 12 includes: The filtering unit is used to retain nodes with visible areas in each parent node layer, and filter out blank nodes without dimensions and auxiliary nodes for debugging. The search unit is used to continue searching upwards when the current parent node contains only two nodes with visible areas and the size of the visible area of ​​one of the nodes is less than a preset area threshold. The node determination unit is used to determine the upper boundary of the form item corresponding to the current parent node when the current parent node has multiple sibling nodes with visible areas or presents a complex structural layout, stop traversing upwards, and select the current parent node as the largest independent structural container node.

[0090] Optionally, the device further includes: The absolute spatial information determination module is used to obtain the spatial coordinates of the target area where the largest independent structural container node is located before using the spatial coordinates of the target area where the node is located as the absolute spatial information of the node, and to calculate the absolute spatial information of the spatial coordinates in the target form in combination with the scroll offset. The judgment module is used to determine whether the spatial difference between the spatial coordinates of the target area where each largest independent structural container node is located and the spatial coordinates of the target area where the largest independent structural container node is located is less than the tolerance threshold. The duplicate region determination module is used to determine a duplicate region when the spatial coordinates of the target region where the largest independent structural container node is located are less than the tolerance threshold compared with the spatial coordinates of the target region where the largest independent structural container node is located in the existing records. In this case, the absolute spatial information of the spatial coordinates of the target region where the largest independent structural container node is located in the target form is not recorded.

[0091] Optionally, the vertical logic block partitioning module 14 includes: The target form upper and lower bound determination unit is used to extract the upper and lower bound coordinates of the input element node in the vertical direction for each node in the node list, and determine the minimum upper bound coordinate and maximum lower bound coordinate of multiple input element nodes contained in the target form. The effective area determination unit is used to expand the global vertical range of the target form according to the minimum upper boundary coordinates and the maximum lower boundary coordinates, and construct the effective area of ​​the form area of ​​the target form.

[0092] Optionally, the vertical logic block partitioning module 14 further includes: The vertical coordinate point set construction unit is used to extract the upper and lower boundary coordinates of all nodes in the node list within the vertical direction range of the effective area, and construct the vertical coordinate point set. The Y-axis coordinate point set construction unit is used to arrange the upper and lower boundary coordinates in the ordinate point set in ascending order to obtain the Y-axis coordinate point set. The Y-axis coordinate point set includes the coordinates of multiple center points. The adjacent coordinates of nodes whose continuous difference between the upper and lower boundary coordinates does not exceed a preset number of pixels are merged into one center point. The vertical logic block list building unit is used to build a vertical logic block list, where each vertical logic block represents a vertical spatial region between two adjacent center points.

[0093] Optionally, the longitudinal structure label determination module 15 includes: The center point determination unit is used to determine the center point that is closest to the upper and lower boundary coordinates of each node; A vertical logic block determination unit is used to determine vertical logic blocks that overlap with the nearest center point based on the nearest center point. The vertical structure label determination unit is used to determine the vertical structure label of the node based on the vertical logic block that overlaps with the nearest center point.

[0094] Optionally, the lateral structure label determination module 16 includes: The horizontal coordinate point set determination unit is used to extract the left and right boundary coordinates of all nodes in the node list within the horizontal direction of the effective area, and construct the horizontal coordinate point set. The cluster center determination unit is used to perform structural induction on the set of horizontal coordinate points through K-means clustering to obtain multiple cluster centers; The horizontal logic block determination unit is used to arrange multiple cluster centers in the horizontal direction, construct intervals between multiple adjacent cluster centers, and each interval is determined as a horizontal logic block. The horizontal structure label determination unit is used to determine the horizontal structure label of each node based on the horizontal logic block.

[0095] Optionally, the device further includes: The first sequence determination module is used to, after determining the coordinate information of the form items of the target form, arrange the coordinates of multiple form items contained in the coordinate information of the form items in ascending order according to the number of the vertical logical blocks to obtain the first sequence. The second sequence determination module is used to arrange the form items within the same vertical logical block according to the order of the horizontal structure labels when multiple form items in the first sequence fall within the same vertical logical block, thereby obtaining the second sequence. The merging module is used to merge the coordinates of all form items belonging to the same vertical logical block for the second sequence, and use them as input information for the large language model to perform form filling tasks.

[0096] The location-based clustering form item coordinate device in this application embodiment is a device, but it can also be a component, integrated circuit, or chip in a terminal. This device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0097] The form item coordinate conversion device based on location clustering in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0098] The form item coordinate system based on location clustering provided in this application can achieve... Figures 1 to 2 The various processes implemented by the form item coordinate device based on location clustering in the method embodiment will not be described again here to avoid repetition.

[0099] Optionally, Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. This application also provides an electronic device; it should be noted that the electronic device in this application includes the mobile electronic device and non-mobile electronic device described above.

[0100] The electronic device includes, but is not limited to, components such as: radio frequency unit, network module, audio output unit, input unit, sensor, display unit, user input unit, interface unit, memory, and processor.

[0101] Those skilled in the art will understand that electronic devices may also include power supplies (such as batteries) that supply power to various components. The power supply may be connected to the processor logic through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0102] As an example, such as Figure 4 As shown, the electronic device 400 includes a memory 410 and a processor 420. The memory 410 and the processor 420 are connected via a bus for communication. The memory 410 stores a computer program that can run on the processor 420 to implement the steps in the location clustering-based form item coordinate method disclosed in the above embodiments of this application.

[0103] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0105] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects.

[0106] Furthermore, this application embodiment also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described location-based clustering method for coordinate mapping of form items, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0107] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0108] This application also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the form item coordinateization method based on location clustering, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0109] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0110] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0112] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for coordinate representation of form items based on location clustering, characterized in that, The method includes: Retrieve all nodes in the target form to obtain a node list, where each node in the node list represents a form item in the target form; Based on the parent node of each node in the target form, determine the largest independent structural container node of that node. The largest independent structural container node is a node that contains only the form item corresponding to that node and the label of that form item. For each node, the spatial coordinates of the target region where the node's largest independent structural container node is located are used as the node's absolute spatial information. The absolute spatial information includes the node's upper and lower boundary coordinates and left and right boundary coordinates. Determine the valid area of ​​the target form, and divide the valid area of ​​the target form vertically to obtain multiple vertical logical blocks; The multiple vertical logical blocks are matched with multiple nodes in the node list to determine the vertical structure label of each node; For each vertical logical block, clustering is performed based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structure label of the node; Based on the horizontal and vertical structure labels of each node, the coordinate information of the form items in the target form is determined.

2. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, The step of determining the largest independent structural container node for each node based on its parent node in the target form includes: In each parent node, nodes with visible areas are retained, while blank nodes without dimensions and auxiliary nodes for debugging are filtered out. If the current parent node contains only two nodes with visible areas, and the size of the visible area of ​​one of these nodes is smaller than a preset area threshold, continue searching upwards; If the current parent node has multiple sibling nodes with visible areas or presents a complex structural layout, determine the upper boundary of the form item corresponding to that node, stop traversing upwards, and select the current parent node as the largest independent structural container node.

3. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, Before using the spatial coordinates of the target region where the node's largest independent structural container node is located as the node's absolute spatial information, the following is also included: Obtain the spatial coordinates of the target area where the largest independent structural container node is located, and calculate the absolute spatial information of the spatial coordinates in the target form in combination with the scroll offset; For the spatial coordinates of the target region where each largest independent structural container node is located, determine whether the spatial difference between the target region where the largest independent structural container node is located and the spatial coordinates of the target region where the existing records are located is less than the tolerance threshold. If the spatial coordinates of the target area where the largest independent structural container node is located are less than the tolerance threshold compared with the spatial coordinates of the target area where the largest independent structural container node is located in the existing records, it is determined to be a duplicate area, and the absolute spatial information of the spatial coordinates of the target area where the largest independent structural container node is located in the target form is not recorded.

4. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, The determination of the valid area of ​​the target form includes: For each node in the node list, extract the upper and lower bound coordinates of the input element node in the vertical direction, and determine the minimum upper bound coordinate and maximum lower bound coordinate of the multiple input element nodes contained in the target form. Based on the minimum upper bound coordinates and the maximum lower bound coordinates, the global vertical range of the target form is expanded to construct the effective area of ​​the form region of the target form.

5. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, The process of dividing the effective area of ​​the target form vertically to obtain multiple vertical logical blocks includes: Within the vertical direction of the effective area, extract the upper and lower boundary coordinates of all nodes in the node list to construct a set of ordinate points; The upper and lower boundary coordinates in the set of vertical coordinate points are arranged in ascending order to obtain the set of Y-axis coordinate points. The set of Y-axis coordinate points includes the coordinates of multiple center points. The adjacent coordinates of nodes whose continuous difference between the upper and lower boundary coordinates does not exceed a preset number of pixels are merged into one center point. Construct a list of vertical logical blocks, where each vertical logical block represents a vertical spatial region between two adjacent center points.

6. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, The step of matching the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node includes: Determine the center point that is closest to the upper and lower boundary coordinates of each node; Based on the nearest center point, determine the vertical logic blocks that overlap with the nearest center point; The vertical structure label of the node is determined based on the vertical logical blocks that overlap with the nearest center point.

7. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, For each vertical logical block, clustering is performed based on the left and right boundary coordinates in the absolute spatial information of each node to generate the horizontal structural labels of the nodes, including: Within the horizontal direction of the effective area, extract the left and right boundary coordinates of all nodes in the node list to construct a set of horizontal coordinate points; By using K-means clustering, the set of horizontal coordinate points is structurally summarized to obtain multiple cluster centers; Arrange multiple cluster centers horizontally to construct intervals between multiple adjacent cluster centers, and define each interval as a horizontal logical block; Based on the horizontal logic block, the horizontal structure label of each node is determined.

8. The method for coordinate representation of form items based on location clustering according to claim 1, characterized in that, After determining the coordinates of the form items in the target form, the following steps are also included: The coordinates of multiple form items contained in the form item coordinate information are arranged in ascending order according to the vertical logical block number to obtain the first sequence; If multiple form items fall within the same vertical logical block in the first sequence, the form items within the same logical block are arranged according to the order of the horizontal structure labels to obtain the second sequence. For the second sequence, the coordinates of all form items belonging to the same vertical logic block are merged and used as input information for the large language model to perform the form filling task.

9. A form item coordinate conversion device based on location clustering, characterized in that, The device includes: The node acquisition module is used to acquire all nodes in the target form and obtain a node list, wherein each node in the node list represents a form item in the target form. The node determination module is used to determine the largest independent structural container node of each node based on its parent node in the target form. The largest independent structural container node is a node that contains only the form item corresponding to the node and the label of the form item. The node location information determination module is used to determine the absolute spatial information of each node by taking the spatial coordinates of the target area where the largest independent structural container node of the node is located. The absolute spatial information includes the upper and lower boundary coordinates and the left and right boundary coordinates of the node. The vertical logic block division module is used to determine the valid area of ​​the target form and divide the valid area of ​​the target form into multiple vertical logic blocks in the vertical direction. The vertical structure label determination module is used to match the plurality of vertical logical blocks with the plurality of nodes in the node list to determine the vertical structure label of each node. The horizontal structure label determination module is used to cluster each vertical logical block based on the left and right boundary coordinates in the absolute spatial information of each node, and generate the horizontal structure label of the node. The form item coordinate information determination module is used to determine the form item coordinate information of the target form based on the horizontal and vertical structure labels of each node.

10. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the form item coordinateization method based on location clustering as described in claims 1-8.

Images