Form generation method and device

Abstract

The invention discloses a form generation method and device, and the method comprises the steps: obtaining a to-be-converted document, and enabling the to-be-converted document to comprise a form region; identifying a form area in the to-be-converted document and content in the form area based on machine learning; and displaying the content in the form area on the webpage according to the layout template corresponding to the to-be-converted document so as to generate a target form. Therefore, the efficiency and accuracy of form conversion are improved, and the cost of manual design and online form development is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and automatic conversion, and in particular to a form generating method and device. Background Art

[0002] With the in-depth development of the digital transformation of the industry, enterprises have put forward higher requirements for the efficiency and accuracy of data processing. Traditionally, enterprises rely on a large number of paper forms to record and manage information. These forms are often designed by office software, with various formats and frequent updates. In order to further improve work efficiency and data management capabilities, enterprises urgently need to convert these paper or electronic forms into online forms to achieve real-time data entry, storage and analysis.

[0003] However, most of the current online form generation tools rely on manual design. For industrial forms with a wide variety of formats and frequent changes, the development workload is huge and it is difficult to keep up with the changes in actual needs. Therefore, it is particularly important to design a method to quickly generate online forms. Summary of the invention

[0004] The main technical problem solved by the present invention is how to automatically identify key information and input areas in a document and convert them into a form that can be edited and stored online, thereby improving the efficiency and accuracy of form conversion and reducing the cost of manual design and development of online forms.

[0005] According to the first aspect, an embodiment provides a form generation method, including:

[0006] Acquire a document to be converted, wherein the document to be converted includes a form area;

[0007] Identify a form area in the document to be converted and content in the form area based on machine learning;

[0008] The content in the form area is displayed on a web page according to the layout template corresponding to the document to be converted, thereby generating a target form.

[0009] In some embodiments, the identifying the form area in the document to be converted based on machine learning includes:

[0010] Acquire a document image corresponding to the document to be converted;

[0011] Binarizing the document image corresponding to the document to be converted to obtain a binarized document image;

[0012] The form area in the binarized document image is detected using a form area detection model.

[0013] In some embodiments, the content in the form area includes a cell, and identifying the content in the form area based on machine learning includes:

[0014] A graph convolutional neural network is used to identify multiple cells in the form area.

[0015] In some embodiments, the form generation method further includes:

[0016] For each of the cells in the form area, determining a filling state of the cell;

[0017] The type of the form element corresponding to the cell is determined according to the filling state of the cell; wherein the type of the form element corresponding to the cell includes a text box and an input box.

[0018] In some embodiments, some of the cells have already been filled with content, and determining the filling status of the cells includes:

[0019] If the cell does not contain the filling content, define the filling state of the cell as no filling content;

[0020] If the filled content in the cell occupies a preset proportion of the space size of the cell, the filling state of the cell is defined as being fully filled;

[0021] If the filling content in the cell does not occupy a preset proportion of the space size of the cell, the filling state of the cell is defined as reserved filling space.

[0022] In some embodiments, determining the type of the form element corresponding to the cell according to the filling state of the cell includes:

[0023] The type of the form element corresponding to the cell whose filling state is reserved filling space or no filling content is determined as an input box, and the type of the form element corresponding to the cell whose filling state is completely filled and meets the preset appearance characteristics is determined as a text box; wherein the appearance characteristics include color characteristics and shape characteristics.

[0024] In some embodiments, the form generation method further includes:

[0025] For each of the cells in the form area, identifying a structural feature of the cell;

[0026] Determine whether there is an association relationship between a cell whose form element type is an input box and a cell whose form element type is a text box based on the structural characteristics of the cell;

[0027] The cells of the form element type being an input box and the cells of the form element type being a text box that have an associated relationship are associated to obtain a plurality of pairs of associated cells; wherein, in the associated cells, the text in the cells of the form element type being a text box is a prompt of the fill content of the cells of the form element type being an input box, and the data inputted in the cells of the form element type being an input box is bound to the text in the cells of the form element type being a text box.

[0028] In some embodiments, the structural features of the cell include size features of the cell, boundary features of the cell, and adjacent features of the cell, and identifying the structural features of the cell includes:

[0029] Using a structure recognition model to identify the size feature of the cell and the boundary feature of the cell; wherein the size feature of the cell includes the size of the cell, and the boundary feature of the cell includes the row number and column number of each cell in the form area;

[0030] A cell having an adjacent relationship with the cell is determined according to the boundary feature of the cell, and the row number and column number of the cell having an adjacent relationship with the cell are used as the adjacent feature of the cell.

[0031] In some embodiments, after generating the target form, the form generation method further includes:

[0032] When receiving a user's operation of filling in a target form in the webpage, data cleaning is performed on the content filled in by the user when filling in the form to obtain cleaned filled content; the data cleaning includes removing spaces and special characters;

[0033] The filled-in content after cleaning is subject to content verification, and the filled-in content after passing the content verification is saved; wherein the content verification includes format verification and range verification.

[0034] According to the second aspect, an embodiment provides a form generating device, including:

[0035] A document acquisition module, used to acquire a document to be converted, wherein the document to be converted includes a form area;

[0036] A region and content identification module, used for identifying a form region in the document to be converted and content in the form region based on machine learning;

[0037] The form display module is used to display the content in the form area on the web page according to the layout template corresponding to the document to be converted, so as to generate a target form.

[0038] According to the form generation method and device of the above embodiment, machine learning is used to identify the form area and the content in the form area in the document to be converted, and the content in the form area is displayed on the web page according to the layout template corresponding to the document to be converted, so as to flexibly generate a target form that meets the requirements, wherein the target form is consistent with the format of the document to be converted, and thus has stronger adaptability and flexibility. The identification of the form area and content based on machine learning reduces the dependence on manual labor, and reduces the development cost and maintenance difficulty. Therefore, the efficiency and accuracy of form conversion are improved, and the cost of manual design and development of online forms is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a form generation method according to an embodiment of the present application;

[0040] Figure 2 A flowchart of an embodiment of identifying a form area in a document to be converted based on machine learning;

[0041] Figure 3 A flowchart of a form generation method according to another embodiment;

[0042] Figure 4 A flowchart of determining the filling state of a cell according to an embodiment;

[0043] Figure 5 A flowchart of a form generation method according to another embodiment;

[0044] Figure 6 A flowchart of identifying structural features of a cell according to an embodiment;

[0045] Figure 7 A flowchart of a form generation method according to another embodiment;

[0046] Figure 8 A schematic diagram of the structure of a form generating device according to an embodiment;

[0047] Fig. 9 A schematic diagram of the structure of a form generating device according to another embodiment. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0049] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0050] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0051] Please refer to Figure 1 In an embodiment of the present invention, a form generation method is provided, including steps S10 to S30, which are described in detail below.

[0052] Step S10: Obtain the document to be converted.

[0053] In some embodiments, the document to be converted may be a paper office document or an electronic office document, wherein the document to be converted includes a form area, for example, the form area is a table.

[0054] Step S20: Identify the form area and the content in the form area in the document to be converted based on machine learning.

[0055] Please refer to Figure 2 In some embodiments, step S20 identifies the form area in the document to be converted based on machine learning, including steps S21 to S23, which are described in detail below.

[0056] Step S21: Acquire the document image corresponding to the document to be converted.

[0057] Step S22: binarizing the document image corresponding to the document to be converted to obtain a binarized document image.

[0058] In some embodiments, in addition to binarizing the document image corresponding to the document to be converted and detecting the form area based on the binarized document image, the document image corresponding to the document to be converted may be grayscaled and the form area may be detected based on the grayscaled document image. The accuracy of subsequent form area detection may be improved by performing different image preprocessing such as grayscale or binarization on the document image corresponding to the document to be converted.

[0059] Step S23: Detect the form area in the binarized document image using the form area detection model.

[0060] In some embodiments, the form area detection model may be a deep learning model for table detection and recognition, such as a CascadeTabNet model and a TableNet model. The CascadeTabNet model uses a cascade approach to perform table detection and structure recognition, while the TableNet model combines a multi-level attention mechanism and a table structure understanding network to achieve efficient training and prediction.

[0061] In some embodiments, for a document to be converted provided by a user, a form area in the document to be converted can be identified through machine learning, wherein, for documents to be converted in various formats such as .docx or .xlsx, machine learning can be used to efficiently and quickly identify the form area. In addition, the document to be converted can be opened in office software, and the form area in the document to be converted can be directly selected to achieve the identification of the form area.

[0062] In some embodiments, the content in the form area includes a cell, and identifying the content in the form area based on machine learning includes:

[0063] Use graph convolutional neural network to identify multiple cells in the form area.

[0064] In some embodiments, the form area includes a table, and the content in the form area includes cells. Therefore, a graph convolutional neural network can be used to parse the table structure, represent the cells and lines in the table as nodes and edges in the graph, and extract features and identify cells through graph convolution operations.

[0065] Please refer to Figure 3 In some embodiments, the form generation method further includes steps S24 to S25, which are described in detail below.

[0066] Step S24: For each cell in the form area, determine the filling status of the cell.

[0067] Please refer to Figure 4 In some embodiments, some cells have already been filled with content, and step S24 determines the filling status of the cells, including steps S241 to S243, which are described in detail below.

[0068] Step S241: If the cell does not contain any filling content, define the filling state of the cell as no filling content.

[0069] Step S242: If the filling content in the cell occupies a preset proportion of the cell space, the filling state of the cell is defined as fully filled.

[0070] Step S243: If the filling content in the cell does not occupy a preset proportion of the cell space, define the filling state of the cell as reserved filling space.

[0071] Step S25: Determine the type of the form element corresponding to the cell according to the filling state of the cell.

[0072] In some embodiments, the types of form elements corresponding to the cells include text boxes and input boxes, wherein the text box is used for the user to input long text content, and the long text content is usually a prompt for the content input in the input box.

[0073] In this embodiment, determining the type of the form element corresponding to the cell according to the filling state of the cell includes:

[0074] The type of the form element corresponding to a cell whose filling state is reserved filling space or no filling content is determined as an input box, and the type of the form element corresponding to a cell whose filling state is fully filled and meets the preset appearance characteristics is determined as a text box.

[0075] In some embodiments, when the filling state of a cell is reserved filling space or no filling content, it means that the cell has filling space for the user to input content, so the type of the form element corresponding to the cell is determined to be an input box. For cells whose filling state is a completely filled cell, it is necessary to further determine whether the cell meets the preset appearance characteristics. Only when the appearance characteristics are also met, the type of the form element corresponding to the cell is determined to be a text box, wherein the appearance characteristics include color characteristics and shape characteristics, and may also include size characteristics.

[0076] In some embodiments, for a cell whose filling state is completely filled, there are two situations. The first situation is that the type of the form element corresponding to the cell is determined to be an input box, and the user can directly enter text in the cell. When the complete filling content is entered in the cell, the filling state of the cell is completely filled. The second situation is that the type of the form element corresponding to the cell is determined to be a text box, and the content in the text box is used to prompt the information in the input box. Therefore, when the filling state of the cell is completely filled, it is necessary to further judge based on the appearance characteristics of the cell. For a text box, its color is different from that of an input box and is usually marked in blue, and its shape is usually a rectangular area of ​​moderate size.

[0077] In some embodiments, for cells whose form element type is determined to be input boxes, there are the following two forms: a first form: a rectangular cell in which the user can directly enter text; a second form: containing prompt text and a selection box, in which there is a partial text prompt in the cell, and after the text prompt there is a corresponding circular area or rectangular area for the user to select, and the circular area or rectangular area represents a check box or radio button of the selection box, and whether it is a selection box is determined by analyzing the shape, color, and check status of these areas.

[0078] Please refer to Figure 5 In some embodiments, the form generation method further includes steps S26 to S28, which are described in detail below.

[0079] Step S26: For each cell in the form area, identify the structural features of the cell.

[0080] In some embodiments, the structural features of the cell include the size features of the cell, the boundary features of the cell, and the adjacent features of the cell. Figure 6 Step S26 of identifying the structural features of the cell includes steps S261 to S262, which are described in detail below.

[0081] Step S261: using the structure recognition model to identify the size features and boundary features of the cell.

[0082] In this embodiment, the structure recognition model includes a convolutional neural network, the size feature of the cell includes the size of the cell, and the boundary feature of the cell includes the row number and column number of each cell in the form area.

[0083] Step S262: Determine the cells that are adjacent to the cell according to the boundary features of the cell, and use the row numbers and column numbers of the cells that are adjacent to the cell as the adjacent features of the cell.

[0084] In some embodiments, since the boundary features of the cells include the row number and column number of each cell in the form area, cells with adjacent row numbers and / or column numbers can be determined as cells with an adjacent relationship.

[0085] Step S27: determining whether there is an association relationship between a cell whose form element type is an input box and a cell whose form element type is a text box based on the structural characteristics of the cell.

[0086] In some embodiments, since there are multiple cells in the form area, the cells whose form element type is an input box are used for data input, and the cells whose table element type is a text box are used for prompts for the input box. In order to associate the content in the text box used for prompts with the content entered in the input box after the data is subsequently filled in, so that the user can analyze the data more intuitively after obtaining the data, it is necessary to determine whether there is an association relationship between the cells whose form element type is an input box and the cells whose form element type is a text box.

[0087] In some embodiments, the structural features of the cell include adjacent features of the cell. For any cell whose form element type is an input box, the cells whose form element type is a text box and meet preset adjacent conditions around the cell are determined based on the adjacent features of the cell. Among the cells whose form element type is a text box and meet the preset adjacent conditions, the cells whose size features meet the size requirements and whose cell boundary features satisfy that the row number is the same as the row number of the input box and the column number is less than the column number of the input box are selected as cells having an associated relationship with the cell whose form element type is an input box.

[0088] Step S28: Associating cells whose form element type is an input box and cells whose form element type is a text box, which have an associated relationship, to obtain multiple pairs of associated cells.

[0089] In this embodiment, in the associated cells, the text in the cell whose form element type is a text box is a prompt of the fill content of the cell whose form element type is an input box, and the data entered in the cell whose form element type is an input box is bound to the text in the cell whose form element type is a text box.

[0090] In some embodiments, for multiple pairs of associated cells, when the form data is subsequently stored in real time, the data entered in the cell whose form element type is an input box will be bound to the text in the cell whose form element type is a text box. This saves the user the time of manually setting the input box name and associating the data of the input box and the text box, making it easier for the form to subsequently collect and analyze the filled-in data.

[0091] Step S30: Displaying the content in the form area on the web page according to the layout template corresponding to the document to be converted, thereby generating a target form.

[0092] In some embodiments, the content in the form area is displayed on the web page using a preset markup language and style sheet language, wherein the markup language is HTML and the style sheet language is CSS. Corresponding HTML elements are dynamically generated on the Web interface based on the content in the form area, and the style and layout of the HTML elements are controlled by CSS. When controlling the style and layout, the layout template corresponding to the document to be converted is referenced, and the interactive function of the elements is also implemented by JavaScript. When using HTML to display the content in the form area, an HTML document structure can be created, the content in the form area can be added to the HTML document structure, and the form attributes can be set to finally generate a target form. The form attributes include type, label, and value domain.

[0093] In some embodiments, for images in the form area, the image can be converted into base64 encoding, and then the encoded string is directly stored in the html tag. For the table in the form area, similar tags such as table will be generated, and the target form will be generated by displaying it on the web page based on the stored html tags.

[0094] In some embodiments, a corresponding target form is generated according to the content in the form area, and a dynamic mapping relationship between the document to be converted and the online form is established. At the same time, the target form can accurately reflect the form structure and logic of the original office document.

[0095] In some embodiments, after generating the target form, please refer to Figure 7 The form generation method also includes steps S31 to S32, which are described in detail below.

[0096] Step S31: when receiving a user's operation of filling in a target form in a webpage, data cleaning is performed on the content filled in by the user when filling in the form to obtain the cleaned filled content.

[0097] In this embodiment, data cleaning includes removing spaces and special characters, where special characters include &, #, and *.

[0098] Step S32: Verify the filled-in content after cleaning, and save the filled-in content that passes the content verification.

[0099] In this embodiment, content verification includes format verification and range verification, wherein format verification refers to the verification that determines whether the letters in the cleaned filled-in content conform to the uppercase format or lowercase format set in advance, and range verification refers to the pre-setting of a range interval to determine whether the numbers in the cleaned filled-in content are within this range interval.

[0100] In some embodiments, the filled content after content verification will be saved in JSON or XML format. The target form is deployed on the web page using the publishing function of the web server, and the target form is shared and accessed through the URL address or embedded coding. At the same time, the form data is updated and synchronized in real time using web technologies such as AJAX or WebSocket.

[0101] The form generation method proposed in this embodiment is intended to meet the needs of industrial digital transformation, and can quickly convert office documents into online forms, which is suitable for the automated processing of complex and changeable forms in the industrial field. By using machine learning to obtain the form area and the content in the form area in the office document to generate the target form, the fully automated conversion from the office document to the online document is realized. In this conversion process, manual intervention is reduced, the degree of automation of the conversion is improved, and the conversion process is more efficient and faster. Existing online form generation tools are often limited to fixed templates and formats, and it is difficult to adapt to the various, different and frequently changing form requirements in the industrial field. By identifying the form area and the content in the form area, online forms that meet the requirements can be flexibly generated, and the format is consistent with the office document, which has stronger adaptability and flexibility. In addition, traditional online form development requires the participation of professional Web developers, and the development cycle is long and costly. The present invention reduces the dependence on Web developers, reduces development costs and maintenance difficulties by providing an automated conversion tool. Enterprises can focus more on business needs themselves rather than technical implementation details.

[0102] In some embodiments, a form generation system is provided. A user accesses the user interface of the form generation system through a browser or other client software. On the user interface, the user selects to upload a locally stored document to be converted. The form generation system supports processing documents to be converted in different file formats and automatically detects the form area and content in the form area in the document to be converted. The user can also open the document to be converted in the office software, directly select the form area in the document to be converted, and copy and paste it to a designated area in the form generation system, so that the user can quickly extract the form area in the document to be converted. Among them, for the directly selected form area, the browser itself can be compatible with the characteristics of identifying the content of office documents, directly generate html-related tags, and restore them to the form generation system. When uploading a locally stored document to be converted, the document to be converted is converted into html-related tags using a parsing library such as Apache POI for Java, and subsequent form generation is performed. The form generation system has a built-in document parsing and rendering engine. When the document to be converted is obtained, the engine will automatically analyze the form area and the content of the form area of ​​the document to be converted, identify the form elements, and accurately rebuild the form elements on the Web interface, maintaining the layout and style corresponding to the document to be converted, and realizing the conversion from the document to be converted to the target form. At the same time, in order to meet the personalized needs of users, the form generation system also provides a wealth of editing tools. Users can directly adjust the layout of the target form on the Web interface, for example, move the width of the form, adjust the size of the form, change the font color, etc. In addition, users can also specify the corresponding field names for the input boxes in the target form. These names can be used as identifiers for data collection, which is convenient for subsequent data statistics and reading. The target form generates a unique URL address or is embedded in an existing web page, allowing others to access and fill in the target form online. The published target form supports multiple users to fill in at the same time, and the data will also be saved to the server in real time to ensure the security and integrity of the data.

[0103] Please refer to Figure 8 An embodiment of the present invention provides a form generation device, including a document acquisition module 10, an area and content identification module 20 and a form display module 30, which are described in detail below.

[0104] The document acquisition module 10 is used to acquire the document to be converted.

[0105] In some embodiments, the document to be converted may be a paper office document or an electronic office document, wherein the document to be converted includes a form area, for example, the form area is a table.

[0106] The region and content identification module 20 is used to identify the form region and the content in the form region in the document to be converted based on machine learning.

[0107] Please refer back to Figure 2 In some embodiments, the region and content identification module 20 identifies the form region in the document to be converted based on machine learning through the following actions, which are described in detail below.

[0108] The region and content identification module 20 obtains a document image corresponding to the document to be converted, performs binarization processing on the document image corresponding to the document to be converted to obtain a binarized document image, and uses a form region detection model to detect a form region in the binarized document image.

[0109] In some embodiments, in addition to binarizing the document image corresponding to the document to be converted and detecting the form area based on the binarized document image, the document image corresponding to the document to be converted may be grayscaled and the form area may be detected based on the grayscaled document image. The accuracy of subsequent form area detection may be improved by performing different image preprocessing such as grayscale or binarization on the document image corresponding to the document to be converted.

[0110] In some embodiments, the form area detection model may be a deep learning model for table detection and recognition, such as a CascadeTabNet model and a TableNet model. The CascadeTabNet model uses a cascade approach to perform table detection and structure recognition, while the TableNet model combines a multi-level attention mechanism and a table structure understanding network to achieve efficient training and prediction.

[0111] In some embodiments, for the document to be converted provided by the user, the region and content identification module 20 can identify the form area in the document to be converted through machine learning, wherein, for documents to be converted in various formats such as .docx or .xlsx, machine learning can be used to efficiently and quickly identify the form area. In addition, the document to be converted can be opened in office software, and the form area in the document to be converted can be directly selected to achieve the identification of the form area.

[0112] In some embodiments, the content in the form area includes cells, and the area and content identification module 20 identifies the content in the form area based on machine learning, including:

[0113] Use graph convolutional neural network to identify multiple cells in the form area.

[0114] In some embodiments, the form area includes a table, and the content in the form area includes cells. Therefore, a graph convolutional neural network can be used to parse the table structure, represent the cells and lines in the table as nodes and edges in the graph, and extract features and identify cells through graph convolution operations.

[0115] Please refer to Fig. 9 In some embodiments, the form generation device may further include a form element type determination module 40, which is used to determine the filling state of each cell in the form area, and determine the type of form element corresponding to the cell according to the filling state of the cell.

[0116] Please refer back to Figure 4 In some embodiments, some cells have already been filled with content, and the form element type determination module 40 can determine the filling status of the cell through the following actions, which are described in detail below.

[0117] If the cell does not contain any fill content, the form element type determination module 40 defines the fill state of the cell as no fill content; if the fill content in the cell occupies a preset proportion of the cell's space size, the form element type determination module 40 defines the fill state of the cell as fully filled; if the fill content in the cell does not occupy a preset proportion of the cell's space size, the form element type determination module 40 defines the fill state of the cell as reserved fill space.

[0118] In some embodiments, the types of form elements corresponding to the cells include text boxes and input boxes, wherein the text box is used for the user to input long text content, and the long text content is usually a prompt for the content input in the input box.

[0119] In this embodiment, the form element type determination module 40 determines the type of the form element corresponding to the cell according to the filling state of the cell, including:

[0120] The type of the form element corresponding to a cell whose filling state is reserved filling space or no filling content is determined as an input box, and the type of the form element corresponding to a cell whose filling state is fully filled and meets the preset appearance characteristics is determined as a text box.

[0121] In some embodiments, when the filling state of a cell is reserved filling space or no filling content, it means that the cell has filling space for the user to input content, so the form element type determination module 40 determines the type of the form element corresponding to the cell as an input box. For a cell whose filling state is a fully filled cell, it is necessary to further determine whether the cell meets the preset appearance characteristics. Only when the appearance characteristics are also met, the type of the form element corresponding to the cell is determined as a text box, wherein the appearance characteristics include color characteristics and shape characteristics, and may also include size characteristics.

[0122] In some embodiments, for a cell whose filling state is completely filled, there are two situations. The first situation is that the type of the form element corresponding to the cell is determined to be an input box, and the user can directly enter text in the cell. When the complete filling content is entered in the cell, the filling state of the cell is completely filled. The second situation is that the type of the form element corresponding to the cell is determined to be a text box, and the content in the text box is used to prompt the information in the input box. Therefore, when the filling state of the cell is completely filled, it is necessary to further judge based on the appearance characteristics of the cell. For a text box, its color is different from that of an input box and is usually marked in blue, and its shape is usually a rectangular area of ​​moderate size.

[0123] In some embodiments, for cells whose form element type is determined to be input boxes, there are the following two forms: a first form: a rectangular cell in which the user can directly enter text; a second form: containing prompt text and a selection box, in which there is a partial text prompt in the cell, and after the text prompt there is a corresponding circular area or rectangular area for the user to select, and the circular area or rectangular area represents a check box or radio button of the selection box, and whether it is a selection box is determined by analyzing the shape, color, and check status of these areas.

[0124] Please refer to Fig. 9 In some embodiments, the form generation device may further include a cell association module 50. For each cell in the form area, the cell association module 50 identifies the structural features of the cell, determines whether there is an association relationship between a cell whose form element type is an input box and a cell whose form element type is a text box based on the structural features of the cell, and associates the cell whose form element type is an input box and the cell whose form element type is a text box with the associated relationship, thereby obtaining multiple pairs of associated cells.

[0125] In some embodiments, the structural features of the cell include the size features of the cell, the boundary features of the cell, and the adjacent features of the cell. Figure 6,The identification of the structural features of the cell is achieved through the following actions, which are explained in detail below.

[0126] The cell association module 50 uses the structure recognition model to identify the size characteristics and boundary characteristics of the cell, determines the cells that are adjacent to the cell according to the boundary characteristics of the cell, and uses the row number and column number of the cells that are adjacent to the cell as the adjacent characteristics of the cell.

[0127] In this embodiment, the structure recognition model includes a convolutional neural network, the size feature of the cell includes the size of the cell, and the boundary feature of the cell includes the row number and column number of each cell in the form area.

[0128] In some embodiments, since the boundary features of the cells include the row number and column number of each cell in the form area, cells with adjacent row numbers and / or column numbers can be determined as cells with an adjacent relationship.

[0129] In some embodiments, since there are multiple cells in the form area, the cells whose form element type is an input box are used for data input, and the cells whose table element type is a text box are used for prompts for the input box. In order to associate the content in the text box used for prompts with the content entered in the input box after the data is subsequently filled in, so that the user can analyze the data more intuitively after obtaining the data, it is necessary to determine whether there is an association relationship between the cells whose form element type is an input box and the cells whose form element type is a text box.

[0130] In some embodiments, the structural features of a cell include adjacent features of the cell. For any cell whose form element type is an input box, the cells whose form element type is a text box and meet preset adjacent conditions around the cell are determined based on the adjacent features of the cell. Among the cells whose form element type is a text box and meet the preset adjacent conditions, the cells whose size features meet the size requirements and whose cell boundary features satisfy that the row number is the same as the row number of the input box and the column number is less than the column number of the input box are selected, and the cell association module 50 regards them as cells having an associated relationship with the cell whose form element type is an input box.

[0131] In this embodiment, in the associated cells, the text in the cell whose form element type is a text box is a prompt of the fill content of the cell whose form element type is an input box, and the data entered in the cell whose form element type is an input box is bound to the text in the cell whose form element type is a text box.

[0132] In some embodiments, for multiple pairs of associated cells, when the form data is subsequently stored in real time, the data entered in the cell whose form element type is an input box will be bound to the text in the cell whose form element type is a text box. This saves the user the time of manually setting the input box name and associating the data of the input box and the text box, making it easier for the form to subsequently collect and analyze the filled-in data.

[0133] The form display module 30 is used to display the content in the form area on the web page according to the layout template corresponding to the document to be converted, so as to generate a target form.

[0134] In some embodiments, the form display module 30 uses a preset markup language and style sheet language to display the content in the form area on a web page, wherein the markup language is html and the style sheet language is CSS. Corresponding html elements are dynamically generated on the web interface according to the content in the form area, and the style and layout of the html elements are controlled by CSS. When controlling the style and layout, the layout template corresponding to the document to be converted is referenced, and the interactive function of the elements is also implemented by JavaScript. When using html to display the content in the form area, an html document structure can be created, the content in the form area can be added to the html document structure, and the form attributes can be set to finally generate a target form. Among them, the form attributes include type, label and value domain.

[0135] In some embodiments, for images in the form area, the form display module 30 can convert the image into base64 encoding, and then store the encoded string directly in the html tag. For tables in the form area, the form display module 30 will generate similar tags such as table, display them on the web page according to the stored html tags, and generate a target form.

[0136] In some embodiments, the form display module 30 generates a corresponding target form according to the content in the form area, and establishes a dynamic mapping relationship between the document to be converted and the online form. At the same time, the target form can accurately reflect the form structure and logic of the original office document.

[0137] In some embodiments, after generating the target form, please refer to Figure 7 , the form display module 30 also includes the following actions, which are described in detail below.

[0138] When receiving the user's operation of filling in the target form in the webpage, the form display module 30 cleans the content filled in by the user when filling in the form to obtain the cleaned filled content, verifies the cleaned filled content, and saves the filled content that passes the content verification.

[0139] In this embodiment, data cleaning includes removing spaces and special characters, where special characters include &, #, and *, etc. Content verification includes format verification and range verification, where format verification refers to the verification that determines whether the letters in the cleaned filled-in content conform to the uppercase format or lowercase format set in advance, and range verification refers to the pre-setting of a range interval to determine whether the numbers in the cleaned filled-in content are within this range interval.

[0140] In some embodiments, the form display module 30 saves the filled content after content verification in JSON or XML format. The target form is deployed on the web page using the publishing function of the web server, and the target form is shared and accessed through the URL address or embedded coding. At the same time, the form data is updated and synchronized in real time using web technologies such as AJAX or WebSocket.

[0141] The form generation method proposed in this embodiment is intended to meet the needs of industrial digital transformation, and can quickly convert office documents into online forms, which is suitable for the automated processing of complex and changeable forms in the industrial field. By using machine learning to obtain the form area and the content in the form area in the office document to generate the target form, the fully automated conversion from the office document to the online document is realized. In this conversion process, manual intervention is reduced, the degree of automation of the conversion is improved, and the conversion process is more efficient and faster. Existing online form generation tools are often limited to fixed templates and formats, and it is difficult to adapt to the various, different and frequently changing form requirements in the industrial field. By identifying the form area and the content in the form area, online forms that meet the requirements can be flexibly generated, and the format is consistent with the office document, which has stronger adaptability and flexibility. In addition, traditional online form development requires the participation of professional Web developers, and the development cycle is long and costly. The present invention reduces the dependence on Web developers, reduces development costs and maintenance difficulties by providing an automated conversion tool. Enterprises can focus more on business needs themselves rather than technical implementation details.

[0142] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0143] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A form generation method, characterized in that: include: Acquire a document to be converted, wherein the document to be converted includes a form area; Identify a form area in the document to be converted and content in the form area based on machine learning; The content in the form area is displayed on a web page according to the layout template corresponding to the document to be converted, thereby generating a target form.

2. The form generation method according to claim 1, characterized in that: The identifying the form area in the document to be converted based on machine learning includes: Acquire a document image corresponding to the document to be converted; Binarizing the document image corresponding to the document to be converted to obtain a binarized document image; The form area in the binarized document image is detected using a form area detection model.

3. The form generation method according to claim 1 or 2, characterized in that: The content in the form area includes cells, and identifying the content in the form area based on machine learning includes: A graph convolutional neural network is used to identify multiple cells in the form area.

4. The form generation method according to claim 3, characterized in that: The form generation method further includes: For each of the cells in the form area, determining a filling state of the cell; The type of the form element corresponding to the cell is determined according to the filling state of the cell; wherein the type of the form element corresponding to the cell includes a text box and an input box.

5. The form generation method according to claim 4, characterized in that: Some of the cells have already been filled with content, and determining the filling status of the cells includes: If the cell does not contain the filling content, define the filling state of the cell as no filling content; If the filled content in the cell occupies a preset proportion of the space size of the cell, the filling state of the cell is defined as being fully filled; If the filling content in the cell does not occupy a preset proportion of the space size of the cell, the filling state of the cell is defined as reserved filling space.

6. The form generation method according to claim 5, characterized in that: The determining the type of the form element corresponding to the cell according to the filling state of the cell includes: The type of the form element corresponding to the cell whose filling state is reserved filling space or no filling content is determined as an input box, and the type of the form element corresponding to the cell whose filling state is completely filled and meets the preset appearance characteristics is determined as a text box; wherein the appearance characteristics include color characteristics and shape characteristics.

7. The form generation method according to claim 6, characterized in that: The form generation method further includes: For each of the cells in the form area, identifying a structural feature of the cell; Determine whether there is an association relationship between a cell whose form element type is an input box and a cell whose form element type is a text box based on the structural characteristics of the cell; The cells of the form element type being an input box and the cells of the form element type being a text box that have an associated relationship are associated to obtain a plurality of pairs of associated cells; wherein, in the associated cells, the text in the cells of the form element type being a text box is a prompt of the fill content of the cells of the form element type being an input box, and the data inputted in the cells of the form element type being an input box is bound to the text in the cells of the form element type being a text box.

8. The form generation method according to claim 7, characterized in that: The structural features of the cell include the size features of the cell, the boundary features of the cell, and the adjacent features of the cell. The identifying the structural features of the cell includes: Using a structure recognition model to identify the size feature of the cell and the boundary feature of the cell; wherein the size feature of the cell includes the size of the cell, and the boundary feature of the cell includes the row number and column number of each cell in the form area; A cell having an adjacent relationship with the cell is determined according to the boundary feature of the cell, and the row number and column number of the cell having an adjacent relationship with the cell are used as the adjacent feature of the cell.

9. The form generation method according to claim 1, characterized in that: After generating the target form, the form generating method further includes: When receiving a user's operation of filling in a target form in the webpage, data cleaning is performed on the content filled in by the user when filling in the form to obtain cleaned filled content; the data cleaning includes removing spaces and special characters; The filled-in content after cleaning is subject to content verification, and the filled-in content after passing the content verification is saved; wherein the content verification includes format verification and range verification.

10. A form generating device, characterized in that: include: A document acquisition module, used to acquire a document to be converted, wherein the document to be converted includes a form area; A region and content identification module, used for identifying a form region in the document to be converted and content in the form region based on machine learning; The form display module is used to display the content in the form area on the web page according to the layout template corresponding to the document to be converted, so as to generate a target form.

Images