OFD document generation method and device, equipment and storage medium

Abstract

The present disclosure provides an OFD document generation method, device and equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of image processing, computer vision, deep learning and the like. The OFD document generation method comprises: acquiring a document image, the document image containing a target object; performing layout analysis on the document image to obtain layout information of the target object; performing layout analysis on the document image based on the layout information to obtain object information of the target object and determine a reading order of the target object; and adding the object information of the target object to a blank OFD document based on the reading order of the target object to obtain a target OFD document. The present disclosure can accurately and efficiently generate an OFD document.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of image processing, computer vision, deep learning, etc., and particularly to an OFD document generation method, apparatus, device, and storage medium. Background Technology

[0002] Open Fixed-layout Document (OFD) is my country's national standard for document format, often referred to as a domestically produced PDF. PDF (Portable Document Format) is a document format standard. Similar to PDF, OFD offers the advantages of a fixed layout and no formatting issues, preserving the original document format across platforms. In the context of domestic document development, various organizations will gradually replace their existing document formats with OFD. For document images, the challenge lies in converting them into OFD documents. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for generating OFD documents.

[0004] According to one aspect of this disclosure, an OFD document generation method is provided, comprising: acquiring a document image, the document image containing a target object; performing layout analysis on the document image to obtain layout information of the target object; performing page layout analysis on the document image based on the layout information to obtain object information of the target object and determining the reading order of the target object; and adding the object information of the target object to a blank OFD document based on the reading order of the target object to obtain a target OFD document.

[0005] According to another aspect of this disclosure, an OFD document generation apparatus is provided, comprising: an acquisition module for acquiring a document image, the document image containing a target object; an analysis module for performing layout analysis on the document image to obtain layout information of the target object; a determination module for performing page layout analysis on the document image based on the layout information to obtain object information of the target object and determine the reading order of the target object; and an adding module for adding the object information of the target object to a blank OFD document according to the reading order of the target object to obtain a target OFD document.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the foregoing aspects.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any of the preceding aspects.

[0008] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to any of the preceding aspects.

[0009] According to the technical solution disclosed herein, OFD documents can be generated accurately and efficiently.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0013] Figure 2 These are schematic diagrams of text blocks with different layouts according to embodiments of this disclosure;

[0014] Figure 3 This is a schematic diagram illustrating an application scenario provided according to embodiments of this disclosure;

[0015] Figure 4 This is a schematic diagram of the overall architecture provided according to embodiments of this disclosure;

[0016] Figure 5 This is a schematic diagram of the OFD document structure provided according to an embodiment of this disclosure;

[0017] Figure 6 This is a schematic diagram according to the second embodiment of the present disclosure;

[0018] Figure 7 This is a schematic diagram according to the third embodiment of the present disclosure;

[0019] Figure 8This is a schematic diagram of an electronic device used to implement the OFD document generation method of the embodiments of this disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] As OFD documents are gradually becoming more widespread, there is a lack of solutions in the relevant technologies for generating OFD documents based on document images.

[0022] To generate OFD documents, this disclosure provides the following embodiments.

[0023] Figure 1 This is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides an OFD document generation method, the method comprising:

[0024] 101. Obtain a document image, wherein the document image contains the target object.

[0025] 102. Perform layout analysis on the document image to obtain the layout information of the target object.

[0026] 103. Perform layout analysis on the document image based on the layout information to obtain the object information of the target object and determine the reading order of the target object.

[0027] 104. Based on the reading order of the target object, add the object information of the target object to a blank OFD document to obtain the target OFD document.

[0028] The document image refers to the image to be converted into an OFD document, which can be obtained by taking a photo or scanning a paper document.

[0029] A target object refers to a document's constituent elements, which can include one or more of the following: text objects, image objects, and path objects. A text object, also known as a text block, consists of one or more characters. Image objects can include one or more of the following: pictures, stamps, and formulas. Path objects include, for example, one or more of the following: curves, lines, and line segments.

[0030] Additionally, the document may contain tables. For tables, they can be split into line segments and text blocks, and then processed based on the processing methods for line segments and text blocks.

[0031] Layout information refers to the formatting and arrangement information of the target object, such as one or more of the following: columns, headers, footers, page numbers, etc.

[0032] Reading order refers to the order in which users (people) read. Generally speaking, users read from left to right and from top to bottom.

[0033] Different layout information can affect the reading order of the target object. Taking a layout using columns as an example... Figure 2 As shown, assuming the target object consists of four text blocks A, B, C, and D, different column settings will affect the reading order of these four text blocks. Specifically, as... Figure 2 The left-hand column layout shows the four text blocks in the following reading order: A, C, B, D. Figure 2 The four text blocks shown in the right-hand column are read in the order of A, B, C, and D.

[0034] Object information refers to relevant information about the target object, such as location information, content information, and category information. For target objects like images, text blocks, seals, and formulas, their location information can be represented by (x, y, w, h), where (x, y) are the coordinates of the top-left vertex of the rectangle containing the target object, and w and h are the width and height of the rectangle, respectively. For target objects that are line segments, the location information can include the coordinates of the two endpoints of the line segment. Category information indicates the category of the target object, such as text blocks or images. Content information is the specific content of the target object. For text blocks, this includes the character content and position of each individual character; for images, it's the image itself; for seals, it's the screenshot corresponding to the seal; and for formulas, it's the screenshot corresponding to the formula. For line segments, only the coordinates of the two endpoints can be recorded, and the line segment is drawn based on these coordinates during display.

[0035] A blank OFD document is a pre-built document that conforms to the OFD standard structure, but the content of the document is blank.

[0036] The target OFD document refers to the final OFD document generated based on the document image.

[0037] For ease of use, the target OFD document is copyable, meaning that the target object in the target OFD document can be copied.

[0038] For OFD documents, the copying order of target objects is consistent with the addition (or writing) order. Assuming the writing order of the four text blocks mentioned above is A, B, C, and D, they will be copied in that order. If the writing order is incorrect, out-of-order copying will occur, leading to errors in the copied content.

[0039] For example, targeting Figure 2 The correct order for adding the content shown on the left should be A, C, B, D. In other words, the correct order should be the reading order of these four text blocks. If the text blocks are not added in the reading order, such as A, B, C, D, the copying order will also be A, B, C, D, resulting in the content being copied in the same order (the normal order should be A, C, B, D).

[0040] To avoid copying content out of order, the corresponding object information is added to the blank OFD document based on the reading order of the target objects. This ensures that the copying order is correct and the copied content is accurate.

[0041] In this embodiment, by processing the document image, the object information of the target object in the document image is obtained. The object information is added to a blank OFD document to generate the target OFD document. This realizes the conversion of the document image into an OFD document, and the implementation process is relatively simple and efficient. In addition, since the object information of the target object is added based on the reading order of the target object, errors can be avoided when copying the target OFD document. Therefore, OFD documents can be generated accurately and efficiently.

[0042] To better understand the embodiments of this disclosure, the application scenarios provided by the embodiments of this disclosure will be described.

[0043] Figure 3 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure. For example... Figure 3 As shown, the system in this scenario may include a user terminal 301 and a server 302. The user terminal 301 may include, for example, a personal computer, a laptop, or a mobile device (such as a mobile phone). The server 302 may be a cloud server or a local server. The user terminal and the server may interact via a communication network, which may include, for example, a wired network and / or a wireless network.

[0044] Users can transmit document images to server 302 via user terminal 301. Server 302 converts the document images into copyable target OFD documents. After that, server 302 can send the target OFD document to user terminal 301 for display. Users can then perform operations such as copying the displayed target OFD document.

[0045] The above example of OFD document conversion (also known as OFD document restoration) using a server illustrates that if the user terminal has the corresponding conversion capability locally, the user terminal can also convert the document image into an OFD document.

[0046] like Figure 4 As shown, the overall architecture for converting a document image into an OFD document can include an image processing module and an OFD generation engine. The image processing module processes the document image to obtain object information and reading order of the target objects in the document image. The OFD generation engine adds the object information of the target objects to a blank OFD document based on the reading order of the target objects, thus obtaining the target OFD document.

[0047] like Figure 4 As shown, the image processing module mainly includes layout analysis and page layout analysis. For layout analysis, the document image can be input into a pre-trained layout analysis model, and the output of the model is the layout information of the target objects in the document image. For page layout analysis, a pre-trained page layout recognition model and an image detection algorithm can be used. The Hough transform algorithm is used as an example of an image detection algorithm. The Hough transform algorithm is a curve (including straight lines and line segments) detection algorithm that is widely used in many fields such as image analysis and pattern recognition. Taking line segment detection as an example, its basic principle is to transform the line segments in the image space to the parameter space, and by detecting the extreme points in the parameter space, determine the descriptive parameters of the line segment, thereby extracting the line segment from the image.

[0048] Page layout recognition models can identify the location and category information of image objects (such as pictures, seals, and formulas) and text objects (text blocks). For text blocks, page layout analysis can also use text recognition models to process the text blocks to obtain the character content and position of individual characters within the text block. Hough transform can be used to identify the position information of line segments.

[0049] Therefore, the image processing module can obtain the following object information:

[0050] For text blocks: the character content and position of a single character within the text block; additionally, the character content can be arranged in order based on the character position.

[0051] For line segments: the positional information of the line segment, specifically including the positional information of the two endpoints of the line segment;

[0052] For images, seals, and formulas: position information for images, seals, and formulas.

[0053] The OFD generation engine can write the above object information into a blank OFD document based on the reading order to obtain the target OFD document.

[0054] Specifically, you can add object information for text objects (such as text blocks) (character content and position of a single character in the text block), object information for image objects (such as pictures, seals, formulas) (position information and corresponding content information), and object information for graphic objects (such as line segments) (position information of the two endpoints of the line segment).

[0055] The structure of a blank OFD document is standard, such as... Figure 5 As shown, the standard OFD document structure includes: the main entry file (OFD.xml) and the document folder (Doc_N). Doc_N includes: the document root node (Document.xml), the document public resource index (PublicRes.xml), the document's own resource index (DocumentRes.xml), the digital signature folder (Signs), and the page folder (Pages). For a specific page (Page_N), it includes: the page content description file (Content.xml), the resource description file (PageRes.xml), and the page resource file (Res).

[0056] For each target object on a page, its object information is recorded in the corresponding page file. Specifically, for text blocks, the obtained object information includes: the character content and position of a single character, which are recorded in the page content description file (Content.xml). For images, seals, and formulas, the position information of these target objects is recorded in the page content description file (Content.xml), and the corresponding content (the image itself, the screenshot of the seal or formula) is recorded in the page resource file (Res). For line segments, the coordinates of the two endpoints of the line segment are recorded in the page content description file (Content.xml).

[0057] In conjunction with the above application scenarios, this disclosure also provides the following embodiments.

[0058] Figure 6 This is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides an OFD document generation method, the method comprising:

[0059] 601. Obtain a document image, wherein the document image contains the target object.

[0060] The document image refers to the image to be converted into an OFD document, which can be obtained by taking a photo or scanning a paper document.

[0061] The target object refers to the constituent elements of a document. This example uses the following as examples: text blocks (text objects), images, stamps, formulas (images, stamps, and formulas can be collectively referred to as image objects), and line segments (graphic objects).

[0062] 602. Perform layout analysis on the document image to obtain the layout information of the target object.

[0063] One approach is to use a pre-trained layout analysis model to perform layout analysis on the input document image, and output layout information.

[0064] Layout information refers to the formatting and arrangement information of the target object, such as one or more of the following: columns, headers, footers, page numbers, etc.

[0065] 603. Based on the layout information, divide the document image into multiple image blocks.

[0066] In this approach, target objects with the same layout information can be grouped into an image block, while different image blocks have different layout information.

[0067] For example, refer to Figure 2 Taking a layout with columns as an example, if the columns are horizontal, they can be divided into two image blocks, one containing text blocks A and C, and the other containing text blocks B and D. Alternatively, if the columns are vertical, they can also be divided into two image blocks, but one image block contains text blocks A and B, and the other contains text blocks C and D.

[0068] 604. For each of the plurality of image blocks, perform layout analysis on each image block to obtain object information of the target object.

[0069] After obtaining multiple image blocks, layout analysis can be performed on each image block to obtain object information of at least one target object in each image block. For example, after performing layout analysis on an image block composed of two text blocks A and C, object information of A and object information of C can be obtained.

[0070] The object information of the target object may include: location information. Accordingly, position detection processing can be performed on each image block to obtain the location information of the target object within each image block.

[0071] In this embodiment, by performing position detection processing on image blocks, the position information of the target object can be obtained, providing an accurate data foundation for accurately generating OFD documents.

[0072] Specifically, for text objects and / or image objects, a pre-trained layout recognition model can be used to process each image block to obtain the position information of the text objects and / or image objects.

[0073] The text object is a text block, and the image object can include one or more of the following: a picture, a seal, or a formula.

[0074] Page layout recognition models can be object detection models, such as the YOLO model or the PicoDet model. YOLO (youonly look once) identifies the category and location of objects in an image with only one look. YOLO combines candidate region recognition and object recognition into one stage, making it a single-stage object detection model with high accuracy and efficiency. PicoDet is also an object detection model, characterized by separable convolution, which reduces computation and storage space, making it more lightweight and faster while maintaining accuracy.

[0075] In this embodiment, the layout recognition model can accurately obtain the position information of target objects such as text blocks, images, seals, and formulas.

[0076] For graphic objects, graphic detection algorithms can be used to process each image block to obtain the position information of the graphic object.

[0077] The graphic object is, for example, a line segment, and the graphic detection algorithm is, for example, the Hough transform algorithm.

[0078] In this embodiment, the position information of graphic objects such as line segments can be accurately obtained through the graphic detection algorithm.

[0079] In addition, for text objects and / or image objects, object information may also include content information. Accordingly, when obtaining object information, the content information of the target object may also be obtained based on the location information of the target object.

[0080] In this embodiment, the corresponding content information is obtained based on the location information of the target object, thus obtaining accurate and comprehensive object information.

[0081] Specifically, for a text object, the content information includes: the character content and position of a single character in the text object; correspondingly, a text image of the text object can be obtained based on the position information of the text object, for example, by taking a screenshot of the image block where the text block is located based on the position information to obtain a text image of the area where the text block is located; a pre-trained text recognition model is used to perform text recognition on the text image to obtain the character content and position of the single character.

[0082] The text recognition model can be an Optical Character Recognition (OCR) model, such as the MaskOCR model. The MaskOCR model is a Transformer-based text recognition model, and the Transformer is a language model based on an encoder and a decoder.

[0083] In this embodiment, the text recognition model can accurately and efficiently obtain the character content and position of a single character in a text block, thereby accurately obtaining the object information of the text object.

[0084] For images, in addition to the location information mentioned above, the object information of an image can also include the image itself. In this case, the image itself can be extracted as the content information of the image based on its location information.

[0085] In this embodiment, the image itself is extracted as content information based on the image's location information, which can accurately obtain the object information of the image.

[0086] For a seal and / or formula, the content information is a screenshot corresponding to the seal and / or formula. At this time, based on the location information of the seal and / or formula, a screenshot can be taken of the image block where the seal and / or formula is located to obtain the screenshot corresponding to the seal and / or formula.

[0087] In this embodiment, screenshot processing based on the location information of the seal and / or formula can accurately and efficiently obtain the screenshot corresponding to the seal and / or formula, and can accurately obtain the object information of the seal and / or formula.

[0088] For target objects such as line segments, a line segment detection algorithm can be used to obtain the positional information of the line segment, including the positional information of the two endpoints of the line segment. That is, for each image block among the multiple image blocks, the line segment detection algorithm is used to perform layout analysis on each image block to obtain the object information of the target object.

[0089] Line segment detection algorithms include, for example, the Hough transform algorithm. The Hough transform algorithm is a curve (including straight lines and line segments) detection algorithm that is widely used in many fields such as image analysis and pattern recognition. Taking line segment detection as an example, its basic principle is to transform the line segments in the image space to the parameter space, and by detecting the extreme points in the parameter space, determine the descriptive parameters of the line segment, thereby extracting the line segment from the image.

[0090] In this embodiment, the line segment detection algorithm can accurately and efficiently obtain the object information of the graphic object (line segment).

[0091] 605. Based on the object information and layout information of the target object, determine the reading order of the target object.

[0092] The object information may include location information. The reading order of the target object can be determined based on the target object's location information and layout information.

[0093] Continuing with the column format as an example, since people typically read from left to right and from top to bottom, for left and right columns, the reading order of the target object in the left column is earlier than the reading order of the target object in the right column; within each column (left or right), the reading order of the target object at the top is earlier than the conventional order of the target object at the bottom. Therefore, refer to... Figure 2 For left-right columns, the reading order of each text block is A, C, B, D; similarly, for top-bottom columns, the reading order of each text block is A, B, C, D.

[0094] In this embodiment, the document image is divided into multiple image blocks based on layout information, and layout analysis is performed on each image block to accurately obtain the object information of the target object. Furthermore, based on the object information and layout information, the reading order of the target object can also be accurately obtained, thereby accurately generating the target OFD document.

[0095] 606. Based on the reading order of the target object, add the object information of the target object to a blank OFD document to obtain the target OFD document.

[0096] After determining the reading order of the target objects, the object information of each target object is written sequentially into a blank OFD document based on the reading order.

[0097] Specifically, for text objects such as text blocks, position information and content information (i.e., the character content and position of a single character in the text block) can be written into the page content description file of a blank OFD document; for image objects such as pictures, seals, and formulas, position information can be written into the page content description text, and content information can be written into the page resource file; for graphic objects such as line segments, position information can be written into the page content description file.

[0098] In this embodiment, by performing layout analysis on the document image, performing page layout analysis based on the layout information, determining the object information and reading order of the target object, and writing the object information sequentially based on the reading order, the document image can be accurately and efficiently converted into an OFD document.

[0099] Figure 7 This is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides an OFD document generation apparatus. Figure 7 As shown, the device 700 includes: an acquisition module 701, an analysis module 702, a determination module 703, and an addition module 704.

[0100] The acquisition module 701 is used to acquire a document image containing a target object; the analysis module 702 is used to perform layout analysis on the document image to obtain layout information of the target object; the determination module 703 is used to perform page layout analysis on the document image based on the layout information to obtain object information of the target object and determine the reading order of the target object; the addition module 704 is used to add the object information of the target object to a blank OFD document according to the reading order of the target object to obtain a target OFD document.

[0101] In this embodiment, by processing the document image, the object information of the target object in the document image is obtained. The object information is added to a blank OFD document to generate the target OFD document. This realizes the conversion of the document image into an OFD document, and the implementation process is relatively simple and efficient. In addition, since the object information of the target object is added based on the reading order of the target object, errors can be avoided when copying the target OFD document. Therefore, OFD documents can be generated accurately and efficiently.

[0102] In some embodiments, the determining module 703 is further configured to: divide the document image into multiple image blocks based on the layout information; perform layout analysis on each image block among the multiple image blocks to obtain object information of the target object; and determine the reading order of the target object based on the object information of the target object and the layout information.

[0103] In this embodiment, the document image is divided into multiple image blocks based on layout information, and layout analysis is performed on each image block to accurately obtain the object information of the target object. Furthermore, based on the object information and layout information, the reading order of the target object can also be accurately obtained, thereby accurately generating the target OFD document.

[0104] In some embodiments, the object information includes: location information; the determining module 703 is further configured to: perform location detection processing on each image block to obtain the location information of the target object.

[0105] In this embodiment, by performing position detection processing on image blocks, the position information of the target object can be obtained, providing an accurate data foundation for accurately generating OFD documents.

[0106] In some embodiments, the target object includes: a text object and / or an image object; the determining module 703 is further configured to: process each image block using a pre-trained layout recognition model to obtain the position information of the text object and / or the image object.

[0107] In this embodiment, the layout recognition model can accurately obtain the position information of target objects such as text blocks, images, seals, and formulas.

[0108] In some embodiments, the target object includes a graphic object; the determining module 703 is further configured to: use a graphic detection algorithm to process each image block to obtain the position information of the graphic object.

[0109] In this embodiment, the position information of graphic objects such as line segments can be accurately obtained through the graphic detection algorithm.

[0110] In some embodiments, the object information further includes content information; the determining module 703 is further configured to: obtain the content information of the target object based on the location information of the target object.

[0111] In this embodiment, the corresponding content information is obtained based on the location information of the target object, thus obtaining accurate and comprehensive object information.

[0112] In some embodiments, the target object includes: the text object; the content information includes: the character content and character position of a single character in the text object; the determining module 703 is further configured to: obtain a text image of the text object based on the position information of the text object; and perform text recognition on the text image using a pre-trained text recognition model to obtain the character content and character position of the single character.

[0113] In this embodiment, the text recognition model can accurately and efficiently obtain the character content and position of a single character in a text block, thereby accurately obtaining the object information of the text object.

[0114] In some embodiments, the target object includes: the image object, and the image object is a picture; the determining module 703 is further configured to: extract the picture itself as the content information of the picture based on the location information of the picture.

[0115] In this embodiment, the image itself is extracted as content information based on the image's location information, which can accurately obtain the object information of the image.

[0116] In some embodiments, the target object includes: the image object, and the image object includes: a seal and / or a formula; the content information includes: a screenshot corresponding to the seal and / or the formula; the determining module 703 is further configured to: perform screenshot processing on the image block where the seal and / or the formula is located based on the location information of the seal and / or the formula, so as to obtain a screenshot corresponding to the seal and / or the formula.

[0117] In this embodiment, screenshot processing based on the location information of the seal and / or formula can accurately and efficiently obtain the screenshot corresponding to the seal and / or formula, and can accurately obtain the object information of the seal and / or formula.

[0118] It is understood that the same or similar content in different embodiments of this disclosure can be referred to each other.

[0119] It is understood that the terms "first" and "second" in the embodiments of this disclosure are only used for distinction and do not indicate the degree of importance or the order of events.

[0120] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0121] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0122] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0123] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0124] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the OFD document generation method. For example, in some embodiments, the OFD document generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the OFD document generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the OFD document generation method by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0131] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0132] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating OFD documents, comprising: Acquire a document image, wherein the document image contains the target object; A pre-trained layout analysis model is used to perform layout analysis on the document image to obtain the layout information of the target object; Based on the layout information, the document image is divided into multiple image blocks. A pre-trained layout recognition model and image detection algorithm are used to perform layout analysis on each image block to obtain the object information of the target object. Based on the object information of the target object and the layout information, the reading order of the target object is determined. The output of the layout recognition model includes the location and category information of image and text objects, and the content information of the corresponding target object is obtained based on the location and category information; the output of the image detection algorithm includes the location information of the image object. Based on the reading order of the target objects, the object information of the target objects is added to a blank OFD document to obtain the target OFD document; The order in which the target objects are copied is the same as the order in which they are added.

2. The method according to claim 1, wherein, The object information includes: location information; The step of performing layout analysis on each image block to obtain object information of the target object includes: For each image block, position detection processing is performed on each image block to obtain the position information of the target object.

3. The method according to claim 2, wherein, The step of performing position detection processing on each of the image blocks to obtain the position information of the target object includes: If the target object includes text objects and / or image objects, a pre-trained layout recognition model is used to process each image block to obtain the positional information of the text objects and / or image objects; or, If the target object includes a graphic object, a graphic detection algorithm is used to process each image block to obtain the position information of the graphic object.

4. The method according to claim 3, wherein, If the target object includes a text object and / or an image object, the object information further includes: content information; The step of performing layout analysis on each image block to obtain object information of the target object also includes: Based on the location information of the target object, the content information of the target object is obtained.

5. The method according to claim 4, wherein, The step of obtaining the content information of the target object based on its location information includes: If the target object includes the text object, based on the location information of the text object, a text image of the text object is obtained; and a pre-trained text recognition model is used to perform text recognition on the text image to obtain the character content and character position of a single character, which are used as the content information of the text object; or, If the target object includes the image object, and the image object is a picture, the picture itself is extracted as the content information of the picture based on the picture's location information; or... If the target object includes the image object, and the image object includes: a seal and / or a formula, based on the location information of the seal and / or formula, a screenshot is taken of the image block where the seal and / or formula is located to obtain a screenshot corresponding to the seal and / or formula, which serves as the content information of the seal and / or formula.

6. An OFD document generation apparatus, comprising: The acquisition module is used to acquire a document image, wherein the document image contains a target object; The analysis module is used to perform layout analysis on the document image using a pre-trained layout analysis model to obtain the layout information of the target object; The determination module is used to divide the document image into multiple image blocks according to the layout information, and to perform layout analysis on each image block using a pre-trained layout recognition model and image detection algorithm to obtain the object information of the target object, and to determine the reading order of the target object based on the object information of the target object and the layout information. The output of the layout recognition model includes the location and category information of image and text objects, and the content information of the corresponding target object is obtained based on the location and category information; the output of the image detection algorithm includes the location information of the image object. An add module is used to add the object information of the target object to a blank OFD document according to the reading order of the target object, so as to obtain the target OFD document; The order in which the target objects are copied is the same as the order in which they are added.

7. The apparatus according to claim 6, wherein, The object information includes: location information; The determining module is further used for: For each image block, position detection processing is performed on each image block to obtain the position information of the target object.

8. The apparatus according to claim 7, wherein, The determining module is further used for: If the target object includes text objects and / or image objects, a pre-trained layout recognition model is used to process each image block to obtain the position information of the text objects and / or image objects; or, If the target object includes a graphic object, a graphic detection algorithm is used to process each image block to obtain the position information of the graphic object.

9. The apparatus according to claim 8, wherein, If the target object includes a text object and / or an image object, the object information further includes: content information; The determining module is also used for: Based on the location information of the target object, the content information of the target object is obtained.

10. The apparatus according to claim 9, wherein, The determining module is further used for: If the target object includes the text object, a text image of the text object is obtained based on the position information of the text object; and a pre-trained text recognition model is used to perform text recognition on the text image to obtain the character content and character position of a single character, which are used as the content information of the text object. or, If the target object includes the image object, and the image object is a picture, the picture itself is extracted as the content information of the picture based on the location information of the picture; or, If the target object includes the image object, and the image object includes: a seal and / or a formula, based on the location information of the seal and / or formula, a screenshot is taken of the image block where the seal and / or formula is located to obtain a screenshot corresponding to the seal and / or formula, which serves as the content information of the seal and / or formula.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images