Document identification method and device based on artificial intelligence, equipment and storage medium

Abstract

The invention provides a document recognition method and device based on artificial intelligence, equipment and a storage medium, and the method comprises the steps: determining a picture in a to-be-recognized document, carrying out the layout recognition of the picture, determining a layout analysis result, and carrying out the recognition of the to-be-recognized document when the layout analysis result has a text region. The text area is detected through the text detection model, the text position is determined, the text area is determined based on the layout analysis result, and then the text position is determined through detection, so that the accuracy of text position detection is improved; according to the method, the character position is subjected to character extraction, and the character extraction content is restored and typeset according to the character position, so that the original intention of the image can be reserved, and a user can more intuitively and better understand the character extraction content.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a document recognition method, apparatus, device and storage medium based on artificial intelligence. Background Art

[0002] Nowadays, in the field of printed circuit board (PCB) engineering, OCR technology has begun to be used for the automatic extraction of marking information, which has greatly improved the efficiency of the production process. However, although OCR technology has made certain progress in this field, it still faces challenges when dealing with complex images including mixed images and texts. Its recognition accuracy is low, and the recognized text content can only form a centralized text, which cannot correspond to the arrangement of the original image, which is not conducive to users' understanding of the recognized content. Summary of the invention

[0003] The embodiments of the present application provide a document recognition method, apparatus, device and storage medium based on artificial intelligence to solve at least one problem existing in the related art. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for document recognition based on artificial intelligence, comprising:

[0005] Determine the image in the document to be recognized;

[0006] Performing layout recognition on the image to determine a layout analysis result;

[0007] When the layout analysis result has a text area, detecting the text area by a text detection model to determine the text position;

[0008] The text position is subjected to text extraction, and the content of the text extraction is restored and typeset according to the text position.

[0009] In one implementation, performing layout recognition on the image and determining a layout analysis result includes:

[0010] Collect various types of Spec documents and determine labels corresponding to different content types in the images of the Spec documents, where the content types include images, tables, and single-line text;

[0011] Using the Spec document and the label as training data, training the YOLO neural network to obtain a layout analysis model;

[0012] The layout analysis model is used to perform layout recognition on the image to determine a layout analysis result, wherein the layout analysis result includes an image area and a text area, and the text area includes an area where a table and a single line of text are located.

[0013] In one embodiment, after determining the layout analysis result, the method further includes:

[0014] Performing a first noise reduction process on the light and shadow and the contour of the image area in the layout analysis result, and perspective-projecting the tilted sub-image in the image area into a front view to obtain a noise-reduced image area;

[0015] Performing a second noise reduction process on the background and the outline of the text area in the layout analysis result to obtain a text area after noise reduction;

[0016] The layout analysis result is updated according to the denoised image area and the denoised text area.

[0017] In one implementation, before detecting the text region using a text detection model, the method further includes:

[0018] Determining a number of text characters in the text area by using a projection algorithm of a text direction prediction model;

[0019] Each text character is matched with a preset character in the mirror library, and a text extension direction of the text area is determined according to the successfully matched target text character.

[0020] In one implementation, when the layout analysis result has a text area, detecting the text area using a text detection model to determine the text position includes:

[0021] When the layout analysis result has a text area, the text area is detected along the text extension direction by the text detection model to determine the text position of each text character in the text area, and the text position is represented by coordinates.

[0022] In one implementation, extracting text from the text position and restoring and typeset the extracted text content according to the text position includes:

[0023] Performing text extraction on the text position through a text extraction model to obtain text extraction content, wherein the text extraction content includes a plurality of the text characters;

[0024] Restore and typeset a plurality of the text characters according to the text positions.

[0025] In one embodiment, the method further comprises:

[0026] When the layout analysis result has a picture area, determining a picture position of the picture area;

[0027] The final layout is determined according to the image position and the restored layout result.

[0028] In a second aspect, an embodiment of the present application provides a document recognition device based on artificial intelligence, comprising:

[0029] A determination module, used to determine the image in the document to be identified;

[0030] An identification module, used to perform layout identification on the image and determine a layout analysis result;

[0031] A detection module, configured to detect the text area by using a text detection model to determine the text position when the layout analysis result has a text area;

[0032] The typesetting module is used to extract text from the text position and restore the extracted text content according to the text position.

[0033] In one embodiment, the detection module is further used for:

[0034] Performing a first noise reduction process on the light and shadow and the contour of the image area in the layout analysis result, and perspective-projecting the tilted image in the image area into a front view to obtain a noise-reduced image area;

[0035] Performing a second noise reduction process on the background and the outline of the text area in the layout analysis result to obtain a text area after noise reduction;

[0036] The layout analysis result is updated according to the denoised image area and the denoised text area.

[0037] In one embodiment, the detection module is further used for:

[0038] Determining a number of text characters in the text area by using a projection algorithm of a text direction prediction model;

[0039] Each text character is matched with a preset character in the mirror library, and a text extension direction of the text area is determined according to the successfully matched target text character.

[0040] In one embodiment, the typesetting module is further used for:

[0041] When the layout analysis result has a picture area, determining a picture position of the picture area;

[0042] The final layout is determined according to the image position and the restored layout result.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned embodiments.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the method in any one of the above-mentioned embodiments is implemented.

[0045] The beneficial effects of the above technical solution include at least:

[0046] By determining the image in the document to be identified, performing layout recognition on the image, and determining the layout analysis result, when the layout analysis result has a text area, the text area is detected by a text detection model to determine the text position, and the text area is determined based on the layout analysis result and then the text position is determined by detection, which is beneficial to improving the accuracy of text position detection; text extraction is performed at the text position, and the extracted text content is restored and typeset according to the text position, which is beneficial to retaining the original intention of the image and allowing users to understand the extracted text content more intuitively and better.

[0047] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0049] Figure 1 This is a schematic diagram of the steps of a document recognition method based on artificial intelligence according to an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a layout analysis result of an embodiment of the present application;

[0051] Figure 3 This is a schematic diagram of a picture area after noise reduction according to an embodiment of the present application;

[0052] Figure 4 A schematic diagram of some preset characters in an embodiment of the present application;

[0053] Figure 5This is a structural block diagram of a document recognition device based on artificial intelligence according to an embodiment of the present application;

[0054] Figure 6 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0056] Reference Figure 1 , a flowchart of a document recognition method based on artificial intelligence according to an embodiment of the present application is shown, and the document recognition method based on artificial intelligence may at least include steps S100-S400:

[0057] S100: Determine the image in the document to be identified.

[0058] S200: Perform layout recognition on the image and determine the layout analysis result.

[0059] S300: When the layout analysis result has a text area, the text area is detected by a text detection model to determine the text position.

[0060] S400: extract text from the text position, and restore and typeset the extracted text content according to the text position.

[0061] The technical solution of the embodiment of the present application determines the picture in the document to be identified, performs layout recognition on the picture, determines the layout analysis result, and when the layout analysis result has a text area, detects the text area through a text detection model to determine the text position, determines the text area based on the layout analysis result, and then detects and determines the text position, which is beneficial to improving the accuracy of text position detection; extracts text from the text position, and restores the layout of the extracted text content according to the text position, which is beneficial to retaining the original intention of the picture, so that users can understand the extracted text content more intuitively and better.

[0062] In one embodiment, it is possible to extract relevant information of the PCB industry in advance based on vertical industry application scenarios, such as taking PCB as an example, collect, annotate and train modeling data to obtain model service capabilities, and then engineer the recognition processing technology, develop application products, adapt the text recognition technology capabilities to different customer application scenarios, and develop corresponding service functions, so as to integrate and obtain a system that can use service functions to implement the method of this application. It should be noted that the system can be run through electronic control units, controllers, processors, etc. of terminals such as computers, mobile phones, tablets, and vehicle-mounted terminals, or through cloud servers; the system is a new generation of document recognition technology based on artificial intelligence based on the combination of OpenCV's OCR recognition algorithm and the self-developed algorithm of this application. This method helps users optimize and improve the entire document content extraction process, and uses open source algorithms plus vertical industry data sets such as the PCB industry to train proprietary algorithm models that meet the demands of vertical industries: such as document classification, document layout analysis, text direction prediction, text detection and text recognition models, to optimize the intermediate process of document recognition, and achieve accurate extraction of document text content.

[0063] In the embodiment of the present application, the Spec document of the PCB is used as the document to be identified for illustration. In some embodiments, it may be other documents. The document to be identified may include pictures, tables, and other text content (such as single-line text) other than the table form. It should be noted that the table and the single-line text can be extracted by OCR technology. The embodiment of the present application focuses on the recognition of the content in the picture. The picture can also have smaller pictures, tables and single-line text, so the picture in the document to be identified is determined.

[0064] In an embodiment of the present application, a dit-base-pcb document classification model is also provided. By setting folders such as form\form_pic\spec_pic\text\text_pic, corresponding sample data is collected as training data, which is obtained by training a neural network model, so that users can classify documents.

[0065] In one embodiment, step S200 includes steps S210-S230:

[0066] S210 , collecting various types of Spec documents, and determining labels corresponding to different content types in images of the Spec documents.

[0067] Optionally, various types of Spec documents are collected in advance, and images with images, tables, and single-line texts are extracted, and then labels corresponding to different content types such as images, tables, and single-line texts in these images are determined.

[0068] S220, using the Spec document and the label as training data, training the YOLO neural network to obtain a layout analysis model.

[0069] Optionally, the images of the Spec document are combined with the labels as training data to train the YOLO neural network to obtain a layout analysis model.

[0070] S230: Perform layout recognition on the image through the layout analysis model to determine the layout analysis result.

[0071] Optionally, the layout analysis model that has been trained is used to perform layout recognition on the image determined in S100 to determine a layout analysis result. Figure 2 As shown, the layout analysis result includes a picture area 201 and a text area. The picture area is the area of ​​smaller pictures (recorded as sub-pictures) contained in the picture, and the text area includes the area where the table 202 and the single-line text 203 are located. It should be noted that Figure 2 Only some of the contents are marked.

[0072] In the embodiment of the present application, the image area and the text area are analyzed through the layout analysis model, which facilitates the accurate cropping and division of the image area and the text area, and facilitates the subsequent different targeted processing of different areas.

[0073] In one implementation, step S230 may further include steps S240-S260:

[0074] S240: Perform a first noise reduction process on the light and shadow and the outline of the image area in the layout analysis result, and perspective-project the tilted sub-image in the image area into the front to obtain a noise-reduced image area.

[0075] Optionally, since the sub-images in the image area may have noise interference, which may affect the recognition effect, the light and shadow and contour of the image area in the layout analysis result are subjected to a first noise reduction process through a pre-trained ygg16-text-orientation text direction prediction model; and when a tilted sub-image is identified in the image area, the tilted sub-image in the image area is perspective-projected as a front face to obtain a noise-reduced image area, such as Figure 3 shown.

[0076] S250: Perform a second noise reduction process on the background and outline of the text area in the layout analysis result to obtain a text area after noise reduction.

[0077] Optionally, a second noise reduction process is performed on the background and outline of the text area in the layout analysis result through the text direction prediction model to obtain the text area after noise reduction to improve the accuracy of the direction prediction. It should be noted that the second noise reduction process includes but is not limited to shadow processing, edge enhancement, angle correction, curvature correction, erasing handwritten text, erasing seals, removing curtain patterns, image quality detection, removing watermarks, removing light spots, removing reflective points, processing resolution distortion, processing motion blur, processing frosted glass, etc. In some embodiments, the integrity can also be detected (large area occlusion, missing corners and edges of the certificate), and the user is reminded when an integrity problem is detected.

[0078] S260: Update the layout analysis result according to the denoised image area and the denoised text area.

[0079] Optionally, after determining the denoised image area and the denoised text area, determining the above layout analysis result according to the denoised image area and the denoised text area is beneficial to improving the accuracy of subsequent recognition.

[0080] In one implementation, the image area and the text area in the image may be further enhanced by an image enhancement algorithm to achieve faithful restoration of text content in different images at different resolutions, thereby enabling better text recognition.

[0081] In one embodiment, before step S300, steps S270-S280 are also included:

[0082] S270: Determine a number of text characters in the text area through a projection algorithm of a text direction prediction model.

[0083] Optionally, a projection algorithm of a text direction prediction model is used to determine individual characters in the text region, thereby obtaining a plurality of text characters.

[0084] S280: Match each text character with a preset character in the image library, and determine a text extension direction of the text area according to the successfully matched target text characters.

[0085] Optionally, several preset characters are configured in the image library, such as Figure 4 The figure shows a schematic diagram of some preset characters. By matching each text character with the preset characters in the image library one by one, the successfully matched target text characters are determined. According to the most directions of the successfully matched target text characters, for example, taking one of the characters as the starting point, assuming that the most matched target text characters are in the right direction, the right direction is the most direction at this time, thereby determining the text extension direction of the text area, that is, the above-mentioned right direction.

[0086] In one embodiment, in step S300, when the layout analysis result has a text region, the text region is detected along the text extension direction by the dbnet-pcb text detection model, so as to determine the text position of each text character in the text region, and the text position is represented by coordinates, such as (x, y). It should be noted that by determining the text region and further determining the text extension direction, and detecting the text region along the text extension direction by the text detection model, a more accurate text position can be obtained, which can reduce the burden of the text extraction model when performing subsequent text recognition, and is conducive to more efficient and more accurate recognition.

[0087] In one implementation, step S400 includes steps S410-S420:

[0088] S410. Perform text extraction on the text position through a text extraction model to obtain text extraction content.

[0089] Optionally, the text position is extracted by using a crnn-pcb text extraction model to obtain accurate text extraction content; wherein the text extraction content includes a number of text characters.

[0090] S420, restoring and typeset a plurality of text characters according to the text positions.

[0091] Optionally, a number of text characters are restored and typeset according to the text positions and displayed, so that users can read the content more clearly and conveniently, and the intention displayed by the original typesetting of the text characters is better restored.

[0092] In one implementation, the artificial intelligence-based document recognition method of the embodiment of the present application may also include step S500, when the layout analysis result has a picture area, determining the picture position of the picture area, and determining the final layout according to the picture position and the restored layout result.

[0093] In an embodiment of the present application, when the layout analysis result has a picture area in addition to a text area, the picture position of the picture area is determined based on the layout analysis result, and further based on the picture position and the restored layout result, the final layout with the text area and the picture area is obtained, and the layout of the picture is completely restored, retaining the intention of the original picture, which is conducive to users understanding the identified content better and more quickly.

[0094] Through the method of the embodiment of the present application, the document recognition technology based on artificial intelligence improved by OpenCV is applied to the Spec document parsing scenario of PCB industry customers, which can improve the efficiency and accuracy of text recognition and help PCB factories improve production efficiency and reduce costs; through the method of the present application, engineering personnel can obtain relevant information and take corresponding actions more quickly to improve the efficiency and quality of the factory production process. Specifically, after testing, the engineering parameter quotation was reduced from 4 to 6 hours to 4 to 6 minutes, the parameter recognition accuracy rate met 99%, and the order received to production preparation process can be improved by 80%.

[0095] Reference Figure 5 , shows a structural block diagram of a document recognition device based on artificial intelligence according to an embodiment of the present application, and the device may include:

[0096] A determination module, used to determine the image in the document to be identified;

[0097] The recognition module is used to perform layout recognition on the image and determine the layout analysis result;

[0098] A detection module, used for detecting the text area through a text detection model to determine the text position when the layout analysis result has a text area;

[0099] The typesetting module is used to extract text at the text position and restore the extracted text content according to the text position.

[0100] In one embodiment, the detection module is further configured to:

[0101] Performing a first noise reduction process on the light and shadow and the outline of the image area in the layout analysis result, and perspective-projecting the tilted image in the image area into the front to obtain a noise-reduced image area;

[0102] Performing a second noise reduction process on the background and the outline of the text area in the layout analysis result to obtain a text area after noise reduction;

[0103] The layout analysis result is updated according to the denoised image area and the denoised text area.

[0104] In one embodiment, the detection module is further configured to:

[0105] Determine a number of text characters in the text area by using a projection algorithm of a text direction prediction model;

[0106] Each text character is matched with a preset character in the image library, and the text extension direction of the text area is determined according to the successfully matched target text character.

[0107] In one embodiment, the typesetting module is further used to:

[0108] When the layout analysis result has a picture area, determining a picture position of the picture area;

[0109] Determine the final layout based on the image position and restored layout results.

[0110] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0111] Reference Figure 6 , shows a structural block diagram of an electronic device according to an embodiment of the present application, the electronic device includes: a memory 310 and a processor 320, the memory 310 stores instructions that can be run on the processor 320, and the processor 320 loads and executes the instructions to implement the document recognition method based on artificial intelligence in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0112] In one embodiment, the electronic device also includes a communication interface 330 for communicating with external devices and performing data exchange transmission. If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be interconnected through a bus and complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0114] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the artificial intelligence-based document recognition method provided in the above embodiment.

[0115] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0116] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0117] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced RISC machines (ARM) architecture.

[0118] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0119] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0122] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0123] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0124] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0125] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0126] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A document recognition method based on artificial intelligence, characterized in that: include: Determine the image in the document to be recognized; Performing layout recognition on the image to determine a layout analysis result; When the layout analysis result has a text area, detecting the text area by a text detection model to determine the text position; The text position is subjected to text extraction, and the content of the text extraction is restored and typeset according to the text position.

2. The document recognition method based on artificial intelligence according to claim 1, characterized in that: The performing layout recognition on the picture and determining the layout analysis result comprises: Collect various types of Spec documents and determine labels corresponding to different content types in the images of the Spec documents, where the content types include images, tables, and single-line text; Using the Spec document and the label as training data, training the YOLO neural network to obtain a layout analysis model; The layout analysis model is used to perform layout recognition on the image to determine a layout analysis result, wherein the layout analysis result includes an image area and a text area, and the text area includes an area where a table and a single line of text are located.

3. The document recognition method based on artificial intelligence according to claim 2, characterized in that: After determining the layout analysis result, the method further includes: Performing a first noise reduction process on the light and shadow and the contour of the image area in the layout analysis result, and perspective-projecting the tilted sub-image in the image area into a front view to obtain a noise-reduced image area; Performing a second noise reduction process on the background and the outline of the text area in the layout analysis result to obtain a text area after noise reduction; The layout analysis result is updated according to the denoised image area and the denoised text area.

4. The document recognition method based on artificial intelligence according to any one of claims 1 to 3, characterized in that: Before detecting the text area by using the text detection model, the following step is also included: Determining a number of text characters in the text area by using a projection algorithm of a text direction prediction model; Each text character is matched with a preset character in the mirror library, and a text extension direction of the text area is determined according to the successfully matched target text character.

5. The document recognition method based on artificial intelligence according to claim 4, characterized in that: When the layout analysis result has a text area, detecting the text area by using a text detection model to determine the text position includes: When the layout analysis result has a text area, the text area is detected along the text extension direction by the text detection model to determine the text position of each text character in the text area, and the text position is represented by coordinates.

6. The document recognition method based on artificial intelligence according to claim 5, characterized in that: The extracting text from the text position and restoring and typeset the extracted text content according to the text position includes: Performing text extraction on the text position through a text extraction model to obtain text extraction content, wherein the text extraction content includes a plurality of the text characters; Restore and typeset a plurality of the text characters according to the text positions.

7. The document recognition method based on artificial intelligence according to claim 6, characterized in that: The method further comprises: When the layout analysis result has a picture area, determining a picture position of the picture area; The final layout is determined according to the image position and the restored layout result.

8. A document recognition device based on artificial intelligence, characterized in that: include: A determination module, used to determine the image in the document to be identified; An identification module, used to perform layout identification on the image and determine a layout analysis result; A detection module, configured to detect the text area by using a text detection model to determine the text position when the layout analysis result has a text area; The typesetting module is used to extract text from the text position and restore the extracted text content according to the text position.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images