Document recognition method, intelligent dialogue method and related devices, equipment and media

Abstract

The present application discloses a document recognition method, an intelligent dialogue method, and related devices, equipment, and media, wherein the document recognition method includes: splitting the document to be recognized to obtain a plurality of pages to be recognized, and selecting a target recognizer from a plurality of document recognizers based on the document attributes of the document to be recognized; performing layout analysis on the page to be recognized to obtain an analysis result, and determining whether to call a formula detector based on the target recognizer, and recognizing the page to be recognized based on the target recognizer to obtain a recognition result; wherein the formula detector is used to detect the page to be recognized to obtain a detection result, and when the detection result includes a formula area, calling a formula parser to parse the formula area to obtain a parsing result; and obtaining a target recognition page based on the recognition result, the analysis result, and the parsing result associated with the detection result. The above scheme can take into account both document recognition efficiency and recognition accuracy.

Description

Technical Field

[0001] The present application relates to the field of retrieval technology, and in particular to a document recognition method, an intelligent dialogue method, and related devices, equipment, and media. Background Art

[0002] With the continuous development of information technology, electronic document recognition has extremely wide applications in many scenarios such as business office, research and study.

[0003] At present, existing document recognition methods, whether implemented through simple programs such as open source toolkits or through professional engines such as OCR engines, are difficult to balance recognition efficiency and recognition accuracy. In view of this, how to balance document recognition efficiency and recognition accuracy as much as possible has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a document recognition method, an intelligent dialogue method and related devices, equipment and media that can take into account both document recognition efficiency and recognition accuracy.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a document recognition method, including: splitting the document to be recognized to obtain several pages to be recognized, and selecting a target recognizer from several document recognizers based on the document attributes of the document to be recognized; performing layout analysis on the page to be recognized to obtain analysis results, and determining whether to call a formula detector based on the target recognizer, and recognizing the page to be recognized based on the target recognizer to obtain recognition results; wherein the formula detector is used to detect the page to be recognized to obtain detection results, and when the detection result includes a formula area, calling a formula parser to parse the formula area to obtain a parsing result; based on the recognition result, the analysis result and the parsing result associated with the detection result, a target recognition page that is consistent with the layout and content of the page to be recognized is obtained.

[0006] In order to solve the above technical problems, the first aspect of the present application provides an intelligent dialogue method, including: obtaining a first dialogue sentence input by a user; searching in a document library based on the first dialogue sentence to obtain a target retrieval sentence; wherein the document library contains a number of target documents, and the target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the document recognition method of the first aspect above; constructing a prompt instruction based at least on the first dialogue sentence and the target retrieval sentence; wherein the prompt instruction is used to instruct a large language model to respond to the first dialogue sentence with reference to the target retrieval sentence; sending the prompt instruction to the large language model, and obtaining a second dialogue sentence output by the large language model to respond to the first dialogue sentence.

[0007] In order to solve the above-mentioned technical problems, the third aspect of the present application provides a document recognition device, including: a recognition preparation module, a document processing module and a content integration module, the recognition preparation module is used to split the document to be recognized to obtain several pages to be recognized, and select a target recognizer from several document recognizers based on the document attributes of the document to be recognized; the document processing module is used to perform layout analysis on the page to be recognized to obtain analysis results, and determine whether to call a formula detector based on the target recognizer, and recognize the page to be recognized based on the target recognizer to obtain recognition results; wherein the formula detector is used to detect the page to be recognized to obtain detection results, and when the detection result includes a formula area, call the formula parser to parse the formula area to obtain a parsing result; the content integration module is used to obtain a target recognition page that is consistent with the layout and content of the page to be recognized based on the recognition result, the analysis result and the parsing result associated with the detection result.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides an intelligent dialogue device, including: a sentence acquisition module, a content retrieval module, an instruction construction module and a sentence generation module, the sentence acquisition module is used to obtain a first dialogue sentence input by a user; the content retrieval module is used to search in a document library based on the first dialogue sentence to obtain a target retrieval sentence; wherein the document library contains a number of target documents, the target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the document recognition device of the third aspect above; the instruction construction module is used to construct a prompt instruction based on at least the first dialogue sentence and the target retrieval sentence; wherein the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target retrieval sentence; the sentence generation module is used to send the prompt instruction to the large language model, and obtain the second dialogue sentence output by the large language model to respond to the first dialogue sentence.

[0009] In order to solve the above-mentioned technical problems, the fifth aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory storing program instructions, and the processor being used to execute the program instructions to implement the document recognition method in the above-mentioned first aspect, or the intelligent dialogue method in the above-mentioned second aspect.

[0010] In order to solve the above technical problems, the sixth aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the document recognition method of the first aspect above, or the intelligent dialogue method of the second aspect above.

[0011] The above scheme splits the document to be identified to obtain several pages to be identified, and selects a target identifier from several document identifiers based on the document attributes of the document to be identified, and then performs layout analysis on the page to be identified to obtain analysis results, and determines whether to call the formula detector based on the target identifier, and identifies the page to be identified based on the target identifier to obtain recognition results, and the formula detector is used to detect the page to be identified to obtain detection results, and when the detection result includes a formula area, the formula parser is called to parse the formula area to obtain a parsing result, so that based on the recognition result, the analysis result and the parsing result associated with the detection result, a target identification page that is consistent with the layout and content of the page to be identified is obtained. Therefore, on the one hand, it is possible to adaptively select a target identifier to identify the document to be identified according to the document attributes of the document to be identified. Compared with uniformly selecting the same document identifier regardless of the actual situation, it is possible to select a more suitable document identifier as much as possible for the document attributes. On the other hand, for formula recognition, combined with the selected target identifier, it is further adaptively selected whether to call the formula detector and the possible associated formula parser, which helps to make up for the performance shortcomings of the target identifier in formula recognition as much as possible. Therefore, both document recognition efficiency and recognition accuracy can be taken into account.

[0012] In addition, since the document recognizer, formula detector, and formula parser are independent of each other, compared to building the formula detector and formula parser into the document recognizer, each toolkit can focus on its own specific task, which helps to improve the flexibility of toolkit calling. If any toolkit needs to be adjusted, it can avoid involving other toolkits as much as possible, which helps to greatly shorten the adjustment cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flowchart of an embodiment of the document recognition method of the present application;

[0014] Figure 2a is a schematic diagram of a framework of an embodiment of a second target detection model;

[0015] Figure 2b It is a process diagram of an embodiment of the document recognition method of the present application;

[0016] Figure 3 This is a flow chart of an embodiment of the intelligent dialogue method of the present application;

[0017] Figure 4 It is a schematic diagram of the framework of an embodiment of a document recognition device of the present application;

[0018] Figure 5 It is a schematic diagram of the framework of an embodiment of the intelligent dialogue device of the present application;

[0019] Figure 6 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0020] Figure 7 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0021] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0022] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0023] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0024] See also Figure 1 , Figure 1 This is a flowchart of an embodiment of the document recognition method of the present application. Specifically, it may include the following steps:

[0025] Step S11: splitting the document to be identified to obtain a number of pages to be identified, and selecting a target identifier from a number of document identifiers based on the document attributes of the document to be identified.

[0026] In one implementation scenario, the document to be identified is an electronic document, which can be obtained by scanning, photocopying, taking a photo, etc., or it can also be obtained after performing an editing operation on an electronic device such as a computer or a mobile phone. As a possible example, take the text to be identified as a PDF document as an example (that is, the file format of the document to be identified is ".pdf"), the PDF document can be generated by scanning, photocopying, taking a photo, etc. of a paper document, or the PDF document can also be obtained by directly performing an editing operation using PDF-related software on the aforementioned electronic device, or it can also be obtained by performing an editing operation on the aforementioned electronic device using office software such as WORD, PPT, etc. and then converting it to a ".pdf" file format, without limitation here. Of course, the document to be identified is not limited to PDF documents, and other situations can be deduced by analogy, and examples are not given one by one here.

[0027] In one implementation scenario, after obtaining the document to be identified, the document to be identified can be split to obtain several pages to be identified, so that the document to be identified can be identified page by page in units. For example, if the document to be identified is an N-page document, it can be split into N pages to be identified. Different from the aforementioned implementation, as another possible example, each page of the document to be identified can also be parsed into a picture as a page to be identified. Of course, the above examples are only several possible examples in actual application, and do not limit other ways of splitting the pages to be identified.

[0028] In one implementation scenario, the document attributes may include at least one of whether it is editable and the number of columns, and the plurality of document identifiers may include a first identifier and a second identifier, the first identifier being applicable to single-column and editable document attributes, the second identifier being applicable to multi-column and / or non-editable document attributes, and the recognition speed of the first identifier being faster than the recognition speed of the second identifier. In the above manner, for different document attributes, document identifiers adapted to various document attributes are prepared in advance for use in any document recognition, which can improve the stability of document recognition as much as possible.

[0029] In a specific implementation scenario, still taking the document to be recognized as a PDF document as an example, the first recognizer may include but is not limited to pdfplumber, PyMuPDF, etc. It should be noted that pdfplumber and PyMuPDF are both open source toolkits, which have high parsing efficiency for single-column, editable PDF documents, and can reduce R&D costs and deployment costs as much as possible compared to private OCR engines.

[0030] In a specific implementation scenario, still taking the case where the document to be identified is a PDF document as an example, the second identifier may include but is not limited to an OCR engine. It should be noted that although the OCR engine is suitable for multi-column and / or non-editable PDF documents, it does not mean that it is not suitable for single-column, editable PDF documents. Rather, multi-column and / or non-editable PDF documents are more difficult to identify than single-column, editable PDF documents. In other words, since the OCR engine can identify multi-column and / or non-editable PDF documents, it can naturally also identify single-column, editable PDF documents. In other words, compared with the aforementioned pdfplumber and PyMuPDF, the OCR engine has the advantage of a wider range of applications, but its recognition speed is slower than that of the aforementioned pdfplumber and PyMuPDF.

[0031] It should be noted that the above examples are only possible examples of the first identifier and the second identifier when the document to be identified is a PDF document, and do not limit the specific type of the document identifier. When the document to be identified is in other file formats, the same can be applied, and no further examples are given here.

[0032] In one implementation scenario, taking the case where several document identifiers include the aforementioned first identifier and the second identifier as an example, in response to the document attribute including a single column and being editable, the first identifier among the several document identifiers can be selected as the target identifier, and in response to the document attribute including at least one of multiple columns and being non-editable, the second identifier among the several document identifiers can be selected as the target identifier. Exemplarily, if the document attribute includes multiple columns, the second identifier can be selected as the target identifier, or, if the document attribute includes being non-editable, the second identifier can be selected as the target identifier, or, if the document attribute includes multiple columns and being non-editable, the second identifier can be selected as the target identifier. It should be noted that the above examples are only possible examples of determining the target identifier according to the document attributes when several document identifiers include the second identifier and the second identifier. In other cases, the same can be applied, and examples are not given one by one here.

[0033] Step S12: Perform layout analysis on the page to be identified to obtain analysis results, and determine whether to call the formula detector based on the target identifier, and identify the page to be identified based on the target identifier to obtain identification results.

[0034] In the disclosed embodiment, the formula detector is used to detect the page to be identified to obtain a detection result, and when the detection result includes a formula area, the formula parser is called to parse the formula area to obtain a parsing result.

[0035] In an implementation scenario, the formula detector can specifically be a one-stage target detection model, and the one-stage target detection model can directly detect the formula area in the page to be identified. Exemplarily, the one-stage target detection model may include but is not limited to yolo_v8, yolo_v10, etc. It should be noted that the main features of YOLO are fast speed and high accuracy, and can achieve fast target detection in real-time scenarios. Specifically, a formula detection data set can be constructed first, and then the one-stage target detection model can be trained and fine-tuned accordingly. In the use stage, the page with identification can be input into the one-stage target detection model, and the page to be identified can be judged based on the detection results (whether there is a formula box bbox) to determine whether there is a formula and a formula area (i.e., a formula box bbox). Of course, in addition to YOLO, other deep learning models can also be used as one-stage target detection models, which are not limited here.

[0036] In one implementation scenario, the formula detector may be a two-stage target detection model, which first detects whether there is a formula in the page to be identified, and then detects the formula area in the page to be identified if a formula is detected. Figure 2a , Figure 2aSchematic diagram of the framework of an embodiment of a two-stage target detection model. Figure 2a As shown, the two-stage target detection model may specifically include a binary classification model and a formula detection network. The binary classification model is used to output the binary classification result of the page to be identified (i.e., the existence of a formula or the absence of a formula). The formula detection network is used to continue processing the page to be identified when the binary classification result is that a formula exists, and obtain the formula area in the page to be identified. It should be noted that the binary classification model may include but is not limited to convolutional neural networks such as mobilenet, and the formula detection network may be implemented based on the Transformer structure. The specific structures of the binary classification model and the formula detection network are not limited here.

[0037] In one implementation scenario, the formula parser may include but is not limited to: TrOCR, LevOCR, Nougat, etc. Of course, in addition to the above examples, the formula parser can also be a custom design. The specific selection of the formula parser is not limited here.

[0038] In one implementation scenario, the analysis results obtained by performing layout analysis on the page to be identified may include, but are not limited to: titles (such as main titles, subtitles, first-level titles, second-level titles, etc.), text, table annotations, illustration annotations, headers, footers, etc. In other words, the analysis results of the layout analysis may specifically include the layout attributes to which each text line in the page to be identified belongs, that is, each text line belongs to a title, or a text, or a table annotation, or an illustration annotation, or a header, or a footer, etc., so as to be used for subsequent restoration of a target identification page with the same layout and content as the page to be identified based on the analysis results of the layout analysis, and unlike the page to be identified, the target identification page supports character retrieval.

[0039] In one implementation scenario, still taking the example of several document identifiers including a first identifier and a second identifier, in response to the target identifier being the first identifier, it can be determined to call the formula detector, and in response to the target identifier being the second identifier, it can be determined whether to call the formula detector based on the recognition performance of the second identifier regarding formula recognition. It should be noted that the recognition performance may include the recognition accuracy of the formula at the two recognition levels of content recognition and location recognition. The above method determines whether to call the formula detector based on the recognition performance of the second identifier regarding formula recognition, which can maximize the accuracy of formula recognition while taking into account recognition efficiency.

[0040] In a specific implementation scenario, in response to the content recognition accuracy in the recognition performance meeting the content recognition requirements, it can be determined not to call the formula detector and the formula parser. That is to say, when the content recognition accuracy meets the content recognition requirements, the recognition result of the second identifier for the page to be identified can be directly used as the final recognition result of the entire page to be identified (including possible formulas). It should be noted that the content recognition requirements may include but are not limited to: the content recognition accuracy is higher than the first threshold (such as 95%, 97%, 99%, etc.). In the above method, when the content recognition accuracy in the recognition performance meets the content recognition requirements, it is determined not to call the formula detector and the formula parser, and the recognition result of the second identifier for the page to be identified can be directly used as the final recognition result of the entire page to be identified, which can avoid calling other toolkits.

[0041] In a specific implementation scenario, in response to the content recognition accuracy rate in the recognition performance not meeting the content recognition requirement and the position recognition accuracy rate meeting the position recognition requirement, it can be determined not to call the formula detector, and the recognition result of the formula position by the second recognizer is selected as the detection result, so that when the recognition result of the formula position by the second recognizer includes the formula area, the formula parser is continued to be called to parse the formula area to obtain the parsing result. In other words, when the content recognition accuracy rate does not meet the content recognition requirement and the position recognition accuracy rate meets the position recognition requirement, it can be considered that the recognition result of the formula position by the second recognizer is still available, so the recognition result of the formula position by the second recognizer can be adopted without calling the formula detector. On this basis, if the recognition result of the formula position by the second recognizer includes the formula area, the image data of the formula area can be directly cut out from the page to be recognized, and then the formula parser is called to parse the image data of the formula area to obtain the parsing result; on the contrary, if the recognition result of the formula position by the second recognizer does not include the formula area, the recognition result of the second recognizer on the page to be recognized can be directly used as the final recognition result of the entire page to be recognized (including the possible formula). It should be noted that the location identification requirements may include, but are not limited to: the location identification accuracy is higher than a second threshold (e.g., 95%, 97%, 99%, etc.). In the above method, in response to the content identification accuracy in the formula performance not meeting the content identification requirements and the location identification accuracy meeting the location identification requirements, it is determined not to call the formula detector, and then the recognition result of the formula location by the second identifier is selected as the detection result, so that when the recognition result of the formula location by the second identifier includes the formula area, the formula parser is continued to be mobilized to parse the formula area to obtain the parsing result, and the recognition result of the formula location by the second identifier can be reused, which helps to improve the efficiency of formula recognition and take into account the accuracy of formula recognition.

[0042] In a specific implementation scenario, in response to the fact that the content recognition accuracy in the recognition performance does not meet the content recognition requirements and the position recognition accuracy does not meet the position recognition requirements, it can be determined to call the formula detector, and accordingly, the formula detector can detect the page to be recognized to obtain a detection result, and when the detection result includes the formula area, the image data of the formula area in the page to be recognized can be extracted, and then the formula parser can be called to parse the image data of the formula area to obtain a parsing result. It should be noted that the specific contents of the content recognition requirements and the position recognition requirements can be referred to the aforementioned related descriptions, which will not be repeated here. In the above method, when both the content recognition accuracy and the position recognition accuracy do not meet the corresponding recognition requirements, directly calling the formula detector can improve the formula recognition accuracy as much as possible.

[0043] In one implementation scenario, after determining whether to call the formula detector and before using the target identifier to identify the page to be identified, in response to the detection result including the formula area, any one of the operations of erasing and replacing the formula area in the page to be identified can be performed to obtain a new page to be identified. It should be noted that the new page to be identified is sent to the target identifier for text recognition. On this basis, based on the page position of the formula area, the parsing result can be backfilled into the recognition result of the new recognition page by the target identifier to obtain a new recognition result, and then the new recognition result is regularized based on the analysis result to obtain the target recognition page. In the above method, by performing operations such as erasing and replacing on the formula area in the page to be identified, a new page to be identified is obtained, so that the parsing result is backfilled into the recognition result of the new page to be identified, and then the analysis result is used to regularize the layout to obtain the target recognition page, which can keep the layout and content of the target recognition page consistent with the page to be identified as much as possible.

[0044] In a specific implementation scenario, different from the above situation, as another possible example, when the detection result includes the absence of a formula area, the target identifier can be directly used to perform text recognition on the page to be recognized to obtain the recognition result. At this time, since there is no formula area, the analysis result associated with the detection result is essentially empty. On this basis, the recognition result can be directly regularized based on the analysis result to obtain the target recognition page.

[0045] In a specific implementation scenario, the aforementioned replacement operation may include but is not limited to: replacing with special characters (such as $$), replacing with images (such as images with the same background color as the page to be identified and the same size as the formula area), etc. The specific implementation of the replacement operation is not limited here.

[0046] Step S13: Based on the recognition result, the analysis result and the parsing result associated with the detection result, a target recognition page that is consistent with the layout and content of the page to be recognized is obtained.

[0047] In the embodiment of the present disclosure, when the detection result includes a formula area, the analysis result associated with the detection result includes the analyzed formula in the formula area, and when the detection result includes an area without a formula, the analysis result associated with the detection result is empty. For details, please refer to the above related description, which will not be repeated here.

[0048] As a possible example, see Figure 2b , Figure 2b Schematic diagram of the process of an embodiment of the document recognition method of the present application. Figure 2bAs shown, the document to be recognized can be split first to obtain several pages to be recognized, and at the same time, the first recognizer or the second recognizer can be selected as the target recognizer based on the document attributes of the document to be recognized. At the same time, the layout analyzer can be used to analyze the layout of the recognized page to obtain the analysis result. Exemplarily, when the document attributes include single column and editable, the first recognizer can be selected as the target recognizer, and when the document attributes include at least one of multi-column and non-editable, the second recognizer can be selected as the target recognizer. When the first recognizer is selected as the target recognizer, the formula detector can be used to detect the page to be recognized to obtain the detection result. When the detection result includes the formula area, the formula parser can be called to parse the formula area to obtain the parsing result, and the aforementioned analysis result, the recognition result of the page to be recognized by the first recognizer and the parsing result are integrated to obtain the target recognition page. Conversely, when the detection result does not include the formula area, the aforementioned analysis result and the recognition result of the page to be recognized by the first recognizer can be directly integrated to obtain the target recognition page. When the second recognizer is selected as the target recognizer, it can be determined whether to call the formula detector based on the recognition performance of the second recognizer on formula recognition. Specifically, when the content recognition requirements are met, the recognition results of the page to be recognized can be directly integrated based on the aforementioned analysis results and the second recognizer to obtain the target recognition page; when only the position recognition requirements are met, it can be determined first whether the recognition results of the second recognizer on the formula position of the page to be recognized include the formula area. If so, the formula parser can be called to parse the formula area to obtain the parsing result, so that the aforementioned analysis results, the recognition results of the second recognizer outside the formula area in the page to be recognized and the aforementioned parsing results can be combined to obtain the target recognition page. Otherwise, the recognition results of the page to be recognized can be directly combined with the aforementioned analysis results and the second recognizer to obtain the target recognition page. In the case where neither is met, the formula detector can be called to first detect the page to be recognized to obtain the detection result. If the detection result includes the formula area, the formula parser can continue to be called to parse the formula area to obtain the parsing result, so that the aforementioned analysis results, the recognition results of the second recognizer outside the formula area in the page to be recognized and the aforementioned parsing results can be combined to obtain the target recognition page. Otherwise, the recognition results of the page to be recognized can be directly combined with the aforementioned analysis results and the second recognizer to obtain the target recognition page.

[0049] The above scheme splits the document to be identified to obtain several pages to be identified, and selects a target identifier from several document identifiers based on the document attributes of the document to be identified, and then performs layout analysis on the page to be identified to obtain analysis results, and determines whether to call the formula detector based on the target identifier, and identifies the page to be identified based on the target identifier to obtain recognition results, and the formula detector is used to detect the page to be identified to obtain detection results, and when the detection result includes a formula area, the formula parser is called to parse the formula area to obtain a parsing result, so that based on the recognition result, the analysis result and the parsing result associated with the detection result, a target identification page that is consistent with the layout and content of the page to be identified is obtained. Therefore, on the one hand, it is possible to adaptively select a target identifier to identify the document to be identified according to the document attributes of the document to be identified. Compared with uniformly selecting the same document identifier regardless of the actual situation, it is possible to select a more suitable document identifier as much as possible for the document attributes. On the other hand, for formula recognition, combined with the selected target identifier, it is further adaptively selected whether to call the formula detector and the possible associated formula parser, which helps to make up for the performance shortcomings of the target identifier in formula recognition as much as possible. Therefore, both document recognition efficiency and recognition accuracy can be taken into account.

[0050] In addition, since the document recognizer, formula detector, and formula parser are independent of each other, compared to building the formula detector and formula parser into the document recognizer, each toolkit can focus on its own specific task, which helps to improve the flexibility of toolkit calling. If any toolkit needs to be adjusted, it can avoid involving other toolkits as much as possible, which helps to greatly shorten the adjustment cycle.

[0051] See also Figure 3 , Figure 3 This is a flow chart of an embodiment of the intelligent dialogue method of the present application. Specifically, it may include the following steps:

[0052] Step S31: Obtain the first dialogue sentence input by the user.

[0053] In an implementation scenario, the user may input the first dialogue sentence through text, voice, or other input methods, and the input method of the first dialogue sentence is not limited herein.

[0054] In an implementation scenario, taking the production and R&D scenario as an example, the first dialogue statement may include but is not limited to: My production requirement is XXXXX, please help me determine whether XX company's XX model equipment can meet the requirement; or, taking the business office scenario as an example, the first dialogue statement may include but is not limited to: XX company failed to deliver on time, please help me confirm whether I can request compensation based on the contract signed with XX company. Of course, the above examples are only possible situations in the actual application process, and other scenarios can be deduced by analogy, so I will not give examples one by one here.

[0055] Step S32: Searching in the document library based on the first dialogue sentence to obtain a target search sentence.

[0056] In the embodiment of the present disclosure, the document library contains several target documents, the target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the implementation method in the above-mentioned document recognition method embodiment. For details, please refer to the above-mentioned public embodiment, which will not be repeated here. It should be noted that the document library can be set according to the actual application. For example, in a business office scenario, the document library may include but is not limited to: documents related to the company's internal rules and regulations, documents related to the company's external contracts, etc., which are not limited here; or, taking the production and research and development scenario as an example, the document library may include but is not limited to: documents related to the company's own products, documents related to the products of upstream and downstream related manufacturers, etc., which are not limited here. Of course, the above examples are only possible situations in the actual application process, and other scenarios can be inferred by analogy, and no examples are given one by one here. In addition, since the target recognition page in the target document in the document library can support character retrieval, it is possible to directly search the target recognition page in each target document according to the first dialogue sentence to obtain the relevant sentence of the first dialogue sentence as the target retrieval sentence. For example, taking the aforementioned first dialogue sentence "My production requirement is XXXXX, please help me determine whether XX company's XX model equipment can meet the requirements" as an example, the target search sentence about "XX company's XX model equipment" can be retrieved in the document library; or, taking the aforementioned first dialogue sentence "XX company failed to deliver on time, please help me confirm whether I can ask for compensation based on the contract signed with XX company" as an example, the target search sentence about "delayed delivery clause in XX company's contract" can be retrieved in the document library. Of course, the above examples are only possible situations in actual application, and other scenarios can be deduced by analogy, so I will not give examples one by one here.

[0057] Step S33: construct a prompt instruction based on at least the first dialogue sentence and the target search sentence.

[0058] In the embodiment of the present disclosure, the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target search sentence. Exemplarily, an instruction template may be predefined, so that after obtaining the target search sentence, the first dialogue sentence and the target search sentence may be filled into the instruction template to obtain the prompt instruction. As a possible example, the instruction template may include but is not limited to the following:

[0059] Please answer the following questions based on the reference materials [fill in the specific content of the "target search statement"] [fill in the "first pair The specific content of the "sentence".

[0060] Taking the first dialogue sentence "My production requirement is XXXXX, please help me determine whether the XX model equipment of XX company can meet the requirement" as an example, the following prompt instructions can be constructed:

[0061] Please answer the following questions based on the reference material [Specific parameters of XX company's XX model equipment are as follows: ...] The production requirement is XXXXX. Please help me determine whether XX company’s XX model equipment can meet the requirements].

[0062] Or, taking the first dialogue sentence "XX Company failed to deliver on time, please help me confirm whether I can ask for compensation based on the contract signed with XX Company" as an example, the following prompt instruction can be constructed:

[0063] Please answer the following questions based on the reference material [Delayed delivery clause in the contract signed by XX Company: ...] The company failed to deliver on time. Please help me confirm whether I can claim compensation based on the contract signed with XX company].

[0064] Of course, the above examples are only possible situations in actual application. Other scenarios can be deduced by analogy, so I will not give examples one by one here.

[0065] Step S34: sending a prompt instruction to the large language model, and obtaining a second dialogue sentence output by the large language model to respond to the first dialogue sentence.

[0066] In one implementation scenario, the large language model may include but is not limited to open source large models such as LLAMA and Bloom; or, the large language model may be obtained by fine-tuning the parameters of an open source large model based on specific training data; or, the large language model may be custom designed. The specific source of the large language model is not limited here.

[0067] In an implementation scenario, after constructing the prompt instruction, the prompt instruction can be sent to the large language model to obtain the second dialogue sentence output by the large language model to respond to the first dialogue sentence. Taking the aforementioned first dialogue sentence "My production requirements are XXXXX, please help me determine whether the XX model equipment of XX company can meet the requirements" as an example, the second dialogue sentence "According to your production requirements, the XX model equipment of XX company has XX parameters..., so it can fully meet your requirements" can be obtained, or, taking the aforementioned first dialogue sentence "XX company failed to deliver on time, please help me confirm whether you can ask for compensation based on the contract signed with XX company" as an example, the second dialogue sentence "Because the contract signed with XX company clearly stipulates delayed delivery..., you can ask for compensation". Of course, the above examples are only possible situations in actual application, and other scenarios can be deduced by analogy, so no more examples are given here.

[0068] The above scheme obtains the first dialogue sentence input by the user, and searches in the document library based on the first dialogue sentence to obtain the target search sentence, and the document library contains several target documents, and the target recognition page in the target document supports character search, and the target recognition page is obtained by the implementation method in the above document recognition method embodiment, and then at least based on the first dialogue sentence and the target search sentence, a prompt instruction is constructed, and the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target search sentence, thereby sending the prompt instruction to the large language model, and obtaining the second dialogue sentence output by the large language model to respond to the first dialogue sentence. Since the target recognition page is obtained by the implementation method in the above document recognition method embodiment, the document accuracy and acquisition speed of the target document can be improved, and then the accuracy and speed of retrieving the target search sentence in the document library can be improved. Therefore, the response speed and response accuracy of the intelligent dialogue can be improved.

[0069] See also Figure 4 , Figure 4It is a schematic diagram of the framework of an embodiment of a document recognition device 40 of the present application. The document recognition device includes: a recognition preparation module 41, a document processing module 42 and a content integration module 43. The recognition preparation module 41 is used to split the document to be recognized to obtain a plurality of pages to be recognized, and select a target recognizer from a plurality of document recognizers based on the document attributes of the document to be recognized; the document processing module 42 is used to perform layout analysis on the page to be recognized to obtain analysis results, and determine whether to call a formula detector based on the target recognizer, and recognize the page to be recognized based on the target recognizer to obtain recognition results; wherein, the formula detector is used to detect the page to be recognized to obtain detection results, and when the detection result includes a formula area, the formula parser is called to parse the formula area to obtain a parsing result; the content integration module 43 is used to obtain a target recognition page that is consistent with the layout and content of the page to be recognized based on the recognition result, the analysis result and the parsing result associated with the detection result.

[0070] In the above scheme, the document recognition device 40 splits the document to be recognized to obtain several pages to be recognized, and selects a target recognizer from several document recognizers based on the document attributes of the document to be recognized, and then performs layout analysis on the page to be recognized to obtain analysis results, and determines whether to call the formula detector based on the target recognizer, and recognizes the page to be recognized based on the target recognizer to obtain recognition results, and the formula detector is used to detect the page to be recognized to obtain detection results, and when the detection result includes a formula area, the formula parser is called to parse the formula area to obtain a parsing result, so that based on the recognition result, the analysis result and the parsing result associated with the detection result, a target recognition page consistent with the layout and content of the page to be recognized is obtained. Therefore, on the one hand, it is possible to adaptively select a target recognizer to recognize the document to be recognized according to the document attributes of the document to be recognized. Compared with uniformly selecting the same document recognizer regardless of the actual situation, it is possible to select a more suitable document recognizer as much as possible according to the document attributes. On the other hand, for formula recognition, combined with the selected target recognizer, it is further adaptively selected whether to call the formula detector and the possible associated formula parser, which helps to make up for the performance shortcomings of the target recognizer in formula recognition as much as possible. Therefore, it is possible to take into account both document recognition efficiency and recognition accuracy. In addition, since the document recognizer, formula detector, and formula parser are independent of each other, compared with building the formula detector and formula parser into the document recognizer, each toolkit can focus on its own business, which helps to improve the flexibility of toolkit calling, and if any toolkit needs to be adjusted, it can avoid involving other toolkits as much as possible, which helps to greatly shorten the adjustment cycle.

[0071] In some disclosed embodiments, document properties include whether the document is editable and at least one of the number of columns, and several document identifiers include a first identifier and a second identifier. The first identifier is applicable to single-column and editable document properties, and the second identifier is applicable to multi-column and / or non-editable document properties, and the recognition speed of the first identifier is faster than that of the second identifier.

[0072] In some disclosed embodiments, the document to be recognized is a PDF document, the first recognizer includes at least one of pdfplumber and PyMuPDF, and the second recognizer is an OCR engine.

[0073] In some disclosed embodiments, the recognition preparation module 41 is specifically used to select a first identifier among several document identifiers as a target identifier in response to the document attributes including a single column and being editable; the document processing module 42 is specifically used to determine to call a formula detector in response to the target identifier being the first identifier.

[0074] In some disclosed embodiments, the recognition preparation module 41 is specifically used to select a second identifier among several document identifiers as a target identifier in response to the document attributes including at least one of multiple columns and non-editable; the document processing module 42 is specifically used to determine whether to call the formula detector based on the recognition performance of the second identifier regarding formula recognition in response to the target identifier being the second identifier.

[0075] In some disclosed embodiments, the document processing module 42 includes a first response submodule for determining not to call the formula detector and not to call the formula parser in response to the content recognition accuracy in the recognition performance satisfying the content recognition requirement.

[0076] In some disclosed embodiments, the document processing module 42 includes a second response submodule for determining not to call the formula detector in response to the fact that the content recognition accuracy in the recognition performance does not meet the content recognition requirements and the position recognition accuracy meets the position recognition requirements; selecting the recognition result of the formula position by the second identifier as the detection result, so as to continue to call the formula parser to parse the formula area to obtain the parsing result when the recognition result of the formula position by the second identifier includes the formula area.

[0077] In some disclosed embodiments, the document processing module 42 includes a third response submodule for determining to call the formula detector in response to the content recognition accuracy not meeting the content recognition requirement and the location recognition accuracy not meeting the location recognition requirement in the recognition performance.

[0078] In some disclosed embodiments, the document processing module 42 includes a fourth response submodule for, in response to the detection result including a formula area, performing any one of an operation of erasing and replacing on the formula area in the page to be identified, to obtain a new page to be identified; the content integration module 43 includes a backfill submodule for backfilling the parsing result into the recognition result of the target identifier for the new page to be identified based on the page position of the formula area, to obtain a new recognition result; the content integration module 43 includes a regularization submodule for regularizing the layout of the new recognition result based on the analysis result, to obtain a target recognition page.

[0079] In some disclosed embodiments, the formula detector is a one-stage target detection model, which directly detects the formula area in the page to be identified; or, the formula detector is a two-stage target detection model, which first detects whether there is a formula in the page to be identified, and then detects the formula area in the page to be identified when the existence of the formula is detected.

[0080] See also Figure 5 , Figure 5 : is a schematic diagram of a framework of an embodiment of an intelligent dialogue device 50 of the present application. The intelligent dialogue device 50 includes: a sentence acquisition module 51, a content retrieval module 52, an instruction construction module 53 and a sentence generation module 54, the sentence acquisition module 51 is used to acquire a first dialogue sentence input by a user; the content retrieval module 52 is used to search in a document library based on the first dialogue sentence to obtain a target retrieval sentence; wherein the document library contains a plurality of target documents, the target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the document recognition device in the above document recognition device embodiment; the instruction construction module 53 is used to construct a prompt instruction based on at least the first dialogue sentence and the target retrieval sentence; wherein the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target retrieval sentence; the sentence generation module 54 is used to send the prompt instruction to the large language model and obtain the second dialogue sentence output by the large language model to respond to the first dialogue sentence.

[0081] In the above scheme, the intelligent dialogue device 50 obtains the first dialogue sentence input by the user, and searches the document library based on the first dialogue sentence to obtain the target search sentence, and the document library contains a number of target documents, and the target recognition page in the target document supports character search, and the target recognition page is obtained by the document recognition device in the above document recognition device embodiment, and then at least based on the first dialogue sentence and the target search sentence, a prompt instruction is constructed, and the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target search sentence, thereby sending the prompt instruction to the large language model, and obtaining the second dialogue sentence output by the large language model to respond to the first dialogue sentence. Since the target recognition page is obtained by the document recognition device in the above document recognition device embodiment, the document accuracy and acquisition speed of the target document can be improved, and then the accuracy and speed of retrieving the target search sentence in the document library can be improved. Therefore, the response speed and response accuracy of the intelligent dialogue can be improved.

[0082] See also Figure 6 , Figure 6 1 is a schematic diagram of a framework of an embodiment of an electronic device 60 of the present application. The electronic device 60 includes a memory 61 and a processor 62 coupled to each other, the memory 61 stores program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above document recognition method embodiments, or the steps in the above intelligent dialogue method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. Specifically, the electronic device 60 may include but is not limited to: a smart phone, a tablet computer, a server, etc., which are not limited here.

[0083] Specifically, the processor 62 is used to control itself and the memory 61 to implement the steps in any of the above document recognition method embodiments, or the steps in the above intelligent dialogue method embodiments. The processor 62 can also be called a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with signal processing capabilities. The processor 62 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 62 can be implemented by an integrated circuit chip.

[0084] In the above scheme, the electronic device 60 can adaptively select the target identifier to identify the document to be identified according to the document attributes of the document to be identified. Compared with uniformly selecting the same document identifier regardless of the actual situation, it can select a more suitable document identifier as much as possible for the document attributes. On the other hand, for formula recognition, combined with the selected target identifier, it further adaptively selects whether to call the formula detector and the formula parser that may be associated, which helps to make up for the performance shortcomings of the target identifier as much as possible in formula recognition. Therefore, it can take into account both document recognition efficiency and recognition accuracy. In addition, since the document identifier, formula detector, and formula parser are independent of each other, compared with the formula detector and formula parser built into the document identifier, each toolkit can be separately focused on a special matter, which helps to improve the flexibility of toolkit calling, and if any toolkit needs to be adjusted, it can also avoid involving other toolkits as much as possible, which helps to greatly shorten the adjustment cycle. When applied to intelligent dialogue, since the target recognition page is obtained by the process steps in the above document recognition method embodiment, the document accuracy and acquisition speed of the target document can be improved, and then the accuracy and speed of retrieving the target search statement in the document library can be improved. Therefore, the response speed and accuracy of intelligent dialogue can be improved.

[0085] See also Figure 7 , Figure 7 The computer-readable storage medium 70 of the present application is a schematic diagram of a framework of an embodiment. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor, and the program instructions 71 are used to implement the steps in any of the above document recognition method embodiments or the steps in the above intelligent dialogue method embodiments.

[0086] In the above scheme, the computer-readable storage medium 70 can, on the one hand, adaptively select a target identifier to identify the document to be identified according to the document attributes of the document to be identified. Compared with uniformly selecting the same document identifier regardless of the actual situation, it can select a more suitable document identifier as much as possible for the document attributes. On the other hand, for formula recognition, in combination with the selected target identifier, it further adaptively selects whether to call the formula detector and the formula parser that may be associated, which helps to make up for the performance shortcomings of the target identifier in formula recognition as much as possible. Therefore, it is possible to take into account both document recognition efficiency and recognition accuracy. In addition, since the document identifier, formula detector, and formula parser are independent of each other, compared with building the formula detector and formula parser into the document identifier, each toolkit can be focused on a specific matter separately, which helps to improve the flexibility of toolkit calling, and if any toolkit needs to be adjusted, it can also avoid involving other toolkits as much as possible, which helps to greatly shorten the adjustment cycle. When applied to intelligent dialogue, since the target identification page is obtained by the process steps in the above document identification method embodiment, the document accuracy and acquisition speed of the target document can be improved, and then the accuracy and speed of retrieving the target search sentence in the document library can be improved. Therefore, the response speed and response accuracy of the intelligent dialogue can be improved.

[0087] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0088] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0093] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A document recognition method, characterized in that: include: Based on the splitting of the document to be identified, a plurality of pages to be identified are obtained, and based on the document attributes of the document to be identified, a target identifier is selected from a plurality of document identifiers; wherein the plurality of document identifiers include a first identifier and a second identifier, the first identifier is applicable to single-column and editable document attributes, and the second identifier is applicable to multi-column and / or non-editable document attributes; Performing layout analysis on the page to be identified to obtain an analysis result, and determining whether to call a formula detector based on the target identifier, and identifying the page to be identified based on the target identifier to obtain an identification result; wherein the formula detector is used to detect the page to be identified to obtain a detection result, and when the detection result includes a formula area, calling a formula parser to parse the formula area to obtain a parsing result; Based on the recognition result, the analysis result and the parsing result associated with the detection result, a target recognition page having the same layout and content as the page to be recognized is obtained; Wherein, determining whether to call a formula detector based on the target identifier includes: In response to the target identifier being the first identifier, determining to call the formula detector; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance of the second identifier meeting the content recognition requirement, determining not to call the formula detector and not to call the formula parser; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance not satisfying the content recognition requirement and the position recognition accuracy rate satisfying the position recognition requirement, determining not to call the formula detector; selecting the recognition result of the formula position by the second identifier as the detection result, so as to continue to call the formula parser to parse the formula area and obtain the parsing result when the recognition result of the formula position by the second identifier includes the formula area; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance not satisfying the content recognition requirement and the position recognition accuracy rate not satisfying the position recognition requirement, it is determined to call the formula detector.

2. The method according to claim 1, characterized in that The document attribute includes at least one of whether the document is editable and the number of columns, and a recognition speed of the first identifier is faster than a recognition speed of the second identifier.

3. The method according to any one of claims 1 to 2, characterized in that: The step of selecting a target identifier from a plurality of document identifiers based on the document attribute of the document to be identified comprises: In response to the document attribute comprising a single column and being editable, a first identifier among the plurality of document identifiers is selected as the target identifier.

4. The method according to any one of claims 1 to 2, characterized in that: The step of selecting a target identifier from a plurality of document identifiers based on the document attribute of the document to be identified comprises: In response to the document attribute including at least one of multiple columns and non-editable, a second identifier among the plurality of document identifiers is selected as the target identifier.

5. The method according to any one of claims 1 to 2, characterized in that: After determining whether to call a formula detector based on the target identifier and before identifying the page to be identified based on the target identifier to obtain an identification result, the method further includes: In response to the detection result including the formula area, performing any one of an operation of erasing and replacing the formula area in the page to be identified to obtain a new page to be identified; The step of obtaining a target recognition page that is consistent with the layout and content of the page to be recognized based on the recognition result, the analysis result, and the parsing result associated with the detection result includes: Based on the page position of the formula area, the parsing result is backfilled into the recognition result of the target identifier for the new page to be recognized, so as to obtain a new recognition result; The new recognition result is layout-regulated based on the analysis result to obtain the target recognition page.

6. The method according to any one of claims 1 to 2, characterized in that: The formula detector is a one-stage target detection model, and the one-stage target detection model directly detects the formula area in the page to be identified; Alternatively, the formula detector is a two-stage target detection model, which first detects whether there is a formula in the page to be identified, and then detects the formula area in the page to be identified when the existence of the formula is detected.

7. An intelligent dialogue method, characterized in that: include: Obtain the first dialogue sentence input by the user; Based on the first dialogue statement, a search is performed in a document library to obtain a target search statement; wherein the document library contains a plurality of target documents, a target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the document recognition method according to any one of claims 1 to 6; constructing a prompt instruction based at least on the first dialogue sentence and the target search sentence; wherein the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target search sentence; The prompt instruction is sent to the large language model, and a second dialogue sentence output by the large language model to respond to the first dialogue sentence is obtained.

8. A document recognition device, characterized in that: include: A recognition preparation module, configured to split the document to be recognized to obtain a plurality of pages to be recognized, and select a target recognizer from a plurality of document recognizers based on the document attributes of the document to be recognized; wherein the plurality of document recognizers include a first recognizer and a second recognizer, wherein the first recognizer is applicable to single-column and editable document attributes, and the second recognizer is applicable to multi-column and / or non-editable document attributes; The document processing module is used to perform layout analysis on the page to be identified to obtain an analysis result, and determine whether to call a formula detector based on the target identifier, and identify the page to be identified based on the target identifier to obtain an identification result; wherein the formula detector is used to detect the page to be identified to obtain a detection result, and when the detection result includes a formula area, call a formula parser to parse the formula area to obtain a parsing result; A content integration module, for obtaining a target recognition page that is consistent with the layout and content of the page to be recognized based on the recognition result, the analysis result and the parsing result associated with the detection result; Wherein, determining whether to call a formula detector based on the target identifier includes: In response to the target identifier being the first identifier, determining to call the formula detector; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance of the second identifier meeting the content recognition requirement, determining not to call the formula detector and not to call the formula parser; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance not satisfying the content recognition requirement and the position recognition accuracy rate satisfying the position recognition requirement, determining not to call the formula detector; selecting the recognition result of the formula position by the second identifier as the detection result, so as to continue to call the formula parser to parse the formula area and obtain the parsing result when the recognition result of the formula position by the second identifier includes the formula area; In response to the target identifier being the second identifier and the content recognition accuracy rate in the recognition performance not satisfying the content recognition requirement and the position recognition accuracy rate not satisfying the position recognition requirement, it is determined to call the formula detector.

9. An intelligent dialogue device, characterized in that: include: A sentence acquisition module, used for acquiring a first dialogue sentence input by a user; a content retrieval module, configured to search in a document library based on the first dialogue sentence to obtain a target retrieval sentence; wherein the document library contains a plurality of target documents, a target recognition page in the target document supports character retrieval, and the target recognition page is obtained by the document recognition device according to claim 8; an instruction construction module, configured to construct a prompt instruction based at least on the first dialogue sentence and the target search sentence; wherein the prompt instruction is used to instruct the large language model to respond to the first dialogue sentence with reference to the target search sentence; A sentence generation module is used to send the prompt instruction to the large language model and obtain a second dialogue sentence output by the large language model to respond to the first dialogue sentence.

10. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the document recognition method described in any one of claims 1 to 6, or the intelligent dialogue method described in claim 7.

11. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the document recognition method described in any one of claims 1 to 6, or the intelligent dialogue method described in claim 7.

Images