Document auditing method, device and equipment and storage medium

Abstract

The invention discloses a document auditing method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: determining auditing content of each auditing item in a to-be-audited document; inputting each auditing item, the auditing requirement of each auditing item and the auditing content into an inference model, and determining the output result as an auditing result, the output result comprises re-auditing, determining target content in the to-be-audited document according to the detail determination model and taking the target content as auditing content, and the output result comprises image auditing, determining image description content in the to-be-audited document according to the image determination model and taking the image description content as the auditing content. Returning to execute the step of inputting each auditing item and the auditing requirement and auditing content of each auditing item into the inference model until an auditing result is determined; and determining an audit result of the to-be-audited document according to the audit result of each audit item. According to the technical scheme, intelligent auditing of the to-be-audited document containing the image is achieved based on the multiple models.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a document review method, apparatus, device and storage medium. Background Technology

[0002] With the development of large language models, there are more and more scenarios in the field of government service approval that use large language models to assist in the approval process.

[0003] In the current technology, large language model-assisted approval is still in the pilot stage, and it is usually based on the review model for policy interpretation and document review.

[0004] However, documents to be reviewed usually contain text and images, and a single review model is usually difficult to perform in-depth analysis of text and images at the same time, resulting in poor review results. Summary of the Invention

[0005] This invention provides a document review method, apparatus, device, and storage medium to achieve efficient and accurate document review.

[0006] In a first aspect, embodiments of the present invention provide a document review method, including:

[0007] In the document to be reviewed, determine the review content corresponding to each review item in the review checklist used for document review;

[0008] Each of the audit items, the audit requirements corresponding to each audit item, and the audit content are input into the inference model in the form of prompt words, so that the inference model audits the corresponding audit content according to the audit requirements corresponding to each audit item and obtains the output result.

[0009] When the output result is determined to include the review result, the output result is determined as the review result of the corresponding review item; when the output result is determined to include re-review, the target content corresponding to the review item is determined in the document to be reviewed based on the detailed determination model, and the target content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined; when the output result is determined to include image review, the image description content corresponding to the review item is determined in the document to be reviewed based on the image determination model, and the image description content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined.

[0010] The audit result of the document to be audited is determined based on the audit result of each audit item.

[0011] The technical solution of this invention provides a document review method, comprising: determining the review content corresponding to each review item in the review checklist used for document review in the document to be reviewed; inputting each review item, the review requirements corresponding to each review item, and the review content into an inference model in the form of prompt words, so that the inference model reviews the corresponding review content according to the review requirements corresponding to each review item, and obtains an output result; when it is determined that the output result includes a review result, determining the output result as the review result of the corresponding review item; when it is determined that the output result includes a re-review, determining the target content corresponding to the review item in the document to be reviewed based on a detailed determination model, and... The target content is used as the review content. The process returns to inputting each review item, the corresponding review requirements, and the review content into the inference model as prompts until the review result is determined. If the output result includes image review, the image description content corresponding to the review item in the document to be reviewed is determined based on the image determination model. This image description content is then used as the review content. The process returns to inputting each review item, the corresponding review requirements, and the review content into the inference model as prompts until the review result is determined. The review result of the document to be reviewed is determined based on the review results of each review item. The above technical solution first determines the similarity between each review item in the review checklist and the corresponding text blocks in the text to be reviewed, performs text matching between each review item and each text block, and determines the review content corresponding to each review item. This extracts the review content corresponding to each review item from the document to be reviewed. Then, by inputting each review item, its corresponding review requirements, and review content as prompts into the inference model, the inference model reviews the corresponding review content according to the review requirements of each review item. When determining the review result for a review item, the model outputs the review result; when it cannot determine the review result for a review item, it outputs "re-review"; when it determines that the review of a review item requires the participation of an image, it outputs "image review". This determines the output result of the inference model for each review item. Furthermore, by combining the output result of the inference model for the review item, the review content determined by the detail determination model or the image determination model, the review result of the review item can be determined. This achieves the review of review items by combining the inference model, the detail determination model, and the image determination model. Finally, based on the review results of each review item, the review result of the document to be reviewed is determined, realizing intelligent review of the document to be reviewed based on multiple intelligent models.

[0012] Furthermore, before determining the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited, the following steps are also included:

[0013] By performing text recognition on the document to be reviewed, the caption text and content text in the document to be reviewed can be obtained;

[0014] The caption text and the corresponding image information are identified as the image to be reviewed, and the content text is identified as the text to be reviewed.

[0015] Furthermore, after determining the content text as the text to be reviewed, the process also includes:

[0016] By segmenting the text to be reviewed, multiple text blocks corresponding to the text to be reviewed are obtained.

[0017] Furthermore, the audit content corresponding to each audit item in the audit checklist used for document auditing is determined in the document to be audited, including:

[0018] The text block and the review item are matched, and the text block with the highest matching degree is determined as the review content corresponding to the review item.

[0019] Furthermore, after determining the caption text and the corresponding image information as the image to be reviewed, the process also includes:

[0020] For each of the images to be reviewed, feature extraction is performed on the caption text and image information contained in the image to be reviewed to obtain the text features and image features corresponding to the image to be reviewed;

[0021] The image features and text features corresponding to the image to be reviewed are mapped to a unified semantic space to obtain the image features corresponding to the image to be reviewed;

[0022] An image feature library is constructed based on each of the images to be reviewed and the corresponding features of each image to be reviewed.

[0023] Further, the image determination model determines the image description content corresponding to the review item in the document to be reviewed, including:

[0024] Based on the review item, at least one review image corresponding to the review item is determined in the image feature library;

[0025] The review item and the review images corresponding to the review item are input into the image determination model, so that the image determination model can determine the target image corresponding to the review item in the review images corresponding to the review item and determine the image description content corresponding to the review item based on the target image.

[0026] Further, determining at least one review image corresponding to the review item in the image feature library based on the review item includes:

[0027] Feature extraction is performed on the review item to obtain the review feature corresponding to the review item;

[0028] Determine the similarity between the review feature and each of the image features to be reviewed in the image feature library, sort each image feature to be reviewed from largest to smallest based on the similarity, and determine the image to be reviewed corresponding to the top N image features to be reviewed as the review image corresponding to the review item, where N is a positive integer.

[0029] Secondly, embodiments of the present invention also provide a document review device, comprising:

[0030] The first determination module is used to determine the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited;

[0031] The review module is used to input each review item, the review requirements corresponding to each review item, and the review content into the inference model in the form of prompt words, so that the inference model reviews the corresponding review content according to the review requirements corresponding to each review item and obtains the output result.

[0032] The execution module is used to: when the output result includes an audit result, determine the output result as the audit result of the corresponding audit item; when the output result includes a re-audit, determine the target content corresponding to the audit item in the document to be audited based on the detail determination model, and use the target content as the audit content, then return to the execution process and input each audit item, the audit requirements corresponding to each audit item, and the audit content into the inference model in the form of prompt words until the audit result is determined; when the output result includes image audit, determine the image description content corresponding to the audit item in the document to be audited based on the image determination model, and use the image description content as the audit content, then return to the execution process and input each audit item, the audit requirements corresponding to each audit item, and the audit content into the inference model in the form of prompt words until the audit result is determined.

[0033] The second determining module is used to determine the audit result of the document to be audited based on the audit result of each audit item.

[0034] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0035] At least one processor; and a memory communicatively connected to said at least one processor;

[0036] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the document review method as described in any of the first aspects.

[0037] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the document review method as described in any of the first aspects.

[0038] Fifthly, this application provides a computer program product including computer instructions that, when executed on a computer, cause the computer to perform the document review method as provided in the first aspect.

[0039] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the document review device, or it may be packaged separately from the processor of the document review device; this application does not impose any limitations on this.

[0040] The descriptions of the second, third, fourth, and fifth aspects in this application can be referred to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referred to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0041] In this application, the name of the aforementioned document review device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0042] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a document review method provided in an embodiment of the present invention;

[0045] Figure 2This is a schematic diagram of the review process in a document review method provided by an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the structure of a document review device provided in an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0049] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0050] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0051] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0052] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc. Moreover, embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0053] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0054] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0055] Figure 1 This is a flowchart illustrating a document review method provided in an embodiment of the present invention. This embodiment is applicable to situations requiring document review, and the method can be executed by a document review device, such as... Figure 1 As shown, the specific steps include the following:

[0056] Step 110: Determine the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited.

[0057] When reviewing documents pending review, an audit checklist is required. The audit checklist includes all audit items that need to be reviewed, and the audit content can be understood as the text content corresponding to the audit items in the document pending review.

[0058] Specifically, the review checklist can first be broken down to obtain all the review items that need to be reviewed. These items are arranged in order, for example, the first review item is: ×××, the second review item is: ×××, and so on. Then, the review content corresponding to each review item can be determined in the document to be reviewed. Since the document to be reviewed usually includes text and image content, and the review methods for text and image content are usually different, to improve the review effect, it is necessary to review the text and image content separately. Therefore, the document to be reviewed needs to be split into text to be reviewed and images to be reviewed.

[0059] In one implementation, before determining the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited, the method further includes:

[0060] By performing text recognition on the document to be reviewed, the caption text and content text in the document to be reviewed are obtained; the caption text and the image information corresponding to the caption text are determined as the image to be reviewed, and the content text is determined as the text to be reviewed.

[0061] Specifically, text recognition can be performed on the document to be reviewed. Since caption text includes obvious caption text features, such as "Figure ×", text containing caption text features can be identified as caption text, and other text can be identified as content text. Furthermore, the content text can be identified as the text to be reviewed. Subsequently, the image information corresponding to the title text and caption text can be identified as the image to be reviewed.

[0062] In practical applications, the document to be reviewed can be an environmental impact assessment report. OCR text recognition technology can be used to convert the environmental impact assessment report into Markdown format, and then text recognition can be performed on the Markdown format environmental impact assessment report to extract the text and images to be reviewed in the environmental impact assessment report.

[0063] By performing text recognition on the document to be reviewed, the text and images to be reviewed in the document are extracted, thus achieving effective segmentation of the text and images contained in the document to be reviewed.

[0064] The text to be reviewed is usually quite long, making it difficult to directly determine the review content corresponding to the review items within the text. Therefore, further breakdown of the text is necessary. Documents to be reviewed typically have a multi-level structure. For example, a three-level document usually includes at least one chapter, each chapter includes at least one section, each section includes at least one unit, and each unit contains corresponding text content.

[0065] In one implementation, after determining the content text as the text to be reviewed, the method further includes:

[0066] By segmenting the text to be reviewed, multiple text blocks corresponding to the text to be reviewed are obtained.

[0067] Specifically, a segmentation strategy can be used to process the text to be reviewed. Specifically, rule-based structured segmentation techniques can be used to segment the text. When the document to be reviewed has a multi-level structure, the text includes multiple levels of headings. First, regular expressions can be used to determine the first-level headings. Based on these first-level headings, the text to be reviewed is segmented. The first-level headings can be numbered 1, 2, 3… and can be in multiple languages. Second, regular expressions are used to determine the second-level headings under each first-level heading. Based on these second-level headings, the text to be reviewed is segmented. The second-level headings can be numbered 1.1, 1.2, 3.3… and so on, until the segmented text content no longer contains headings, resulting in hierarchical Markdown text.

[0068] When the text content under the smallest heading in a hierarchical Markdown text still exceeds the input limit of the large language model, the text content under the smallest heading can be segmented. At this time, the text can be segmented according to the input limit of the large language model, and the text content under the smallest heading can be segmented into at least two text blocks to ensure that each text block does not exceed the input limit of the large language model.

[0069] It should be noted that during the process of segmenting the text to be reviewed, contextual information, such as title number and title path, can be preserved to determine the position of any text block in the text to be reviewed, and to facilitate the restoration of the text context of the text block.

[0070] By segmenting the text to be reviewed, the text is divided into multiple hierarchical Markdown text blocks that do not exceed the input limit of the large language model.

[0071] In one embodiment, after determining the caption text and the corresponding image information as the image to be reviewed, the method further includes:

[0072] For each of the images to be reviewed, feature extraction is performed on the caption text and image information contained in the image to be reviewed to obtain the text features and image features corresponding to the image to be reviewed; the image features and text features corresponding to the image to be reviewed are mapped to a unified semantic space to obtain the image features corresponding to the image to be reviewed; an image feature library is constructed based on each image to be reviewed and the image features corresponding to each image to be reviewed.

[0073] The Contrastive Language–Image Pretraining (CLIP) model can be used to vectorize images and text, that is, it can be used to extract features from image information and caption text to determine the vectorized image corresponding to the image information and the vectorized caption corresponding to the caption text.

[0074] Specifically, feature extraction can be performed on the image information and the caption text separately. The image information can be input into CLIP's Vision Transformer (ViT). ViT can extract the image features of the image information and encode the image information into multi-dimensional image features, such as 512 dimensions, to extract the high-dimensional visual feature vector corresponding to the image information. The caption text can also be input into CLIP's text encoder, such as a Transformer with a GPT-2 architecture. The Transformer with a GPT-2 architecture can extract the text features of the caption information and encode the caption text into multi-dimensional text features with the same dimensions as the multi-dimensional image features, such as 512 dimensions, to extract the high-dimensional text feature vector corresponding to the caption text.

[0075] Furthermore, by mapping multidimensional image features and multidimensional text features to a unified semantic space, the features of the image to be reviewed can be determined. Specifically, CLIP's cross-modal alignment capability can be used to map multidimensional image features and multidimensional text features to a unified semantic space, that is, to map multidimensional image features and multidimensional text features to a 512-dimensional space, so that semantically related image-text pairs are oriented close in space, while semantically unrelated image-text pairs have large differences in space, thus achieving geometric alignment of multidimensional image features corresponding to semantically related image information and multidimensional text features corresponding to caption text.

[0076] By mapping the image features corresponding to the image information contained in the image to be reviewed and the text features corresponding to the caption text to a unified semantic space, it is convenient to carry out bidirectional retrieval of "image → text" and "text → image", providing a traceable chain of evidence for image verification.

[0077] Of course, the multi-dimensional image features corresponding to the semantically aligned image information and the multi-dimensional text features corresponding to the caption text can be determined as the image features to be reviewed for the image information and the caption text corresponding to the image to be reviewed. Furthermore, an image feature library can be constructed based on each image to be reviewed and the image features corresponding to each image to be reviewed.

[0078] By performing a unified semantic space mapping on the image features obtained from extracting the image information contained in the image to be reviewed and the text features obtained from extracting the caption text contained in the image to be reviewed, the review features corresponding to the image to be reviewed are obtained, thus realizing the feature extraction of the image to be reviewed.

[0079] In one embodiment, step 110 may specifically include:

[0080] The text block and the review item are matched, and the text block with the highest matching degree is determined as the review content corresponding to the review item.

[0081] Specifically, text matching between review items and text blocks can be performed by determining the semantic similarity between the review items and each text block. The greater the semantic similarity between the review items and text blocks, the higher the matching degree between the review items and text blocks. Therefore, after determining the semantic similarity between the review items and each text block, the text block corresponding to the maximum semantic similarity can be identified as the text block with the highest matching degree, and then the text block with the highest matching degree can be identified as the review content corresponding to the review item.

[0082] In this embodiment of the invention, by performing text matching between each audit item in the audit list and each text block in the text to be audited corresponding to the content to be audited, the audit content corresponding to each audit item is determined, thereby realizing the extraction of the audit content corresponding to each audit item from the document to be audited.

[0083] Step 120: Input each of the audit items, the audit requirements corresponding to each audit item, and the audit content into the inference model as prompt words, so that the inference model audits the corresponding audit content according to the audit requirements corresponding to each audit item and obtains the output result.

[0084] The review details include the review requirements for each review item, which can be understood as the admission conditions that the review item must meet.

[0085] After identifying each audit item in the audit checklist, the corresponding audit requirements can be determined in the audit details based on each audit item in the audit checklist.

[0086] It should be noted that audit items and their corresponding audit requirements can also be presented in the review specification document. The review specification document can include audit items, audit points, and audit criteria. By searching the review specification document using keywords corresponding to each audit item, the audit requirements for each audit item can be determined. For example, if the audit item is whether the registered address of the construction unit and the actual construction address of the project are consistent, the audit answer is that they must be consistent. In practical applications, the audit requirements corresponding to each audit item can be extracted chapter by chapter from the review specification document using the Refine method.

[0087] After determining the audit requirements for each audit item, corresponding audit items can be constructed based on each audit item and its corresponding audit requirements, thus enabling the construction of audit items based on the audit items included in the audit list and the audit requirements corresponding to each audit item.

[0088] Figure 2 This is a schematic diagram of the review process in a document review method provided by an embodiment of the present invention, such as... Figure 2As shown, the review of documents to be reviewed is achieved through an inference model, a detail determination model, and an image determination model. The inference model can be a Supervisor Agent, the detail determination model can be a Refine Agent, and the image determination model can be a VL Agent.

[0089] Specifically, each audit item, its corresponding audit requirements, and audit content can be input into the inference model as prompt words. Specifically, for each audit item, the audit item, its corresponding audit requirements, and audit content can be filled into a preset prompt word template to obtain the corresponding prompt words, thus achieving automated construction of prompt words for each audit item. For example, if the audit item is: whether the registered address of the construction unit and the specific construction address of the project are consistent, the audit requirement is that the registered address of the construction unit and the specific construction address of the project must be consistent, and the audit content is ×××××, the prompt word can be constructed as: "Based on the audit content ×××××, determine whether the registered address of the construction unit and the specific construction address of the project are consistent, and determine the audit result as passed if the registered address of the construction unit and the specific construction address of the project are consistent; otherwise, determine the audit result as failed."

[0090] Furthermore, the inference model can be based on inputting the prompts corresponding to the review items. The inference model can review the review content contained in the prompts according to the review requirements. If the inference model can determine the review result, the output result includes the review result. If the inference model cannot determine the review result based on the review content contained in the prompts, it indicates that the review content contained in the prompts is unclear or incorrect, and the review of the review item cannot be achieved. Therefore, the output result can be determined as a re-review. If the inference model determines that the review requires image verification based on the review content contained in the prompts, it indicates that the review of the review item requires the participation of images. Therefore, the output result can be determined as image review, thus determining the output result corresponding to the review item.

[0091] In this embodiment of the invention, each audit item, the corresponding audit requirements, and the audit content are input into the inference model as prompts. The inference model then audits the corresponding audit content according to the audit requirements of each audit item. When the audit result corresponding to the audit item is determined, the audit result is output. When the audit result corresponding to the audit item cannot be determined, a re-audit is output. When the audit of the audit item requires the participation of an image, an image audit is output. This process determines the output result of the inference model for each audit item.

[0092] Step 130: When it is determined that the output result includes the review result, the output result is determined as the review result of the corresponding review item; when it is determined that the output result includes re-review, the target content corresponding to the review item is determined in the document to be reviewed based on the detail determination model, and the target content is used as the review content. The process returns to the inference model by inputting each review item, the review requirements corresponding to each review item, and the review content in the form of prompt words until the review result is determined; when it is determined that the output result includes image review, the image description content corresponding to the review item is determined in the document to be reviewed based on the image determination model, and the image description content is used as the review content. The process returns to the inference model by inputting each review item, the review requirements corresponding to each review item, and the review content in the form of prompt words until the review result is determined.

[0093] Specifically, when the output of the inference model includes the review result, the output can be determined as the review result of the corresponding review item. When the output of the inference model includes re-review, it indicates that the previously determined review content cannot determine the review result corresponding to the review item, and it is necessary to re-determine the target content corresponding to the review item in the text to be reviewed. Therefore, the detailed determination model can redetermine the target content corresponding to the review item in the text to be reviewed. The detailed determination model is also a large language model. The detailed determination model can determine the semantic similarity between the review item and each text block in the document to be reviewed, perform text matching between the review item and each text block, sort the text blocks from largest to smallest according to semantic similarity, and determine the top M text blocks as the target content corresponding to the review item in sequence. The target content is then used as the review content, and the process returns to the inference model by inputting the review item, the review requirements corresponding to the review item, and the review content as prompt words until the review result of the review item is determined.

[0094] The output of the inference model includes the following: When reviewing images, it indicates that the review of the review item requires the participation of images. Therefore, the model can determine the image description content corresponding to the review item in the document to be reviewed based on the image, and use the image description content as the review content. Then, it returns to the execution process, inputting the review item, the review requirements corresponding to the review item, and the review content into the inference model as prompt words until the review result is determined.

[0095] In one implementation, determining the image description content corresponding to the review item in the document to be reviewed based on an image recognition model includes:

[0096] Based on the review item, at least one review image corresponding to the review item is determined in the image feature library; the review item and each review image corresponding to the review item are input into the image determination model, so that the image determination model determines the target image corresponding to the review item in each review image corresponding to the review item and determines the image description content corresponding to the review item based on the target image.

[0097] Further, determining at least one review image corresponding to the review item in the image feature library based on the review item includes:

[0098] Feature extraction is performed on the review item to obtain the review feature corresponding to the review item; the similarity between the review feature and each of the image features to be reviewed in the image feature library is determined; based on the similarity, each image feature to be reviewed is sorted from largest to smallest; the image to be reviewed corresponding to the top N image features to be reviewed is determined as the review image corresponding to the review item, where N is a positive integer.

[0099] Specifically, firstly, feature extraction can be performed on the review items to obtain the review features corresponding to the review items. Specifically, high-dimensional text features of the review items can be extracted; for example, high-dimensional text features with the same dimension as the image to be reviewed can be extracted. Secondly, the similarity between the review features and the features of each image to be reviewed in the image feature library can be determined. Based on the similarity, the features of each image to be reviewed are sorted from largest to smallest. The images corresponding to the top N features in the sorting are determined as the review images corresponding to the review items, thus achieving the initial screening of the images corresponding to the review items.

[0100] For example, if the review item is "Review the project floor plan and confirm the following information: 1. Is there a residential area within 50 meters to the west, and is it correctly marked? 2. Is there a shopping mall within 60 meters to the south, and is it correctly marked? 3. Are these the only affected objects within a 200-meter radius?", then by determining the similarity between the review features corresponding to the review item and the features of each image to be reviewed in the image feature library, N review images corresponding to the review item can be determined. This allows for the identification of multiple review images required for the review item from the images to be reviewed.

[0101] Furthermore, the review items and their corresponding review images can be input into the image determination model. This model can be a visual model. Therefore, the review items and their corresponding review images can be input into the visual model, which can then determine the target image corresponding to each review item from among the review images. Specifically, it can sequentially compare the review items and their corresponding review images, identifying the images that match the review items as the target images. The visual model can also perform visual descriptions of the target images based on the review items, determining the image description content related to the review items, thus determining the image description content corresponding to the review items. This image description content can then be used as the review content, and the process can return to the inference model by inputting the review items, their corresponding review requirements, and the review content as prompts until the review result is determined.

[0102] In practical applications, the Supervisor Agent is used for unified orchestration, which pre-delegates image review to the VLA Agent and detailed review to the Refine Agent, thus achieving a closed loop of "coarse retrieval → fine reasoning → evidence backfilling".

[0103] It should be noted that after determining the audit content, target content, or target image corresponding to the audit item, it can be stored in the form of audit item + attribute + audit content, target content, or target image. The attribute should at least include the location, evidence fragment, and confidence level of the audit content, target content, or target image.

[0104] In this embodiment of the invention, the review result of the review item is determined by combining the output result of the reasoning model for the review item, the review content determined by the detail determination model or the image determination model, thereby realizing the review of the review item based on multiple intelligent models.

[0105] Step 140: Determine the audit result of the document to be audited based on the audit results of each audit item.

[0106] Specifically, after determining the audit results for each audit item, an audit report can be constructed based on each audit item and its corresponding audit results.

[0107] In addition, if there are any audit items whose audit results are still undetermined after the above audit, these audit items will be identified as items pending review, and the audit report will also include these items pending review.

[0108] In this embodiment of the invention, the review result of the document to be reviewed is determined according to the review result of each review item, thereby realizing intelligent review of the document to be reviewed based on multiple intelligent models.

[0109] The document review method provided in this embodiment of the invention includes: determining the review content corresponding to each review item in the review checklist used for document review in the document to be reviewed; inputting each review item, the review requirements corresponding to each review item, and the review content into an inference model in the form of prompt words, so that the inference model reviews the corresponding review content according to the review requirements corresponding to each review item, and obtains an output result; when it is determined that the output result includes a review result, determining the output result as the review result of the corresponding review item; when it is determined that the output result includes a re-review, determining the target content corresponding to the review item in the document to be reviewed based on a detailed determination model, and inputting the review content into an inference model in the form of prompt words. The target content is used as the review content. The process returns to the inference model by inputting each review item, the corresponding review requirements, and the review content as prompts until the review result is determined. If the output result includes image review, the image description content corresponding to the review item in the document to be reviewed is determined based on the image determination model, and the image description content is used as the review content. The process returns to the inference model by inputting each review item, the corresponding review requirements, and the review content as prompts until the review result is determined. The review result of the document to be reviewed is determined based on the review result of each review item. The above technical solution first determines the similarity between each review item in the review checklist and the corresponding text blocks in the text to be reviewed, performs text matching between each review item and each text block, and determines the review content corresponding to each review item. This extracts the review content corresponding to each review item from the document to be reviewed. Then, by inputting each review item, its corresponding review requirements, and review content as prompts into the inference model, the inference model reviews the corresponding review content according to the review requirements of each review item. When determining the review result for a review item, the model outputs the review result; when it cannot determine the review result for a review item, it outputs "re-review"; when it determines that the review of a review item requires the participation of an image, it outputs "image review". This determines the output result of the inference model for each review item. Furthermore, by combining the output result of the inference model for the review item, the review content determined by the detail determination model or the image determination model, the review result of the review item can be determined. This achieves the review of review items by combining the inference model, the detail determination model, and the image determination model. Finally, based on the review results of each review item, the review result of the document to be reviewed is determined, realizing intelligent review of the document to be reviewed based on multiple intelligent models.

[0110] Figure 3 This is a schematic diagram of a document review device provided in an embodiment of the present invention. This device is applicable to documents requiring review, improving review efficiency and accuracy. The device can be implemented through software and / or hardware and is generally integrated into electronic devices, such as computer equipment.

[0111] like Figure 3 As shown, the device includes:

[0112] The first determination module 310 is used to determine the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited;

[0113] The review module 320 is used to input each review item, the review requirements corresponding to each review item, and the review content into the inference model in the form of prompt words, so that the inference model reviews the corresponding review content according to the review requirements corresponding to each review item and obtains the output result.

[0114] The execution module 330 is configured to: when the output result includes an audit result, determine the output result as the audit result of the corresponding audit item; when the output result includes a re-audit, determine the target content corresponding to the audit item in the document to be audited based on the detail determination model, and use the target content as the audit content, and return to the execution process to input each audit item, the audit requirements corresponding to each audit item, and the audit content into the inference model in the form of prompt words until the audit result is determined; when the output result includes an image audit, determine the image description content corresponding to the audit item in the document to be audited based on the image determination model, and use the image description content as the audit content, and return to the execution process to input each audit item, the audit requirements corresponding to each audit item, and the audit content into the inference model in the form of prompt words until the audit result is determined.

[0115] The second determining module 340 is used to determine the audit result of the document to be audited based on the audit result of each audit item.

[0116] The document review device provided in this embodiment determines the review content corresponding to each review item in the review checklist used for document review in the document to be reviewed; it inputs each review item, the review requirements corresponding to each review item, and the review content into a reasoning model in the form of prompt words, so that the reasoning model reviews the corresponding review content according to the review requirements corresponding to each review item and obtains an output result; when it is determined that the output result includes a review result, the output result is determined as the review result of the corresponding review item; when it is determined that the output result includes a re-review, the detailed determination model determines the target content corresponding to the review item in the document to be reviewed, and sets the target content... The content is used as the review content. The process returns to the inference model by inputting each review item, the corresponding review requirements, and the review content as prompts until the review result is determined. If the output result includes image review, the image description content corresponding to the review item is determined in the document to be reviewed based on the image determination model, and the image description content is used as the review content. The process returns to the inference model by inputting each review item, the corresponding review requirements, and the review content as prompts until the review result is determined. The review result of the document to be reviewed is determined based on the review result of each review item. The above technical solution first determines the similarity between each review item in the review checklist and the corresponding text blocks in the text to be reviewed, performs text matching between each review item and each text block, and determines the review content corresponding to each review item. This extracts the review content corresponding to each review item from the document to be reviewed. Then, by inputting each review item, its corresponding review requirements, and review content as prompts into the inference model, the inference model reviews the corresponding review content according to the review requirements of each review item. When determining the review result for a review item, the model outputs the review result; when it cannot determine the review result for a review item, it outputs "re-review"; when it determines that the review of a review item requires the participation of an image, it outputs "image review". This determines the output result of the inference model for each review item. Furthermore, by combining the output result of the inference model for the review item, the review content determined by the detail determination model or the image determination model, the review result of the review item can be determined. This achieves the review of review items by combining the inference model, the detail determination model, and the image determination model. Finally, based on the review results of each review item, the review result of the document to be reviewed is determined, realizing intelligent review of the document to be reviewed based on multiple intelligent models.

[0117] Based on the above embodiments, the device further includes:

[0118] The recognition module is used to obtain the caption text and content text in the document to be reviewed by performing text recognition on the document to be reviewed; to determine the caption text and the image information corresponding to the caption text as the image to be reviewed, and to determine the content text as the text to be reviewed.

[0119] Based on the above embodiments, the identification module is further configured to:

[0120] After identifying the content text as the text to be reviewed, the text to be reviewed is segmented to obtain multiple text blocks corresponding to the text to be reviewed.

[0121] Based on the above embodiments, the first determining module 310 is specifically used for:

[0122] The text block and the review item are matched, and the text block with the highest matching degree is determined as the review content corresponding to the review item.

[0123] Based on the above embodiments, the identification module is further configured to:

[0124] After identifying the caption text and the corresponding image information as images to be reviewed, for each image to be reviewed, feature extraction is performed on the caption text and the image information contained in the image to be reviewed, to obtain the text features and image features corresponding to the image to be reviewed; the image features and text features corresponding to the image to be reviewed are mapped to a unified semantic space to obtain the image features corresponding to the image to be reviewed; and an image feature library is constructed based on each image to be reviewed and the image features corresponding to each image to be reviewed.

[0125] Based on the above embodiments, the execution module 330 is specifically used for:

[0126] Based on the review item, at least one review image corresponding to the review item is determined in the image feature library; the review item and each review image corresponding to the review item are input into the image determination model, so that the image determination model determines the target image corresponding to the review item in each review image corresponding to the review item and determines the image description content corresponding to the review item based on the target image.

[0127] In one embodiment, determining at least one review image corresponding to the review item in the image feature library based on the review item includes:

[0128] Feature extraction is performed on the review item to obtain the review feature corresponding to the review item;

[0129] Determine the similarity between the review feature and each of the image features to be reviewed in the image feature library, sort each image feature to be reviewed from largest to smallest based on the similarity, and determine the image to be reviewed corresponding to the top N image features to be reviewed as the review image corresponding to the review item, where N is a positive integer.

[0130] The document review device provided in this embodiment of the invention can execute the document review method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the document review method.

[0131] It is worth noting that in the above-described embodiments of the document review device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0132] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0133] like Figure 4 As shown, electronic device 4 is represented in the form of a general-purpose computing electronic device. The components of electronic device 4 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0134] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0135] Electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 4, including volatile and non-volatile media, removable and non-removable media.

[0136] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 As not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0137] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0138] Electronic device 4 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 4, and / or with any device that enables electronic device 4 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 4 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 4 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with electronic device 4 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0139] Processing unit 16 executes various functional applications and page displays by running programs stored in system memory 28, such as implementing the document review method provided in this embodiment of the invention, which includes:

[0140] In the document to be reviewed, determine the review content corresponding to each review item in the review checklist used for document review;

[0141] Each of the audit items, the audit requirements corresponding to each audit item, and the audit content are input into the inference model in the form of prompt words, so that the inference model audits the corresponding audit content according to the audit requirements corresponding to each audit item and obtains the output result.

[0142] When the output result is determined to include the review result, the output result is determined as the review result of the corresponding review item; when the output result is determined to include re-review, the target content corresponding to the review item is determined in the document to be reviewed based on the detailed determination model, and the target content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined; when the output result is determined to include image review, the image description content corresponding to the review item is determined in the document to be reviewed based on the image determination model, and the image description content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined.

[0143] The audit result of the document to be audited is determined based on the audit result of each audit item.

[0144] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the document review method provided in any embodiment of the present invention.

[0145] This invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements, for example, the document review method provided in this invention, which includes:

[0146] In the document to be reviewed, determine the review content corresponding to each review item in the review checklist used for document review;

[0147] Each of the audit items, the audit requirements corresponding to each audit item, and the audit content are input into the inference model in the form of prompt words, so that the inference model audits the corresponding audit content according to the audit requirements corresponding to each audit item and obtains the output result.

[0148] When the output result is determined to include the review result, the output result is determined as the review result of the corresponding review item; when the output result is determined to include re-review, the target content corresponding to the review item is determined in the document to be reviewed based on the detailed determination model, and the target content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined; when the output result is determined to include image review, the image description content corresponding to the review item is determined in the document to be reviewed based on the image determination model, and the image description content is used as the review content. The process then returns to the inference model, inputting each review item, the review requirements corresponding to each review item, and the review content as prompts until the review result is determined.

[0149] The audit result of the document to be audited is determined based on the audit result of each audit item.

[0150] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0151] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0152] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0153] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0155] Furthermore, the acquisition, storage, use, and processing of data in the technical solution of this invention all comply with relevant laws and regulations.

[0156] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A document review method, characterized in that, include: In the document to be reviewed, determine the review content corresponding to each review item in the review checklist used for document review; Each of the audit items, the audit requirements corresponding to each audit item, and the audit content are input into the inference model in the form of prompt words, so that the inference model audits the corresponding audit content according to the audit requirements corresponding to each audit item and obtains the output result. When it is determined that the output result includes the review result, the output result is determined as the review result of the corresponding review item; when it is determined that the output result includes re-review, the model determines the target content corresponding to the review item in the document to be reviewed based on the details, and uses the target content as the review content. Then, the model returns to input each review item, the review requirements corresponding to each review item, and the review content into the inference model in the form of prompt words until the review result is determined. When it is determined that the output result includes image review, the image determination model determines the image description content corresponding to the review item in the document to be reviewed, and uses the image description content as the review content. Then, the model returns to input each review item, the review requirements corresponding to each review item, and the review content as prompt words into the inference model until the review result is determined. The audit result of the document to be audited is determined based on the audit result of each audit item.

2. The document review method according to claim 1, characterized in that, Before determining the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited, the following is also included: By performing text recognition on the document to be reviewed, the caption text and content text in the document to be reviewed can be obtained; The caption text and the corresponding image information are identified as the image to be reviewed, and the content text is identified as the text to be reviewed.

3. The document review method according to claim 2, characterized in that, After determining the content text as the text to be reviewed, the process also includes: By segmenting the text to be reviewed, multiple text blocks corresponding to the text to be reviewed are obtained.

4. The document review method according to claim 3, characterized in that, In the document to be reviewed, determine the review content corresponding to each review item in the review checklist, including: The text block and the review item are matched, and the text block with the highest matching degree is determined as the review content corresponding to the review item.

5. The document review method according to claim 2, characterized in that, After determining the caption text and the corresponding image information as the image to be reviewed, the process also includes: For each of the images to be reviewed, feature extraction is performed on the caption text and image information contained in the image to be reviewed to obtain the text features and image features corresponding to the image to be reviewed; The image features and text features corresponding to the image to be reviewed are mapped to a unified semantic space to obtain the image features corresponding to the image to be reviewed; An image feature library is constructed based on each of the images to be reviewed and the corresponding features of each image to be reviewed.

6. The document review method according to claim 5, characterized in that, The image-based model determines the image description content corresponding to the review item in the document to be reviewed, including: Based on the review item, at least one review image corresponding to the review item is determined in the image feature library; The review item and the review images corresponding to the review item are input into the image determination model, so that the image determination model can determine the target image corresponding to the review item in the review images corresponding to the review item and determine the image description content corresponding to the review item based on the target image.

7. The document review method according to claim 6, characterized in that, Based on the review item, at least one review image corresponding to the review item is determined in the image feature library, including: Feature extraction is performed on the review item to obtain the review feature corresponding to the review item; Determine the similarity between the review feature and each of the image features to be reviewed in the image feature library, sort each image feature to be reviewed from largest to smallest based on the similarity, and determine the image to be reviewed corresponding to the top N image features to be reviewed as the review image corresponding to the review item, where N is a positive integer.

8. A document review device, characterized in that, include: The first determination module is used to determine the audit content corresponding to each audit item in the audit checklist used for document auditing in the document to be audited; The review module is used to input each review item, the review requirements corresponding to each review item, and the review content into the inference model in the form of prompt words, so that the inference model reviews the corresponding review content according to the review requirements corresponding to each review item and obtains the output result. An execution module is used to determine the output result as the audit result of the corresponding audit item when it is determined that the output result includes the audit result; When the output result is determined to include a re-review, the model determines the target content corresponding to the review item in the document to be reviewed based on the details, and uses the target content as the review content. Then, the model returns to the process of inputting each review item, the review requirements corresponding to each review item, and the review content into the inference model in the form of prompt words until the review result is determined. When it is determined that the output result includes image review, the image determination model determines the image description content corresponding to the review item in the document to be reviewed, and uses the image description content as the review content. Then, the model returns to input each review item, the review requirements corresponding to each review item, and the review content as prompt words into the inference model until the review result is determined. The second determining module is used to determine the audit result of the document to be audited based on the audit result of each audit item.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the document review method as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the document review method as described in any one of claims 1-7.

Images