Information extraction method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of information extraction, and in particular to an information extraction method and device, electronic equipment and a storage medium, which comprise the following steps: processing a demonstration document to obtain a target document image; performing region detection on the target document image to extract a question and answer region in the target document image; extracting the text content of the question and answer region to obtain initial text; and performing structured processing on the initial text by using a trained text structuring model to obtain structured target text, wherein the text structuring model is constructed based on a pre-trained large language model, and the pre-trained large language model is adjusted by a low-rank adaptation technology. Thus, by adjusting the model by the low-rank adaptation technology, the model can be adapted to different question and answer formats by optimizing only a small number of low-rank parameters, the model does not need to be trained from scratch, the computing cost is greatly reduced, and the document structured information can be efficiently and accurately obtained from the demonstration document.

Description

Technical Field

[0001] This application relates to the field of information extraction technology, and in particular to an information extraction method, apparatus, electronic device and storage medium. Background Technology

[0002] Presentation documents are important carriers of information, typically containing various types of page areas, such as title areas, question and answer areas, etc. When users need to use the text information contained in the presentation document, information extraction is required. However, due to the diverse layouts and unstructured content of presentation documents, the efficiency of information extraction in real-world scenarios is very low. Even when text information is extracted, it is often in an unstructured form, making it difficult to use directly.

[0003] Therefore, how to efficiently and accurately extract structured information from presentation documents is a problem that needs to be solved. Summary of the Invention

[0004] This application provides an information extraction method, apparatus, electronic device, and storage medium that can efficiently and accurately extract structured information from presentation documents.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, an information extraction method is provided, including: The presentation document is processed to obtain the target document image; Region detection is performed on the target document image to extract the question-and-answer region in the target document image; The text content of the question-and-answer area is extracted to obtain the initial text; The initial text is processed using a trained text structuring model to obtain a structured target text. The text structuring model is built based on a pre-trained large language model, and the pre-trained large language model is adjusted using low-rank adaptation techniques.

[0006] In the above solution, the text structuring model built based on the pre-trained large language model can quickly identify the text content in the initial text. At the same time, by adjusting the large language model through low-rank adaptation technology, only a few low-rank parameters need to be optimized to adapt to the question-and-answer formats of different scenarios. There is no need to train the large language model from scratch, which greatly reduces the computational cost and can efficiently and accurately obtain document structure information from the demonstration document.

[0007] Optionally, before using a trained text structuring model to perform structuring processing on the initial text to obtain the structured target text, the method further includes: Obtain the low-rank decomposition matrix constructed using the dot product attention mechanism; The low-rank decomposition matrix is ​​added to each converter layer of the pre-trained large language model; The parameters of the low-rank decomposition matrix in the large language model are trained to obtain the text structure model.

[0008] In the above scheme, by obtaining the low-rank decomposition matrix constructed by the dot product attention mechanism and adding it to each converter layer of the pre-trained large language model, the attention mechanism of the model can be optimized, enabling the model to focus on key information when processing text and improve the accuracy of information extraction.

[0009] Optionally, obtaining the low-rank decomposition matrix constructed through the dot product attention mechanism includes: Construct context vectors; The context vector is subjected to average pooling to obtain a feature matrix of a set dimension; An attention matrix is ​​constructed based on the diagonal elements of the feature matrix of the defined dimension. The product of the first matrix, the attention matrix, and the second matrix is ​​taken as the low-rank decomposition matrix. Wherein, the dimension of the first matrix is ​​the product of the feature vector dimension of the converter layer and the set rank, and the dimension of the second matrix is ​​the product of the set rank and the feature vector dimension, wherein the value of the feature vector dimension is greater than the value of the set rank.

[0010] In the above scheme, the rank of the matrix is ​​reduced while retaining key information, thereby improving the computational efficiency and performance of the model. The dimension of the matrix is ​​determined based on the feature vector dimension of the transformer layer and the set rank, so that the low-rank decomposition matrix can better match the structure of the pre-trained large language model, improving the model's ability to extract and represent text features.

[0011] Optionally, the step of performing region detection on the target document image to extract the question-and-answer region in the target document image includes: A trained page detection model is used to perform region detection on the target document image in order to extract the question-and-answer region in the target document image; The page detection model is built and trained based on a neural network model and is used to detect the page structure of an image and identify the types of regions contained in the page structure.

[0012] The above solution can utilize the image recognition capabilities of neural network models to accurately locate the question-and-answer regions in a document, providing an accurate range for subsequent text extraction and processing, and improving the accuracy of information extraction.

[0013] Optionally, before employing a trained page detection model to perform region detection on the target document image to extract the question-and-answer region in the target document image, the method further includes: The neural network model is trained using a pre-set document layout analysis dataset to obtain the initial model parameters of the initial page detection model. The document layout analysis dataset contains multiple first document images, each of which contains page layout annotation information. Obtain a page structure dataset, which includes multiple second document images, each of which includes different types of page regions, and each page region is labeled with a corresponding page region category label; Based on the page structure dataset, the initial page detection model is trained to obtain the trained page detection model.

[0014] The above approach uses a document layout analysis dataset containing page layout annotation information and a page structure dataset containing different types of page regions and category labels for training. This provides the model with a large amount of labeled data, allowing the model to learn various document layouts and page region features during training. This improves the model's adaptability to various types of presentation documents and the accuracy of question-and-answer region detection.

[0015] Optionally, after processing the presentation document to obtain the target document image, the method further includes: If the question-and-answer region is not detected in the target document image, the target document image is determined to be an anomaly investigation page; The anomaly detection page is identified using an optical character recognition model. If the optical character recognition model fails to recognize the content of the anomaly investigation page, a preset multimodal large model is used to identify the anomaly investigation page.

[0016] In the above scheme, a hierarchical recognition strategy can avoid OCR recognition failure. A pre-set multimodal model serves as a fallback solution, preventing processing interruptions due to the limitations of a single recognition tool. This reduces text extraction errors and improves the accuracy of the initial text. Compared to using a large multimodal model to extract text content entirely, the speed of text extraction is significantly improved.

[0017] Optionally, the step of extracting the text content of the question-and-answer area to obtain the initial text includes: The text content of the question-and-answer region is extracted using a pre-defined multimodal large model to obtain the initial text.

[0018] In the above solution, the use of a multimodal large model can accurately identify and extract the text content of the question and answer area, reduce manual intervention, and improve the overall processing speed.

[0019] Optionally, processing the presentation document to obtain the target document image includes: The presentation document is converted to a portable document format. The portable document format document is divided into pages to obtain the target document image corresponding to each page of the portable document format document.

[0020] In the above solution, the presentation document is converted to PDF and split into target document images page by page, which unifies the document processing format and makes it easier to improve the precision of the processing by splitting into pages.

[0021] Optionally, after processing the presentation document to obtain the target document image, the method further includes: If an electrical diagram exists in the target document image, a scaled image is determined based on the target document image and preset scaling parameters; Based on the scaled image and the preset scaling parameters, determine the mapping position of each pixel in the scaled image in the original image; Calculate the pixel value corresponding to each of the mapped positions in the target image; The scaled image is then sharpened to obtain the target image.

[0022] The above solution can enhance the edges of electrical diagrams and make the text in the electrical diagrams clear and readable.

[0023] Optionally, the step of sharpening the scaled image to obtain the target image includes: A blurred image is obtained by blurring the scaled image using a Gaussian function. Calculate the difference between each pixel in the scaled image and each pixel in the blurred image; The target image is obtained by calculating the pixel values ​​of the pixels in the target image based on the preset sharpening intensity, the preset scaling image weight, the difference, and the pixel values ​​of the pixels in the scaled image.

[0024] The above solution can enhance the clarity of electrical diagram details, solve the problem of image blurring after scaling, and facilitate engineers' viewing or subsequent electrical diagram analysis.

[0025] Optionally, calculating the pixel value corresponding to each of the mapped positions in the target image includes: A bilinear interpolation resampling method is used to calculate the pixel value corresponding to each of the mapped positions in the target image.

[0026] In the above scheme, bilinear interpolation allows for a natural transition of pixels at non-integer coordinate positions, ensuring the integrity and accuracy of the scaled electrical diagram.

[0027] Secondly, an information extraction device is provided, comprising: The first processing module is used to process the presentation document to obtain the target document image; The detection module is used to perform region detection on the target document image to extract the question-and-answer region in the target document image; The extraction module is used to extract the text content of the question-and-answer area to obtain the initial text; The second processing module is used to perform structuring processing on the initial text using a trained text structuring model to obtain structured target text. The text structuring model is built based on a pre-trained large language model, and the pre-trained large language model has been adjusted using low-rank adaptation techniques.

[0028] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the information extraction method as described in any one of the first aspects above.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the information extraction method as described in any one of the first aspects above.

[0030] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the information extraction method described in any one of the first aspects.

[0031] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0032] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0033] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating the information extraction method provided in the first embodiment of this application; Figure 2 This is a schematic diagram of a PPT format presentation document provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the information extraction method provided in the second embodiment of this application; Figure 4 This is a flowchart illustrating the information extraction method provided in the third embodiment of this application; Figure 5 This is a schematic diagram of the structure of a low-rank decomposition matrix provided in an embodiment of this application; Figure 6 This is a schematic diagram of the model training and inference process provided in the embodiments of this application; Figure 7 This is a structural block diagram of the information extraction device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0034] The embodiments of the technical solutions of this application will now be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application. When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0035] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0036] The following is an illustrative description of the scenarios that may be involved in the embodiments of this application.

[0037] When the after-sales department receives a repair order, it is forwarded to the quality department. The quality department conducts an in-depth analysis of the cause of the malfunction based on the repair order and corrects the responsible party. During the in-depth analysis by the quality department, the quality engineer records the troubleshooting steps, the test contents, and the cause of the malfunction, and creates a PPT (PowerPoint) presentation document as a customer complaint report.

[0038] For example, there are currently 9,000 historical customer complaint reports, and 300 new customer complaint reports are added each month. When conducting in-depth analysis of a new repair order, it is necessary to query 6 cases related to the current work order from the historical customer complaint reports. Since the historical customer complaint reports are unstructured data, querying 6 related cases takes at least 2 hours. With 300 repair orders per month, querying historical customer complaints will consume a lot of manpower and time every year.

[0039] To facilitate efficient handling of customer complaint cases and achieve cost reduction and efficiency improvement, this invention proposes an information extraction method, device, electronic device, and storage medium that utilizes a structured method to batch parse existing and new customer complaint reports.

[0040] In related technologies, the following problems often exist when processing PPT format presentation documents: For example, using a multimodal large model to structure the content of a PowerPoint presentation is not very effective. The structured result often contains summaries of the model rather than a true structure of the PowerPoint content, and multimodal large models are time-consuming. Furthermore, the FTA (Fault Tree Analysis) images and electrical diagrams obtained from PowerPoint presentations are often unclear, making it difficult to discern circuit pins, text, and labels on each node, thus affecting user experience.

[0041] To address at least one of the aforementioned problems, embodiments of this application propose an information extraction method, apparatus, electronic device, and storage medium.

[0042] It should be noted that the execution subject of the information extraction method in this embodiment can be an information extraction device, hereinafter referred to as "device". The device can be configured in any type of electronic device, such as a server, a laptop, a super mobile personal computer, or other electronic devices capable of implementing the information extraction method. This application embodiment does not limit this.

[0043] See Figure 1 This is a flowchart illustrating the information extraction method provided in the first embodiment of this application. For example... Figure 1 As shown, the information extraction method may include the following steps: Step 101: Process the presentation document to obtain the target document image.

[0044] The presentation document can be in PowerPoint format, or it can be any other format such as Keynote or Google Slides; there are no restrictions on this.

[0045] As an example, the presentation document could be a presentation document about a customer complaint report. For instance, the presentation document could include an overview of the repair order, troubleshooting steps, corrective measures from the responsible party, and comparisons of historical cases, for internal use within the quality department or for sharing information with other departments; there are no restrictions on this.

[0046] like Figure 2 As shown, Figure 2 This is a sample slide of a PowerPoint presentation. The presentation is about "Root Cause Analysis," themed "4D," and features a simple line drawing of a windmill and power lines in the upper right corner, representing the energy and electricity sector. The presentation is presented in a tabular format, divided into three sections, each containing questions and answers.

[0047] The first part, "Occurrence root cause (What caused the actual failure?)," contains two pairs of questions and answers: Q1: Why do components fail? A1: Because of the failure of electrical parameters.

[0048] Q2: Why did the electrical parameters fail? A2: During the probe testing process, XXX.

[0049] The second part, "Detection or Escape root cause (What allowed the failure to occur without being detected internally?)," contains one pair of questions and answers. Q1: Why did the faulty chip get out? A1: The chip underwent ATE testing at the factory, XXX.

[0050] The third part is "System root cause (What part of the organization's process failed to address the issue / or is the organization lacking a specific process, method, etc.?)", which includes one pair of questions and answers: Q1: Why do these kinds of customer complaints occur? A1: / The target document image can be the image to be identified obtained by processing the presentation document.

[0051] Optionally, the presentation document can be converted to Portable Document Format (PDF). Then, the PDF document can be divided into pages to obtain the target document image for each page. It should be noted that, for easier subsequent image detection, the PDF document can be exported page by page as images to obtain the target document image for each page.

[0052] Step 102: Perform region detection on the target document image to extract the question-and-answer region in the target document image.

[0053] The question-and-answer area can refer to an image area that includes the text "question-answer".

[0054] Optionally, a trained page detection model can be used to perform region detection on the target document image in order to extract the question-and-answer region in the target document image.

[0055] The page detection model is built and trained based on a neural network model and is used to detect the page structure of an image and identify the types of regions contained in the page structure.

[0056] Specifically, after training, the page detection model possesses the ability to analyze the structural layout of images. Upon receiving an input target document image, it can analyze the page layout within the image and identify the types of various regions on the page, such as title regions, item regions, summary regions, FTA regions, question-and-answer regions, and other regions. Furthermore, the page detection model can also crop various regions on the page, for example, cropping the question-and-answer region, without limitation.

[0057] like Figure 2 As shown, Figure 2The area in the table represents the question and answer area in the target document image, containing multiple sets of "questions and answers".

[0058] As an example, the neural network model can be the YOLOv10 model, or it can be other types of object detection models, without limitation here.

[0059] As one possible approach, the following steps can be taken when training the page detection model: First, train the neural network model using a pre-defined document layout analysis dataset to obtain the initial model parameters of the initial page detection model. Then, obtain the page structure dataset, which includes multiple second document images. Finally, train the initial page detection model based on the page structure dataset to obtain the final page detection model.

[0060] The document layout analysis dataset contains multiple first document images.

[0061] The first document image contains layout annotations. These annotations are attached to the first document image and typically include the coordinate range of each region in the image (such as the pixel positions of the top left and bottom right corners) and the category of the region (such as text blocks, images, tables, etc.).

[0062] The second document image includes different types of page regions, each labeled with a corresponding page region category tag, such as title, project, other, summary, FTA, Q&A, etc.

[0063] The initial page detection model is a neural network model that has been initially trained on a document layout analysis dataset. It has basic document layout recognition capabilities (such as distinguishing between text areas and chart areas).

[0064] The page structure dataset can be used to optimize the initial page detection model. It contains multiple second document images, each containing features of various page regions. For example, the second document images are pre-annotated using the Labelme tool, and the page structure dataset includes multiple categories such as title, item, other, summary, FTA, and question-and-answer. For instance, 678 images can be annotated, with 544 used as the training set, and 67 images each for the validation and test sets. It should be noted that the above example is merely illustrative and is not intended to limit this disclosure.

[0065] Specifically, a neural network model is trained using a pre-defined document layout analysis dataset. Since this dataset contains multiple first document images with layout annotations, the neural network model can learn basic document layout features through these annotations to obtain an initial page detection model and corresponding initial model parameters.

[0066] The preset document layout analysis dataset can be the DocSynth300K dataset, and there is no limitation here.

[0067] It should be noted that DocSynth300K is a large-scale, diverse synthetic dataset for document layout analysis generated by the Mesh-candidate BestFit algorithm. It contains 300,000 diverse documents, covering various layouts such as single-column and multi-column, as well as styles such as papers and magazines. It is diverse in style and has high visual realism. Using its pre-trained neural network model can significantly improve the accuracy and generalization of document layout detection.

[0068] As a specific example, in this application, the DocSynth300K dataset can be used as the document layout analysis dataset. The model parameters obtained by training YOLOv10 with the DocSynth300K dataset can be used as the initial page detection model and the corresponding initial model parameters, which can significantly improve the detection accuracy of the page detection model.

[0069] Furthermore, after training the initial page detection model, it can be trained again using a page structure dataset until the prediction accuracy of the initial page detection model meets the usability requirements. The initial page detection model can then be used as the final page detection model.

[0070] Step 103: Extract the text content of the question and answer area to obtain the initial text.

[0071] The initial text can be unstructured text extracted from the question-and-answer area.

[0072] Optionally, if no question-and-answer region is detected in the target document image, the target document image is determined to be an anomaly investigation page. Then, an optical character recognition model can be used to identify the anomaly investigation page. If the optical character recognition model fails to identify the content of the anomaly investigation page, a preset multimodal large model is used to identify the anomaly investigation page.

[0073] Understandably, after obtaining the target document image, if it is detected that the target document image does not contain a question-and-answer area, but rather other types of areas, such as a project area, summary area, FTA area, etc., then the target document image can be used as an anomaly investigation page. Then, an optical character recognition model is further used to identify and detect the content in the anomaly investigation page.

[0074] It should be noted that if no question-and-answer area is detected in the target document image, but other functional areas (such as project area, summary area, FTA area, etc.) are included, the page type of the target document image can be defined as an anomaly investigation page.

[0075] For example, the anomaly investigation page may include project progress, summary points, fault breakdown, etc., and may contain various types of page elements such as lists, tables, flowcharts, electrical diagrams, etc., without limitation.

[0076] Among them, the OCR model (Optical Character Recognition) is a model that can convert text information in images (such as printed text, handwritten text, text in screenshots, scanned documents, and text in flowcharts) into editable and storable digital text.

[0077] Optionally, a preset multimodal large model can be used to extract the text content of the question-and-answer region to obtain the initial text. It is understandable that if a question-and-answer region is detected in the target document image, the target document image can be used as the question-and-answer page. The question-and-answer page can be extracted using only the multimodal large model. Since the content of the question-and-answer page is in a table, the multimodal large model extraction is more effective. If only OCR extraction is used, misalignment of the question-and-answer content may occur.

[0078] Among them, a multimodal large model is a model that can simultaneously process and understand two or more different types of information and realize cross-modal information association and generation. A multimodal large model can receive mixed modal data (such as image data of the input question-and-answer area and preset text extraction prompts at the same time).

[0079] Optionally, the input to the multimodal large model includes image data of the question-and-answer region and preset text extraction prompts.

[0080] Among them, the preset text extraction prompts are used to guide the multimodal large model to recognize and output the text content contained in the question-and-answer area.

[0081] Step 104: The trained text structuring model is used to perform structuring processing on the initial text to obtain the structured target text.

[0082] The target text can be structured text with a preset format. The preset format can be defined as key-value pairs, tables, JSON, or other structured forms, and is not limited here.

[0083] The text structuring model is built on a pre-trained large language model, which is then adjusted using Low-Rank Adaptation (LoRA).

[0084] Specifically, text structuring models can be based on pre-trained large language models. LoRA is used to fine-tune the large language model, optimizing only the low-rank factorization matrix parameters while preserving the semantic understanding capabilities of the pre-trained large language model. During processing, the text structuring model receives the initial text and uses its learned structuring rules to transform the unstructured initial text into structured target text.

[0085] For example, a large language model tuned by LoRA can be trained using a question-answering pair dataset. For instance, the question-answering pair dataset might include 3104 training samples and 312 test samples. An example of the training data is shown below: Model input: <|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nConvert the given text into question-and-answer pairs and output them in JSON format. The given text is:\nThe text information in the image is as follows:\n\n1. Occurrence root cause (What caused the actual failure?)\nQ1: Why can't the vehicle connect to the high voltage?\nA1: Because the high voltage circuit is open.\nQ2: Why is the high voltage circuit open?\nA2: Because an abnormally large current appeared in the high voltage line, and the fuse of the MSD blew.\nQ3: Why did an abnormally large current appear in the high voltage line?\nA3: It is suspected that the charging pile is abnormal and an abnormally large current has entered the battery system.\n\n2. Detection or Escape root cause (What allowed the failure to occur without being detected internally?)\nQ1: Why was the MSD and relay abnormality not intercepted when the product was taken off the production line? A1: Because there were no abnormalities when the product was taken off the production line, an abnormally large current entered by the charging pile during customer use caused the MSD fuse to blow and the relay to be damaged. <|im_end|>\n<|im_start|>assistant\n Model output label data: {'Root Cause': {'Q1': 'Why can't the vehicle connect to the high voltage?', 'A1': 'Because the high voltage circuit is open.', 'Q2': 'Why is the high voltage circuit open?', 'A2': 'Because an abnormally large current appeared in the high voltage line, causing the MSD fuse to blow.', 'Q3': 'Why did an abnormally large current appear in the high voltage line?', 'A3': 'Suspected charging pile malfunction, causing an abnormally large current to enter the battery system.'}, 'Outflow Root Cause': {'Q1': 'Why wasn't the MSD and relay malfunction detected when the product was off the production line?', 'A1': 'Because there was no malfunction when the product was off the production line, but an abnormally large current entered by the charging pile during customer use caused the MSD fuse to blow and the relay to be damaged.'}}<|im_end|> It should be noted that the above training data examples are merely illustrative and are not intended to limit this application.

[0086] In this embodiment, the presentation document is first processed to obtain a target document image. Then, region detection is performed on the target document image to extract the question-and-answer regions. Next, the text content of the question-and-answer regions is extracted to obtain initial text. Then, a trained text structuring model is used to perform structuring processing on the initial text to obtain structured target text. Thus, the text structuring model built based on a pre-trained large language model can quickly identify the text content in the initial text. Furthermore, by adjusting the model through low-rank adaptation techniques, only a small number of low-rank parameters need to be optimized to adapt to question-and-answer formats in different scenarios, eliminating the need to train the model from scratch, significantly reducing computational costs, and enabling efficient and accurate extraction of document structure information from presentation documents.

[0087] See Figure 3 This is a flowchart illustrating the information extraction method provided in the second embodiment of this application. Figure 3 As shown, the information extraction method may include the following steps: Step 201: Process the presentation document to obtain the target document image.

[0088] It should be noted that the specific implementation of step 201 can be referred to the above embodiments, and will not be repeated here.

[0089] Step 202: If an electrical diagram exists in the target document image, determine the scaled image based on the target document image and preset scaling parameters.

[0090] It should be noted that the electrical diagrams in the target document images may have blurry pins, rendering them unusable. Therefore, further improvement in the clarity of the target document images is needed.

[0091] The preset scaling parameters can refer to magnification factors, such as 1.5, 2, or 3, and are not limited here.

[0092] Here, a scaled image can refer to an image obtained by adjusting the target document image using scaling parameters.

[0093] For example, if the size of the target document image is Preset scaling parameters The size of the scaled image is .

[0094] Step 203: Based on the scaled image and preset scaling parameters, determine the mapping position of each pixel in the scaled image in the original image.

[0095] The correspondence between the pixel position (x', y') in the scaled image and its mapping position (x, y) in the original image is: x = x' * s, y = y' * s. s is the scaling parameter, representing any pixel in the scaled image. The mapping location to the original image is:

[0096] Step 204: Calculate the pixel value corresponding to each mapped position in the target image.

[0097] Optionally, a bilinear interpolation resampling method can be used to calculate the pixel value corresponding to each mapped position in the target image.

[0098] Specifically, the formula calculated using bilinear interpolation is as follows:

[0099]

[0100] In the formula, , , To scale the image The pixel value of the point. The bilinear interpolation resampling method described above can improve image clarity but has problems such as blurred edges and small text sticking together in electrical diagrams.

[0101] Step 205: Sharpen the scaled image to obtain the target image.

[0102] The target image can be an image obtained by sharpening a scaled image, which has higher clarity than the scaled image.

[0103] Optionally, a blurred image can be obtained by blurring the scaled image using a Gaussian function. Then, the difference between each pixel in the scaled image and each pixel in the blurred image is calculated. Finally, based on the preset sharpening intensity, the preset scaled image weight, the difference, and the pixel values ​​of the pixels in the scaled image, the pixel values ​​of the pixels in the target image are calculated to obtain the target image.

[0104] The blurred image can be an image obtained by blurring a scaled image.

[0105] The preset sharpening intensity can be 1.3 or other settings, which are not limited here.

[0106] The formula for calculating the Gaussian function is as follows:

[0107] In the formula, This represents the offset relative to the center point of the Gaussian kernel. Control the diffusion level of the Gaussian blur; for example, it can be set to 0.9.

[0108] Optionally, the Gaussian kernel size can be set to... This means convolving the scaled image with a 5×5 Gaussian kernel G(m,n). The pixel value of a pixel in the blurred image can be... Scaling pixels in an image pixels in a blurred image The difference can be .

[0109] Optionally, when calculating the pixel values ​​of pixels in the target image based on the preset sharpening intensity, preset scaling image weights, differences, and pixel values ​​of pixels in the scaled image, the following calculation formula can be used:

[0110] In the formula, Represents the pixels in the target image pixel values, This indicates the preset image scaling weights. This is the preset sharpening intensity.

[0111] In this embodiment, the demonstration document is first processed to obtain a target document image. If an electrical diagram exists in the target document image, a scaled image is determined based on the target document image and preset scaling parameters. Then, based on the scaled image and the preset scaling parameters, the mapping position of each pixel in the scaled image in the original image is determined. The pixel value corresponding to each mapping position in the target image is calculated. Finally, the scaled image is sharpened to obtain the target image. This preserves details such as circuit routing and component symbols in the electrical diagram, avoiding graphic distortion or element misalignment caused by scaling. Sharpening through Gaussian blur difference calculation enhances the clarity of electrical diagram details, resolving the problem of image blurring after scaling.

[0112] See Figure 4 This is a flowchart illustrating the information extraction method provided in the third embodiment of this application. Figure 4 As shown, the information extraction method may include the following steps: Step 301: Process the presentation document to obtain the target document image.

[0113] Step 302: Perform region detection on the target document image to extract the question-and-answer region in the target document image.

[0114] Step 303: Extract the text content of the question and answer area to obtain the initial text.

[0115] It should be noted that the specific implementation of steps 301-303 can be referred to the above embodiments, and will not be repeated here.

[0116] Step 304: Obtain the low-rank decomposition matrix constructed using the dot product attention mechanism.

[0117] Among them, the dot product attention mechanism is a core mechanism in deep learning used to capture the correlation information of input data. It calculates the dot product of the query vector (Q) and the key vector (K), and combines normalization and weighted summation of the value vector (V) to allocate different input features, thereby efficiently mining the dependencies in the data.

[0118] It's important to note that the purpose of using a low-rank decomposition matrix is ​​to approximate the original matrix with fewer parameters, reducing storage and computational costs while preserving the matrix's core information. Low-rank decomposition matrices are obtained by factoring a high-dimensional matrix into products of two or more low-dimensional matrices, thus significantly reducing parameter size and computational complexity while retaining core information.

[0119] Optionally, the steps for obtaining the low-rank decomposition matrix constructed through the dot product attention mechanism may include: constructing a context vector, then performing average pooling on the context vector to obtain a feature matrix of a set dimension, then constructing an attention matrix based on the diagonal elements of the feature matrix of the set dimension, and then using the product of the first matrix, the attention matrix and the second matrix as the low-rank decomposition matrix.

[0120] It should be noted that the context vector is a vector obtained by the model after semantically encoding the current input text. The context vector not only contains the literal semantics of the input text, but can also capture word associations and contextual information within the text.

[0121] Specifically, the context vector can be represented as: , This represents the attention weight.

[0122] After constructing the context vector, average pooling can be performed on the context vector to obtain a feature matrix of a set dimension. The formula is as follows: ) in, .

[0123] in, The calculation formula is expressed as:

[0124] In the formula, express Dimensions , ,and is the learnable projection matrix in the pre-trained large language model, Q, K, and V are the query vector, key vector, and value vector, respectively, X is the feature vector input to the attention mechanism, and is the original input data for computation.

[0125] As an example, a low-rank decomposition matrix can be .

[0126] Where B is the first matrix and A is the second matrix. This represents the characteristic matrix.

[0127] in, .

[0128] The first matrix has a dimension that is the product of the feature vector dimension of the Transformer layer and the set rank. As an example, let r represent the numerical value of the set rank, and let r = r + ... This represents the dimension of the feature vectors of the Transformer layer. The dimension of the first matrix can be represented as... The first matrix can be initialized with all zeros.

[0129] The dimension of the second matrix is ​​the product of the set rank and the dimension of the eigenvectors. The dimension of the second matrix can be expressed as... The second matrix can be initialized using a Gaussian function.

[0130] In this embodiment, the feature vector dimension is greater than the set rank. It should be noted that, in this application, the feature vector dimension can be much larger than the set rank, significantly reducing the number of training parameters and thus accelerating the fitting of the pre-trained large language model to the question-answer pair data.

[0131] Step 305: Add a low-rank decomposition matrix to each converter layer of the pre-trained large language model.

[0132] It should be noted that by adding a low-rank decomposition matrix to each converter layer of the pre-trained large language model, LoRA can be used to fine-tune the pre-trained large language model, thereby improving the large language model's ability to follow instructions and extract structured data. The features output by the converter layer of the fine-tuned large language model are denoted as h.

[0133] The formula for calculating the output feature h is as follows:

[0134] in, This represents the original projection matrix in the pre-trained large language model. , , ), where x is the input vector.

[0135] like Figure 5 As shown, Figure 5 The left side W represents a d-row, d-column real matrix. Figure 5 The second matrix on the right is A=N(0, σ 2 The expression indicates that the second matrix A is initialized with a mean of 0 and a variance of σ. 2 It follows a normal distribution. Figure 5 The first matrix B=0 on the right indicates that the first matrix B is initialized to all zeros. Figure 5 The input vector x below is a vector of dimension d, which is the input data of the converter layer. Figure 5 The h above is the output vector obtained after the model calculation.

[0136] Step 306: Train the parameters of the low-rank decomposition matrix in the large language model to obtain the text structure model.

[0137] It is understandable that by training the parameters of the low-rank decomposition matrix in a large language model, the large language model can be enabled to transform unstructured text (initial text) into a structured format.

[0138] Specifically, first fix the basic parameters of the pre-trained large language model (as described above). , , Only low-rank factorization matrices (such as B, A, and diag(β) mentioned above) are selected as trainable parameters. Then, labeled structured training data can be input, and the parameters of the low-rank factorization matrices are iteratively updated with the error between the structured results output by the pre-trained large language model and the labels as the optimization objective. The structured training data includes unstructured text and corresponding structured labels.

[0139] It should be noted that in the above scheme, only a few parameters of the low-rank decomposition matrix need to be updated to adjust the feature extraction capability of the text structure model, which reduces the computational cost of fine-tuning and can accurately adapt to specific tasks.

[0140] Step 307: The trained text structuring model is used to perform structuring processing on the initial text to obtain the structured target text.

[0141] It should be noted that the specific implementation of step 307 can be referred to the above embodiments, and will not be repeated here.

[0142] In this embodiment, the demonstration document is first processed to obtain a target document image. Then, region detection is performed on the target document image to extract question-and-answer regions. Next, the text content of these regions is extracted to obtain initial text. Then, a low-rank decomposition matrix constructed using a dot-product attention mechanism is obtained. This low-rank decomposition matrix is ​​then added to each transformer layer of a pre-trained large language model. Finally, the parameters of the low-rank decomposition matrix in the large language model are trained to obtain a text structuring model. This trained text structuring model is then used to perform structuring processing on the initial text to obtain structured target text. Therefore, by obtaining the low-rank decomposition matrix constructed using the dot-product attention mechanism and adding it to each transformer layer of the pre-trained large language model, the model's attention mechanism can be optimized. This makes the low-rank adaptation matrix input-aware and context-sensitive, enabling the model to focus on key information when processing text and improving the accuracy of information extraction.

[0143] The following is combined with Figure 6The above embodiments are illustrated below. In some scenarios, the presentation document (such as a PPT) is first converted into a document image, which serves as the basic input for subsequent model training and inference. Then, based on the document image, a page structure dataset is established to train the "initial page detection model" (recognizing page structure). Simultaneously, a large language model training dataset is established to train the "large language model fine-tuned via LoRA (the initial text structuring model)". Then, the "initial page detection model" and the "LoRA fine-tuned large language model" are respectively fed into the model training cluster to complete model training, evaluation, and optimization. Finally, algorithm engineers encapsulate the model. For question-and-answer regions: the multimodal large model is directly called to extract text, obtaining the initial text. For anomaly detection page regions: the OCR model (primary) is prioritized for recognition; if OCR fails, the multimodal large model (backup) is used as a fallback for recognition. The initial text extracted by the multimodal large model in the question-and-answer region is fed into the text structuring model (i.e., the usable model trained from the LoRA fine-tuned large language model), deployed and applied through the server cluster, and outputs the structured target text after inference.

[0144] Corresponding to the information extraction method described in the above embodiments, Figure 7 This is a structural block diagram of the information extraction device provided in the embodiments of this application.

[0145] Reference Figure 7 The information extraction device 400 includes: The first processing module 410 is used to process the presentation document to obtain the target document image; The detection module 420 is used to perform region detection on the target document image to extract the question-and-answer region in the target document image; Extraction module 430 is used to extract the text content of the question and answer area to obtain initial text; The second processing module 440 is used to perform structuring processing on the initial text using a trained text structuring model to obtain structured target text. The text structuring model is constructed based on a pre-trained large language model, and the pre-trained large language model is adjusted using low-rank adaptation techniques.

[0146] Optionally, the second processing module also includes: The first acquisition unit is used to acquire the low-rank decomposition matrix constructed through the dot product attention mechanism; Add a unit to add the low-rank decomposition matrix to each converter layer of the pre-trained large language model; The first training unit is used to train the parameters of the low-rank decomposition matrix in the large language model to obtain the text structure model.

[0147] Optionally, the first acquisition unit is specifically used for: Construct context vectors; The context vector is subjected to average pooling to obtain a feature matrix of a set dimension; An attention matrix is ​​constructed based on the diagonal elements of the feature matrix of the defined dimension. The product of the first matrix, the attention matrix, and the second matrix is ​​taken as the low-rank decomposition matrix. Wherein, the dimension of the first matrix is ​​the product of the feature vector dimension of the converter layer and the set rank, and the dimension of the second matrix is ​​the product of the set rank and the feature vector dimension, wherein the value of the feature vector dimension is greater than the value of the set rank.

[0148] Optionally, the detection module includes: The first detection unit is used to perform region detection on the target document image using a trained page detection model in order to extract the question-and-answer region in the target document image. The page detection model is built and trained based on a neural network model and is used to detect the page structure of an image and identify the types of regions contained in the page structure.

[0149] Optionally, the first detection unit is also used for: The neural network model is trained using a pre-set document layout analysis dataset to obtain the initial model parameters of the initial page detection model. The document layout analysis dataset contains multiple first document images, each of which contains page layout annotation information. Obtain a page structure dataset, which includes multiple second document images, each of which includes different types of page regions, and each page region is labeled with a corresponding page region category label; Based on the page structure dataset, the initial page detection model is trained to obtain the trained page detection model.

[0150] Optionally, after processing the presentation document to obtain the target document image, the method further includes: If the question-and-answer region is not detected in the target document image, the target document image is determined to be an anomaly investigation page; The anomaly detection page is identified using an optical character recognition model. If the optical character recognition model fails to recognize the content of the anomaly investigation page, a preset multimodal large model is used to identify the anomaly investigation page.

[0151] Optionally, the step of extracting the text content of the question-and-answer area to obtain the initial text includes: The text content of the question-and-answer region is extracted using a pre-defined multimodal large model to obtain the initial text.

[0152] Optional, the first processing module is specifically used for: The presentation document is converted to a portable document format. The portable document format document is divided into pages to obtain the target document image corresponding to each page of the portable document format document.

[0153] Optionally, the first processing module is also used for: If an electrical diagram exists in the target document image, a scaled image is determined based on the target document image and preset scaling parameters; Based on the scaled image and the preset scaling parameters, determine the mapping position of each pixel in the scaled image in the original image; Calculate the pixel value corresponding to each of the mapped positions in the target image; The scaled image is then sharpened to obtain the target image.

[0154] Optionally, the first processing module is also used for: A blurred image is obtained by blurring the scaled image using a Gaussian function. Calculate the difference between each pixel in the scaled image and each pixel in the blurred image; The target image is obtained by calculating the pixel values ​​of the pixels in the target image based on the preset sharpening intensity, the preset scaling image weight, the difference, and the pixel values ​​of the pixels in the scaled image.

[0155] Optionally, the first processing module is also used for: A bilinear interpolation resampling method is used to calculate the pixel value corresponding to each of the mapped positions in the target image.

[0156] In this embodiment, the presentation document is first processed to obtain a target document image. Then, region detection is performed on the target document image to extract the question-and-answer regions. Next, the text content of the question-and-answer regions is extracted to obtain initial text. Then, a trained text structuring model is used to perform structuring processing on the initial text to obtain structured target text. Thus, the text structuring model built based on a pre-trained large language model can quickly identify the text content in the initial text. Furthermore, by adjusting the model through low-rank adaptation techniques, only a small number of low-rank parameters need to be optimized to adapt to question-and-answer formats in different scenarios, eliminating the need to train the model from scratch, significantly reducing computational costs, and enabling efficient and accurate extraction of document structure information from presentation documents.

[0157] in addition, Figure 7 The information extraction device shown can be a software unit, hardware unit, or a combination of software and hardware built into an existing electronic device, or it can be integrated into the electronic device as a separate component, or it can exist as a standalone electronic device.

[0158] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0159] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 8 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 8 (Only one is shown in the diagram) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, wherein the processor 50 executes the computer program 52 to implement the steps in any of the above-described information extraction method embodiments.

[0160] The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0161] The processor 50 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0162] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may be an external storage device of the electronic device 5, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., equipped on the electronic device 5. Further, the memory 51 may include both internal storage units and external storage devices of the electronic device 5. The memory 51 is used to store operating systems, applications, boot loaders, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0163] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0164] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0165] If the integrated unit is implemented as a software functional unit and used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0169] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0170] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An information extraction method, characterized in that, include: The presentation document is processed to obtain a target document image, wherein the presentation document is a presentation document about a customer complaint report; Region detection is performed on the target document image to extract the question-and-answer region in the target document image, wherein the question-and-answer region is used to carry fault question-and-answer content; The text content of the question-and-answer area is extracted to obtain the initial text; The initial text is processed using a trained text structuring model to obtain a structured target text. The text structuring model is built based on a pre-trained large language model, and the pre-trained large language model is adjusted using low-rank adaptation techniques. If the question-and-answer region is not detected in the target document image, and the target document image contains a functional region, the target document image is determined to be an anomaly investigation page. The functional region includes a project area, a summary area, or an FTA area. The anomaly detection page is identified using an optical character recognition model. If the optical character recognition model fails to recognize the content of the anomaly investigation page, a preset multimodal large model is used to identify the anomaly investigation page. This includes, before using a trained text structuring model to perform structuring processing on the initial text to obtain structured target text, the following steps are also included: Obtain the low-rank decomposition matrix constructed using the dot product attention mechanism; The low-rank decomposition matrix is ​​added to each converter layer of the pre-trained large language model; The parameters of the low-rank decomposition matrix in the large language model are trained to obtain the text structuring model. The step of obtaining the low-rank decomposition matrix constructed through the dot product attention mechanism includes: Construct context vectors; The context vector is subjected to average pooling to obtain a feature matrix of a set dimension; An attention matrix is ​​constructed based on the diagonal elements of the feature matrix of the defined dimension. The product of the first matrix, the attention matrix, and the second matrix is ​​taken as the low-rank decomposition matrix. Wherein, the dimension of the first matrix is ​​the product of the feature vector dimension of the converter layer and the set rank, and the dimension of the second matrix is ​​the product of the set rank and the feature vector dimension, wherein the value of the feature vector dimension is greater than the value of the set rank.

2. The method according to claim 1, characterized in that, The step of performing region detection on the target document image to extract the question-and-answer region in the target document image includes: A trained page detection model is used to perform region detection on the target document image in order to extract the question-and-answer region in the target document image; The page detection model is built and trained based on a neural network model and is used to detect the page structure of an image and identify the types of regions contained in the page structure.

3. The method according to claim 2, characterized in that, Before employing a trained page detection model to perform region detection on the target document image to extract the question-and-answer region in the target document image, the method further includes: The neural network model is trained using a pre-set document layout analysis dataset to obtain the initial model parameters of the initial page detection model. The document layout analysis dataset contains multiple first document images, each of which contains page layout annotation information. Obtain a page structure dataset, which includes multiple second document images, each of which includes different types of page regions, and each page region is labeled with a corresponding page region category label; Based on the page structure dataset, the initial page detection model is trained to obtain the trained page detection model.

4. The method according to claim 1, characterized in that, The step of extracting the text content of the question-and-answer area to obtain the initial text includes: The text content of the question-and-answer region is extracted using a pre-defined multimodal large model to obtain the initial text.

5. The method according to claim 4, characterized in that, The process of processing the presentation document to obtain the target document image includes: The presentation document is converted to a portable document format. The portable document format document is divided into pages to obtain the target document image corresponding to each page of the portable document format document.

6. The method according to claim 5, characterized in that, After processing the presentation document to obtain the target document image, the process further includes: If an electrical diagram exists in the target document image, a scaled image is determined based on the target document image and preset scaling parameters; Based on the scaled image and the preset scaling parameters, determine the mapping position of each pixel in the scaled image in the original image; Calculate the pixel value in the target image corresponding to each of the mapped positions; The scaled image is then sharpened to obtain the target image.

7. The method according to claim 6, characterized in that, The step of sharpening the scaled image to obtain the target image includes: A blurred image is obtained by blurring the scaled image using a Gaussian function. Calculate the difference between each pixel in the scaled image and each pixel in the blurred image; The target image is obtained by calculating the pixel values ​​of the pixels in the target image based on the preset sharpening intensity, the preset scaling image weight, the difference, and the pixel values ​​of the pixels in the scaled image.

8. The method according to claim 7, characterized in that, The calculation of the pixel value corresponding to each of the mapped positions in the target image includes: A bilinear interpolation resampling method is used to calculate the pixel value corresponding to each of the mapped positions in the target image.

9. An information extraction device, characterized in that, include: The first processing module is used to process the presentation document to obtain a target document image, wherein the presentation document is a presentation document about a customer complaint report; The detection module is used to perform region detection on the target document image to extract the question and answer region in the target document image, wherein the question and answer region is used to carry fault question and answer content; The extraction module is used to extract the text content of the question-and-answer area to obtain the initial text; The second processing module is used to perform structuring processing on the initial text using a trained text structuring model to obtain structured target text. The text structuring model is built based on a pre-trained large language model, and the pre-trained large language model is adjusted using low-rank adaptation techniques. If the question-and-answer region is not detected in the target document image, and the target document image contains a functional region, the target document image is determined to be an anomaly investigation page. The functional region includes a project area, a summary area, or an FTA area. The anomaly detection page is identified using an optical character recognition model. If the optical character recognition model fails to identify the content of the anomaly investigation page, a preset multimodal large model is used to identify the anomaly investigation page. Before using a trained text structuring model to perform structuring processing on the initial text to obtain the structured target text, the process further includes: Obtain the low-rank decomposition matrix constructed using the dot product attention mechanism; The low-rank decomposition matrix is ​​added to each converter layer of the pre-trained large language model; The parameters of the low-rank decomposition matrix in the large language model are trained to obtain the text structuring model. The step of obtaining the low-rank decomposition matrix constructed through the dot product attention mechanism includes: Construct context vectors; The context vector is subjected to average pooling to obtain a feature matrix of a set dimension; An attention matrix is ​​constructed based on the diagonal elements of the feature matrix of the defined dimension. The product of the first matrix, the attention matrix, and the second matrix is ​​taken as the low-rank decomposition matrix. Wherein, the dimension of the first matrix is ​​the product of the feature vector dimension of the converter layer and the set rank, and the dimension of the second matrix is ​​the product of the set rank and the feature vector dimension, wherein the value of the feature vector dimension is greater than the value of the set rank.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to perform the steps of the method according to any one of claims 1 to 8.

Images