Digital archive automatic quality inspection method and system based on image recognition

Abstract

The invention discloses a digital archive automatic quality inspection method and system based on image recognition, and relates to the technical field of image recognition and processing based on artificial intelligence, and the method comprises the steps: collecting image data and text data, and carrying out the preprocessing; a quality inspection multi-mode converter model is constructed, the image quality score and the text quality score are fused, and a quality inspection result is generated; performing structured processing on image features and text features in the quality inspection result by using a VGG convolutional neural network to generate a structured document; an enhanced BERT model is constructed to extract image features and text features in the structured document, a semantic analysis report is generated, and an image quality problem is automatically marked; according to the method, the image quality score and the text quality score are fused through the quality inspection multi-modal converter model, and deep integration and analysis of cross-modal data are realized.

Description

Technical Field

[0001] The present invention relates to the field of image recognition and processing based on artificial intelligence, and in particular to an automatic quality inspection method and system for digital archives based on image recognition. Background Art

[0002] With the continuous advancement of information technology, the digital management of archives has gradually become an important development direction in the field of archives management. Compared with traditional paper archives, digital archives have the advantages of easy preservation, convenient retrieval and efficient management. However, the quality control and quality inspection of digital archives still face many challenges. At this stage, image recognition technology is increasingly used in the quality inspection of digital archives. The recognition and analysis of archive content are mainly achieved through image processing and optical character recognition technology. Typical technologies include the use of convolutional neural networks for image feature extraction and classification, and the use of natural language processing algorithms for text data analysis and processing.

[0003] Although existing technologies have made certain progress, there are still many shortcomings in the quality inspection process of digital archives. First, image recognition technology is highly dependent on image quality, and the recognition accuracy of low-quality or noisy images is low, which affects the overall quality inspection effect. Second, existing technologies lack the ability to comprehensively analyze multimodal data and cannot fully combine the information between images and texts for comprehensive quality assessment. In addition, existing quality inspection methods mostly rely on manual operations, making it difficult to achieve efficient automated quality inspection. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an automatic quality inspection method for digital archives based on image recognition to solve the problem of automated quality inspection.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an automatic quality inspection method for digital archives based on image recognition, which includes collecting image data and text data and performing preprocessing; extracting features from the preprocessed image data and text data respectively through a ResNet model and an NLP algorithm to generate an image quality score and a text quality score; constructing a quality inspection multimodal converter model to fuse the image quality score and the text quality score to generate a quality inspection result; using a VGG convolutional neural network to perform structured processing on the image features and text features in the quality inspection result to generate a structured document; constructing an enhanced BERT model to extract image features and text features in the structured document, generate a semantic analysis report, and automatically annotate image quality issues.

[0008] As a preferred solution of the method for automatic quality inspection of digital archives based on image recognition according to the present invention, the preprocessing includes:

[0009] Improve the clarity, denoise, normalize the image size, and enhance the local contrast of the image data;

[0010] Perform word segmentation, cleaning, and text normalization on text data.

[0011] As a preferred solution of the automatic quality inspection method for digital archives based on image recognition described in the present invention, wherein: the ResNet model and the NLP algorithm are used to extract features from the pre-processed image data and text data respectively to generate image quality scores and text quality scores. The specific steps are as follows:

[0012] Use the pre-trained ResNet model to extract image features from image data;

[0013] By extracting image features, we can obtain the image clarity, brightness, and contrast scores, and generate an image quality score, which is expressed as:

[0014]

[0015] Where E is the image quality score, N is the number of images, S(L i ) is the clarity score of the i-th image, B(L i ) is the brightness score of the i-th image, C(L i ) is the contrast score of the i-th image, L i is the i-th image, i is the index of the image position;

[0016] Use NLP algorithm to extract text features from preprocessed text data;

[0017] By extracting text features, we can obtain the complexity, keyword coverage, and spelling and grammatical accuracy scores of the text, and generate a text quality score, which is expressed as:

[0018]

[0019] Among them, G is the text quality score, M is the number of text paragraphs, C(O j ) is the complexity score of the jth paragraph, F(O j ) is the keyword coverage score of paragraph j, P(O j ) is the spelling and grammatical accuracy score of paragraph j, j is the text paragraph position, O j is the jth text paragraph.

[0020] As a preferred solution of the automatic quality inspection method for digital archives based on image recognition described in the present invention, a quality inspection multimodal converter model is constructed to fuse the image quality score and the text quality score to generate the quality inspection result. The specific steps are as follows:

[0021] Define the input as image quality score and text quality score, and the output as quality inspection result;

[0022] Normalize image quality scores and text quality scores;

[0023] The image quality score and text quality score are fused through the cross-modal attention mechanism to generate the quality inspection result, which is expressed as:

[0024] U = α × E + β × G;

[0025] Where U is the quality inspection result, α is the coefficient for adjusting the image quality score E in the quality inspection result U, and β is the coefficient for adjusting the text quality score G in the quality inspection result U.

[0026] As a preferred solution of the automatic quality inspection method for digital archives based on image recognition described in the present invention, wherein: the VGG convolutional neural network is used to perform structured processing on the image features and text features in the quality inspection results to generate a structured document. The specific steps are:

[0027] Format the image features and text features in the quality inspection results and input them into the VGG convolutional neural network;

[0028] The convolutional layer of the VGG convolutional neural network extracts low-level and high-level features of the image and identifies document elements in the image;

[0029] Using the extracted low-level image features, high-level image features, and text features extracted by the NLP algorithm, each document element is identified and classified separately;

[0030] Arrange the identified and classified document elements in order and hierarchy to form a structured document;

[0031] Compare the original image data and text data to check whether the document elements in the structured document are consistent.

[0032] As a preferred solution of the automatic quality inspection method for digital archives based on image recognition described in the present invention, the following specific steps are used to construct an enhanced BERT model, extract image features and text features from structured documents, generate a semantic analysis report, and automatically annotate image quality issues:

[0033] Define the cross-modal attention layer that enhances the BERT model;

[0034] Extract text features and image features from structured documents and input the text features into the cross-modal attention layer;

[0035] Using the cross-modal attention mechanism, image features and text features are deeply integrated and analyzed to generate comprehensive analysis results;

[0036] Through the generated comprehensive analysis results, the information consistency between image features and text features is checked;

[0037] Based on the results of the information consistency check, annotation is performed to identify errors in image and text pairing and generate a semantic analysis report;

[0038] Based on the information in the semantic analysis report, image quality issues are automatically marked and saved, and correction suggestions are generated.

[0039] As a preferred solution of the automatic quality inspection method for digital archives based on image recognition described in the present invention, the cross-modal attention mechanism is used to deeply fuse and analyze image features and text features to generate comprehensive analysis results. The specific steps are as follows:

[0040] In the cross-modal attention mechanism, image features are used as the query of the attention mechanism, and text features are used as keys and values;

[0041] Generate attention weights by computing the dot product between the query and the key;

[0042] Apply attention weights to the values ​​to generate deep fusion values;

[0043] According to the depth fusion value, the final comprehensive analysis result is generated, and the expression is:

[0044]

[0045] Among them, Φ is the comprehensive analysis result, EBERT is the result of semantic analysis of text features and context information by the enhanced BERT model, Θ is the recognized text features, Ω is the context information, Γ is the number of document elements, λ is the index of the document element, VGG is the analysis result of the VGG model on the document element, Δ λ is the feature of the λth document element.

[0046] In a second aspect, the present invention provides an automatic quality inspection system for digital archives based on image recognition, comprising an acquisition module, a scoring module, a quality inspection module, a structuring module and an annotation module; the acquisition module is used to acquire image data and text data and perform preprocessing; the scoring module is used to extract features of the preprocessed image data and text data respectively through a ResNet model and an NLP algorithm to generate image quality scores and text quality scores; the quality inspection module is used to construct a quality inspection multimodal converter model to fuse the image quality score and the text quality score to generate a quality inspection result; the structuring module is used to use a VGG convolutional neural network to perform structured processing on the image features and text features in the quality inspection results to generate a structured document; the annotation module is used to construct an enhanced BERT model to extract image features and text features in the structured document, generate a semantic analysis report, and automatically annotate image quality issues.

[0047] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for automatic quality inspection of digital archives based on image recognition as described in the first aspect of the present invention is implemented.

[0048] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for automatic quality inspection of digital archives based on image recognition as described in the first aspect of the present invention.

[0049] The beneficial effects of the present invention are as follows: the present invention integrates image quality scores and text quality scores through the quality inspection multimodal converter model, realizes the deep integration and analysis of cross-modal data, and further standardizes the processing to ensure that the image and text features are compared on the same scale, and uses the cross-modal attention mechanism to generate quality inspection results. The quality inspection multimodal converter model not only improves the quality inspection accuracy of image and text data, but also makes the quality inspection process more comprehensive, intelligent and efficient through the generation of comprehensive scores, reduces manual intervention and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 This is a flow chart of the automatic quality inspection method for digital archives based on image recognition in Example 1.

[0052] Figure 2 This is a module diagram of the automatic quality inspection system for digital archives based on image recognition in Example 1. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0055] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0056] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a method for automatic quality inspection of digital archives based on image recognition, comprising the following steps:

[0057] S1. Collect image data and text data and perform preprocessing.

[0058] Furthermore, the image data and text data of the archives are collected. The image data is obtained by high-precision scanners or high-resolution camera equipment and saved in high-quality lossless compression formats (such as TIFF);

[0059] Text data is extracted from images using OCR technology;

[0060] Use a high-pass filter to enhance edge details in the image;

[0061] Use median filtering to remove noise from the image;

[0062] Resize images to a uniform size to ensure consistency in subsequent processing;

[0063] Use adaptive histogram equalization to enhance the contrast of the image;

[0064] Perform word segmentation on text data, that is, dividing a continuous sequence of characters into meaningful words or phrases. Word segmentation facilitates subsequent text analysis and processing.

[0065] Clean the text data to remove irrelevant information and noise, such as removing punctuation marks, special characters, extra spaces, and stop words (such as "de", "shi", etc.). The cleaning step can improve the quality of the text data and the accuracy of analysis;

[0066] Normalize the text, including converting case, replacing synonyms, and lemmatization (such as restoring the past tense or progressive form of a verb to its base form), to ensure the consistency and comparability of the text data;

[0067] It should be noted that through detailed preprocessing of the image data and text data, the high quality and consistency of the data are ensured. High-precision scanning and OCR technology are used to extract text data, ensuring the accuracy of the images and text. High-pass filters and median filters are used to enhance image edge details and remove noise respectively, and the image size and contrast are adjusted to ensure the consistency of image quality. The steps of text tokenization, cleaning, and normalization improve the accuracy of text data analysis, and extracting keywords ensures the prominence of text information. This step not only lays a solid foundation for subsequent feature extraction and quality scoring, but also significantly improves the accuracy and efficiency of the entire quality inspection process.

[0068] S2. Extract features from the preprocessed image data and text data through the ResNet model and NLP algorithm respectively, and generate image quality scores and text quality scores.

[0069] Furthermore, use TensorFlow or PyTorch as the ResNet model framework;

[0070] Use a pre-trained ResNet model to extract image features (sharpness, brightness, contrast) from the image data;

[0071] Extract image features through a certain layer of the ResNet model (such as the pooling layer before the fully connected layer) for subsequent image quality assessment;

[0072] ResNet is a powerful image recognition model that can efficiently extract and analyze image features, improving the performance and accuracy of image processing and recognition tasks;

[0073] Through the extracted image features, obtain the sharpness, brightness, and contrast scores of the image (by training a simple linear regression model or neural network to map these image features to quality scores), and generate the image quality score. The expression is:

[0074]

[0075] where E is the image quality score, N is the number of images, S(L i ) is the sharpness score of the i-th image, B(Li ) is the brightness score of the i-th image, C(L i ) is the contrast score of the i-th image, L i is the i-th image, i is the index of the image position;

[0076] Image quality scoring indicators can help optimize the quality inspection process of digital archives and ensure the high quality and consistency of archival images;

[0077] Use natural language processing (NLP) algorithms to extract text features from preprocessed text data, perform semantic analysis on the extracted text features, and identify keywords, sentence structures, and semantic relationships;

[0078] Natural language processing (NLP) is a branch of artificial intelligence that aims to enable computers to understand, interpret, and generate human language. Natural language processing (NLP) algorithms process and analyze large amounts of natural language data by combining computational linguistics and artificial intelligence techniques.

[0079] Through the extracted text features, the complexity, keyword coverage, spelling and grammatical accuracy scores of the text are obtained, and the text quality score is generated. The expression is:

[0080]

[0081] Among them, G is the text quality score, M is the number of text paragraphs, C(O j ) is the complexity score of the jth paragraph, F(O j ) is the keyword coverage score of paragraph j, P(O j ) is the spelling and grammatical accuracy score of paragraph j, j is the text paragraph position, O j is the jth text paragraph;

[0082] It should be noted that the ResNet model and NLP algorithm are used to extract image and text data features respectively, generate image and text quality scores, and significantly improve the quality inspection effect of digital archives. The ResNet model performs well in image feature extraction and can accurately evaluate the clarity, brightness and contrast of images to ensure consistent image quality. Using TensorFlow or PyTorch as a framework can further improve the training and application efficiency of the model. In text data processing, the NLP algorithm effectively identifies the complexity of the text, keyword coverage, spelling and grammatical accuracy through steps such as word segmentation and semantic analysis, and generates text quality scores. This step comprehensively uses deep learning and natural language processing technologies to achieve efficient quality inspection of image and text data, ensuring the high quality and consistency of archives.

[0083] S3. Build a quality inspection multimodal converter model to fuse image quality scores and text quality scores to generate quality inspection results.

[0084] Furthermore, the input is defined as image quality score and text quality score, and the output is the quality inspection result;

[0085] The image quality score and the text quality score are normalized so that the image features and text features in the image quality score and the text quality score are on the same scale;

[0086] The image quality score and text quality score are fused through the cross-modal attention mechanism to generate the quality inspection result, which is expressed as:

[0087] U = α × E + β × G;

[0088] Among them, U is the quality inspection result, α is the coefficient for adjusting the image quality score E in the quality inspection result U. By adjusting the value of α, the influence of the image quality score in the final comprehensive score can be controlled. If the quality of the image is very important to the overall quality inspection result, then the value of α can be set to a larger value. On the contrary, if the image quality has little impact, then the value of α can be smaller. β is the coefficient for adjusting the text quality score G in the quality inspection result U. By adjusting the value of β, the influence of the text quality score in the final comprehensive score can be controlled. If the quality of the text is very important to the overall quality inspection result, then the value of β can be set to a larger value. On the contrary, if the text quality has little impact, then the value of β can be smaller.

[0089] It should be noted that by constructing a quality inspection multimodal converter model, the image quality score and text quality score are standardized and fused to generate quality inspection results. The cross-modal attention mechanism is used to ensure that image and text features are deeply integrated at the same scale, thereby improving the comprehensiveness and accuracy of the quality inspection results. By adjusting the weight coefficient, the impact of different scores on the quality inspection results is controlled, making the scoring more flexible and accurate.

[0090] S4. Use the VGG convolutional neural network to perform structured processing on the image features and text features in the quality inspection results to generate structured documents.

[0091] Furthermore, the image features and text features in the quality inspection results are formatted and input into the VGG convolutional neural network;

[0092] VGG convolutional neural network is mainly used for image classification, feature extraction and other computer vision tasks. It is widely used in various image processing applications due to its efficient feature extraction capabilities.

[0093] Format the image features in the quality inspection results to adapt to the input requirements of the VGG network;

[0094] The convolutional layer of the VGG convolutional neural network extracts low-level features (such as edges, textures, etc.) and high-level features (such as shape, structure, etc.) of the image to identify document elements (such as titles, paragraphs, tables, etc.) in the image;

[0095] Using the extracted low-level and high-level features of the image and the text features extracted by the NLP algorithm, each document element is identified and classified separately to ensure that each element is accurately labeled and classified;

[0096] Arrange the identified and classified document elements in order and hierarchy to form a structured document, for example, according to the hierarchy of titles, paragraphs, and tables;

[0097] Compare the original image data and text data to check whether the document elements in the structured document are consistent, ensuring the integrity and accuracy of the structured document;

[0098] It should be noted that the use of VGG convolutional neural network to perform structured processing on the image features and text features in the quality inspection results can generate more accurate structured documents. After the image and text features are formatted and input into the VGG network, the convolution layer of the network can extract low-level and high-level features, identify the document elements in the image, and accurately label and classify each element. Through sequential and hierarchical arrangement, a complete structured document is formed, and the original image and text data are compared to ensure the consistency and integrity of the document elements. This not only improves the accuracy of quality inspection, but also ensures the clarity of the document structure and the completeness of the information, ultimately achieving efficient digital archive quality inspection and management.

[0099] S5. Build an enhanced BERT model to extract image and text features from structured documents, generate a semantic analysis report, and automatically annotate image quality issues.

[0100] Furthermore, we define a cross-modal attention layer to enhance the BERT model. The purpose of this layer is to fuse image and text features through a cross-modal attention mechanism. The cross-modal attention layer can dynamically adjust the attention weights based on the correlation between the image and text, thereby enhancing the model's understanding of the joint features of image and text.

[0101] Extract text features and image features from structured documents and input the text features into the cross-modal attention layer;

[0102] Using the cross-modal attention mechanism, image features and text features are deeply integrated and analyzed to generate comprehensive analysis results;

[0103] In the cross-modal attention mechanism, image features are used as the query of the attention mechanism, and text features are used as keys and values;

[0104] By calculating the dot product between the query and the key, the attention weights are generated (these weights represent the correlation between image features and text features), and the expression is:

[0105]

[0106] Where W is the attention weight, Q is the query matrix, K is the key matrix, and d k is the dimension of the key, used to scale the dot product result, K T is the transpose of the key matrix K, T is the transpose, and softmax is the probability distribution function applied to the result to obtain the probability distribution;

[0107] Applying the attention weights to the values ​​generates a deep fusion value, expressed as:

[0108] Output = W × Qv;

[0109] Among them, Output is the deep fusion value, which represents the joint representation of image features and text features after deep fusion, and Qv is the value matrix, which represents the values ​​of image features and text features.

[0110] According to the depth fusion value, the final comprehensive analysis result is generated, and the expression is:

[0111]

[0112] Among them, Φ is the comprehensive analysis result, EBERT is the result of semantic analysis of text features and context information by the enhanced BERT model, Θ is the recognized text features, Ω is the context information, Γ is the number of document elements, λ is the index of the document element, VGG is the analysis result of the VGG model on the document element, Δ λ is the feature of the λth document element;

[0113] Through the generated comprehensive analysis results, the information consistency between image features and text features is checked to ensure that the information in the image is consistent with the text description;

[0114] Based on the results of the information consistency check, annotation is performed to identify errors in the image and text pairing (for example, identifying whether the text description matches the image content, or whether there are any errors), and a semantic analysis report is generated, detailing the errors and inconsistencies.

[0115] Automatically mark image quality issues and save them based on the information in the semantic analysis report, and generate correction suggestions;

[0116] Specifically, after generating the semantic analysis report, further automatic processing is performed based on the information in the report;

[0117] Automatically annotate image quality issues, which includes annotating images based on errors and issues identified in semantic analysis reports;

[0118] Save annotation reports, detailing quality issues and their locations for each image;

[0119] Generate correction suggestions based on the annotation report to guide subsequent correction and processing work. These suggestions are generated based on the information consistency check results and image quality annotations in the semantic analysis report;

[0120] Specifically,

[0121] Provide detailed descriptions of image quality issues noted in the semantic analysis report, such as blurry images, insufficient brightness, poor contrast, inconsistency between text and image, etc.

[0122] Provide specific examples and locations for each problem, allowing users to quickly locate the problem;

[0123] For each identified quality issue, provide specific correction methods, such as:

[0124] For blurred images, it is recommended to use image enhancement algorithms (such as denoising, sharpening, etc.) to repair them;

[0125] For images with insufficient brightness, it is recommended to adjust the brightness parameters and use image processing software to perform brightness correction;

[0126] If the text and image are inconsistent, it is recommended to review the text and image content to ensure that the information on the two matches;

[0127] Prioritize remediation suggestions based on the severity and scope of the problem, putting the most urgent and impactful remediation suggestions at the top to ensure that critical issues are addressed first;

[0128] Provide recommendations for tools and resources needed to implement the recommended corrections, such as recommending specific software or algorithms for image processing, or providing relevant online tutorials and guides;

[0129] Provide specific steps for each correction suggestion to ensure that users can complete the correction process step by step. This includes detailed step-by-step descriptions and example operations to help users complete the correction work efficiently.

[0130] It is recommended that users re-perform quality inspection after correction to ensure that the problem has been resolved;

[0131] Provide a feedback mechanism where users can record problems and results during the correction process to further optimize the quality inspection process;

[0132] It should be noted that the construction of an enhanced BERT model, the extraction of image features and text features in structured documents, and the generation of semantic analysis reports can effectively integrate cross-modal data. Through the cross-modal attention mechanism, the attention weights are dynamically adjusted to enhance the model's understanding of the joint features of images and texts. The generated comprehensive analysis results ensure the information consistency between images and texts, can automatically identify and annotate image quality issues, and save detailed annotation reports. These annotations are used to generate correction suggestions to guide subsequent correction and processing work. This process greatly improves the accuracy and efficiency of quality inspection, reduces manual intervention, and ensures the comprehensiveness and consistency of archive quality.

[0133] This embodiment also provides an automatic quality inspection system for digital archives based on image recognition, including: an acquisition module, a scoring module, a quality inspection module, a structuring module and an annotation module; the acquisition module is used to acquire image data and text data and perform preprocessing; the scoring module is used to extract features from the preprocessed image data and text data respectively through a ResNet model and an NLP algorithm to generate image quality scores and text quality scores; the quality inspection module is used to construct a quality inspection multimodal converter model, fuse the image quality score and the text quality score, and generate a quality inspection result; the structuring module is used to use a VGG convolutional neural network to perform structured processing on the image features and text features in the quality inspection results to generate a structured document; the annotation module is used to construct an enhanced BERT model to extract image features and text features in the structured document, generate a semantic analysis report, and automatically annotate image quality issues.

[0134] This embodiment also provides a computer device suitable for the automatic quality inspection method of digital archives based on image recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the automatic quality inspection method of digital archives based on image recognition proposed in the above embodiment.

[0135] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0136] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatic quality inspection of digital archives based on image recognition as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0137] In summary, the present invention integrates image quality scores and text quality scores through the quality inspection multimodal converter model, realizes the deep integration and analysis of cross-modal data, further standardizes the processing, ensures that the image and text features are compared on the same scale, and uses the cross-modal attention mechanism to generate quality inspection results. The quality inspection multimodal converter model not only improves the quality inspection accuracy of image and text data, but also makes the quality inspection process more comprehensive, intelligent and efficient through the generation of comprehensive scores, reduces manual intervention and improves work efficiency.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatic quality inspection of digital archives based on image recognition, characterized by: include, Collect image data and text data and perform preprocessing; The ResNet model and NLP algorithm are used to extract features from the preprocessed image data and text data, respectively, to generate image quality scores and text quality scores; Build a quality inspection multimodal converter model to fuse image quality scores and text quality scores to generate quality inspection results; Use VGG convolutional neural network to perform structural processing on image features and text features in quality inspection results to generate structured documents; Build an enhanced BERT model to extract image and text features from structured documents, generate semantic analysis reports, and automatically annotate image quality issues.

2. The method for automatic quality inspection of digital archives based on image recognition according to claim 1, characterized in that: The pretreatment includes: Improve the clarity, denoise, normalize the image size, and enhance the local contrast of the image data; Perform word segmentation, cleaning, and text normalization on text data.

3. The method for automatic quality inspection of digital archives based on image recognition according to claim 2, characterized in that: The ResNet model and the NLP algorithm are used to extract features from the pre-processed image data and text data respectively to generate image quality scores and text quality scores. The specific steps are as follows: Use the pre-trained ResNet model to extract image features from image data; By extracting image features, we can obtain the image clarity, brightness, and contrast scores, and generate an image quality score, which is expressed as: Where E is the image quality score, N is the number of images, S(L i ) is the clarity score of the i-th image, B(L i ) is the brightness score of the i-th image, C(L i ) is the contrast score of the i-th image, L i is the i-th image, i is the index of the image position; Use NLP algorithm to extract text features from preprocessed text data; By extracting text features, we can obtain the complexity, keyword coverage, and spelling and grammatical accuracy scores of the text, and generate a text quality score, which is expressed as: Among them, G is the text quality score, M is the number of text paragraphs, C(O j ) is the complexity score of the jth paragraph, F(O j ) is the keyword coverage score of paragraph j, P(O j ) is the spelling and grammatical accuracy score of paragraph j, j is the text paragraph position, O j is the jth text paragraph.

4. The method for automatic quality inspection of digital archives based on image recognition according to claim 3, characterized in that: The quality inspection multimodal converter model is constructed to fuse the image quality score and the text quality score to generate the quality inspection result. The specific steps are as follows: Define the input as image quality score and text quality score, and the output as quality inspection result; Normalize image quality scores and text quality scores; The image quality score and text quality score are fused through the cross-modal attention mechanism to generate the quality inspection result, which is expressed as: U = α × E + β × G; Where U is the quality inspection result, α is the coefficient for adjusting the image quality score E in the quality inspection result U, and β is the coefficient for adjusting the text quality score G in the quality inspection result U.

5. The method for automatic quality inspection of digital archives based on image recognition according to claim 4, characterized in that: The VGG convolutional neural network is used to perform structural processing on the image features and text features in the quality inspection results to generate a structured document. The specific steps are: Format the image features and text features in the quality inspection results and input them into the VGG convolutional neural network; The convolutional layer of the VGG convolutional neural network extracts low-level and high-level features of the image and identifies document elements in the image; Using the extracted low-level image features, high-level image features, and text features extracted by the NLP algorithm, each document element is identified and classified separately; Arrange the identified and classified document elements in order and hierarchy to form a structured document; Compare the original image data and text data to check whether the document elements in the structured document are consistent.

6. The method for automatic quality inspection of digital archives based on image recognition according to claim 5, characterized in that: The enhanced BERT model is constructed to extract image features and text features from structured documents, generate semantic analysis reports, and automatically annotate image quality issues. The specific steps are: Define the cross-modal attention layer that enhances the BERT model; Extract text features and image features from structured documents and input the text features into the cross-modal attention layer; Using the cross-modal attention mechanism, image features and text features are deeply integrated and analyzed to generate comprehensive analysis results; Through the generated comprehensive analysis results, the information consistency between image features and text features is checked; Based on the results of the information consistency check, annotation is performed to identify errors in image and text pairing and generate a semantic analysis report; Based on the information in the semantic analysis report, image quality issues are automatically marked and saved, and correction suggestions are generated.

7. The automatic quality inspection method for digital archives based on image recognition according to claim 6, characterized in that: The cross-modal attention mechanism is used to deeply fuse and analyze image features and text features to generate comprehensive analysis results. The specific steps are: In the cross-modal attention mechanism, image features are used as the query of the attention mechanism, and text features are used as keys and values; Generate attention weights by computing the dot product between the query and the key; Apply attention weights to the values ​​to generate deep fusion values; According to the depth fusion value, the final comprehensive analysis result is generated, and the expression is: Among them, Φ is the comprehensive analysis result, EBERT is the result of semantic analysis of text features and context information by the enhanced BERT model, Θ is the recognized text features, Ω is the context information, Γ is the number of document elements, λ is the index of the document element, VGG is the analysis result of the VGG model on the document element, Δ λ is the feature of the λth document element.

8. An automatic quality inspection system for digital archives based on image recognition, based on the automatic quality inspection method for digital archives based on image recognition according to any one of claims 1 to 7, characterized in that: Including, acquisition module, scoring module, quality inspection module, structuring module and annotation module; The acquisition module is used to acquire image data and text data and perform preprocessing; The scoring module is used to extract features from the pre-processed image data and text data using the ResNet model and the NLP algorithm, respectively, to generate image quality scores and text quality scores; The quality inspection module is used to build a quality inspection multimodal converter model, fuse the image quality score and the text quality score to generate a quality inspection result; The structuring module is used to perform structural processing on the image features and text features in the quality inspection results using the VGG convolutional neural network to generate a structured document; The annotation module is used to build an enhanced BERT model to extract image features and text features in structured documents, generate a semantic analysis report, and automatically annotate image quality issues.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the automatic quality inspection method for digital archives based on image recognition according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the automatic quality inspection method for digital archives based on image recognition according to any one of claims 1 to 7 are implemented.

Images