Image recognition-based digital archive automatic quality inspection method and system

Abstract

The application discloses a kind of based on image recognition's digital archives automatic quality inspection method and system, it is related to the image recognition and processing technical field based on artificial intelligence, including, acquisition image data and text data, and pre-processing;Quality inspection multi-modal converter model is constructed, image quality score and text quality score are fused, and quality inspection result is generated;Image features and text features in quality inspection result are structured using VGG convolutional neural network Process, generate structured document;Enhanced BERT model is constructed to extract image features and text features in structured document, generate semantic analysis report, and automatically mark image quality problem;The application fuses image quality score and text quality score by quality inspection multi-modal converter model, realizes the depth integration and analysis of cross-modal data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition and processing based on artificial intelligence, and in particular to a digital archive automatic quality inspection method and system based on image recognition. BACKGROUND

[0002] With the continuous progress of information technology, the digital management of archives has gradually become an important development direction in the field of archives management. Digital archives have advantages such as easy storage, convenient retrieval, and efficient management compared to traditional paper archives. However, the quality control and quality inspection of digital archives still face many challenges. At present, image recognition technology is increasingly widely used in the quality inspection of digital archives. It mainly realizes the recognition and analysis of archive content through image processing and optical character recognition technology. Typical technologies include using convolutional neural networks for image feature extraction and classification, and using natural language processing algorithms for text data analysis and processing.

[0003] Although the existing technology has made some progress, there are still many deficiencies in the process of digital archive quality inspection. First, the image recognition technology has high dependence on image quality, and the recognition accuracy of low-quality or noisy images is low, which affects the overall quality inspection effect. Second, the existing technology lacks comprehensive analysis capability for multi-modal data, and cannot fully combine the information between images and texts for comprehensive quality evaluation. In addition, the existing quality inspection methods rely on manual operation and are difficult to realize efficient automatic quality inspection. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a digital archive automatic quality inspection method based on image recognition to solve the problem of automatic quality inspection.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a digital archive automatic quality inspection method based on image recognition, which includes collecting image data and text data and preprocessing; extracting features from the preprocessed image data and text data through ResNet model and NLP algorithm respectively, and generating image quality score and text quality score; constructing a quality inspection multi-modal converter model to fuse the image quality score and the text quality score, and generating a quality inspection result; using a VGG convolutional neural network to structure process the image features and text features in the quality inspection result, and generating a structured document; constructing an enhanced BERT model to extract image features and text features in the structured document, generating a semantic analysis report, and automatically labeling image quality problems.

[0008] As a preferred scheme of the image recognition-based automatic quality inspection method for digital archives provided by the application, the preprocessing comprises,

[0009] The image data is subjected to definition enhancement, denoising, image size standardization and image local contrast enhancement.

[0010] The text data is subjected to word segmentation, cleaning and text normalization.

[0011] As a preferred scheme of the image recognition-based automatic quality inspection method for digital archives provided by the application, the preprocessing comprises,

[0012] The pre-trained ResNet model is used to extract image features from the image data.

[0013] The definition, brightness and contrast scores of the image are obtained by extracting the image features, and an image quality score is generated, and the expression is:

[0014]

[0015] Wherein, E is the image quality score, N is the number of images, S(L i ) is the definition score of the i-th image, B(L i ) is the brightness score of the i-th image, C(L i ) is the contrast score of the i-th image, L i is the i-th image, and i is the index of the image position.

[0016] The NLP algorithm is used to extract text features from the preprocessed text data.

[0017] The complexity, keyword coverage and spelling and grammar accuracy scores of the text are obtained by extracting the text features, and a text quality score is generated, and the expression is:

[0018]

[0019] Wherein, G is the text quality score, M is the number of text paragraphs, C(O j ) is the complexity score of the j-th paragraph, F(O j ) is the keyword coverage score of the j-th paragraph, P(O j ) is the spelling and grammar accuracy score of the j-th paragraph, j is the text paragraph position, and O j is the j-th text paragraph.

[0020] As a preferred scheme of the image recognition-based automatic quality inspection method for digital archives, the image quality score and the text quality score are fused to generate a quality inspection result, and the specific steps are,

[0021] The input is defined as the image quality score and the text quality score, and the output is the quality inspection result;

[0022] The image quality score and the text quality score are standardized;

[0023] The image quality score and the text quality score are fused through a cross-modal attention mechanism to generate a quality inspection result, and the expression is:

[0024] U = alpha * E + beta * G;

[0025] Wherein, U is the quality inspection result, alpha is the coefficient of adjusting the image quality score E in the quality inspection result U, and beta is the coefficient of adjusting the text quality score G in the quality inspection result U.

[0026] As a preferred scheme of the image recognition-based automatic quality inspection method for digital archives, the image features and text features in the quality inspection result are structured by using a VGG convolutional neural network to generate a structured document, and the specific steps are,

[0027] The image features and text features in the quality inspection result are formatted and input into the VGG convolutional neural network;

[0028] The low-level features and high-level features of the image are extracted through the convolutional layer of the VGG convolutional neural network to identify the document elements in the image;

[0029] The extracted image low-level features and image high-level features and the text features extracted by the NLP algorithm are used to identify and classify each document element;

[0030] The identified and classified document elements are arranged in order and hierarchy to form a structured document;

[0031] The original image data and text data are compared to check whether the document elements in the structured document are consistent.

[0032] As a preferred scheme of the image recognition-based automatic quality inspection method for digital archives, the image features and text features in the structured document are extracted by constructing an enhanced BERT model to generate a semantic analysis report and automatically label image quality problems, and the specific steps are,

[0033] A cross-modal attention layer of the enhanced BERT model is defined;

[0034] Text features and image features in a structured document are extracted, and the text features are input into a cross-modal attention layer;

[0035] Image features and text features are deeply fused and analyzed using a cross-modal attention mechanism to generate a comprehensive analysis result;

[0036] The generated comprehensive analysis result is used to check the information consistency between image features and text features;

[0037] According to the result of the information consistency check, annotations are made to identify errors in image and text pairing, and a semantic analysis report is generated;

[0038] According to the information in the semantic analysis report, image quality problems are automatically annotated and saved, and correction suggestions are generated.

[0039] As a preferred scheme of the image recognition-based digital archive automatic quality inspection method, the cross-modal attention mechanism is used to deeply fuse and analyze image features and text features to generate a comprehensive analysis result, and the specific steps are,

[0040] In the cross-modal attention mechanism, image features are used as attention mechanisms, and text features are used as keys and values;

[0041] Attention weights are generated by calculating the dot product between the query and the key;

[0042] The attention weights are applied to the value to generate a deep fusion value;

[0043] According to the deep fusion value, the final comprehensive analysis result is generated, and the expression is:

[0044]

[0045] Where Φ is the comprehensive analysis result, EBERT is the result of the enhanced BERT model analyzing the text features and the context information, Θ is the recognized text features, Ω is the context information, Γ is the number of document elements, λ is the index of the document element, VGG is the analysis result of the VGG model on the document element, Δ λ is the feature of the λth document element.

[0046] In a second aspect, the present application provides an image recognition-based automatic quality inspection system for digital archives, comprising a collection module, a scoring module, a quality inspection module, a structured module and a labeling module; the collection module is configured to collect image data and text data and perform preprocessing; the scoring module is configured to extract features from the preprocessed image data and text data respectively by using a ResNet model and an NLP algorithm, and generate image quality scores and text quality scores; the quality inspection module is configured to construct a quality inspection multi-modal converter model, fuse the image quality scores and the text quality scores, and generate a quality inspection result; the structured module is configured to perform structured processing on image features and text features in the quality inspection result by using a VGG convolutional neural network, and generate a structured document; and the labeling module is configured to construct an enhanced BERT model to extract the image features and the text features in the structured document, generate a semantic analysis report, and automatically label image quality problems.

[0047] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the image recognition-based automatic quality inspection method for digital archives according to the first aspect of the present application is implemented.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, any step of the image recognition-based automatic quality inspection method for digital archives according to the first aspect of the present application is implemented.

[0049] The present application has the following advantages: the quality inspection multi-modal converter model of the present application fuses the image quality scores and the text quality scores, realizes deep integration and analysis of cross-modal data, further performs standardized processing, ensures that the image and text features are compared on the same scale, and generates a quality inspection result by using a cross-modal attention mechanism, so that the quality inspection multi-modal converter model not only improves the quality inspection accuracy of image and text data, but also makes the quality inspection process more comprehensive, intelligent and efficient by generating a comprehensive score, reduces manual intervention and improves work efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0051] Figure 1 The flowchart of the image recognition-based automatic quality inspection method for digital archives in embodiment 1.

[0052] Figure 2 The module diagram of the image recognition-based automatic quality inspection system for digital archives in Embodiment 1. DETAILED DESCRIPTION

[0053] In order to make the above objectives, features and advantages of the present application more apparent, specific embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0054] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details given herein, that the present application can be practiced with other than the described embodiments, and that variations from the particular embodiments described herein can be made and still be within the scope of the present application.

[0055] Secondly, the term "one embodiment" or "an embodiment" as used herein means that a particular implementation can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the purview of one of ordinary skill in the art to effect such a feature, structure, or characteristic in connection with other embodiments whether or not the other embodiments are described or otherwise set forth in this description.

[0056] Embodiment 1, Reference Figure 1 and Figure 2 The first embodiment of the present application provides an image recognition-based automatic quality inspection method for digital archives, which includes the following steps:

[0057] S1, collect image data and text data and perform preprocessing.

[0058] Further, the image data and text data of the archives are collected, the image data is acquired by a high-precision scanner or a high-resolution camera device, and is saved in a high-quality lossless compression format (such as TIFF);

[0059] The text data is extracted from the image by an OCR technique;

[0060] A high-pass filter is used to enhance the edge details of the image;

[0061] A median filter is used to remove noise in the image;

[0062] The image is adjusted to a uniform size to ensure consistency in subsequent processing;

[0063] An adaptive histogram equalization is used to enhance the contrast of the image;

[0064] The text data is subjected to word segmentation processing, i.e., continuous character sequences are divided into meaningful words or phrases, which facilitates subsequent text analysis and processing;

[0065] Cleaning the text data, removing irrelevant information and noise, such as removing punctuation, special characters, extra spaces and stop words (such as "of", "is", etc.), cleaning steps can improve the quality of text data and the accuracy of analysis;

[0066] Normalizing the text, including case conversion, synonym replacement, word reduction (such as converting the past tense or present tense of verbs to the base form), etc., to ensure the consistency and comparability of the text data;

[0067] It should be noted that through detailed preprocessing of image data and text data, the quality and consistency of the data are ensured, high-precision scanning and OCR technology are used to extract text data, ensuring the accuracy of images and text, high-pass filter and median filter are used to enhance image edge details and remove noise respectively, adjusting image size and contrast to ensure the consistency of image quality, text segmentation, cleaning and normalization steps improve the accuracy of text data analysis, and extracting keywords ensures the importance of text information. This step not only lays a solid foundation for subsequent feature extraction and quality scoring, but also significantly improves the accuracy and efficiency of the entire quality inspection process.

[0068] S2, extract features from preprocessed image data and text data respectively through ResNet model and NLP algorithm, generate image quality score and text quality score.

[0069] Further, TensorFlow or PyTorch is used as the ResNet model framework;

[0070] Using a pre-trained ResNet model to extract image features (sharpness, brightness, contrast) from image data;

[0071] Extract image features through a certain layer of the ResNet model (such as the pooling layer before the fully connected layer) to facilitate subsequent image quality evaluation;

[0072] ResNet is a powerful image recognition model that can efficiently extract and analyze image features, improving the performance and accuracy of image processing and recognition tasks;

[0073] Through the extracted image features, obtain the sharpness, brightness and contrast scores of the image (by training a simple linear regression model or neural network to map these image features to quality scores), and generate the image quality score, the expression is:

[0074]

[0075] Where E is the image quality score, N is the number of images, S(L i ) is the sharpness score of the i-th image, B(Li ) is the brightness score of the i-th image, C(L i ) is the contrast score of the i-th image, L i is the i-th image, and i is the index of the image position.

[0076] Through the image quality score index, the quality inspection process of the digitized archives can be optimized to ensure the high quality and consistency of the archive images.

[0077] The preprocessed text data is subjected to text feature extraction using the natural language processing (NLP) algorithm, and the extracted text features are subjected to semantic analysis to identify keywords, sentence structure, and semantic relationships.

[0078] Natural language processing (NLP) is a branch of artificial intelligence that aims to enable computers to understand, interpret, and generate human language. The NLP algorithm combines computational linguistics and artificial intelligence techniques to process and analyze large amounts of natural language data.

[0079] Through the extracted text features, the complexity, keyword coverage, and spelling and grammar accuracy scores of the text are obtained, and a text quality score is generated, expressed as:

[0080]

[0081] where G is the text quality score, M is the number of text paragraphs, C(O j ) is the complexity score of the j-th paragraph, F(O j ) is the keyword coverage score of the j-th paragraph, P(O j ) is the spelling and grammar accuracy score of the j-th paragraph, j is the position of the text paragraph, and O j is the j-th text paragraph.

[0082] It should be noted that the ResNet model and the NLP algorithm are used to extract image and text data features, respectively, to generate image and text quality scores, significantly improving the quality inspection effect of digitized archives. The ResNet model performs well in image feature extraction, accurately evaluating the clarity, brightness, and contrast of images to ensure consistent image quality. TensorFlow or PyTorch is used as the framework to further improve the training and application efficiency of the model. The NLP algorithm effectively identifies the complexity, keyword coverage, and spelling and grammar accuracy of the text through steps such as word segmentation and semantic analysis, generating a text quality score. This step combines deep learning and natural language processing techniques to achieve efficient quality inspection of image and text data, ensuring the high quality and consistency of archives.

[0083] S3, construct a quality inspection multi-modal converter model, fuse the image quality score and the text quality score to generate a quality inspection result.

[0084] Further, define the input as the image quality score and the text quality score, and the output as the quality inspection result;

[0085] Standardize the image quality score and the text quality score, so that the image features and the text features in the image quality score and the text quality score are on the same scale;

[0086] Fuse the image quality score and the text quality score through the cross-modal attention mechanism to generate the quality inspection result, the expression is:

[0087] U = a x E + b x G;

[0088] Wherein, U is the quality inspection result, a is the coefficient of adjusting the image quality score E in the quality inspection result U, by adjusting the value of a, the influence of the image quality score in the final comprehensive score can be controlled, if the quality of the image is very important to the overall quality inspection result, the value of a can be set larger, otherwise, if the image quality has less influence, the value of a can be smaller, b is the coefficient of adjusting the text quality score G in the quality inspection result U, by adjusting the value of b, the influence of the text quality score in the final comprehensive score can be controlled, if the quality of the text is very important to the overall quality inspection result, the value of b can be set larger, otherwise, if the text quality has less influence, the value of b can be smaller;

[0089] It should be noted that by constructing the quality inspection multi-modal converter model, the image quality score and the text quality score are fused after standardization, the cross-modal attention mechanism is used to ensure that the image and text features are integrated in the same scale, the comprehensiveness and accuracy of the quality inspection result are improved, the influence of different scores on the quality inspection result is controlled by adjusting the weight coefficient, so that the score is more flexible and accurate.

[0090] S4, use the VGG convolutional neural network to structure the image features and the text features in the quality inspection result to generate a structured document.

[0091] Further, the image features and the text features in the quality inspection result are formatted and input into the VGG convolutional neural network;

[0092] The VGG convolutional neural network is mainly used for image classification, feature extraction and other computer vision tasks, and is widely used in various image processing applications due to its efficient feature extraction capability;

[0093] The image features in the quality inspection result are formatted to adapt to the input requirements of the VGG network;

[0094] Low-level features (such as edges, textures, etc.) and high-level features (such as shapes, structures, etc.) of the image are extracted by the convolutional layers of the VGG convolutional neural network, and document elements (such as titles, paragraphs, tables, etc.) in the image are recognized;

[0095] Using the extracted low-level image features and high-level image features and text features extracted by NLP algorithms, each document element is identified and classified respectively, ensuring that each element is accurately labeled and classified;

[0096] The identified and classified document elements are arranged in order and hierarchy to form a structured document, such as arranging in the hierarchy of title, paragraph, table;

[0097] Comparing the original image data and text data, checking whether the document elements in the structured document are consistent, ensuring the integrity and accuracy of the structured document;

[0098] It should be noted that using the VGG convolutional neural network to structure the image features and text features in the quality inspection results can generate more accurate structured documents. After formatting the image and text features into the VGG network, the convolutional layers of the network can extract low-level and high-level features, recognize document elements in the image, and accurately label and classify each element. By arranging in order and hierarchy, a complete structured document is formed. Comparing the original image and text data ensures the consistency and integrity of the document elements. This not only improves the accuracy of quality inspection, but also ensures the clarity of the document structure and the completeness of the information, ultimately achieving efficient digital archive quality inspection and management.

[0099] S5, construct an enhanced BERT model to extract image features and text features in the structured document, generate a semantic analysis report, and automatically label image quality problems.

[0100] Further, a cross-modal attention layer of the enhanced BERT model is defined; the purpose of this layer is to fuse image features and text features through a cross-modal attention mechanism. The cross-modal attention layer can dynamically adjust the attention weight based on the correlation between the image and the text, thereby enhancing the model's understanding of the joint features of the image and the text;

[0101] Extract text features and image features from the structured document, and input the text features into the cross-modal attention layer;

[0102] Using the cross-modal attention mechanism, the image features and text features are deeply fused and analyzed to generate a comprehensive analysis result;

[0103] In the cross-modal attention mechanism, the image features are used as the query of the attention mechanism, and the text features are used as the key and value;

[0104] The attention weights are generated by computing the dot product between the query and the keys, which represent the relevance between the image features and the text features, expressed as:

[0105]

[0106] where W is the attention weight, Q is the query matrix, K is the key matrix, d k is the dimension of the key, used to scale the dot product result, K T is the transpose of the key matrix K, T is the transpose, softmax is the probability distribution function applied to the result to obtain the probability distribution;

[0107] The attention weights are applied to the values to generate the deep fusion values, expressed as:

[0108] Output = W x Qv;

[0109] where Output is the deep fusion value, representing the joint representation of the image features and the text features after deep fusion, Qv is the value matrix, which represents the values of the image features and the text features here;

[0110] According to the deep fusion values, the final comprehensive analysis result is generated, expressed as:

[0111]

[0112] where Φ is the comprehensive analysis result, EBERT is the result of the enhanced BERT model performing semantic analysis on the text features and the context information, Θ is the recognized text features, Ω is the context information, Γ is the number of document elements, λ is the index of the document element, VGG is the analysis result of the VGG model on the document element, Δ λ is the feature of the λth document element;

[0113] By generating the comprehensive analysis result, the information consistency between the image features and the text features is checked to ensure that the information in the image is consistent with the text description;

[0114] According to the result of the information consistency check, the labeling is performed to identify errors in the image and text pairing (such as identifying whether the text description matches the image content or whether there are errors), and a semantic analysis report is generated to describe the errors and inconsistent parts in detail;

[0115] According to the information in the semantic analysis report, the image quality problems are automatically labeled and saved, and correction suggestions are generated;

[0116] Specifically, after generating the semantic analysis report, further automatic processing is performed according to the information in the report;

[0117] Automatically labeling image quality issues, which includes labeling images based on the errors and issues identified in the semantic analysis report;

[0118] Saving the labeling report, which details the quality issues of each image and their locations;

[0119] Generating correction suggestions based on the labeling report, which are generated based on the consistency checks in the semantic analysis report and the image quality labeling, to guide subsequent correction and handling work;

[0120] Specifically,

[0121] Detailed description of the image quality issues labeled in the semantic analysis report, such as image blur, insufficient brightness, poor contrast, text and image inconsistency, etc.;

[0122] Provide specific examples and locations for each issue, allowing users to quickly locate the problem;

[0123] Provide specific correction methods for each identified quality issue, such as:

[0124] For image blur, suggest using image enhancement algorithms (such as denoising, sharpening, etc.) for repair;

[0125] For images with insufficient brightness, suggest adjusting the brightness parameters and using image processing software for brightness correction;

[0126] For text and image inconsistency, suggest re-auditing the text and image content to ensure information matching;

[0127] According to the severity and impact of the problem, prioritize the correction suggestions, with the most urgent and most impactful correction suggestions at the top, ensuring that critical issues are addressed first;

[0128] Provide tool and resource recommendations needed to implement the correction suggestions, such as recommending specific software or algorithms for image processing, or providing relevant online tutorials and guides;

[0129] Provide specific operation steps for each correction suggestion to ensure users can follow the guidance to complete the correction process step by step, including detailed step descriptions and example operations to help users efficiently complete the correction work;

[0130] Suggest that users re-qualify after correction to ensure that the problem has been solved;

[0131] Provide a feedback mechanism for users to record problems and results during the correction process to further optimize the quality inspection process;

[0132] It should be noted that the construction of the enhanced BERT model, the extraction of image features and text features in the structured document, and the generation of the semantic analysis report can effectively fuse cross-modal data, dynamically adjust the attention weight through the cross-modal attention mechanism, enhance the understanding of the model to the joint features of image and text, and generate a comprehensive analysis result that ensures the information consistency between image and text, can automatically identify and label image quality problems, and save detailed labeling reports, use these labels to generate correction suggestions to guide subsequent correction and processing work, which greatly improves the accuracy and efficiency of quality inspection, reduces manual intervention, and ensures the comprehensiveness and consistency of the quality of the archives.

[0133] The embodiment also provides a digital archive automatic quality inspection system based on image recognition, which comprises an acquisition module, a scoring module, a quality inspection module, a structured module and a labeling module; the acquisition module is used for acquiring image data and text data and performing preprocessing; the scoring module is used for extracting features of the preprocessed image data and text data respectively through a ResNet model and an NLP algorithm, and generating image quality scores and text quality scores; the quality inspection module is used for constructing a quality inspection multi-modal converter model, fusing the image quality scores and the text quality scores, and generating a quality inspection result; the structured module is used for performing structured processing on image features and text features in the quality inspection result by using a VGG convolutional neural network, and generating a structured document; and the labeling module is used for constructing an enhanced BERT model, extracting image features and text features in the structured document, generating a semantic analysis report, and automatically labeling image quality problems.

[0134] The embodiment also provides a computer device suitable for the case of the digital archive automatic quality inspection method based on image recognition, which comprises a memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the digital archive automatic quality inspection method based on image recognition proposed in the above embodiment.

[0135] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0136] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the method for automatically checking a digital archive based on image recognition as described above. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0137] In summary, the application fuses the image quality score and the text quality score through the quality inspection multi-modal converter model, realizes deep integration and analysis of cross-modal data, further standardizes the processing, ensures that the image and text features are compared on the same scale, and generates a quality inspection result by using a cross-modal attention mechanism. The quality inspection multi-modal converter model not only improves the quality inspection accuracy of image and text data, but also makes the quality inspection process more comprehensive, intelligent and efficient by generating a comprehensive score, reduces manual intervention, and improves work efficiency.

[0138] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. An image recognition-based automatic quality inspection method for digitized archives, characterized in that: The application relates to a quality inspection method and device for image-text paired data. Image data and text data are collected and preprocessed; Image quality scores and text quality scores are generated by extracting features from the preprocessed image data and text data respectively through ResNet models and NLP algorithms; A quality inspection multi-modal converter model is constructed to fuse the image quality scores and the text quality scores to generate a quality inspection result; A structured document is generated by structuring the image features and the text features in the quality inspection result using a VGG convolutional neural network; An enhanced BERT model is constructed to extract the image features and the text features in the structured document, generate a semantic analysis report, and automatically label image quality problems; The specific steps of constructing the quality inspection multi-modal converter model to fuse the image quality scores and the text quality scores to generate the quality inspection result are as follows: The input is defined as the image quality scores and the text quality scores, and the output is defined as the quality inspection result; The image quality scores and the text quality scores are standardized; The image quality scores and the text quality scores are fused through a cross-modal attention mechanism to generate the quality inspection result, and the expression is as follows: ； wherein, is the quality inspection result, is the adjusted image quality score is the coefficient in the quality inspection result is the coefficient in the quality inspection result is the adjusted text quality score is the coefficient in the quality inspection result is the coefficient in the quality inspection result The specific steps of using the VGG convolutional neural network to structure the image features and the text features in the quality inspection result to generate the structured document are as follows: The image features and the text features in the quality inspection result are formatted and input into the VGG convolutional neural network; Low-level features and high-level features of the image are extracted through the convolutional layers of the VGG convolutional neural network to identify the document elements in the image; The extracted low-level image features and high-level image features and the text features extracted by the NLP algorithm are used to identify and classify each document element respectively; The identified and classified document elements are arranged in order and hierarchy to form the structured document; The consistency of the document elements in the structured document is checked by comparing the original image data and the text data; The specific steps of constructing the enhanced BERT model to extract the image features and the text features in the structured document, generate a semantic analysis report, and automatically label image quality problems are as follows: A cross-modal attention layer of the enhanced BERT model is defined; The text features and the image features in the structured document are extracted, and the text features are input into the cross-modal attention layer; The image features and the text features are deeply fused and analyzed through the cross-modal attention mechanism to generate a comprehensive analysis result; The information consistency between the image features and the text features is checked through the generated comprehensive analysis result; According to the result of the information consistency check, errors in the image-text pairing are identified and labeled to generate a semantic analysis report; According to the information in the semantic analysis report, image quality problems are automatically labeled and saved to generate a correction suggestion.

2. The image recognition based digital archive automatic quality inspection method of claim 1, wherein: The preprocessing includes: The image data is subjected to clarity enhancement, denoising, image size standardization, and image local contrast enhancement; The text data is subjected to word segmentation, cleaning, and text standardization.

3. The image recognition based digital archive automatic quality inspection method of claim 1, wherein: The specific steps of extracting features from the preprocessed image data and text data through ResNet models and NLP algorithms to generate image quality scores and text quality scores are as follows: Pretrained ResNet models are used to extract image features from the image data; By extracting image features, the clarity, brightness and contrast scores of the image are obtained, and the image quality score is generated, the expression is: ； wherein, is an image quality score, is a number of images, is a sharpness score of the image, is a brightness score of the image, is a contrast score of the image, is a colorfulness score of the image, is an index of an image position; The preprocessed text data is subjected to text feature extraction using NLP algorithm; By extracting text features, the complexity, keyword coverage and spelling and grammar accuracy scores of the text are obtained, and the text quality score is generated, the expression is: ； wherein, is a text quality score, is a number of text passages, is a complexity score of a first passage, is a keyword coverage score of a first passage, is a spelling and grammar accuracy score of a first passage, is a text passage position, is a first text passage.

4. The image recognition based digital archive automatic quality inspection method of claim 1, wherein: The image features and text features are deeply fused and analyzed by using the cross-modal attention mechanism to generate a comprehensive analysis result, the specific steps are, In the cross-modal attention mechanism, the image features are used as the query of the attention mechanism, and the text features are used as the key and value; By calculating the dot product between the query and the key, the attention weight is generated; The attention weight is applied to the value to generate a deep fusion value; According to the deep fusion value, the final comprehensive analysis result is generated, the expression is: ； wherein, is the comprehensive analysis result, is the result of enhancing the semantic analysis of the text features and context information by the BERT model, is the recognized text feature, is the context information, is the number of document elements, is the index of the document element, is the analysis result of the VGG model on the document element, is the feature of the th document element.

5. An image recognition-based automatic quality inspection system for digitized archives, based on the image recognition-based automatic quality inspection method for digitized archives according to any one of claims 1 to 4, characterized in that: It includes a collection module, a scoring module, a quality inspection module, a structured module and a labeling module; The collection module is used to collect image data and text data and perform preprocessing; The scoring module is used to extract features from the preprocessed image data and text data by ResNet model and NLP algorithm respectively, and generate image quality score and text quality score; The quality inspection module is used to construct a quality inspection multi-modal converter model to fuse the image quality score and the text quality score to generate a quality inspection result; The structured module is used to use the VGG convolutional neural network to structure the image features and text features in the quality inspection result to generate a structured document; The labeling module is used to construct an enhanced BERT model to extract image features and text features from the structured document to generate a semantic analysis report and automatically label image quality problems. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the image recognition based digital archive automatic quality inspection method of any one of claims 1-4.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the image recognition based digital archive automatic quality inspection method of any one of claims 1-4.

Images