An archive image metadata intelligent extraction method and system

Abstract

The application provides an archive image metadata intelligent extraction method and system, the method comprises: acquiring an archive image to be extracted; performing multi-source metadata capture on the archive image to obtain multi-modal candidate metadata; detecting and positioning a potential tampering area based on the homogeneity difference feature to obtain a tampering area positioning map; performing layout semantic structure analysis on the archive image to divide it into multiple semantic areas; performing spatial mapping of the tampering area positioning map and each semantic area, determining the tampered semantic area based on the mapping result, and reducing the confidence of the extracted candidate metadata in the tampered semantic area to obtain adjusted candidate metadata; generating fused metadata according to the evidence weight of each semantic area; and performing knowledge graph-based semantic enhancement processing on the fused metadata to obtain final metadata, thereby realizing high-confidence and high-integrity archive metadata extraction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for intelligent extraction of archival image metadata. Background Technology

[0002] With the comprehensive advancement of archival digitization, massive amounts of paper archives are being converted into digital image formats for storage, management, and utilization. Automatically and accurately extracting metadata (such as issuing authority, issuance date, document number, and seal information) from archival images is the core foundation for building intelligent archival systems and achieving efficient retrieval and in-depth utilization of archival resources.

[0003] However, existing archival image metadata extraction technologies still have the following shortcomings in practical applications: First, the information capture dimension is singular, resulting in a high rate of missing key information. Mainstream technical solutions often process information from different sources independently, such as parsing standard metadata fields like EXIF ​​separately, performing optical character recognition (OCR) to extract text separately, or analyzing the visual features of images separately. This single-modal processing approach lacks a cross-modal information fusion mechanism and cannot establish connections between different information sources. For example, when handwritten annotations in an archival image describe the same thing as printed titles, traditional methods cannot effectively link them, leading to the omission of key metadata.

[0004] Second, the system suffers from insufficient adaptability to image quality and rigid recognition strategies. Archival images vary greatly in origin and quality, ranging from clear, standardized electronic archives to faded, wrinkled, and unevenly lit historical documents. Existing OCR systems typically use fixed confidence thresholds for text filtering, failing to dynamically adjust processing strategies based on image quality. High thresholds lead to significant loss of valuable information in historical archives, while low thresholds introduce noise interference when processing clear archives, making it difficult to balance recognition coverage and accuracy.

[0005] Third, there is a lack of understanding of the semantic structure of the archival layout. Traditional techniques treat archival images as collections of pixels or simple sequences of text, neglecting the structured semantics of the archival layout. In archival images, different areas such as titles, body text, seals, and interlocking seals have drastically different evidentiary value and legal effect. A uniform processing method results in the identification priority of key metadata such as seals and issuing authorities being the same as ordinary text content, reducing the accuracy and reliability of extracting important fields.

[0006] Fourth, the ability to resolve semantic ambiguity is weak, and the handling of missing values ​​is simplistic. Existing technologies lack effective association between extracted metadata and domain knowledge, making it difficult to resolve issues such as synonym ambiguity (e.g., "responsible person" vs. "handler," "signer" vs. "approver") and confusion regarding hierarchical relationships (e.g., "contract" includes "purchase contract" and "sales contract"), leading to missed or false detections during retrieval. Furthermore, for fields that cannot be identified due to image quality or occlusion, simple blanking or default value filling is typically used, without utilizing the inherent logical relationships within the archives for reasoning and completion. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a method and system for intelligent extraction of archival image metadata, which solves the technical problems in the prior art.

[0008] On the one hand, this invention provides the following technical solution: an intelligent extraction method for archival image metadata, the method comprising: Obtain the image of the file to be extracted; Multi-source metadata capture is performed on the archive images to obtain multimodal candidate metadata; Extract homogeneity difference features from the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map; The document image is analyzed for its layout semantic structure, divided into multiple semantic regions, and a preset evidence weight is assigned to each semantic region according to the layout attributes of each semantic region. The tampered region location map is spatially mapped to each of the semantic regions. Based on the mapping result, the tampered semantic regions are determined, and the confidence of the candidate metadata extracted from the tampered semantic regions is reduced to obtain the adjusted candidate metadata. Based on the evidence weights of each semantic region, multimodal feature fusion and adaptive threshold decision are performed on the adjusted candidate metadata to generate fused metadata; The fused metadata is subjected to semantic enhancement processing based on knowledge graphs to obtain the final metadata with confidence level and source tagging after entity disambiguation and reasoning completion.

[0009] Compared with existing technologies, the beneficial effects of this invention are as follows: by establishing a multi-source metadata parallel capture mechanism, standard metadata fields, visual features, printed text and seal information are extracted simultaneously, and multimodal feature fusion is performed based on the evidence weight of each semantic region. This breaks the limitation of single-modal information silos in existing technologies, and can effectively establish the association between different information sources (such as associating handwritten annotations and printed titles with the same matter), significantly reducing the omission rate of key metadata.

[0010] By performing semantic structure analysis on the archival images, the images are divided into multiple semantic regions and differentiated evidence weights are assigned according to the layout attributes of each region. This allows key fields with high legal validity, such as seals and issuing authorities, to receive higher processing priority. This solves the problem of existing technologies that process all regions uniformly and ignore the semantic structure of the layout, effectively improving the accuracy and reliability of extracting important metadata fields.

[0011] By invoking a pre-built knowledge graph of the archival domain, entity linking and disambiguation are performed on the fused metadata, effectively resolving issues such as synonym ambiguity (e.g., "responsible person" and "handler") and confusion of hierarchical relationships (e.g., "contract" includes "purchase contract" and "sales contract"), thus improving the accuracy and recall of retrieval. Simultaneously, rules and relationships within the knowledge graph are used to reason and complete missing fields, overcoming the shortcomings of existing technologies that simply leave blanks or fill in default values, significantly improving the completeness of the metadata.

[0012] By extracting features from archival images that reflect the homogeneity differences between tampered and real areas, potential tampered areas are detected and located. The tampered area location map is then spatially mapped to the semantic region. The confidence weight of candidate metadata extracted from the tampered area is reduced, ensuring the credibility of the extracted metadata from the source and making up for the lack of authenticity verification capability in existing technologies.

[0013] Furthermore, the step of capturing multi-source metadata for the archive image includes: Establish a parallel processing pipeline to perform the following operations synchronously: Parse the standard metadata fields embedded in the archive image to obtain basic information; Global and local visual features of the archive images are extracted using a convolutional neural network. The archive image is initially screened using multi-engine OCR, and a set of candidate text blocks and their corresponding confidence scores are output. The archive image is subjected to color space conversion and morphological filtering to separate the seal area and perform radial distortion correction in order to extract seal information.

[0014] Furthermore, the multi-engine OCR, when performing initial text screening on the archive image, also includes an adaptive quality enhancement step: The archive image is subjected to a quality pre-assessment, which includes at least calculating a sharpness index that characterizes the sharpness of image edges, an illumination index that characterizes the uniformity of illumination, and a noise index that characterizes the intensity of random noise. A comprehensive quality score is generated based on the sharpness index, the illumination index, and the noise index. The archive image is divided into different quality scenes according to the comprehensive quality score, and the corresponding OCR engine and preprocessing strategy are dynamically selected for different quality scenes. The confidence threshold for OCR recognition is adaptively adjusted based on the comprehensive quality score, wherein the threshold is lowered and context verification is enabled for archival images of historical archive type, and the threshold is increased for archival images of electronic archive type to ensure accuracy; The overall quality score is calculated using the following formula:

[0015] in, For the overall quality score, The aforementioned sharpness index, The illumination index is... The noise index is... , For preset weighting coefficients, This is the normalization function.

[0016] Furthermore, the step of extracting seal information includes: The archive image is converted from the RGB color space to the HSV color space, and a red region mask is generated based on the preset red hue range, saturation threshold and brightness threshold to detect candidate areas for the stamp. The circular or elliptical regions in the candidate seal region are detected by the Maximum Stable Extreme Region (MSER) ​​algorithm, and the roundness of each detected circular or elliptical region is calculated for shape verification to filter out non-seal noise. Color clustering is performed on the candidate seal regions retained after shape verification to separate the seal text from the background pattern, and connected component analysis is performed on the overlapping text regions to segment the connected characters. For the candidate regions of the seal after color clustering and connected component analysis, their elliptical contours are detected, and the major and minor axes of the elliptical contours are determined. Based on the ratio of the major axis to the minor axis, a polar coordinate mapping relationship is established, and the elliptical region is corrected into a planar front view to obtain the corrected seal image. The corrected seal image is input into a dedicated recognition model, which outputs the seal text sequence and position information. The position information includes at least the inner ring text, the outer ring text, and the central pattern.

[0017] Furthermore, obtaining the tampered area location map is achieved through a two-stream network consisting of an RGB stream branch and a noise stream branch. The processing flow of the two-stream network includes: Feature extraction is performed on the archive image through the RGB stream branch to obtain a multi-level RGB feature map; Multi-scale visual coding of the archive image is performed through the noise stream branch to obtain multi-level global noise features; The tampered region location map is generated by gradually fusing the multi-level RGB feature map with the multi-level global noise features through three sequentially arranged complementary modules, following a coarse-to-fine strategy.

[0018] Furthermore, the tampered region location map is a pixel-level binary classification result, and each pixel in the tampered region location map is marked as a tampered pixel or a real pixel; The step of spatially mapping the tampered region location map to each of the semantic regions, determining the tampered semantic regions based on the mapping results, and reducing the confidence of candidate metadata extracted from the tampered semantic regions specifically includes: Based on the pixel-level binary classification results of the tampered region location map, the pixel coordinates in the tampered region location map are compared with the boundary coordinates of each semantic region, and the proportion of the area of ​​pixels marked as tampered in each semantic region to the total area of ​​the semantic region is calculated. If the proportion within a certain semantic region exceeds a preset threshold, then the semantic region is determined to be a tampered semantic region. A suspected tampering marker is attached to the candidate metadata extracted from the tampered semantic region, or its confidence level is multiplied by a preset attenuation coefficient to reduce the confidence level of the candidate metadata.

[0019] Furthermore, multimodal feature fusion of the adjusted candidate metadata includes: Cross-modal conflict detection is performed on similar information of different modalities in the adjusted candidate metadata; When a conflict is detected, based on the document type of the archive image, the candidate metadata modalities are weighted, fused, and conflict resolved according to a preset modal credibility priority rule to generate a fused metadata field. The modal credibility priority rule includes at least the following: the weight of seal information is greater than the weight of printed text, and the weight of printed text is greater than the weight of handwritten content; the same type of information includes at least one of time information, subject information, and event information; For the aforementioned core information conflict, the coreference resolution score is calculated using the following formula:

[0020] in, To resolve fractions by common reference, These are the entity name strings identified from the text fragment. For entities respectively and The set of context words of the text segment. There are two candidate entities. For string similarity functions, Context similarity function and Preset weighting coefficients.

[0021] For the aforementioned time information conflict, the final merged time is calculated using the following formula:

[0022] in, For the first Candidate times for each modality, For document types based on the archive image Dynamically adjusted credibility weights For indicator functions, This refers to the final time after the fusion.

[0023] Secondly, this invention provides the following technical solution: an intelligent extraction system for archival image metadata, the system comprising: The capture module is used to acquire the archive image to be extracted; and to capture multi-source metadata of the archive image to obtain multimodal candidate metadata. The module is used to extract homogeneity difference features from the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map. The allocation module is used to perform layout semantic structure analysis on the archive image, divide it into multiple semantic regions, and assign preset evidence weights to each semantic region according to the layout attributes of each semantic region. The module is used to spatially map the tampered region location map with each of the semantic regions, determine the tampered semantic regions based on the mapping results, and reduce the confidence of the candidate metadata extracted from the tampered semantic regions to obtain the adjusted candidate metadata. The generation module is used to perform multimodal feature fusion and adaptive threshold decision on the adjusted candidate metadata based on the evidence weights of each semantic region, and generate fused metadata. The processing module is used to perform knowledge graph-based semantic enhancement processing on the fused metadata to obtain the final metadata with confidence and source tags after entity disambiguation and reasoning completion.

[0024] Thirdly, the invention provides the following technical solution: a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described intelligent extraction method for archive image metadata.

[0025] Fourthly, the invention provides the following technical solution: a storage medium storing a computer program, which, when executed by a processor, implements the above-described intelligent extraction method for archive image metadata. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of the intelligent extraction method for archival image metadata provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of a dual-stream network structure provided in the first embodiment of the present invention; Figure 3 This is a structural block diagram of the intelligent extraction system for archival image metadata provided in the second embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of a computer provided in the third embodiment of the present invention.

[0028] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain embodiments of the present invention, and should not be construed as limiting the present invention.

[0030] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0031] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0032] Example 1 In the first embodiment of the present invention, please refer to Figure 1 and Figure 2 As shown, a method for intelligent extraction of archival image metadata includes the following steps S01 to S06: S01, acquire the archive image to be extracted; perform multi-source metadata capture on the archive image to obtain multimodal candidate metadata; Specifically, the step of capturing multi-source metadata for the archive image includes: Establish a parallel processing pipeline to perform the following operations synchronously: Parse the standard metadata fields embedded in the archive image to obtain basic information; Global and local visual features of the archive images are extracted using a convolutional neural network. The archive image is initially screened using multi-engine OCR, and a set of candidate text blocks and their corresponding confidence scores are output. The archive image is subjected to color space conversion and morphological filtering to separate the seal area and perform radial distortion correction in order to extract seal information.

[0033] Specifically, the multi-engine OCR, when performing initial text screening on the archive image, also includes an adaptive quality enhancement step: The archive image is subjected to a quality pre-assessment, which includes at least calculating a sharpness index that characterizes the sharpness of image edges, an illumination index that characterizes the uniformity of illumination, and a noise index that characterizes the intensity of random noise. A comprehensive quality score is generated based on the sharpness index, the illumination index, and the noise index. The archive image is divided into different quality scenes according to the comprehensive quality score, and the corresponding OCR engine and preprocessing strategy are dynamically selected for different quality scenes. The confidence threshold for OCR recognition is adaptively adjusted based on the comprehensive quality score, wherein the threshold is lowered and context verification is enabled for archival images of historical archive type, and the threshold is increased for archival images of electronic archive type to ensure accuracy; The overall quality score is calculated using the following formula:

[0034] in, For the overall quality score, The aforementioned sharpness index, The illumination index is... The noise index is... , For preset weighting coefficients, This is the normalization function.

[0035] Specifically, the steps for extracting seal information include: The archive image is converted from the RGB color space to the HSV color space, and a red region mask is generated based on the preset red hue range, saturation threshold and brightness threshold to detect candidate areas for the stamp. The circular or elliptical regions in the candidate seal region are detected by the Maximum Stable Extreme Region (MSER) ​​algorithm, and the roundness of each detected circular or elliptical region is calculated for shape verification to filter out non-seal noise. Color clustering is performed on the candidate seal regions retained after shape verification to separate the seal text from the background pattern, and connected component analysis is performed on the overlapping text regions to segment the connected characters. For the candidate regions of the seal after color clustering and connected component analysis, their elliptical contours are detected, and the major and minor axes of the elliptical contours are determined. Based on the ratio of the major axis to the minor axis, a polar coordinate mapping relationship is established, and the elliptical region is corrected into a planar front view to obtain the corrected seal image. The corrected seal image is input into a dedicated recognition model, which outputs the seal text sequence and position information. The position information includes at least the inner ring text, the outer ring text, and the central pattern.

[0036] In this embodiment, a four-channel parallel processing pipeline is established for the input file image to be extracted, and the following operations are performed synchronously: 1. Standard metadata channel: This process parses standard metadata fields embedded in archival images, including but not limited to fields defined by standards such as EXIF ​​(Exchangeable Image File Format), IPTC (International Press Telecommunication Council), and XMP (Extensible Metadata Platform). Extracted basic information includes: camera model, shooting timestamp, image resolution, color space, and compression format. For scanned archival images, scanning resolution and scanning device information will also be extracted. The output data of this channel is a structured set of key-value pairs.

[0037] 2. Visual feature channels: Feature extraction from archival images is performed using a pre-trained convolutional neural network. The network architecture can use ResNet-50 or EfficientNet as the backbone, pre-trained on the ImageNet dataset, and then fine-tuned on the archival image dataset. Extracted global visual features include: document type (official documents, contracts, certificates, invoices, etc.), layout type (official document layout, letter layout, tabular layout, etc.), and overall image quality level. Extracted local visual features include: table region bounding boxes, heatmaps of handwritten marks, and bounding boxes of seal locations. The output data format of this channel is a multi-dimensional feature vector and region coordinates with category labels.

[0038] 3. Printed text channel: Multi-engine OCR is used for initial text screening of archival images. Specifically, this embodiment configures three OCR engines: a general printed font recognition engine (suitable for standard printed fonts such as Song, Hei, and Kai), a handwritten font recognition engine (suitable for handwritten fonts such as Xing and Cao), and an enhanced recognition engine for special fonts (such as traditional and variant characters). Each engine runs independently and outputs a set of candidate text blocks. Each candidate text block contains three attributes: text content, bounding box coordinates, and recognition confidence.

[0039] 4. Dedicated channel for seals: Specialized extraction of seal information is performed on archival images. Since archival seals are usually red, circular or oval, and carry key legal information such as the name of the issuing authority and the type of seal, this channel is designed with a complete processing link from detection to identification.

[0040] This process involves performing layered, progressive quality enhancement processing on archival images before or simultaneously with OCR recognition to address the adaptive recognition problem of archival images of varying quality. This processing consists of three stages: Phase 1: Image quality pre-assessment; Calculate the following three quality metrics for archival images: (1) Sharpness index: The sharpness of image edges is calculated based on the Laplacian operator. The calculation formula is:

[0041] in, For the Laplace operator, and These are the height and width of the image, respectively. For pixels The grayscale value at that location. This indicator reflects the richness of edge information in the image; a higher value indicates a clearer image.

[0042] (2) Illumination index: Divide the image into M×N equal-sized image blocks, calculate the local mean of each image block, and calculate the standard deviation of each local mean. and the global mean of the image. Comparison:

[0043] The value range of this illumination index is [0,1]. The closer the value is to 1, the more uniform the illumination distribution. The closer the value is to 0, the more serious the illumination unevenness problem (such as local dark or overexposed areas).

[0044] (3) Noise index: The image is smoothed using median filtering, and the ratio of residual energy before and after filtering is calculated:

[0045] in, This is the image after median filtering. This represents the L2 norm. This indicator reflects the intensity of random noise; a higher value indicates more severe noise.

[0046] A comprehensive quality score is generated based on the above three indicators. :

[0047] in, , The preset weighting coefficients are set to 0.4, 0.35, and 0.25 in this embodiment, respectively. For the normalization function, Map to the interval [0,1].

[0048] Phase Two: Dynamic Engine Selection; Based on the overall quality score The archival images are divided into three categories: High-quality images ( (i.e., sharpness greater than or equal to the threshold and uniform illumination): enable the high-speed print engine and process using standard resolution. In this embodiment, The value is 0.8; low quality images ( (i.e., sharpness below a threshold or the presence of wrinkles): switch to the handwriting-specific engine and enable super-resolution preprocessing to enhance image details. In this embodiment, The value is 0.5; Mixed quality region ( The system employs a sliding window strategy, calling the adaptation engine separately for different regions. The window size is dynamically determined based on the image resolution; in this embodiment, it defaults to 64×64 pixels.

[0049] Phase 3: Confidence-adaptive calibration; An image quality-confidence mapping table is established, and the effective threshold of the OCR results is dynamically adjusted based on the pre-assessment results of the quality. This embodiment uses a piecewise function for threshold setting:

[0050] in This is an adaptive confidence threshold function that dynamically adjusts the effective threshold for OCR recognition based on image quality. This is an image quality parameter. For historical archives (faded, blurry), lower the threshold to 0.60 but enable contextual verification to help correct recognition errors; for electronic archives (clear, standardized), increase the threshold to 0.95 to ensure accuracy.

[0051] This step involves specifically extracting the red seal from the archival image. The specific sub-steps are as follows: First, the archival image is converted from the RGB color space to the HSV color space. The HSV color space describes colors in a way that is more in line with human perception, making it easier to separate the red stamp area.

[0052] Define the red area mask :

[0053] in, , , These represent the hue, saturation, and brightness channels, respectively. In this embodiment, This is the lower threshold value for the red hue range, with a value of 10. This is the upper limit threshold for the red hue range, with a value of 160. The minimum threshold for saturation is set to 80. The minimum brightness threshold is set to 50 to filter out interference areas that are too dark or have low saturation. coordinates The mask value of the red area at that location, a value of 1 indicates that the pixel belongs to the candidate area of ​​the red stamp; coordinates The hue component value of the pixel in the HSV color space; coordinates The saturation component value of the pixel in the HSV color space; coordinates The value of the luminance component of the pixel in the HSV color space; This is an indicator function.

[0054] Secondly, an improved Maximum Stable Extreme Region (MSER) ​​algorithm is used to detect circular or elliptical regions within the masked area. While the traditional MSER algorithm is robust to changes in illumination, it is prone to generating numerous fragmented regions. This embodiment, based on MSER detection, introduces a shape verification step: calculating the circularity of the detected region. ,in For the area, The perimeter of the region is defined as . Regions with a circularity below the threshold (0.75 in this embodiment) are identified as non-stamp noise regions and filtered out.

[0055] For the candidate seal regions retained after shape verification, a color-based clustering method is used to separate the seal text from the background pattern. This embodiment employs the K-means clustering algorithm, grouping pixels into three categories based on color features: red seal text, red seal border / pattern, and white / light-colored background. Then, connected component analysis is performed on overlapping text regions to identify and segment adhering characters.

[0056] Seal text is typically arranged in a circular pattern, exhibiting significant radial and perspective distortion. Directly inputting it into a conventional OCR engine will result in extremely low recognition rates. Therefore, this step performs radial correction on the seal image.

[0057] First, detect the elliptical outline of the stamp area and determine the coordinates of the ellipse's center. , ), major axis radius a and minor axis radius b.

[0058] Secondly, for any point within the seal area ( Establish polar coordinate mapping relationship:

[0059] in, The angle of rotation of the ellipse; This is the radial distance in polar coordinates, representing the distance from the pixel to the center of the ellipse; The angle in polar coordinates represents the azimuth angle of a pixel relative to the center of the ellipse. The arctangent function in the four quadrants returns a point. The azimuth angle is determined. Interpolation reconstruction is performed in polar coordinate space, stretching the elliptical region into a standard circle, while flattening the arc-shaped text into a horizontal arrangement, resulting in the corrected front view of the seal.

[0060] The corrected seal image is input into a dedicated recognition model for text recognition. This dedicated recognition model is fine-tuned and trained using commonly used seal font samples such as seal script, Song typeface, and regular script, based on a general OCR model, and incorporates a corpus of common archival seal text (such as "XX Contract Seal"). The recognition output includes the seal text sequence and the positional information of each character. The positional information is divided into at least three categories: inner ring text, outer ring text, and central pattern (such as a five-pointed star).

[0061] S02, extract the homogeneity difference features in the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map; Specifically, obtaining the location map of the tampered area is achieved through a two-stream network consisting of an RGB stream branch and a noise stream branch. The processing flow of the two-stream network includes: Feature extraction is performed on the archive image through the RGB stream branch to obtain a multi-level RGB feature map; Multi-scale visual coding of the archive image is performed through the noise stream branch to obtain multi-level global noise features; The tampered region location map is generated by gradually fusing the multi-level RGB feature map with the multi-level global noise features through three sequentially arranged complementary modules, following a coarse-to-fine strategy.

[0062] In this embodiment, to ensure the authenticity and credibility of the extracted metadata, this step verifies the authenticity of the archive image, detecting and locating potentially tampered areas within the image. See also... Figure 2 In this embodiment, a dual-stream network consisting of an RGB stream branch and a noise stream branch is used to achieve tamper detection and location.

[0063] RGB Stream Branch: Using a fully convolutional neural network as the backbone, this embodiment employs a fully convolutional feature extraction network based on multi-scale depthwise separable convolution and inverse residual structures. This network consists of five sub-blocks: The first sub-block is a multi-scale saliency feature enhancement module, which uses multi-scale depthwise separable convolution to aggregate multi-scale image features through convolution kernels of different scales. During downsampling, noise is filtered out and foreground information is highlighted while background interference is suppressed. The second to fifth sub-blocks are four sub-modules of the hierarchical detail feature extraction module, progressively enhancing the capabilities of contour feature representation, spatial information extraction, and high-level semantic context capture. Each sub-module integrates an inverse residual structure and a spatial attention mechanism, reducing network parameters while enhancing the ability to focus on detailed features in key regions.

[0064] The network outputs five layers of feature maps, denoted as f1(x), f2(x), f3(x), f4(x), and f5(x), with sizes of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image size, respectively. f1(x) serves as a shallow edge detail feature and participates in the initial fusion of the suspected tampering artifact highlighting module. f2(x) through f5(x) participate in the coarse-to-fine feature fusion of three sequentially arranged complementary modules.

[0065] Noise Flow Branch: First, the archival image is filtered using a steganalysis model (SRM). The SRM filter contains three types of fixed convolutional kernels: edge enhancement filter, noise analysis filter, and tamper detection filter, all with a kernel size of 5×5. After filtering, a noise residual map is obtained, which suppresses the semantic content information of the image and highlights local noise distribution anomalies.

[0066] Secondly, the noise residual map is divided into multi-scale blocks, namely image block sequences of 32×32, 16×16 and 8×8 sizes.

[0067] Then, image patch sequences of different scales are input into the corresponding number of layers of the Visual Transformer (ViT) encoder: 32×32 scale sequences are input into a 2-layer ViT encoder, 16×16 scale sequences are input into a 4-layer ViT encoder, and 8×8 scale sequences are input into a 6-layer ViT encoder. Through the self-attention mechanism, the ViT encoder can capture the global noise distribution features at each scale.

[0068] The system outputs three sets of multi-level global noise features, denoted as ν1(x), ν2(x), and ν3(x), with their receptive fields decreasing in size.

[0069] Specifically, through three sequentially arranged complementary modules, following a "coarse-fine-fine" strategy, the multi-level RGB feature maps of the RGB flow branch and the multi-level global noise features of the noise flow branch are gradually fused. The three sequentially arranged complementary modules are: Suspicious Tampering Artifact Highlighting Module (STP), Fine Tampering Artifact Significance Module (FTS), and Tampering Artifact Edge Refinement Module (TER).

[0070] The Suspicious Tampering Artifact Highlighting Module aims to generate a preliminary map of suspected tampering regions. First, the deepest RGB feature f5(x) is subjected to 1×1 convolution and dilated spatial pyramid pooling (ASPP) operations. ASPP employs parallel branches with dilation rates of 1, 6, 12, and 18 to obtain a multi-scale global context with a large receptive field. The two types of features are then fused using a Feature Fusion Block (FFB). The FFB uses an element-wise multiplication followed by two 3×3 convolutions, which more effectively suppresses background noise compared to simple summation or channel concatenation.

[0071] Secondly, the fused features are upsampled to the same size as feature f4(x), and then fused with f4(x) compressed by 1×1 convolution through FFB to capture the high-level semantic context and obtain p1(x).

[0072] Next, the first stage output of the noise flow branch... ν 1( x The output of the 32×32 scale 2-layer ViT encoder is reshaped and upsampled to the same size as p1(x), and then fused with p1(x) to obtain m1(x).

[0073] Finally, through full attention block (FLA) m 1( x Feature enhancement is performed. FLA utilizes horizontal and vertical pooling and attention mechanisms to enable each spatial location to perceive contextual information along the same horizontal or vertical line, enhancing the representational ability of homogeneous difference features while preserving detailed information about tampering artifacts. The output of FLA is added to the input residual to obtain a preliminary suspected tampering localization map m1′(x).

[0074] The Fine-grained Artifact Saliency Module aims to generate a more refined tamper localization map by acquiring richer spatial detail information. First, the intermediate layer RGB feature f3(x) is enhanced with spatial dimension weighting using a Spatial Attention Block (SAB). The SAB performs max pooling and average pooling along the channel axis, concatenates the two single-channel spatial maps, and then generates a spatial attention map through a 3×3 convolution and a sigmoid function. This spatial dimension weighting is achieved by multiplying the attention map with the original feature.

[0075] Secondly, the features enhanced by SAB and the output of the previous module are aggregated through a cross-fusion block (CFB). m 1′( x CFB performs cross-fusion at two different resolution scales, then captures multi-scale information through 3×3 convolution, and finally outputs it through FFB fusion.

[0076] Next, the second-stage output of the noise flow branch... ν 2( x (Output from a 4-layer ViT encoder at a 16×16 scale) incorporates the above-mentioned aggregated features.

[0077] Finally, by enhancing the feature representation using FLA, a fine-grained tamper artifact saliency map m2′(x) is generated.

[0078] Tampering artifact edge refinement module: This module aims to further optimize the edge quality of tampered regions by capturing synaptic edge features. First, it utilizes the shallowest features from the RGB stream. f 2( x(with strong contour detail information), the number of channels is compressed by 1×1 convolution, and then fused with high-level spatial location information (from the upsampled result of f3′(x) after SAB enhancement in the previous module) through FFB. While preserving edge details, position guidance is used to suppress edge interference of non-tampered objects.

[0079] Secondly, the fused features mentioned above are combined with the output m2′(x) of the previous module through the Edge Refinement Block (ERB). After upsampling m2′(x), the ERB is fused with shallow features, and an attention vector is generated using global average pooling and attention mechanisms to guide the selection and refinement of edge features.

[0080] Furthermore, the third-level output ν3(x) (output by a 6-layer ViT encoder on an 8×8 scale), which incorporates the noise flow branch, provides guidance for advanced semantic noise feature development.

[0081] Finally, by enhancing the feature representation using FLA, a final tampered region localization map m3′(x) with fine edges is generated.

[0082] The localization map is a pixel-level binary classification result, where each pixel is labeled as a tampered pixel (value 1) or a real pixel (value 0).

[0083] When training the two-stream network, a combination of FOCAL loss, DICE loss, and Structural Similarity (SSIM) loss is used as the total loss function:

[0084] In this embodiment, Let be the total loss function for training the two-stream network. For FOCAL loss, For DICE losses, For structural similarity (SSIM) loss, , and The values ​​are 0.8, 0.1, and 0.1, respectively. The FOCAL loss (β=0.25, k=2) is used to handle the class imbalance between tampered pixels and real pixels, making the network pay more attention to the boundary pixels that are difficult to classify; the DICE loss is used to optimize the overlap between the predicted region and the real tampered region; the SSIM loss focuses on the structural information between image patches, which helps to mine the spatial features of tampering artifacts.

[0085] Training was performed using the AdamW optimizer with an initial learning rate of 1×10⁻⁶. 4. Decay to 1×10 using a cosine annealing strategy. 7. Calculate the loss for each of the three modules' outputs separately. The total loss is the weighted sum of the losses from the three modules:

[0086] in, The total loss is the weighted average of the three modules. To modify the artifact edge refinement module on the input image The final output location map, To finely modify the artifact saliency module on the input image The output is a detailed localization map. The module for highlighting suspicious tampering artifacts on the input image The output preliminary positioning map. This is to truly tamper with the area mask.

[0087] S03, perform layout semantic structure analysis on the archive image, divide it into multiple semantic regions, and assign preset evidence weights to each semantic region according to the layout attributes of each semantic region; Specifically, the semantic structure of the archival images is analyzed to identify the structured layout of the images.

[0088] In this embodiment, based on the global visual features extracted from the visual feature channels in step S01, a classifier is used to determine the document type of the archive image. Document types include at least: official documents, contracts, certificates, invoices, and letters. Based on the recognition result, the corresponding layout template is loaded from a pre-set layout template library. The layout template defines the standard layout structure for that type of document.

[0089] Based on a layout template, the archival image is divided into multiple semantic regions and semantically labeled. In this embodiment, the semantic regions include at least: Title area: Usually located at the top center of the page, containing the document title or file name; Document Number Area: Usually located below the title area or in the header of the page, it contains the document number. Body area: The main part of the page, containing the main text content; Signature area: Usually located at the bottom of the main text, it contains the name of the issuing authority and its seal; Seam area: The area at the edge of the page, which may contain traces of a seal across the seam.

[0090] Based on the layout attributes of each semantic region—that is, the type of information it carries in the archival document and its legal effect—differentiated evidentiary weights are assigned to each semantic region. The evidentiary weight reflects the relative credibility of the metadata extracted from that region in the subsequent multimodal fusion process. The allocation rules in this embodiment are as follows: The signature area has the highest evidence weight, set to 1.0, because it contains legal information such as the issuing authority and seal; The weight of evidence in the title section is second, set to 0.9, because it contains the core topic of the document; The evidence weight for the document number area is set to 0.85; The weight of evidence in the main text area is set to 0.7 because it mainly contains descriptive information; The weight of evidence in the seam area is set to 0.5 because it only provides auxiliary verification information.

[0091] Construct a region adjacency graph, where nodes represent semantic regions and edges represent spatial relationships between regions (e.g., "the title area is above the body text area," "the signature area is below the body text area"). Utilize spatial constraints to validate OCR recognition results; for example, date-formatted text should appear at the end of the body text area or near the signature area. If date text is detected in the title area, it is marked as a suspicious result.

[0092] S04, spatially map the tampered region location map with each of the semantic regions, determine the tampered semantic regions based on the mapping results, and reduce the confidence of the candidate metadata extracted from the tampered semantic regions to obtain the adjusted candidate metadata; Specifically, the tampered region location map is a pixel-level binary classification result, and each pixel in the tampered region location map is marked as a tampered pixel or a real pixel; The step of spatially mapping the tampered region location map to each of the semantic regions, determining the tampered semantic regions based on the mapping results, and reducing the confidence of candidate metadata extracted from the tampered semantic regions specifically includes: Based on the pixel-level binary classification results of the tampered region location map, the pixel coordinates in the tampered region location map are compared with the boundary coordinates of each semantic region, and the proportion of the area of ​​pixels marked as tampered in each semantic region to the total area of ​​the semantic region is calculated. If the proportion within a certain semantic region exceeds a preset threshold, then the semantic region is determined to be a tampered semantic region. A suspected tampering marker is attached to the candidate metadata extracted from the tampered semantic region, or its confidence level is multiplied by a preset attenuation coefficient to reduce the confidence level of the candidate metadata.

[0093] In this embodiment, the obtained tampered region location map is spatially mapped to the divided semantic regions. Specifically: Pixel ratio calculation: The pixel coordinates in the tampered region location map are compared region by region with the boundary coordinates of each semantic region. The proportion of the area of ​​pixels marked as tampered pixels in each semantic region to the total area of ​​that semantic region is calculated. In this embodiment, a pixel is marked as a tampered pixel when its value in the tampered region location map is 1.

[0094] Tampering determination: If the proportion of pixels marked as tampered within a certain semantic region exceeds a preset threshold (0.15 in this embodiment), then the semantic region is determined to be a tampered semantic region. For critical areas such as the signature area, the threshold can be further reduced to 0.10 to achieve more stringent authenticity verification.

[0095] De-weighting: The candidate metadata extracted from the tampered semantic region is subjected to confidence de-weighting. Specifically, a suspected tampering marker is attached to the candidate metadata, or its confidence is multiplied by a preset attenuation coefficient (0.5 in this embodiment) to reduce the influence of the candidate metadata in subsequent fusion decisions.

[0096] After the above processing, the adjusted candidate metadata is obtained.

[0097] S05, based on the evidence weights of each semantic region, perform multimodal feature fusion and adaptive threshold decision on the adjusted candidate metadata to generate fused metadata; Specifically, multimodal feature fusion of the adjusted candidate metadata includes: Cross-modal conflict detection is performed on similar information of different modalities in the adjusted candidate metadata; When a conflict is detected, based on the document type of the archive image, the candidate metadata modalities are weighted, fused, and conflict resolved according to a preset modal credibility priority rule to generate a fused metadata field. The modal credibility priority rule includes at least the following: the weight of seal information is greater than the weight of printed text, and the weight of printed text is greater than the weight of handwritten content; the same type of information includes at least one of time information, subject information, and event information; For the aforementioned core information conflict, the coreference resolution score is calculated using the following formula:

[0098] in, To resolve fractions by common reference, These are the entity name strings identified from the text fragment. For entities respectively and The set of context words of the text segment. There are two candidate entities. For string similarity functions, Context similarity function and Preset weighting coefficients.

[0099] For the aforementioned time information conflict, the final merged time is calculated using the following formula:

[0100] in, For the first Candidate times for each modality, For document types based on the archive image Dynamically adjusted credibility weights For indicator functions, This refers to the final time after the fusion.

[0101] In this embodiment, the specific process of multimodal feature fusion is as follows: Cross-modal conflict detection: Comparing similar information from different modalities. Similar information includes time information (EXIF timestamp, OCR-recognized date text, seal date), subject information (company name in the title, names of Party A and Party B in the body, name of issuing authority in the seal), and event information (visual classification results, keywords recognized by OCR, special seal type in the seal).

[0102] Conflict resolution strategy: When a conflict is detected, it is resolved through weighted fusion according to a preset modality credibility priority rule. The rules are as follows: The weight of seal information is greater than the weight of printed text; Printed text has a higher weight than handwritten content; The weight of handwritten content is greater than that of EXIF ​​metadata information.

[0103] Subject Information Fusion: For conflicts in subject information (such as organization names), a coreference resolution method is used. For two candidate entities... and Calculate the coreference elimination fraction:

[0104] in, This is a string similarity function based on edit distance or the Jaccard coefficient; This is a contextual semantic similarity function based on word vectors or BERT embeddings; and To preset the weighting coefficients, in this embodiment, the values ​​are 0.4 and 0.6 respectively, reflecting the higher importance of contextual semantics compared to string surface matching. When When the preset fusion threshold is exceeded (0.75 in this embodiment), the two entities are merged to retain the most complete representation of the information.

[0105] Time information fusion: Extracting candidate times from various modalities to resolve conflicts in time information. The differences between candidate times are calculated. The temporal reliability weights for each modality are determined based on the document type of the archival image. The final time is output by fusion according to the following formula:

[0106] in, For indicator functions, when The value is 1 if the condition is valid (i.e., time information is detected in the modality), and 0 otherwise. Time reliability weight. The weights are dynamically adjusted based on document type. The table below provides examples of weight allocation for several typical document types:

[0107] Information fusion: The visual classification results (e.g., layout classification as "Purchase Contract"), keywords extracted by OCR (e.g., "Purchase" and "Contract" appearing in the text), and the special seal type in the seal (e.g., "Contract Special Seal") are cross-checked for consistency. If the three points are consistent, it is confirmed to be the same item type; if there is inconsistency, a multi-source voting mechanism is initiated, and the majority consensus result prevails.

[0108] Specifically, for candidate metadata after multimodal feature fusion, the selection threshold for whether to adopt the metadata is dynamically adjusted based on multidimensional factors (adaptive threshold dynamic decision-making), as follows: Quality dimension adjustment: Adjust the base threshold based on the image quality level. (For quality scoring) High-quality archives are selected based on strict thresholds (e.g., above 0.9) and consistency of multimodal information before acceptance; for For medium-quality archives, a balanced threshold (e.g., 0.75) is used, allowing for missing single-modal information but requiring it to be marked as "single-source extraction"; for For low-quality files, a lenient threshold (such as 0.6) is used, and a strong context completion mechanism is automatically enabled.

[0109] Regional Dimension Adjustment: Adjust the threshold according to the semantic region type. Lower the threshold for the title area (e.g., by 0.05-0.1) to ensure the acquisition of the document's subject information; raise the threshold for the signature area (e.g., by 0.05-0.1) to ensure the accuracy and reliability of legal validity information.

[0110] Chronological Adjustment: Thresholds are adjusted based on the age of the archives. Using 1949 and 2010 as two dividing lines, the threshold is lowered for archives before 1949 (due to their age and varying preservation conditions), and higher for archives after 2010 (mostly digitized and of high quality). The specific adjustment range is determined based on statistical data from actual application scenarios.

[0111] S06. Perform semantic enhancement processing on the fused metadata based on the knowledge graph to obtain the final metadata with entity disambiguation and inference completion, marked with confidence and source.

[0112] In this embodiment, a knowledge graph of the archival field is pre-constructed to provide structured knowledge support for semantic enhancement processing. The knowledge graph is stored in a dedicated graph database. The construction of the knowledge graph includes three aspects: entity type definition, relationship Schema design, and knowledge source organization, which are as follows: Entity type definition: The following entity types are defined in the knowledge graph: Archival entities: fonds, file folder, document, page; Content entities: organization, person, date, matter, amount; Relationship entities: subordination, signing, approval, circulation, association.

[0113] Relationship Schema design: The following relationship types are defined: Hierarchical relationship: fonds → contains → file folder → contains → document → contains → page; Business relationship: document ← issued by ← organization, person ← holds the position of ← approval role, matter ← involves ← amount; Time relationship: document ← generated on the date, document ← effective on the date, document ← archived on the date.

[0114] Knowledge source: The knowledge sources of the knowledge graph include archival industry standards (such as DA / T series standards), organization code libraries, historical figure libraries, and business rule libraries. Business rule libraries store rules such as "request - reply" association pairs, official documents must have an issuing authority, receiving date = issuing date + circulation time, etc.

[0115] Among them, link the text fragments in the fused metadata to the entities in the knowledge graph to solve the semantic ambiguity problem. This process includes three stages: candidate entity generation, candidate entity ranking, and ambiguity resolution, which are as follows: Candidate entity generation: For the text fragment m, generate a candidate entity set based on string matching and synonym expansion. First, perform exact matching and fuzzy matching (such as similarity calculation based on edit distance), and filter out the preliminary candidate set with a string similarity higher than the preset threshold. Then perform synonym expansion, for example, expand "NDRC" to "National Development and Reform Commission", and add the expanded entities to the candidate set.

[0116] Candidate entity ranking: Calculate the comprehensive matching score of the candidate entities, considering the following dimensions: Context similarity: Extract the co-occurrence degree of the words around the text fragment and the descriptive words of the candidate entities, and calculate the semantic similarity using a pre-trained language model; Temporal consistency: Determine the degree of match between the existence time of an entity in the knowledge graph and the creation time of its file; Spatial consistency: Determine the degree of matching between the hierarchical level of an institutional entity and the type of archive (e.g., central documents correspond to central-level institutions).

[0117] Disambiguation: For the highest-scoring candidate entity, if the score exceeds a preset threshold, a link is established, transforming the unstructured text fragment into a structured knowledge entity. For ambiguous entities (such as the same name corresponding to multiple historical figures), joint disambiguation is performed by combining archival information on time, institution, and position. For entities not present in the knowledge graph, a new entity discovery process is initiated, features are extracted, and recommendations are generated for inclusion in the database.

[0118] Handling of questionable source regions: For text fragments originating from semantic regions marked as tampered with, the candidate entity ranking score is multiplied by an additional attenuation factor (e.g., 0.5) during entity linking to reflect the low credibility of the source information.

[0119] Specifically, for missing fields in the fused metadata that cannot be extracted or identified due to image quality issues, occlusion, or original document defects, the association relationships in the knowledge graph are used for reasoning and completion. This embodiment comprehensively employs three complementary reasoning strategies: rule-based reasoning, association-based reasoning, and statistical reasoning, as detailed below: Rule-based reasoning: Inferences are made based on the required field rules for file types and business rules. For example: If the "issuance date" is missing, the issuance date can be deduced by referring to the "receipt date" and the file transfer rules (receipt date = issuance date + transfer time). If the "Issuing Authority" is missing, it can be inferred from the seal recognition information (Seal Area Institution → Issuance → Document). If the "Approver" field is missing but the name information exists in the signature area, establish a mapping relationship between the name and the approver role; The "Issuing Authority" field in official documents is mandatory. If it is missing, it should be inferred from the authority that issued the seal and signature, and secondly from the authority name in the title.

[0120] Associative reasoning: Inferences are made by leveraging the shared metadata characteristics of files within the same case file. The formula is:

[0121] in, For the file The inferred values ​​of the missing fields to be filled in. Documents within the same case file Known values ​​of the same field in the same context. For the document A collection of all documents belonging to the same case file. For aggregation functions, a majority voting strategy or an interpolation strategy is used to summarize the values ​​in the set. In this embodiment, a majority voting strategy is used. For the same field in other files within the same case file, the value with the highest frequency is taken as the inferred value. For ordered files (such as those arranged by file number), an interpolation strategy can be used.

[0122] Furthermore, for the "request for instructions - reply" pair, reasoning is based on the chronological order: the reply date must be later than the request date. If the request date is missing, it can be deduced backwards from the reply date and the average document circulation time.

[0123] Statistical inference: For frequently missing fields, fill them in based on statistical patterns of similar documents in historical data. For example, if a statistical analysis of documents issued by an organization over the years shows that the organization's documents are usually signed by a specific signatory, then when the signatory field of a new document is missing, this statistical result will be used as a reference value.

[0124] Confidence Level and Source Tagging: A confidence level and source tag are configured for each extracted or inferred metadata field to achieve traceable management of metadata quality. In this embodiment, the source tag and corresponding confidence level are as follows:

[0125] After the above steps, the final output is the archive image's metadata, which includes confidence level and source tagging.

[0126] In summary, the intelligent extraction method for archival image metadata has the following effects: This invention establishes a four-channel parallel capture pipeline to simultaneously extract standard metadata, visual features, printed text, and seal information. Combined with a multimodal feature alignment and fusion mechanism, it breaks down silos of single-modal information. Compared to traditional methods that suffer from low information coverage and easy omission of key fields, this invention significantly improves key information coverage and effectively reduces the omission rate of key fields. Furthermore, through preset modal credibility priority rules and dynamic weight adjustment based on document type, it achieves prioritized and evidence-based information integration, resolving the credibility judgment problem when multiple sources of information conflict.

[0127] This invention employs a layered, progressive OCR quality enhancement strategy. Through a three-tiered mechanism of "quality pre-assessment—dynamic engine selection—adaptive confidence calibration," the processing chain is dynamically adjusted based on image sharpness, illumination uniformity, and noise level. For faded, wrinkled, or unevenly illuminated historical archives, a dedicated handwriting engine is automatically switched to and the threshold is lowered to ensure no information loss. For clear electronic archives, a high-speed engine is used and the threshold is increased to ensure accuracy. This method significantly improves the metadata extraction effect of archival images of different quality levels, balancing the recognition coverage of historical archives with the recognition accuracy of electronic archives.

[0128] This invention utilizes a two-stream network composed of RGB and noise stream branches to extract homogeneous difference features between tampered and real regions, achieving pixel-level detection and localization of potential tampered regions in archival images. A three-level progressive strategy—"highlighting suspicious tampering artifacts—making fine tampering artifacts significant—refining tampering artifact edges"—ensures the accuracy and edge precision of tampered region localization. More importantly, this invention innovatively spatially maps the tampering localization results with the page layout semantic parsing results, automatically performing confidence-weighted processing on the metadata extracted from the tampered region, ensuring the authenticity and credibility of the metadata from the source. This is a core capability not possessed by existing archival metadata extraction technologies.

[0129] This invention performs structured semantic analysis of archival images, identifies document types and loads corresponding layout templates, and divides the image into multiple semantic regions such as title area, document number area, body text area, signature area, and seam area, assigning differentiated weights based on the evidentiary value of each region. The legal validity information in the signature area has a higher weight than the thematic information in the title area, and the title area has a higher weight than the descriptive information in the body text area. This prioritized information integration method effectively improves the extraction priority and accuracy of important fields such as seals, issuing authorities, and effective dates.

[0130] This invention constructs a knowledge graph covering the entire field of archival management, linking unstructured text fragments to structured knowledge entities. This effectively solves the problems of synonym ambiguity and confusion of hierarchical relationships, improving entity disambiguation accuracy and retrieval recall. Furthermore, for missing fields that cannot be identified due to image quality or occlusion, this invention integrates three strategies—rule-based reasoning, association-based reasoning, and statistical reasoning—for interpretable reasoning completion. Each completed field is assigned confidence and source tags, significantly improving the completeness, traceability, and credibility of metadata.

[0131] Example 2 like Figure 3 As shown, a second embodiment of the present invention provides an intelligent extraction system for archival image metadata, the system comprising: The capture module 10 is used to acquire the archive image to be extracted; and to capture multi-source metadata of the archive image to obtain multimodal candidate metadata. The module 20 is used to extract homogeneity difference features in the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map. The allocation module 30 is used to perform layout semantic structure analysis on the archive image, divide it into multiple semantic regions, and assign a preset evidence weight to each semantic region according to the layout attributes of each semantic region. The module 40 is used to spatially map the tampered region location map with each of the semantic regions, determine the tampered semantic regions based on the mapping results, and reduce the confidence of the candidate metadata extracted from the tampered semantic regions to obtain the adjusted candidate metadata. The generation module 50 is used to perform multimodal feature fusion and adaptive threshold decision on the adjusted candidate metadata based on the evidence weights of each semantic region, and generate fused metadata. The processing module 60 is used to perform knowledge graph-based semantic enhancement processing on the fused metadata to obtain the final metadata with confidence and source tags after entity disambiguation and reasoning completion.

[0132] Example 3 like Figure 4 As shown, in the third embodiment of the present invention, the present invention provides the following technical solution: a computer, including a memory 202, a processor 201, and a computer program stored in the memory 202 and executable on the processor 201, wherein the processor 201 executes the computer program to implement the intelligent extraction method of archive image metadata as described above.

[0133] Specifically, the processor 201 may include a central processing unit, a specific integrated circuit, or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0134] Memory 202 may include a large-capacity memory for data or instructions. For example, and not limitingly, memory 202 may include a hard disk drive, floppy disk drive, solid-state drive, flash memory, optical disk drive, magneto-optical disk drive, magnetic tape drive, or Universal Serial Bus drive, or a combination of two or more of these. Where appropriate, memory 202 may include removable or non-removable media. Where appropriate, memory 202 may be internal or external to a data processing device. In a particular embodiment, memory 202 is non-volatile memory. In a particular embodiment, memory 202 includes read-only memory and random access memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM, an erasable PROM, an electrically erasable PROM, an electrically rewritable ROM, or flash memory, or a combination of two or more of these. Where appropriate, the RAM may be static random access memory (SRAM) or dynamic random access memory (DRAM), wherein DRAM may be fast page-mode DRAM, extended data output DRAM, synchronous DRAM, etc.

[0135] The memory 202 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 201.

[0136] The processor 201 reads and executes computer program instructions stored in the memory 202 to implement the above-mentioned intelligent extraction method of archival image metadata.

[0137] In some embodiments, the computer may further include a communication interface 203 and a bus 200. For example, Figure 4 As shown, the processor 201, memory 202, and communication interface 203 are connected through bus 200 and complete communication with each other.

[0138] The communication interface 203 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 203 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0139] Bus 200 includes hardware, software, or both, that couples computer components together. Bus 200 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, local bus. For example, and not limitingly, bus 200 may include a graphics acceleration interface or other graphics bus, an enhanced industry standard architecture bus, a front-side bus, HyperTransport interconnect, an industry standard architecture bus, a wireless bandwidth interconnect, a low pin count bus, a memory bus, a WeChat architecture bus, a peripheral component interconnect bus, a PCI Express bus, a Serial Advanced Technology Attached Bus, a Video Electronics Standards Association local bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 200 may include one or more buses. Although specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0140] Example 4 In the fourth embodiment of the present invention, in conjunction with the above-described intelligent extraction method for archival image metadata, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described intelligent extraction method for archival image metadata.

[0141] Those skilled in the art will understand that the data in the flowchart, or the logic and / or steps otherwise described herein, such as a sequence of instructions or a program list that can be considered as executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device. For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0142] More specific examples of readable media include: electrical connections with one or more wires, portable computer disk drives, random access memory, read-only memory, erasable and editable read-only memory, fiber optic devices, and portable optical disc read-only memory. Furthermore, computer-readable media can even be printed on paper or other suitable media, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0143] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for intelligent extraction of archival image metadata, characterized in that, The method includes: Obtain the image of the file to be extracted; Multi-source metadata capture is performed on the archive images to obtain multimodal candidate metadata; Extract homogeneity difference features from the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map; The document image is analyzed for its layout semantic structure, divided into multiple semantic regions, and a preset evidence weight is assigned to each semantic region according to the layout attributes of each semantic region. The tampered region location map is spatially mapped to each of the semantic regions. Based on the mapping result, the tampered semantic regions are determined, and the confidence of the candidate metadata extracted from the tampered semantic regions is reduced to obtain the adjusted candidate metadata. Based on the evidence weights of each semantic region, multimodal feature fusion and adaptive threshold decision are performed on the adjusted candidate metadata to generate fused metadata; The fused metadata is subjected to semantic enhancement processing based on knowledge graphs to obtain the final metadata with confidence level and source tagging after entity disambiguation and reasoning completion.

2. The method for intelligent extraction of archival image metadata according to claim 1, characterized in that, The step of capturing multi-source metadata of the archive image includes: Establish a parallel processing pipeline to perform the following operations synchronously: Parse the standard metadata fields embedded in the archive image to obtain basic information; Global and local visual features of the archive images are extracted using a convolutional neural network. The archive image is initially screened using multi-engine OCR, and a set of candidate text blocks and their corresponding confidence scores are output. The archive image is subjected to color space conversion and morphological filtering to separate the seal area and perform radial distortion correction in order to extract seal information.

3. The method for intelligent extraction of archival image metadata according to claim 2, characterized in that, The multi-engine OCR also includes an adaptive quality enhancement step when performing initial text screening on the archive image: The archive image is subjected to a quality pre-assessment, which includes at least calculating a sharpness index that characterizes the sharpness of image edges, an illumination index that characterizes the uniformity of illumination, and a noise index that characterizes the intensity of random noise. A comprehensive quality score is generated based on the sharpness index, the illumination index, and the noise index. The archive image is divided into different quality scenes according to the comprehensive quality score, and the corresponding OCR engine and preprocessing strategy are dynamically selected for different quality scenes. The confidence threshold for OCR recognition is adaptively adjusted based on the comprehensive quality score, wherein the threshold is lowered and context verification is enabled for archival images of historical archive type, and the threshold is increased for archival images of electronic archive type to ensure accuracy; The overall quality score is calculated using the following formula: in, For the overall quality score, The aforementioned sharpness index, The illumination index is... The noise index is... , For preset weighting coefficients, This is the normalization function.

4. The method for intelligent extraction of archival image metadata according to claim 2, characterized in that, The steps for extracting seal information include: The archive image is converted from the RGB color space to the HSV color space, and a red region mask is generated based on the preset red hue range, saturation threshold and brightness threshold to detect candidate areas for the stamp. The circular or elliptical regions in the candidate seal region are detected by the Maximum Stable Extreme Region (MSER) ​​algorithm, and the roundness of each detected circular or elliptical region is calculated for shape verification to filter out non-seal noise. Color clustering is performed on the candidate seal regions retained after shape verification to separate the seal text from the background pattern, and connected component analysis is performed on the overlapping text regions to segment the connected characters. For the candidate regions of the seal after color clustering and connected component analysis, their elliptical contours are detected, and the major and minor axes of the elliptical contours are determined. Based on the ratio of the major axis to the minor axis, a polar coordinate mapping relationship is established, and the elliptical region is corrected into a planar front view to obtain the corrected seal image. The corrected seal image is input into a dedicated recognition model, which outputs the seal text sequence and position information. The position information includes at least the inner ring text, the outer ring text, and the central pattern.

5. The method for intelligent extraction of archival image metadata according to claim 1, characterized in that, The location map of the tampered area is obtained through a two-stream network consisting of an RGB stream branch and a noise stream branch. The processing flow of the two-stream network includes: Feature extraction is performed on the archive image through the RGB stream branch to obtain a multi-level RGB feature map; Multi-scale visual coding of the archive image is performed through the noise stream branch to obtain multi-level global noise features; The tampered region location map is generated by gradually fusing the multi-level RGB feature map with the multi-level global noise features through three sequentially arranged complementary modules, following a coarse-to-fine strategy.

6. The method for intelligent extraction of archival image metadata according to claim 1, characterized in that, The tampered region location map is a pixel-level binary classification result, and each pixel in the tampered region location map is marked as a tampered pixel or a real pixel; The step of spatially mapping the tampered region location map to each of the semantic regions, determining the tampered semantic regions based on the mapping results, and reducing the confidence of candidate metadata extracted from the tampered semantic regions specifically includes: Based on the pixel-level binary classification results of the tampered region location map, the pixel coordinates in the tampered region location map are compared with the boundary coordinates of each semantic region, and the proportion of the area of ​​pixels marked as tampered in each semantic region to the total area of ​​the semantic region is calculated. If the proportion within a certain semantic region exceeds a preset threshold, then the semantic region is determined to be a tampered semantic region. A suspected tampering marker is attached to the candidate metadata extracted from the tampered semantic region, or its confidence level is multiplied by a preset attenuation coefficient to reduce the confidence level of the candidate metadata.

7. The method for intelligent extraction of archival image metadata according to claim 1, characterized in that, Multimodal feature fusion of the adjusted candidate metadata includes: Cross-modal conflict detection is performed on similar information of different modalities in the adjusted candidate metadata; When a conflict is detected, based on the document type of the archive image, the candidate metadata modalities are weighted, fused, and conflict resolved according to a preset modal credibility priority rule to generate a fused metadata field. The modal credibility priority rule includes at least the following: the weight of seal information is greater than the weight of printed text, and the weight of printed text is greater than the weight of handwritten content; the same type of information includes at least one of time information, subject information, and event information; For the aforementioned core information conflict, the coreference resolution score is calculated using the following formula: in, To resolve fractions by common reference, These are the entity name strings identified from the text fragment. For entities respectively and The set of context words of the text segment. There are two candidate entities. For string similarity functions, Context similarity function and Preset weighting coefficients; For the aforementioned time information conflict, the final merged time is calculated using the following formula: in, For the first Candidate times for each modality, For document types based on the archive image Dynamically adjusted credibility weights For indicator functions, This refers to the final time after the fusion.

8. A smart system for extracting archival image metadata, characterized in that, The system includes: The capture module is used to acquire the archive image to be extracted; and to capture multi-source metadata of the archive image to obtain multimodal candidate metadata. The module is used to extract homogeneity difference features from the archive image that reflect the tampered area and the real area, detect and locate potential tampered areas based on the homogeneity difference features, and obtain a tampered area location map. The allocation module is used to perform layout semantic structure analysis on the archive image, divide it into multiple semantic regions, and assign preset evidence weights to each semantic region according to the layout attributes of each semantic region. The module is used to spatially map the tampered region location map with each of the semantic regions, determine the tampered semantic regions based on the mapping results, and reduce the confidence of the candidate metadata extracted from the tampered semantic regions to obtain the adjusted candidate metadata. The generation module is used to perform multimodal feature fusion and adaptive threshold decision on the adjusted candidate metadata based on the evidence weights of each semantic region, and generate fused metadata. The processing module is used to perform knowledge graph-based semantic enhancement processing on the fused metadata to obtain the final metadata with confidence and source tags after entity disambiguation and reasoning completion.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent extraction method for archival image metadata as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the intelligent extraction method for archival image metadata as described in any one of claims 1 to 7.

Images