Intelligent retrieval method and system for association of images and contents in PDF (Portable Document Format) document

Abstract

The invention discloses an intelligent retrieval method and system for association of an image and content in a PDF document, and relates to the technical field of data process.The method comprises the steps that a document preprocessing strategy is called to preprocess the PDF document, and an image processing result and a content processing result are obtained; performing association analysis on the image processing result and the content processing result according to an association index mechanism to obtain an image-text index structure; and performing retrieval matching on the target retrieval request of the PDF document by taking the image-text index structure as a benchmark to obtain target retrieval information. According to the method, the technical problem that the retrieval efficiency and accuracy are insufficient due to the fact that an existing PDF document retrieval method cannot effectively associate the image and the text content is solved, and the technical effects that intelligent associated retrieval of the image and the text content is achieved by constructing the image-text index structure, and the retrieval efficiency and accuracy are improved are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an intelligent retrieval method and system for associating images with content in PDF documents. Background Art

[0002] With the wide application of digital office and electronic documents, PDF documents are widely used for storing and sharing files containing rich graphic and text content due to their strong compatibility and stable format. However, most of the existing PDF document retrieval technologies only support retrieval based on text keywords, have extremely limited retrieval capabilities for image content, and cannot effectively associate the internal connection between images and text. This causes users to often need to manually flip through the document when looking for text information associated with a specific image or looking for the corresponding image according to text descriptions, consuming a large amount of time and energy, with low retrieval efficiency and poor user experience. Summary of the Invention

[0003] This application provides an intelligent retrieval method and system for associating images with content in PDF documents, which is used to solve the technical problem that the existing PDF document retrieval methods cannot effectively associate image and text content, resulting in insufficient retrieval efficiency and accuracy.

[0004] In the first aspect of this application, an intelligent retrieval method for associating images with content in PDF documents is provided. The method includes: invoking a document preprocessing strategy to preprocess a PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result; performing association analysis on the image processing result and the content processing result according to an association index mechanism to obtain a graphic-text index structure; and performing a retrieval match on a target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information.

[0005] In the second aspect of this application, an intelligent retrieval system for associating images with content in PDF documents is provided. The system includes: a document preprocessing module, which is used to invoke a document preprocessing strategy to preprocess a PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result; a graphic-text association analysis module, which is used to perform association analysis on the image processing result and the content processing result according to an association index mechanism to obtain a graphic-text index structure; and a retrieval match module, which is used to perform a retrieval match on a target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] The intelligent retrieval method and system for associating images with content in a PDF document provided by this application relate to the technical field of data processing. By means of a document preprocessing strategy, the images and text content in the PDF are extracted to generate an image processing result and a content processing result. Based on an association indexing mechanism, a graphic-text index structure is constructed, and based on this index structure, a target retrieval request is matched to output a corresponding retrieval result, solving the technical problem that the existing retrieval methods for PDF documents cannot effectively associate images with text content, resulting in insufficient retrieval efficiency and accuracy, and achieving the technical effect of realizing intelligent associative retrieval of images and text content by constructing a graphic-text index structure, improving retrieval efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0009] Figure 1 It is a schematic flowchart of the intelligent retrieval method for associating images with content in a PDF document provided by an embodiment of this application;

[0010] Figure 2 It is a schematic structural diagram of the intelligent retrieval system for associating images with content in a PDF document provided by an embodiment of this application.

[0011] Description of the reference numerals: Document preprocessing module 11, graphic-text association analysis module 12, retrieval matching module 13. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] This application provides an intelligent retrieval method and system for associating images with content in a PDF document to solve the technical problem that the existing retrieval methods for PDF documents cannot effectively associate images with text content, resulting in insufficient retrieval efficiency and accuracy.

[0013] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0014] It should be noted that the terms "first", "second", etc. in the description of this application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0015] Embodiment 1, as Figure 1 shown, this application provides an intelligent retrieval method for associating images and content in PDF documents. The method includes:

[0016] P10: Invoke the document preprocessing strategy to preprocess the PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result.

[0017] Furthermore, step P10 of the embodiment of this application further includes:

[0018] P11: Discriminate and extract the PDF document with the object structure as the discrimination constraint to obtain an extraction result; P12: Process the image objects in the extraction result according to the image strategy in the document preprocessing strategy to obtain the image processing result; where, it includes:

[0019] P12-1: Extract the first object and the second object in the image object; P12-2: Perform standardization processing on the first object and the second object in turn according to the image standardization plan in the image strategy to obtain a first standard image and a second standard image respectively; P12-3: Calculate the first hash value of the first standard image and the second hash value of the second standard image in turn; P12-4: Compare the first hash value with the second hash value to obtain the Hamming distance; P12-5: If the Hamming distance does not reach the predetermined threshold, divide the first object and the second object to obtain the image processing result.

[0020] It should be understood that, to construct the basic data source for the subsequent graphic-text association index, first, the preset document preprocessing strategy is invoked to systematically analyze the input PDF document to obtain a document processing result including an image processing result and a content processing result. Among them, the image processing result can be obtained through a multi-level image object extraction and structure comparison process, and the content processing result includes text semantic units, paragraph structures, and their layout position information, etc.

[0021] Specifically, to achieve the standardized extraction of image processing results, an object structure discrimination mechanism is first introduced to perform a preliminary structural analysis of the PDF document with the object structure as the discrimination constraint. The object structure here refers to the organizational form and hierarchical relationship of various elements (such as text, images, tables, etc.) in the PDF document. Through the discrimination of the object structure, the image objects and other relevant elements in the document can be accurately identified, thereby obtaining a preliminary extraction result. This process can be based on the internal format and structural characteristics of the PDF document, effectively avoiding the mis-extraction of non-target objects and ensuring the accuracy and integrity of the extraction result.

[0022] Next, the image objects in the extraction result are processed according to the image strategy in the document preprocessing strategy to obtain the image processing result. The specific processing flow is as follows:

[0023] First, the first object and the second object in the image object are extracted. The first object and the second object here can be different regions, different types of image elements in the image object, or different versions of the same image, etc. For example, in a PDF document containing multiple illustrations, the first object may be the main illustration, while the second object may be an auxiliary illustration or legend related to the main illustration.

[0024] Then, an image standardization plan is introduced to perform image standardization processing on the first object and the second object respectively to obtain the first standard image and the second standard image with unified size, gray scale range, and image ratio. The image standardization plan is a plan for preprocessing images, aiming to convert images into a unified format and quality standard. The plan may include image normalization operations such as gray scale conversion, edge enhancement, and size unification (such as normalizing to 128×128 pixels) to eliminate the differences in resolution, brightness, or compression format of the original images. Through the standardization processing, the first standard image and the second standard image can be obtained, thus eliminating the format differences and quality differences between the images and providing a consistent basis for subsequent feature extraction and comparison.

[0025] Next, the first hash value of the first standard image and the second hash value of the second standard image are calculated in sequence. The hash value is a digital summary generated from the image data through a specific algorithm, which can uniquely identify the content of the image. By calculating the hash value, the similarity between two images can be quickly compared. In this step, the calculation of the hash value is based on the standardized image, and the first hash value and the second hash value respectively represent the feature compression representations of the two standard images.

[0026] Subsequently, the Hamming distance is obtained by comparing the first hash value and the second hash value. The Hamming distance refers to the number of different bits between two hash values and is used to measure the similarity between two hash values. The smaller the Hamming distance, the more similar the two hash values are, and thus the more similar the corresponding image contents are. By calculating the Hamming distance, the similarity between the first object and the second object can be quickly determined, providing a basis for further image processing.

[0027] If the Hamming distance does not reach the predetermined threshold, the first object and the second object are divided to obtain the image processing result. The predetermined threshold is a parameter preset according to the actual application scenario and the requirements for image similarity, and is used to determine whether two images are similar enough. If the Hamming distance does not reach the predetermined threshold, it indicates that the similarity between the first object and the second object is insufficient, and they need to be divided, that is, they are processed as independent image objects respectively. This can avoid combining and processing image objects with low similarity, thus ensuring the accuracy and effectiveness of the image processing result.

[0028] Through the above image processing link, this application can accurately preprocess and analyze the image objects in the PDF document, providing a high-quality data basis for subsequent image and content association analysis. This method takes into account both structural analysis and perceptual feature extraction, and has good adaptability and scalability, especially suitable for document processing scenarios with complex mixed text and images and diverse image styles.

[0029] Furthermore, step P10 of this application embodiment further includes:

[0030] P13: Processing the text object in the extraction result according to the text strategy in the document preprocessing strategy to obtain the content processing result; wherein, it includes:

[0031] P13-1: Extracting the third object in the text object; P13-2: Performing semantic analysis on the third object according to the text strategy to obtain semantic features; P13-3: Constructing a semantic vector based on the semantic features and combining it with the third object to obtain a mapping relationship; P13-4: Forming the content processing result based on the mapping relationship.

[0032] Optionally, step P10 not only includes the structured preprocessing of the image objects in the PDF document, but also further includes the semantic feature extraction and semantic structure mapping operations for the text objects to form a complete content processing result.

[0033] Specifically, after completing the preprocessing of the image object, the text object in the extraction result is processed according to the text strategy in the document preprocessing strategy to obtain the content processing result. First, the third object in the text object is extracted. The "third object" here refers to the key text paragraphs or sentences in the text object, and these text contents usually have a direct relevance to the image object, such as the title, descriptive text, or legend of the image. By analyzing the structural and semantic features of the text object, these key text paragraphs can be accurately identified. For example, using the text segmentation algorithm in natural language processing (NLP) technology, combined with the format features of the text (such as font size, bold, italic, etc.) and semantic clues (such as keywords, context semantic coherence, etc.), the third object related to the image object is screened out from the extracted text object.

[0034] Next, semantic analysis is performed on the third object according to the text strategy to obtain semantic features. Semantic analysis refers to the in-depth parsing of text content through natural language processing technology to extract the semantic information of the text. The specific operations include performing word segmentation on the third object to divide the text into independent lexical units; performing part-of-speech tagging to determine the part of speech of each word; and performing syntactic analysis to parse the structure and grammatical relationship of the sentence. On this basis, semantic understanding algorithms are used to extract the semantic features of the text. For example, by using word embedding models (such as Word2Vec, BERT, etc.), the words are mapped into the semantic space to generate semantic vectors of the words, and the semantic features of the text paragraph are obtained by aggregating these vectors. For example, for a text paragraph describing the content of an image, the keywords related to the image theme and their semantic relationships can be extracted through semantic analysis, so as to obtain the semantic features of the text paragraph.

[0035] Then, based on the obtained semantic features, semantic vectors are constructed, and a mapping relationship is obtained in combination with the third object. Semantic vectors are high-dimensional vectors that can represent the semantic information of the text. By converting the semantic features into semantic vectors, the semantic similarity comparison between texts can be realized. In this application, the extracted semantic features can be input into a pre-trained word embedding model to generate corresponding semantic vectors, and then the third object is associated with the corresponding semantic vector to form a mapping relationship. For example, for the title text of an image, its semantic vector can reflect the semantic content of the title, and the mapping relationship between the title text and the semantic vector provides a basis for subsequent image-text association analysis.

[0036] Finally, a content processing result is formed based on the above mapping relationship. The content processing result is text data represented in a structured and semantic form, which not only contains the original content of the text, but also contains the semantic features and semantic vector information of the text. In this way, the text object is converted into a form convenient for subsequent processing and analysis, and can be effectively associated and analyzed with the image processing result. For example, the content processing result can be a data structure containing text paragraphs, semantic vectors, and mapping relationships for subsequent image-text association retrieval.

[0037] Through the implementation of the above steps, the extraction, representation, and structured expression of semantic information in the PDF document can be realized without destroying the original layout structure, providing an executable processing basis for subsequent graphic-text alignment, intelligent indexing, and efficient retrieval.

[0038] P20: Perform an association analysis on the image processing result and the content processing result according to the association index mechanism to obtain a graphic-text index structure.

[0039] Further, step P20 of the embodiment of the present application further includes:

[0040] P21: Obtain any image group in the image processing result; P22: Collect multi-dimensional features of the first arbitrary image in the any image group to obtain a first arbitrary feature parameter; P23: Analyze the first arbitrary feature parameter to determine a target feature parameter; P24: Activate the label classifier in the association index mechanism to perform a classification analysis on the target feature parameter to obtain an arbitrary category label of the any image group; P25: Traverse the content processing result with the arbitrary category label as a traversal constraint to obtain a traversal result; P26: Establish the graphic-text index structure according to the traversal result.

[0041] Specifically, on the basis of completing the image processing result and the content processing result, a semantic association structure between images and texts is constructed, that is, a graphic-text index structure. This structure is a key support data organization form for subsequent intelligent retrieval. Through feature matching and semantic classification analysis based on images and texts, semantic coupling of two types of heterogeneous information can be realized, thereby completing one-to-one mapping or one-to-many and one-to-one linkage organization of graphics and texts.

[0042] During the specific execution process, first obtain any image group from the image processing result. Here, the any image group refers to a set of image objects obtained after preprocessing, and these image objects may include main images, local detail images, or other relevant image elements. The acquisition of the image group can be achieved by classifying or grouping the image processing result. For example, image objects are grouped into different groups according to the source, type, or similarity of the images.

[0043] Next, multi-dimensional feature collection is performed on the first arbitrary image in any image group to obtain the first arbitrary feature parameters. Multi-dimensional feature collection refers to extracting features from different aspects of an image to comprehensively characterize the content and attributes of the image. The specific operations include visual feature extraction, where local feature points of the image are extracted using image processing algorithms (such as SIFT, SURF, ORB, etc.), and these feature points can reflect the texture, shape, and edge information of the image; at the same time, the color histogram, texture features (such as GLCM), and shape features (such as contour information) of the image are calculated to obtain the visual features of the image. In addition, semantic feature extraction is also required. By combining the context information and semantic labels of the image, the semantic features of the image are extracted. For example, through image recognition technology (such as deep learning models), the image is classified or labeled to obtain the semantic category (such as people, scenery, charts, etc.) and keyword descriptions of the image. Finally, spatial feature extraction is carried out to analyze the position information of the image in the document, including the coordinate position, size of the image, and the spatial relationship with other elements (such as text). These spatial features can be used for subsequent association analysis to judge the proximity between the image and the text.

[0044] After the first arbitrary feature parameters are collected, these feature parameters are analyzed to determine the target feature parameters, that is, the subset of features that play a key role in characterizing the image content and semantics. Exemplarily, the multi-dimensional features collected can be screened through feature selection algorithms (such as those based on information gain, principal component analysis PCA, etc.) to remove redundant or irrelevant features and retain the features that are most valuable for image classification and association analysis. For example, for an image containing people and scenery, the target feature parameters may include the facial features of the people, the texture features of the scenery, and the overall semantic category of the image.

[0045] Subsequently, the target feature parameters are classified and analyzed by activating the label classifier in the association index mechanism to obtain the arbitrary category labels of any image group. The label classifier is a pre-trained machine learning model that can classify an image according to the input feature parameters and assign corresponding category labels to it. For example, a classifier constructed using a convolutional neural network (CNN) can classify an image into different semantic categories (such as people, scenery, charts, etc.) and output the corresponding category labels. Through classification analysis, one or more category labels can be assigned to each image group, and these labels will serve as an important basis for subsequent association analysis.

[0046] Next, traverse the content processing results with any category label as the traversal constraint to obtain the traversal results. The content processing results refer to the preprocessed text data, including the semantic features and semantic vectors of the text objects. The traversal process is to search for the relevant text content in the text data according to the category label of the image. The specific operations include semantic matching, which matches the category label of the image with the semantic features of the text to find the text paragraphs or sentences related to the image category. For example, if the category label of the image is "scenery", then search for the text content containing the semantic features related to "scenery" in the text data. At the same time, combine the spatial position information of the image and the text in the document to analyze the proximity between the image and the text. If the text content has a proximity relationship with the image in space (such as being near the image or in the same paragraph as the image), it is considered that they have a high relevance.

[0047] Finally, establish a graphic-text index structure according to the traversal results for storing the association relationship between the image and the text. Specifically, that is to associate the category label, feature parameters of the image and the corresponding text content and store them in a structured manner. For example, an index table can be constructed, where each item contains the identifier of the image, the category label, the feature vector, and the identifier and semantic vector of the text paragraph associated with it. Through this index structure, the retrieval and matching between the image and the text can be quickly realized, improving the retrieval efficiency and accuracy.

[0048] Furthermore, step P23 of the embodiment of the present application further includes:

[0049] P23-1: Collect multi-dimensional features to obtain the second arbitrary feature parameters of the second arbitrary image in the arbitrary image group; P23-2: Analyze the first arbitrary feature parameters and the second arbitrary feature parameters to determine the target feature parameters; wherein, before analyzing the first arbitrary feature parameters and the second arbitrary feature parameters to determine the target feature parameters, it includes:

[0050] P23-21a: Perform discrete cosine transform on the first arbitrary image and the second arbitrary image in sequence to obtain the first transform coefficient and the second transform coefficient respectively; P23-22a: Based on the first DC coefficient and the first AC coefficient in the first transform coefficient, form the first arbitrary feature parameters; P23-23a: Based on the second DC coefficient and the second AC coefficient in the second transform coefficient, form the second arbitrary feature parameters.

[0051] Optionally, the determination process of the target feature parameters can be further refined to enhance the accuracy and robustness of image feature analysis. In particular, in-depth computational processing is performed on the structural similarity and content feature differences between different images in the image group to generate an image feature vector that combines local and global information, which is used for the subsequent determination of the target feature parameters and the input preparation of the image classifier.

[0052] Specifically, first, multi-dimensional feature collection is performed on the second arbitrary image in any image group to obtain the second arbitrary feature parameters. Here, the "second arbitrary image" refers to another image object in the image group, which, together with the first arbitrary image, is used to further analyze the features of the image group. The method and steps of multi-dimensional feature collection are similar to those of the feature collection of the first arbitrary image in P22, including visual feature extraction, semantic feature extraction, and spatial feature extraction, etc., to comprehensively represent the content and attributes of the second arbitrary image.

[0053] Next, before analyzing the first arbitrary feature parameters and the second arbitrary feature parameters to determine the target feature parameters, the first arbitrary image and the second arbitrary image are sequentially subjected to discrete cosine transform (DCT). The discrete cosine transform is a transformation method widely used in image compression and feature extraction. Its function is to map the image from the spatial domain to the frequency domain, so that the main energy in the image is concentrated in a small number of low-frequency components, which is convenient for compression and analysis. By dividing the image into blocks of a fixed size (such as 8×8) and performing a two-dimensional DCT operation on each block area, the first transformation coefficients of the first image and the second transformation coefficients of the second image are obtained respectively. These transformation coefficients reflect the energy distribution of the image in different frequency components, where the direct current coefficient (DC) represents the average brightness of the image, and the alternating current coefficient (AC) reflects the details and texture information of the image.

[0054] Immediately afterwards, based on the first direct current coefficient and the first alternating current coefficient in the first transformation coefficients, the first arbitrary feature parameters are formed. Exemplarily, the first direct current coefficient is used as the overall brightness feature of the image, and some key coefficients (such as low-frequency alternating current coefficients) in the first alternating current coefficient are used as the texture and detail features of the image. According to the preset frequency domain feature construction strategy, the DC value and several most significant AC values are combined into an ordered vector to form the first arbitrary feature parameters. In this way, the coefficients after the DCT transformation can be combined with the previously collected multi-dimensional feature parameters to further enrich the feature representation of the first arbitrary image.

[0055] Similarly, based on the second DC coefficient and the second AC coefficient in the second transformation coefficient, a second arbitrary feature parameter is constructed. Similar to the first arbitrary image, the second DC coefficient is used as the overall brightness feature of the image, and the key coefficients in the second AC coefficient are used as the texture and detail features of the image. Through this combination, the feature representation of the second arbitrary image is also enhanced. During this process, energy threshold screening can be performed on the AC coefficients, and the first several terms with the cumulative energy reaching 95% are retained to reduce redundant calculations.

[0056] After the construction of the above two frequency-domain feature parameters, comprehensive analysis is performed on the first arbitrary feature parameter and the second arbitrary feature parameter. For example, methods such as Euclidean distance, cosine similarity, or projection similarity evaluation method based on principal component analysis (PCA) are used to calculate the feature correlation and difference degree between the two sets of features, and then a representative and highly discriminative feature subset is determined as the target feature parameter.

[0057] This process not only considers the visual and semantic features of the image, but also analyzes the frequency features of the image through DCT, enabling the target feature parameter to more comprehensively represent the content and attributes of the image. Finally, these target feature parameters will be used for subsequent classification analysis and the construction of the graphic-text index structure, providing more accurate and efficient support for realizing the intelligent association retrieval of images and content in PDF documents.

[0058] Furthermore, step P23-2 of the embodiment of the present application further includes:

[0059] P23-21: Obtain the first difference between the first DC coefficient and the second DC coefficient; P23-22: Obtain the second difference between the first AC coefficient and the second AC coefficient; P23-23: Perform variant weighted calculation on the first difference and the second difference to obtain the first deviation index; P23-24: Optimize to obtain the target pair with the maximum first deviation index as the target; P23-25: Take the mean of the first target image group and the second target image group in the target pair to form the target feature parameter.

[0060] In a possible embodiment of the present application, to further improve the discriminability and structural stability of the image feature parameter, the determination process of the target feature parameter can be further refined, automatically screening the image pair with the most significant difference representativeness from the image group, and generating the final target feature parameter with its structural mean.

[0061] First, obtain the first direct current coefficient (DC1) and the second direct current coefficient (DC2) obtained after discrete cosine transform of the first arbitrary image and the second arbitrary image, and calculate their difference to obtain the first difference. This difference reflects the difference between the two images in terms of overall brightness or gray level reference, and can be used to evaluate the degree of change in the overall structure or the morphology of the main region of the image. By calculating the difference between the direct current coefficients of the first arbitrary image and the second arbitrary image, the difference in overall brightness between the two images can be quantified. For example, if the first direct current coefficient is DC1 and the second direct current coefficient is DC2, then the first difference can be expressed as .

[0062] Immediately afterwards, further obtain the first set of alternating current coefficients (AC1) and the second set of alternating current coefficients (AC2), which respectively represent the detailed change information of the non-direct current parts of the two images in the frequency domain. Calculate the difference for each corresponding alternating current coefficient one by one to form a difference vector as the second difference. This difference vector reveals the differences in high-frequency information such as texture and edge structure of the images.

[0063] Next, introduce a variant weighted calculation mechanism, and jointly participate in the calculation of the first difference and the second difference to generate a numerical index comprehensively measuring the difference degree between the two images, denoted as the first deviation index ( ). The specific implementation of variant weighting can adopt the following model:

[0064] ; where α and β are weight coefficients used to adjust the influence balance between the overall brightness difference and the detail variability; is the variance or mean square deviation of the alternating current difference vector, representing the different degrees of the image in terms of structural complexity. The larger the value of this deviation index, the stronger the expression difference between the image pairs, and the better the category discrimination ability.

[0065] Subsequently, with the goal of maximizing the first deviation index, optimize to obtain the target pair. The target pair refers to the image pair with the largest deviation index in the image group. By maximizing the first deviation index, the image pair with the largest difference in brightness and texture can be found. This process can be achieved by traversing all image pairs in the image group and calculating the first deviation index of each pair of images. Finally, select the image pair with the largest first deviation index as the target pair.

[0066] Finally, based on the determined target group pairs, all feature parameters of the first target image group and the second target image group are extracted respectively, and the mean value is calculated dimension by dimension. That is, for the frequency-domain feature vector composed of the DC and AC coefficient combinations, the average of the corresponding values of the two vectors is taken in each dimension, and finally a target feature parameter vector representing the overall features of the image group is formed. This vector not only comprehensively retains the average structural features between representative images, but also avoids the deviation of the overall feature expression caused by abnormal samples, enhancing the expression ability of the classification model for the structural commonalities of the image group. These target feature parameters will be used for subsequent classification analysis and the construction of the graphic index structure, providing more accurate and efficient support for realizing the intelligent association retrieval of images and content in PDF documents.

[0067] Furthermore, step P24 of the embodiment of the present application further includes:

[0068] P24-1: Classify and analyze the target DC coefficient through the label classifier to obtain the target object type; P24-2: Classify and analyze the target AC coefficient through the label classifier to obtain the target scene type; P24-3: The target object type and the target scene type form the arbitrary category label.

[0069] Specifically, to enhance the fine-grained expression ability of the image semantic label, the generation process of the arbitrary category label can be further refined. By classifying and analyzing the target DC coefficient and the target AC coefficient respectively, the target object type and the target scene type are obtained, and the two are combined into the arbitrary category label.

[0070] Specifically, first, classify and analyze the target DC coefficient through the label classifier to obtain the target object type. The target DC coefficient is the DC coefficient part extracted from the target feature parameters, which mainly reflects the overall brightness information of the image. The label classifier is a pre-trained machine learning model that can classify images according to the input feature parameters. By classifying and analyzing the target DC coefficient, the main object types contained in the image can be identified. For example, if the features of the target DC coefficient match the features of the "person" category most, the classifier will output "person" as the target object type.

[0071] Next, classify and analyze the target AC coefficient through the label classifier to obtain the target scene type. The target AC coefficient is the AC coefficient part extracted from the target feature parameters, which mainly reflects the texture and detail information of the image. Similarly, using the label classifier to analyze the target AC coefficient can identify the scene type where the image is located. For example, if the features of the target AC coefficient match the features of the "landscape" category most, the classifier will output "landscape" as the target scene type.

[0072] Finally, combine the target object type and the target scene type into an arbitrary category label. The arbitrary category label is a comprehensive label that contains not only the information of the main object in the image but also the information of the scene where the image is located. This combination method can more comprehensively describe the content of the image and provide richer semantic information for subsequent association analysis. For example, if the target object type is "person" and the target scene type is "landscape", the arbitrary category label can be expressed as "person - landscape".

[0073] Through the design of the dual - channel classification path in this step, it is no longer limited to the traditional single - dimensional image classification method, but constructs a multi - level and multi - label semantic recognition framework based on frequency - domain features, effectively improving the semantic richness and discrimination accuracy of image nodes in the graphic - text index structure, enabling the system to have stronger adaptability and intelligent analysis capabilities when processing diverse document structures (such as scientific research reports, patent specifications, teaching materials).

[0074] P30: Perform a retrieval match on the target retrieval request of the PDF document based on the graphic - text index structure to obtain the target retrieval information.

[0075] Furthermore, step P30 of the embodiment of the present application further includes:

[0076] P31: Determine whether there is an object in the target retrieval request; P32: If there is an object, cooperate with the graphic - text index structure to perform a retrieval match on the object in the PDF document to obtain the target retrieval information; P33: If there is no object, determine whether there is a scene in the target retrieval request; P34: If there is a scene, cooperate with the graphic - text index structure to perform a retrieval match on the scene in the PDF document to obtain the target retrieval information.

[0077] It should be understood that in the retrieval stage, first receive the target retrieval request input by the user, and perform multi - modal element decomposition on the request statement based on the natural language parser to extract the possible object nouns, scene description words, and context - limiting information therein; then call the intent recognition engine to perform semantic annotation on the parsing result to generate a retrieval vector containing an object candidate set, a scene candidate set, and a query confidence threshold.

[0078] When the retrieval process enters the determination phase, the object existence discrimination module is called to quickly scan the above-mentioned retrieval vector: If the retrieval vector contains at least one high-confidence entity mapped to the "target object type" label in the graphic-text index structure, it is determined that "the target retrieval request has an object", and then the object-graphic-text coupling retrieval channel is immediately activated to extract object-related feature information from the retrieval request, such as the name of the object, descriptive keywords, or image feature parameters; Using the image category labels (including the target object type and the target scene type) stored in the graphic-text index structure, search for the image group that matches the object features in the retrieval request. For example, if the object specified in the retrieval request is "person", search for the image group with the label "person" in the graphic-text index structure; Perform further retrieval matching on the matched image group, and combine the feature parameters of the image (such as the target feature parameters) and the semantic vector of the text content to determine the image and text content most relevant to the retrieval request. The final obtained target retrieval information includes the matched image and its related text description.

[0079] If the object existence discrimination result is negative, the scene existence discrimination module is continuously triggered. If a high-confidence description word matching the "target scene type" label appears in the retrieval vector, it is regarded as "the target retrieval request has a scene"; Then, the scene-graphic-text coupling retrieval channel is used instead, and the graph-text joint traversal is performed in the graphic-text index structure with the scene type label as the anchor point, and the image nodes and corresponding text nodes that meet the scene semantic constraints are screened out through the joint sorting of the in-page layout proximity, the semantic vector included angle, and the index confidence, and the structured target retrieval information is also returned.

[0080] In the case where neither of the two-level determinations is triggered, the remaining keywords in the retrieval vector are used as the text priority retrieval conditions, and the full-text inverted index and semantic vector recall module are called to perform complementary matching on the content processing results, ensuring that the most relevant retrieval feedback can be given even if there is a lack of clear object or scene identification.

[0081] The above branch-type retrieval strategy is uniformly supported by the graphic-text index structure, and performs dynamic matching of object priority, scene priority, or text priority for different query intentions respectively, which not only ensures the accuracy and interpretability of the retrieval results, but also avoids the semantic ambiguity and recall deficiency problems of traditional single-channel retrieval under multi-modal requests.

[0082] Furthermore, after determining whether the target retrieval request has a scene, step P30 of the embodiment of the present application further includes:

[0083] P35: If not, perform semantic analysis on the target retrieval request according to the text strategy to obtain a retrieval semantic vector; P36: Collaborate with the graphic-text index structure to perform retrieval matching on the retrieval semantic vector in the PDF document to obtain the target retrieval information.

[0084] Specifically, to comprehensively respond to multi-type target retrieval requests, a supplementary retrieval path based on semantic vectors is further introduced to handle natural language queries in retrieval requests that contain neither explicit object types nor scene descriptions.

[0085] Specifically, after determining whether the target retrieval request contains a scene, if it is determined that the retrieval request contains neither a target object type label that can be mapped to the graphic-text index structure nor a target scene type label, a semantic analysis mechanism based on a text strategy is triggered to perform semantic feature extraction and embedding modeling operations on the original retrieval request.

[0086] In this process, natural language processing techniques are used to perform semantic analysis on the text content in the retrieval request, extract the key semantic information therein, and convert it into a semantic vector. This process can be achieved through pre-trained language models (such as BERT, Word2Vec, etc.), mapping the text in the retrieval request to a high-dimensional semantic space to obtain a retrieval semantic vector. This semantic vector retains the core semantic intention and logical structure features of the query statement and is suitable for alignment with the vector space in the graphic-text index structure.

[0087] Subsequently, using this retrieval semantic vector as the main query vector and collaborating with the semantic space information in the graphic-text index structure, a semantic similarity matching operation is performed across the entire PDF document. Exemplarily, all text nodes and image nodes in the graphic-text index structure are traversed, and the semantic vectors pre-stored or generated online are calculated for similarity with the retrieval semantic vector (such as cosine similarity, Euclidean distance, or Mahalanobis distance), and sorted according to the matching scores to filter out the graphic-text content segments closest to the query semantics. If some image nodes in the graphic-text index structure are not bound to text nodes, their semantic content can be complemented through the image-text alignment path to achieve unsupervised semantic complementation matching.

[0088] Finally, the results above the similarity threshold are packaged as target retrieval information for output, including the image ID and page location of the matching image, the paragraph number and original content of the matching text segment, the matching similarity score, and its context path in the document structure. Multiple high-similarity candidate paths can be retained simultaneously for the user to select, supporting multi-round interactive iterative query optimization. This process makes full use of the image category labels, feature parameters, and semantic vectors of the text content stored in the graphic-text index structure. Through precise matching operations, it can quickly locate the images and text content most relevant to the user's retrieval request. The final obtained target retrieval information not only meets the user's retrieval needs but also improves the accuracy and efficiency of the retrieval, providing efficient support for realizing intelligent associated retrieval of images and content in PDF documents.

[0089] In summary, the embodiments of the present application at least have the following technical effects:

[0090] Through multi-dimensional feature extraction and semantic association analysis of the images and text content in the PDF document, the present application constructs a graphic-text index structure, realizes the deep semantic linkage between images and text, and significantly improves the retrieval accuracy and matching ability of graphic-text information; the target retrieval request matching carried out based on this index structure has an efficient response ability and avoids the waste of resources caused by full-text traversal; at the same time, this method supports the recognition of multi-type target requests such as images, text, and scenes and the selection of branch retrieval paths, enhancing the adaptability of the system in complex query contexts; the introduction of discrete cosine transform, multi-dimensional difference analysis, and variant weighting strategies in the image processing process further improves the discriminability of image features and the accuracy of index classification; the overall process realizes the standardized and automated processing of graphic-text information, and can complete the construction of high-quality graphic-text correspondence relationships without manual annotation, having good adaptability, scalability, and engineering practical value.

[0091] It achieves the technical effect of realizing the intelligent associated retrieval of image and text content by constructing a graphic-text index structure, and improving the retrieval efficiency and accuracy.

[0092] Embodiment 2, based on the same inventive concept as the intelligent retrieval method for the association between images and content in the PDF document in the foregoing embodiment, as Figure 2 shown, the present application provides an intelligent retrieval system for the association between images and content in a PDF document. The system in the embodiments of the present application and the method embodiments are based on the same inventive concept. Among them, the system includes:

[0093] A document preprocessing module 11, which is used to retrieve a document preprocessing strategy to preprocess the PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result.

[0094] A graphic-text association analysis module 12, which is used to perform association analysis on the image processing result and the content processing result according to an association index mechanism to obtain a graphic-text index structure.

[0095] A retrieval matching module 13, which is used to perform retrieval matching on the target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information.

[0096] Furthermore, the document preprocessing module 11 is further used to perform the following steps:

[0097] Perform discrimination extraction on the PDF document with the object structure as the discrimination constraint to obtain an extraction result; process the image objects in the extraction result according to the image strategy in the document preprocessing strategy to obtain the image processing result; wherein, it includes: extracting the first object and the second object in the image object; sequentially performing standardization processing on the first object and the second object according to the image standardization plan in the image strategy to obtain a first standard image and a second standard image respectively; sequentially calculating a first hash value of the first standard image and a second hash value of the second standard image; comparing the first hash value with the second hash value to obtain a Hamming distance; if the Hamming distance does not reach a predetermined threshold, divide the first object and the second object to obtain the image processing result.

[0098] Further, the document preprocessing module 11 is further configured to perform the following steps:

[0099] Process the text objects in the extraction result according to the text strategy in the document preprocessing strategy to obtain the content processing result; wherein, it includes: extracting a third object in the text object; performing semantic analysis on the third object according to the text strategy to obtain semantic features; constructing a semantic vector based on the semantic features and combining it with the third object to obtain a mapping relationship; forming the content processing result based on the mapping relationship.

[0100] Further, the graphic-text association analysis module 12 is further configured to perform the following steps:

[0101] Obtain any image group in the image processing result; collect multi-dimensional features of the first arbitrary image in the any image group to obtain a first arbitrary feature parameter; analyze the first arbitrary feature parameter to determine a target feature parameter; activate the label classifier in the association index mechanism to perform classification analysis on the target feature parameter to obtain an arbitrary category label of the any image group; traverse the content processing result with the arbitrary category label as the traversal constraint to obtain a traversal result; establish the graphic-text index structure according to the traversal result.

[0102] Further, the graphic-text association analysis module 12 is further configured to perform the following steps:

[0103] Collect multi-dimensional features to obtain the second arbitrary feature parameter of the second arbitrary image in the arbitrary image group; analyze the first arbitrary feature parameter and the second arbitrary feature parameter to determine the target feature parameter; wherein, before analyzing the first arbitrary feature parameter and the second arbitrary feature parameter to determine the target feature parameter, it includes: performing discrete cosine transform on the first arbitrary image and the second arbitrary image in sequence to obtain a first transform coefficient and a second transform coefficient respectively; based on the first DC coefficient and the first AC coefficient in the first transform coefficient, form the first arbitrary feature parameter; based on the second DC coefficient and the second AC coefficient in the second transform coefficient, form the second arbitrary feature parameter.

[0104] Further, the graphic-text association analysis module 12 is further configured to perform the following steps:

[0105] Obtain a first difference between the first DC coefficient and the second DC coefficient; obtain a second difference between the first AC coefficient and the second AC coefficient; perform mutation weighted calculation on the first difference and the second difference to obtain a first deviation index; with the maximum of the first deviation index as the target, optimize to obtain a target pair; take the mean of the first target image group and the second target image group in the target pair to form the target feature parameter.

[0106] Further, the graphic-text association analysis module 12 is further configured to perform the following steps:

[0107] Perform classification analysis on the target DC coefficient through the label classifier to obtain the target object type; perform classification analysis on the target AC coefficient through the label classifier to obtain the target scene type; the target object type and the target scene type form the arbitrary category label.

[0108] Further, the retrieval and matching module 13 is further configured to perform the following steps:

[0109] Judge whether there is an object in the target retrieval request; if there is, cooperate with the graphic-text index structure to perform retrieval and matching on the object in the PDF document to obtain the target retrieval information; if not, judge whether there is a scene in the target retrieval request; if there is, cooperate with the graphic-text index structure to perform retrieval and matching on the scene in the PDF document to obtain the target retrieval information.

[0110] Further, the retrieval and matching module 13 is further configured to perform the following steps:

[0111] If not, semantic analysis is performed on the target retrieval request according to the text strategy to obtain a retrieval semantic vector; and the retrieval semantic vector is retrieved and matched in the PDF document in cooperation with the graphic-text index structure to obtain the target retrieval information.

[0112] It should be noted that the above order of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of the present specification have been described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0114] This specification and the drawings are only exemplary descriptions of the present application and are considered to have covered any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and modifications.

Claims

1. An intelligent retrieval method for associating images with content in a PDF document, characterized in that Including: Retrieve the document preprocessing strategy to preprocess the PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result; Perform correlation analysis on the image processing result and the content processing result according to the correlation index mechanism to obtain a graphic-text index structure; Perform retrieval matching on the target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information.

2. The intelligent retrieval method for associating images with content in a PDF document according to claim 1, characterized in that, Retrieve the document preprocessing strategy to preprocess the PDF document to obtain a document processing result, including: Perform discrimination extraction on the PDF document with the object structure as the discrimination constraint to obtain an extraction result; Process the image objects in the extraction result according to the image strategy in the document preprocessing strategy to obtain the image processing result; Among them, including: Extract the first object and the second object in the image object; Perform standardization processing on the first object and the second object in turn according to the image standardization plan in the image strategy to obtain the first standard image and the second standard image respectively; Calculate the first hash value of the first standard image and the second hash value of the second standard image in turn; Compare the first hash value with the second hash value to obtain the Hamming distance; If the Hamming distance does not reach the predetermined threshold, divide the first object and the second object to obtain the image processing result.

3. The intelligent retrieval method for associating images with content in a PDF document according to claim 2, wherein, Retrieve the document preprocessing strategy to preprocess the PDF document to obtain a document processing result, including: Process the text objects in the extraction result according to the text strategy in the document preprocessing strategy to obtain the content processing result; Among them, including: Extract the third object in the text object; Perform semantic analysis on the third object according to the text strategy to obtain semantic features; Construct a semantic vector based on the semantic features and combine it with the third object to obtain a mapping relationship; Form the content processing result based on the mapping relationship.

4. The intelligent retrieval method for associating images with content in a PDF document according to claim 1, wherein Perform correlation analysis on the image processing result and the content processing result according to the correlation index mechanism to obtain a graphic-text index structure, including: Obtain any image group in the image processing result; Collect multi-dimensional features of the first arbitrary image in the arbitrary image group to obtain the first arbitrary feature parameter; Analyze the first arbitrary feature parameter to determine the target feature parameter; Activate the label classifier in the correlation index mechanism to perform classification analysis on the target feature parameter to obtain the arbitrary category label of the arbitrary image group; Traverse the content processing result with the arbitrary category label as the traversal constraint to obtain a traversal result; Establish the graphic-text index structure according to the traversal result.

5. The intelligent retrieval method for associating images with content in a PDF document according to claim 4, wherein, Analyze the first arbitrary feature parameter to determine the target feature parameter, including: Collect multi-dimensional features to obtain the second arbitrary feature parameter of the second arbitrary image in the arbitrary image group; Analyze the first arbitrary feature parameter and the second arbitrary feature parameter to determine the target feature parameter; Among them, before analyzing the first arbitrary feature parameter and the second arbitrary feature parameter to determine the target feature parameter, including: Perform discrete cosine transform on the first arbitrary image and the second arbitrary image in sequence to obtain a first transform coefficient and a second transform coefficient respectively; Based on the first DC coefficient and the first AC coefficient in the first transform coefficient, construct the first arbitrary feature parameter; Based on the second DC coefficient and the second AC coefficient in the second transform coefficient, construct the second arbitrary feature parameter.

6. The intelligent retrieval method for associating images with content in a PDF document according to claim 5, wherein, Analyze the first arbitrary feature parameter and the second arbitrary feature parameter to determine the target feature parameter, including: Obtain a first difference between the first DC coefficient and the second DC coefficient; Obtain a second difference between the first AC coefficient and the second AC coefficient; Perform mutation weighted calculation on the first difference and the second difference to obtain a first deviation index; Taking the maximum of the first deviation index as the target, optimize to obtain a target pair; Take the mean of the first target image group and the second target image group in the target pair to form the target feature parameter.

7. The intelligent retrieval method for associating images with content in a PDF document according to claim 4, characterized in that, Activate the classification analysis of the target feature parameter by the label classifier in the correlation indexing mechanism to obtain an arbitrary category label of the arbitrary image group, including: Perform classification analysis on the target DC coefficient through the label classifier to obtain a target object type; Perform classification analysis on the target AC coefficient through the label classifier to obtain a target scene type; The target object type and the target scene type form the arbitrary category label.

8. The intelligent retrieval method for the association between an image and content in a PDF document according to claim 3, wherein Perform retrieval matching on the target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information, including: Determine whether there is an object in the target retrieval request; If there is, cooperate with the graphic-text index structure to perform retrieval matching on the object in the PDF document to obtain the target retrieval information; If not, determine whether there is a scene in the target retrieval request; If there is, cooperate with the graphic-text index structure to perform retrieval matching on the scene in the PDF document to obtain the target retrieval information.

9. The intelligent retrieval method for associating images with content in a PDF document according to claim 8, wherein After determining whether there is a scene in the target retrieval request, it further includes: If not, perform semantic analysis on the target retrieval request according to the text strategy to obtain a retrieval semantic vector; Cooperate with the graphic-text index structure to perform retrieval matching on the retrieval semantic vector in the PDF document to obtain the target retrieval information.

10. An intelligent retrieval system for associating images with content in a PDF document, characterized in that, The system includes: A document preprocessing module, which is used to retrieve the document preprocessing strategy to preprocess the PDF document to obtain a document processing result, where the document processing result includes an image processing result and a content processing result; A graphic-text correlation analysis module, which is used to perform correlation analysis on the image processing result and the content processing result according to the correlation indexing mechanism to obtain a graphic-text index structure; A retrieval matching module, which is used to perform retrieval matching on the target retrieval request of the PDF document based on the graphic-text index structure to obtain target retrieval information.

Images