PDF (Portable Document Format) image quick retrieval system fusing content awareness

Abstract

The invention discloses a PDF image quick retrieval system fused with content awareness, which belongs to the field of document retrieval, and comprises a retrieval task receiving module used for receiving a PDF image retrieval task of a target user; the retrieval expansion module is used for performing verification expansion on the retrieval target words to generate a retrieval keyword list; the content perception retrieval module is used for performing content association retrieval on the PDF file to obtain a plurality of associated image blocks; the deep retrieval module is used for performing deep retrieval on the plurality of associated image blocks to obtain a PDF image retrieval result; and the retrieval feedback module is used for feeding back the PDF image retrieval result to the target user. According to the method and the device, the technical problems of low PDF document image retrieval efficiency and insufficient retrieval result accuracy in the prior art are solved, and the technical effect of improving the accuracy and the efficiency of PDF image retrieval through a mode of combining content perception and depth feature twin constraint is achieved.

Description

Technical Field

[0001] The present invention relates to the field of document retrieval, and in particular to a PDF image fast retrieval system integrating content perception. Background Art

[0002] As a widely used electronic document format, PDF documents have become an important carrier for information dissemination and storage. Existing PDF image retrieval technologies mainly rely on simple text matching or basic image feature extraction. These methods have the following shortcomings: on the one hand, due to the lack of in-depth understanding of the semantics of PDF document content, the accuracy of retrieval results is low; on the other hand, traditional feature extraction methods cannot effectively capture the deep semantic information of images, resulting in low retrieval efficiency. Therefore, the PDF document image retrieval efficiency in the existing technology is low and the retrieval result accuracy is insufficient. Summary of the invention

[0003] This application aims to solve the technical problems of low efficiency and insufficient accuracy of PDF document image retrieval in the prior art by providing a content-aware PDF image fast retrieval system.

[0004] The present application discloses a content-aware PDF image fast retrieval system, comprising: a retrieval task receiving module, used to receive a PDF image retrieval task of a target user, wherein the PDF image retrieval task comprises a PDF file and a retrieval target word; a retrieval expansion module, used to verify and expand the retrieval target word according to a retrieval word verifier and a retrieval expansion multi-channel, and generate a retrieval keyword list; a content-aware retrieval module, used to perform content-related retrieval on the PDF file based on content alignment constraints in combination with a retrieval keyword list, and obtain multiple related image blocks; a deep retrieval module, used to perform deep retrieval on multiple related image blocks in combination with a retrieval keyword list according to a predetermined retrieval window constraint and a deep feature twin constraint, and obtain a PDF image retrieval result; and a retrieval feedback module, used to feed back the PDF image retrieval result to the target user.

[0005] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0006] The PDF image retrieval task receiving module receives the PDF image retrieval task of the target user, including the PDF file and the retrieval target word, so as to ensure that the user's retrieval needs and the scope of the document to be retrieved can be accurately obtained; the retrieval expansion module uses the retrieval word verifier and the retrieval expansion multi-channel to verify and expand the retrieval target word, and generates a retrieval keyword list. Through multi-dimensional keyword expansion, the retrieval scope is effectively expanded and the retrieval recall rate is improved; the content-aware retrieval module performs content-related retrieval on the PDF file based on the content alignment constraint and the retrieval keyword list, obtains multiple related image blocks, and improves the relevance of the preliminary retrieval results; the deep retrieval module performs deep retrieval on multiple related image blocks according to the predetermined retrieval window constraint and the deep feature twin constraint and the retrieval keyword list to obtain the PDF image retrieval result, and further improves the retrieval accuracy; the retrieval feedback module feeds back the PDF image retrieval result to the target user, completes the entire retrieval process, and ensures that the retrieval result can be presented to the user in a timely and accurate manner. The technical solution solves the technical problems of low efficiency of PDF document image retrieval and insufficient accuracy of retrieval results in the prior art, and achieves the technical effect of improving the accuracy and efficiency of PDF image retrieval by combining content perception and deep feature twin constraints.

[0007] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A structural schematic diagram of a content-aware PDF image fast retrieval system is provided for an embodiment of the present application;

[0009] Figure 2 A flowchart of establishing an n-th associated image block in a content-aware PDF image fast retrieval system is provided for an embodiment of the present application.

[0010] Explanation of the accompanying drawings: retrieval task receiving module 11, retrieval expansion module 12, content-aware retrieval module 13, deep retrieval module 14, retrieval feedback module 15. DETAILED DESCRIPTION

[0011] The overall idea of ​​the technical solution provided by this application is as follows:

[0012] The embodiment of the present application provides a fast PDF image retrieval system that integrates content perception. First, the user's retrieval requirements are obtained through the retrieval task receiving module, laying the foundation for the subsequent retrieval process; secondly, the retrieval expansion module is used to expand the multi-channel keywords and expand the retrieval dimension; then the semantic association between text and image is established through the content-aware retrieval module to achieve preliminary content-related retrieval; then the deep retrieval module is used to perform precise matching based on the retrieval window constraints and deep feature twin constraints to improve the retrieval accuracy; finally, the retrieval feedback module presents the retrieval results to the user, which significantly improves the accuracy of PDF image retrieval while ensuring the retrieval efficiency.

[0013] After introducing the basic principles of the present application, the non-limiting implementation methods of the present application will be specifically introduced below in conjunction with the drawings in the specification.

[0014] like Figure 1 As shown, the embodiment of the present application provides a content-aware PDF image fast retrieval system, the system comprising:

[0015] The retrieval task receiving module 11 is used to receive a PDF image retrieval task from a target user, wherein the PDF image retrieval task includes a PDF file and a retrieval target word.

[0016] Specifically, the retrieval task receiving module 11 is responsible for receiving the PDF image retrieval task submitted by the target user. The PDF image retrieval task includes two key parts: the PDF file to be retrieved and the retrieval target word input by the user, which indicates the need for image retrieval of a specific PDF file. Among them, the PDF file to be retrieved is a complete electronic document containing a large amount of image and text information. These images are distributed on different pages and different positions of the PDF file, and the image contents are different; the retrieval target word is a keyword or phrase entered by the target user that he hopes to find in the PDF image. By providing the retrieval target word, the target user clarifies the retrieval intention and determines the retrieval direction and focus.

[0017] The search expansion module 12 is used to verify and expand the search target words according to the search word verifier and the search expansion multi-channel to generate a search keyword list.

[0018] Specifically, the search expansion module 12 includes two sub-parts: a search term verifier and a search expansion multi-channel. Among them, the search term verifier is responsible for performing searchability verification on the search target word and judging whether the target word meets the search requirements and specifications. Through the verification of the search term verifier, illegal, erroneous or unrecognizable search target words can be filtered out, thereby improving the search efficiency and accuracy. The search expansion multi-channel performs multi-dimensional and multi-angle semantic expansion and association analysis on the search target word after it passes the verification. The search expansion multi-channel includes multiple parallel processing channels such as a search term splitting channel, a search synonym expansion channel, and a search synonym expansion channel. Through the parallel expansion of multiple channels, the search vocabulary can be enriched and expanded from multiple dimensions to improve the comprehensiveness and accuracy of the search.

[0019] After the search term checker completes the check and the search expansion multi-channel completes the expansion, the search expansion module 12 combines the expanded multiple synonyms and near synonyms with the original search target words, removes duplicates, and finally generates a complete search keyword list. The list includes multiple keywords that are highly relevant to the target user's search intent and have rich semantics, providing a key vocabulary foundation for subsequent content-aware retrieval, deep retrieval and other steps.

[0020] The content-aware retrieval module 13 is used to perform content-related retrieval on the PDF file based on content registration constraints and in combination with the retrieval keyword list to obtain a plurality of related image blocks.

[0021] Specifically, the content-aware retrieval module 13 first analyzes and processes the PDF file based on the content registration constraints. The content registration constraints refer to a series of rules and conditions preset by the system for measuring the relevance between the content of the PDF file and the search keyword list. These rules and conditions may include measurement indicators of multiple dimensions such as the distribution density of keywords, the location information of keywords, and the proximity between keywords and images. The content-aware retrieval module 13 traverses each page of the PDF file, extracts the text and image information therein, and performs relevance evaluation and scoring according to the content registration constraints.

[0022] After completing the evaluation of the content registration constraint, the content-aware retrieval module 13 will combine the search keyword list to perform content-related retrieval on the PDF file. Specifically, the content-aware retrieval module 13 matches and compares each keyword in the search keyword list with the content of the PDF file to find the image area that contains the keyword and satisfies the content registration constraint. These image areas are distributed on different pages and different locations of the PDF file. The content-aware retrieval module 13 accurately locates and extracts these image areas to obtain multiple associated image blocks.

[0023] After content-related retrieval, the content-aware retrieval module 13 obtains a plurality of associated image blocks associated with the retrieval keyword list, which contain image information that is potentially of interest to the target user.

[0024] The deep search module 14 is used to perform a deep search on the multiple associated image blocks according to a predetermined search window constraint and a deep feature twin constraint in combination with the search keyword list to obtain a PDF image search result.

[0025] Specifically, the predetermined retrieval window constraint refers to a method of dividing the associated image block into multiple retrieval windows of fixed size and position, and performing local feature extraction and analysis within the retrieval window. Through the retrieval window, the deep retrieval module 14 can focus more specifically on the local details and features of the image, thereby improving the accuracy and reliability of the retrieval. The deep retrieval module 14 divides and maps the associated image block according to the preset multiple retrieval window size and position schemes to obtain a series of local image blocks within the retrieval window.

[0026] After obtaining the local image block in the retrieval window, the deep retrieval module 14 will further apply the deep feature twin constraint to extract the deep semantic features of the image block, and match and calculate the similarity with the retrieval keyword list. The deep feature twin constraint is a feature extraction and comparison method based on deep learning. By training the twin neural network, the deep semantic association between the image block and the keyword is learned to obtain a highly abstract and semantically rich feature representation. By calculating the similarity between the image block features and the retrieval keyword features, the deep retrieval module 14 can accurately evaluate the relevance of the image block to the target user's retrieval intent, and achieve more accurate and comprehensive image retrieval.

[0027] After completing the processing of the predetermined search window constraints and the deep feature twin constraints, the deep search module 14 will comprehensively analyze the relevance scores and rankings of all image blocks in the search window, and select the image blocks with the highest scores and the strongest relevance according to the set thresholds and rules to form the final PDF image search results. These image search results are highly matched with the search intentions of the target users and contain the core image information that the users are interested in.

[0028] The retrieval feedback module 15 is used to feed back the PDF image retrieval result to the target user.

[0029] Specifically, the retrieval feedback module 15 receives the PDF image retrieval results transmitted by the deep retrieval module 14, and performs necessary format conversion, data encapsulation and visualization processing on them, so that they can be displayed in a way that is easy for the target user to accept and understand. These processes may include extracting key frames of image retrieval results, generating thumbnail previews of image retrieval results, and mapping the positions of image retrieval results with original PDF files, so as to generate an intuitive and friendly image retrieval result display interface. After completing the processing and encapsulation of the image retrieval results, the retrieval feedback module 15 selects appropriate feedback channels and display forms according to the system access method and interaction preferences of the target user, so as to achieve fast and accurate PDF image retrieval. For example, for the target user on the Web side, the retrieval feedback module 15 pushes the image retrieval results to the user's browser in the form of a web page; for the target user on the mobile side, the retrieval feedback module 15 displays the image retrieval results on the user's mobile phone screen through mobile applications or applets; for the target user accessed by API, the retrieval feedback module 15 returns the image retrieval results to the user's application in a structured data format (such as JSON, XML, etc.).

[0030] Furthermore, the embodiment of the present application also includes:

[0031] The search target word is subjected to a searchability check according to the search term checker to obtain a target word check result; when the target word check result is passed, the search expansion multi-channel is activated, wherein the search expansion multi-channel includes a search term splitting channel, a search synonym expansion channel and a search synonym expansion channel; the search target word is input into the search term splitting channel to obtain a plurality of search split words; based on the plurality of search split words, a plurality of groups of synonym expansion results are obtained according to the search synonym expansion channel; based on the plurality of search split words, a plurality of groups of synonym expansion results are obtained according to the search synonym expansion channel; the plurality of groups of synonym expansion results are combined according to the plurality of groups of synonym expansion results and the plurality of groups of synonym expansion results, and the search target word is combined to construct the search keyword list.

[0032] In a feasible implementation, first, the search expansion module 12 uses a search term verifier to perform a searchability check on the search target word. The search term verifier has a set of predefined verification rules and algorithms built in it, which are used to evaluate the legitimacy, integrity and identifiability of the search target word. By performing lexical analysis, grammatical analysis and semantic analysis on the search target word, the search term verifier can determine whether the search target word meets the search requirements of the system, and give the corresponding target word verification result. When the search target word passes the verification of the search term verifier, that is, the target word verification result is passed, the search expansion module 12 will start the search expansion multi-channel to perform multi-dimensional expansion on the search target word. The search expansion multi-channel includes three parallel processing channels, namely, a search term splitting channel, a search synonym expansion channel and a search synonym expansion channel. These three channels perform semantic association analysis and expansion on the search target word from different angles, respectively, to generate multiple groups of semantically related expansion results.

[0033] Specifically, the search term splitting channel splits the search target term into fine-grained parts to obtain multiple independent search split terms. This splitting can reveal the internal structure of the search target term and help discover more key information contained in the search target term. For example, for the search target term "image segmentation algorithm", the search term splitting channel splits it into three search split terms, "image", "segmentation", and "algorithm", so that subsequent expansion can be carried out at a finer granularity. The search synonym expansion channel uses the pre-built synonym dictionary and knowledge base to match synonyms and equivalent expressions for the search split terms to obtain multiple groups of synonym expansion results. By introducing synonyms, this channel can expand the semantic coverage of the search terms and improve the recall rate of the search. For example, for the search split term "segmentation", the synonym expansion channel can obtain a series of synonym expansion results such as "cutting", "separation", and "division". The working principle of the search synonym expansion channel is similar to that of the synonym expansion channel, but it focuses on the conceptual synonyms and hyponyms of the search split terms. By identifying semantically related but not completely equivalent words, the synonym expansion channel can introduce more generalized or specialized search terms to enrich the diversity of search results. For example, for the search split word "algorithm", the synonym expansion channel can obtain a series of synonym expansion results such as "method", "strategy", and "process".

[0034] After the three expansion channels have completed their expansion, the search expansion module 12 summarizes, removes duplicates and combines all synonym expansion results and near-synonym expansion results, and at the same time combines the original search target words to build a diversified and comprehensive search keyword list. The list not only includes the search target words directly given by the target user, but also covers a large number of semantically related expansion keywords.

[0035] Furthermore, the embodiment of the present application also includes:

[0036] Perform a searchability assessment based on the search target word to obtain the target word searchability; input the target word searchability into the search word verifier to generate the target word verification result, wherein the search word verifier includes a search word verification rule, and the search word verification rule includes outputting the target word verification result as passed when the target word searchability is greater than / equal to a preset searchability, and outputting the target word verification result as failed when the target word searchability is less than the preset searchability.

[0037] In a preferred embodiment, the search term verifier in the search expansion module 12 uses a method based on retrievability evaluation and verification rules to perform retrievability verification on the search target word, and then obtain the target word verification result. Specifically, the search term verifier first performs a retrievability evaluation based on the search target word to obtain a quantitative target word searchability. The target word searchability is a comprehensive measurement indicator used to measure the possibility of the search target word being effectively recognized and indexed by the search system. The target word searchability is obtained by comprehensively considering multiple factors such as the lexical characteristics, grammatical characteristics, and semantic characteristics of the search target word. Among them, in the retrievability evaluation process, the search term verifier focuses on the following two aspects, namely lexical characteristics and grammatical characteristics. For lexical characteristics, the search target word should try to avoid using overly uncommon, rare or specialized words, as well as special characters or symbols that cannot be recognized by the search system. This is because overly unpopular words will exceed the coverage of the search dictionary, resulting in the inability to be effectively indexed; and special characters or symbols may interfere with the lexical analysis and grammatical analysis process of the search, affecting the search performance. Therefore, by scoring the lexical characteristics of the search target words, the more common and standardized the vocabulary, the higher the score. For grammatical characteristics, the search target words should comply with the basic grammatical rules of the target language to avoid spelling errors, grammatical errors or non-standard expressions. This is because incorrect or non-standard search target words may cause the search to fail to accurately understand their semantics, thereby affecting the accuracy and efficiency of the search. Therefore, by analyzing the grammatical characteristics of the search target words, examining whether their spelling, punctuation use, grammatical structure, etc. are standardized, the more correct and standardized the grammar, the higher the score.

[0038] After completing the target word searchability evaluation, the search word verifier compares the target word searchability with the preset searchability, and gives the final target word verification result according to the comparison result. Specifically, the search word verifier is provided with a search word verification rule for mapping the target word searchability to the target word verification result. According to the search word verification rule, when the target word searchability is greater than or equal to the preset searchability threshold, the verifier outputs a passed target word verification result, indicating that the search target word meets the searchability requirements; when the target word searchability is less than the preset searchability threshold, the verifier outputs a failed target word verification result, indicating that the search target word does not meet the searchability requirements and needs further optimization and adjustment. Among them, the preset searchability threshold is an empirical value set according to the actual search capability of the system and user needs, which determines the verification standard and strictness of the search word verifier. The higher the preset searchability, the higher the searchability requirements for the search target word, and the greater the difficulty of passing the verification; conversely, the lower the preset searchability, the looser the searchability requirements for the search target word, and the less difficult it is to pass the verification. Therefore, in practical applications, it is necessary to repeatedly test and tune the preset searchability based on specific field characteristics, user groups and other factors to balance retrieval performance and user experience.

[0039] By introducing the target word searchability evaluation and search word verification rules, we can effectively judge the searchability of the search target word, filter out unqualified or low-quality search words, and provide a good data foundation for subsequent search expansion. At the same time, the search word verifier provides a basis and direction for further optimization and improvement of the search target word.

[0040] Furthermore, the embodiment of the present application also includes:

[0041] When the target word verification result is not passed, a target word warning signal is generated; the target word warning signal is sent to the target user to obtain user feedback data; and the retrieval target word is optimized according to the user feedback data.

[0042] In a preferred embodiment, the search expansion module 12 also has the search target word optimization and feedback functions, and can dynamically adjust and improve the search target word according to the target word verification result, thereby improving the search quality and user experience.

[0043] Specifically, when the target word verification result output by the search word checker is not passed, it means that the current search target word does not meet the searchability requirements, which will have a negative impact on the subsequent search expansion and image retrieval. In order to correct this situation in time, the search expansion module 12 automatically generates a target word warning signal, indicating that the search target word has a searchability risk and needs to be further optimized. After generating the target word warning signal, the search expansion module 12 sends the signal to the interactive interface of the target user, prompting the user that there is a problem with the current search target word, and requests the user to provide optimization opinions and feedback. This interactive feedback mechanism can make full use of the user's domain knowledge and language knowledge, and provide valuable references for optimizing the search target word. After receiving the feedback data from the target user, the search expansion module 12 analyzes and processes the feedback data, and automatically or semi-automatically optimizes and adjusts the original search target word. Among them, the optimization strategy includes spelling correction, grammar correction, synonym replacement, phrase simplification and other methods, the purpose is to improve the searchability and standardization of the search target word on the basis of retaining the original search intention. The optimized search target word will re-enter the search word checker for searchability verification. If the verification passes, it means that the optimization is successful, and the new search target word can replace the original search target word and enter the subsequent search expansion process; if the verification still fails, the optimization can continue until a satisfactory search target word is obtained.

[0044] Through the optimization and feedback mechanism of retrieval target words, the quality and searchability of retrieval target words can be effectively improved, and the problems such as retrieval failure or performance degradation caused by inappropriate retrieval words can be reduced.

[0045] Furthermore, the implementation of this application also includes:

[0046] Perform multiple authentications on the PDF file to obtain a PDF authentication result; when the PDF authentication result is passed, split the PDF file to obtain multiple PDF images; extract the nth search keyword according to the search keyword list, where n is a positive integer; based on the content registration constraint, perform content association retrieval on the multiple PDF images according to the nth search keyword, establish an nth associated image block, and add the nth associated image block to the multiple associated image blocks.

[0047] In a feasible implementation, the content-aware retrieval module 13 is responsible for performing content-based association retrieval on the PDF file, so as to obtain multiple associated image blocks related to the search keyword list. First, the content-aware retrieval module 13 performs multiple authentications on the input PDF file to obtain a PDF authentication result to ensure the legality, integrity and processability of the file. Multiple authentications can include file format verification, file integrity verification, file permission check and other aspects, with the purpose of filtering out PDF files that do not meet the requirements or may cause processing errors. Only PDF files that have passed multiple authentications can enter the subsequent processing flow. When the PDF authentication result is that the PDF file has passed multiple authentications, the content-aware retrieval module 13 performs splitting processing on the PDF file. The purpose of this processing is to convert the PDF file into a series of independent image files for image-level content analysis and retrieval. Each split image file corresponds to a page or an independent image area in the PDF file, retaining the image content and layout information in the original PDF file.

[0048] After splitting the PDF file, the content-aware retrieval module 13 will obtain multiple PDF images. Next, the content-aware retrieval module 13 extracts the retrieval keywords one by one from the retrieval keyword list and uses them to guide the subsequent content-related retrieval. Specifically, the nth retrieval keyword is extracted from the retrieval keyword list, where n is a positive integer, indicating the position or serial number of the retrieval keyword in the list. After extracting the nth retrieval keyword, the content-aware retrieval module 13 performs content-related retrieval on multiple PDF images according to the preset content alignment constraint. Among them, the content alignment constraint is used to measure the relevance between the image content and the retrieval keyword. The content-aware retrieval module 13 comprehensively considers these constraints and calculates the relevance score between each PDF image and the nth retrieval keyword. After the content-related retrieval, the content-aware retrieval module 13 selects a group of images with the highest relevance to the nth retrieval keyword from multiple PDF images to form the nth associated image block. The images in this block have a strong relevance to the nth retrieval keyword in terms of content, and serve as important retrieval results for the keyword. The content-aware retrieval module 13 adds the nth associated image block to a global multiple associated image blocks for storing the associated image blocks of all search keywords. The content-aware retrieval module 13 will repeat the above extraction and content-associated retrieval process for each search keyword in the search keyword list until all search keywords in the list are processed. Finally, the content-aware retrieval module 13 will obtain multiple associated image blocks, including multiple groups of image blocks related to different search keywords. These image blocks will be output as the results of content-aware retrieval to the subsequent deep retrieval module 14 for further analysis and screening.

[0049] Further, such as Figure 2As shown, the embodiment of the present application also includes:

[0050] The method comprises the steps of traversing the plurality of PDF images and extracting a first PDF image; performing content perception according to the first PDF image to obtain content feature information of the first image; performing association evaluation on the content feature information of the first image according to the nth search keyword to obtain a content registration coefficient of the first image; judging whether the content registration coefficient of the first image satisfies the content registration constraint; if the content registration coefficient of the first image satisfies the content registration constraint, setting the first PDF image as a first associated image, and adding the first associated image to the nth associated image block.

[0051] In a preferred embodiment, the content-aware retrieval module 13 adopts an image-by-image analysis method based on content registration constraints to establish the nth associated image block when performing content-associated retrieval.

[0052] Specifically, the content-aware retrieval module 13 first extracts a single PDF image from multiple PDF images one by one, which is recorded as the first PDF image. Then, the content-aware retrieval module 13 performs an in-depth content-aware analysis on the first PDF image, extracts its inherent semantic features and structural features, and forms the first image content feature information. The first image content feature information includes multiple aspects such as the main object of the image, the scene context, the color distribution, and the texture features, reflecting the content semantics and visual attributes of the first PDF image. After obtaining the first image content feature information, the content-aware retrieval module 13 performs an association evaluation on the first image content feature information according to the nth search keyword currently being processed. Specifically, the content-aware retrieval module 13 calculates the correlation, matching degree and other indicators between the first image content feature information and the nth search keyword, and obtains a quantized first image content registration coefficient. The higher the first image content registration coefficient, the stronger the content association between the first PDF image and the nth search keyword, and vice versa. After obtaining the first image content registration coefficient, the content-aware retrieval module 13 compares it with the preset content registration constraint to determine whether the first PDF image meets the requirements of content association retrieval. The content registration constraint is one or more threshold conditions used to define the minimum standard of content relevance. For example, a content registration coefficient lower limit is set, and only when the first image content registration coefficient is greater than or equal to the lower limit, the first PDF image is considered to have sufficient content relevance to the nth search keyword.

[0053] If the first image content registration coefficient meets the content registration constraint, the content-aware retrieval module 13 will mark the first PDF image as the first associated image, indicating that the image has passed the screening of the content-associated retrieval and has become a valid retrieval result for the nth search keyword. At the same time, the content-aware retrieval module 13 adds the first associated image to the nth associated image block, and together with other associated images that also meet the content registration constraint, they constitute a complete associated image block for the nth search keyword. The content-aware retrieval module 13 repeats the above-mentioned process of content perception, association evaluation, and content registration judgment for each of the multiple PDF images until all single PDF images have been processed. After the entire process is completed, the nth associated image block will contain all PDF images related to the content of the nth search keyword as the output result of the content-associated retrieval.

[0054] Through content registration constraints and image content feature analysis, the content relevance of each PDF image to the search keywords can be accurately and efficiently determined, and the most matching related images can be screened out from the image data, which greatly improves the pertinence and practicality of image retrieval and lays a solid foundation for achieving high-performance content-aware retrieval.

[0055] Furthermore, the embodiment of the present application also includes:

[0056] The predetermined retrieval window constraint is sent to the target user to obtain a selected retrieval window, wherein the predetermined retrieval window constraint includes multiple predetermined retrieval windows; each associated image in the nth associated image block is divided according to the selected retrieval window to obtain multiple window associated images; according to the nth search keyword, deep feature twin recognition is performed on the multiple window associated images to obtain multiple deep feature twin coefficients; whether the multiple deep feature twin coefficients satisfy the deep feature twin constraint is determined to generate a superior deep feature twin distribution; according to the superior deep feature twin distribution, the multiple window associated images are sorted to obtain the nth keyword image retrieval result, and the nth keyword image retrieval result is added to the PDF image retrieval result.

[0057] In a feasible implementation, when performing a deep search on a plurality of associated image blocks, the deep search module 14 adopts a hierarchical search method based on a predetermined search window constraint and a deep feature twin constraint to obtain a final PDF image search result.

[0058] First, the deep retrieval module 14 sends a set of predefined retrieval window constraints to the target user for the user to select and confirm. The predetermined retrieval window constraints are a set of pre-designed image area division schemes, including multiple predetermined retrieval windows, each of which corresponds to different retrieval granularity and focus. For example, the image is divided into different sub-regions according to the image's size, aspect ratio, position and other attributes, or the image is divided into different object regions according to the image's content semantics. The target user can select a most suitable division scheme from multiple predetermined retrieval windows as the selected retrieval window according to his or her retrieval needs and preferences, providing a basis for subsequent deep retrieval. After obtaining the selected retrieval window, the deep retrieval module 14 performs window division on each associated image in the nth associated image block. Specifically, the deep retrieval module 14 divides each associated image into multiple independent sub-images according to the size and position parameters of the selected retrieval window, and marks these sub-images as window-associated images to obtain multiple window-associated images. Each window-associated image corresponds to a local area of ​​the original associated image, retaining the image content and feature information in the area. By dividing the windows, the deep retrieval module 14 can analyze and compare the local features of the image more finely, thereby improving the accuracy and flexibility of the retrieval.

[0059] After obtaining multiple window-associated images, the deep retrieval module 14 will perform deep feature twin recognition on each window-associated image according to the nth search keyword. Deep feature twin recognition is an image feature extraction and comparison method based on deep learning. Through models such as convolutional neural networks, the deep semantic features of the image are learned, and the similarity between these features and the search keywords is calculated. Specifically, the deep retrieval module 14 inputs the window-associated image and the nth search keyword into the pre-trained twin network respectively to obtain their respective deep feature vectors, and then calculates the matching degree between the two vectors through metric functions such as cosine similarity to obtain a quantized deep feature twin coefficient. The higher the deep feature twin coefficient, the more similar the window-associated image is to the nth search keyword at the semantic level, and vice versa, the lower the semantic relevance. After calculating the deep feature twin coefficients of all window-associated images, the deep retrieval module 14 screens and filters these coefficients according to the preset deep feature twin constraints. The deep feature twin constraints are a set of threshold conditions used to define the minimum requirements for deep feature matching. The deep search module 14 determines whether each deep feature twin coefficient satisfies the constraint condition one by one, and retains all coefficients that meet the condition to form a winning deep feature twin distribution. The winning deep feature twin distribution reflects the image area in the window-associated image that best matches the nth search keyword, providing an important basis for the subsequent generation of search results.

[0060] Based on the superior deep feature twin distribution, the deep retrieval module 14 will further screen and sort the multiple window-associated images, and select a group of images that are most relevant to the nth search keyword as the nth keyword image retrieval result. Specifically, the deep retrieval module 14 sets a set of sorting rules and selection strategies based on the statistical characteristics of the superior deep feature twin distribution, such as mean, variance, peak, etc., and evaluates the relevance and importance of each window-associated image in turn. For example, the top k window-associated images with the highest twin coefficients are preferentially selected, or a group of window-associated images with the most concentrated twin coefficient distribution are selected as the nth keyword image retrieval result, and then the deep retrieval module 14 adds it to the PDF image retrieval result, and together with the retrieval results of other search keywords, it constitutes a complete PDF image retrieval output.

[0061] Furthermore, the embodiment of the present application also includes:

[0062] Traverse the multiple window-associated images and extract the first window-associated image; perform multi-feature expectation analysis based on the nth search keyword to determine the nth expected underlying image feature and the nth expected deep semantic feature; perform multi-feature recognition based on the first window-associated image to obtain the first window underlying image feature and the first window deep semantic feature; perform twin recognition based on the nth expected underlying image feature and the first window underlying image feature to obtain the first underlying image feature twin coefficient; perform twin recognition based on the nth expected deep semantic feature and the first window deep semantic feature to obtain the first deep semantic feature twin coefficient; perform weighted calculation on the first underlying image feature twin coefficient and the first deep semantic feature twin coefficient based on the depth feature twin weight constraint to obtain the first depth feature twin coefficient, and add the first depth feature twin coefficient to the multiple depth feature twin coefficients.

[0063] In a preferred embodiment, when performing deep feature twin recognition on multiple window-associated images, the deep retrieval module 14 adopts a multi-level, multi-perspective feature extraction and matching method to obtain more comprehensive and accurate deep feature twin coefficients.

[0064] Specifically, the deep retrieval module 14 first extracts a single image from multiple window-associated images one by one, recorded as the first window-associated image. Then, the deep retrieval module 14 performs a multi-feature expectation analysis on the nth search keyword currently being processed. Multi-feature expectation analysis is a method based on statistical learning and knowledge reasoning. By extracting features and summarizing patterns of a large number of image samples related to the nth search keyword, a set of expected feature vectors reflecting the semantic connotation of the nth search keyword is obtained. These expected feature vectors are divided into two levels: one is the nth expected underlying image feature, which represents the common expression of images related to the nth search keyword in the underlying visual features (such as color, texture, shape, etc.); the other is the nth expected deep semantic feature, which represents the common expression of images related to the nth search keyword in high-level semantic features (such as object category, scene theme, emotional tendency, etc.). The deep retrieval module 14 uses these two levels of expected feature vectors to guide the subsequent underlying feature twin recognition and deep feature twin recognition, respectively.

[0065] After obtaining the expected feature vector of the nth search keyword, the deep search module 14 performs multi-feature recognition on the first window associated image. Multi-feature recognition is an image feature extraction method based on deep learning, which automatically learns and extracts the representation of the image at different feature levels through models such as convolutional neural networks. Specifically, the deep search module 14 inputs the first window associated image into the pre-trained underlying feature extraction network and the deep feature extraction network to obtain the first window underlying image features and the first window deep semantic features, respectively. The first window underlying image features characterize the characteristics of the first window associated image in terms of visual attributes such as color, texture, and shape, while the first window deep semantic features characterize the characteristics of the first window associated image in terms of semantic attributes such as object category, scene theme, and emotional tendency. These two levels of image features will be twinned with the expected feature vector of the nth search keyword to evaluate similarity and matching.

[0066] Subsequently, the deep retrieval module 14 inputs the first window underlying image features and the nth expected underlying image features into the pre-trained underlying twin network, and obtains the first underlying image feature twin coefficient through network forward propagation and similarity measurement, which indicates the degree of matching between the two underlying feature vectors. Similarly, the deep retrieval module 14 inputs the first window deep semantic features and the nth expected deep semantic features into the pre-trained deep twin network, and obtains the first deep semantic feature twin coefficient through network forward propagation and similarity measurement, which indicates the degree of matching between the two deep feature vectors. These two twin coefficients respectively reflect the similarity between the first window associated image and the nth search keyword in the underlying visual features and deep semantic features, and provide an important basis for subsequent comprehensive evaluation. After obtaining the first underlying image feature twin coefficient and the first deep semantic feature twin coefficient, the deep retrieval module 14 performs weighted fusion on these two coefficients according to the preset deep feature twin weight constraint to obtain the comprehensive first deep feature twin coefficient. The deep feature twin weight constraint is a set of weight parameters used to balance the importance of underlying features and deep features in twin matching. Generally, deep semantic features should be given greater weight in twin matching due to their higher level of abstraction and semantic expression ability; while the underlying image features can be used as a supplement and correction to the deep features, and their weights can be appropriately reduced. The deep retrieval module 14 sets reasonable deep feature twin weight constraints according to specific application scenarios and empirical rules, and uses them to weighted sum the first underlying image feature twin coefficient and the first deep semantic feature twin coefficient to obtain the first deep feature twin coefficient. The first deep feature twin coefficient comprehensively considers the feature matching of both the underlying and deep levels, and can more comprehensively and accurately evaluate the correlation between the first window-associated image and the nth search keyword. Afterwards, the deep retrieval module 14 adds the first deep feature twin coefficient to multiple deep feature twin coefficients, and together with the deep feature twin coefficients of other window-associated images, it constitutes a complete twin matching result for the nth search keyword.

[0067] In summary, the content-aware PDF image fast retrieval system provided by the embodiments of the present application has the following technical effects:

[0068] The retrieval task receiving module is used to receive the PDF image retrieval task of the target user, wherein the PDF image retrieval task includes a PDF file and a retrieval target word. The module is responsible for receiving the PDF file and the retrieval target word input by the user and providing basic data for the retrieval task. The retrieval expansion module is used to verify and expand the retrieval target word according to the retrieval word verifier and the retrieval expansion multi-channel, generate a retrieval keyword list, expand the keyword list through retrieval word verification and multi-channel expansion, and improve the retrieval coverage. The content-aware retrieval module is used to perform content-related retrieval on the PDF file based on the content registration constraint and the retrieval keyword list to obtain multiple associated image blocks, and perform content-related retrieval on the PDF file based on the content registration constraint to obtain related image blocks. The deep retrieval module is used to perform deep retrieval on multiple associated image blocks based on the predetermined retrieval window constraint and the deep feature twin constraint and the retrieval keyword list to obtain the PDF image retrieval result, and use the retrieval window constraint and the deep feature twin constraint for accurate matching to provide high-quality retrieval results. The retrieval feedback module is used to feed back the PDF image retrieval result to the target user, feed back the final retrieval result to the user, complete the retrieval process, and improve the accuracy and efficiency of PDF image retrieval.

[0069] Any steps of the system described above may be stored as computer instructions or programs in an unlimited computer memory, and may be called and recognized by an unlimited computer processor to implement any system in the embodiments of the present application, without any unnecessary restrictions.

[0070] Furthermore, the first or second mentioned above may not only represent an order relationship, but may also represent a specific concept, and / or refer to multiple elements that can be selected individually or in full. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these changes and variations.

Claims

1. A content-aware PDF image fast retrieval system, characterized by: The system comprises: A retrieval task receiving module, the retrieval task receiving module is used to receive a PDF image retrieval task of a target user, wherein the PDF image retrieval task includes a PDF file and a retrieval target word; A search expansion module, the search expansion module is used to verify and expand the search target word according to the search word verifier and the search expansion multi-channel to generate a search keyword list; A content-aware retrieval module, the content-aware retrieval module is used to perform content-related retrieval on the PDF file based on content registration constraints and in combination with the retrieval keyword list to obtain a plurality of related image blocks; A deep search module, the deep search module is used to perform a deep search on the multiple associated image blocks according to a predetermined search window constraint and a deep feature twin constraint in combination with the search keyword list to obtain a PDF image search result; A retrieval feedback module is used to feed back the PDF image retrieval result to the target user.

2. The content-aware PDF image fast retrieval system according to claim 1, characterized in that: The search expansion module is used to verify and expand the search target word according to the search word verifier and the search expansion multi-channel to generate a search keyword list, including: Performing a searchability check on the search target word according to the search word checker to obtain a target word check result; When the target word verification result is passed, the retrieval expansion multi-channel is activated, wherein the retrieval expansion multi-channel includes a retrieval word splitting channel, a retrieval synonym expansion channel and a retrieval near synonym expansion channel; Inputting the search target word into the search word splitting channel to obtain a plurality of search split words; Based on the multiple search split words, and according to the search synonym expansion channel, multiple groups of synonym expansion results are obtained; Based on the multiple search split words, and according to the search synonym expansion channel, multiple groups of synonym expansion results are obtained; The search keyword list is constructed by combining the multiple groups of synonym expansion results and the multiple groups of near-synonym expansion results in combination with the search target words.

3. The content-aware PDF image fast retrieval system as claimed in claim 2, characterized in that: Performing a searchability check on the search target word according to the search word checker to obtain a target word check result includes: Performing a searchability evaluation based on the search target word to obtain the searchability of the target word; The target word searchability is input into the search word verifier to generate the target word verification result, wherein the search word verifier includes a search word verification rule, and the search word verification rule includes that when the target word searchability is greater than / equal to a preset searchability, the target word verification result output is passed, and when the target word searchability is less than the preset searchability, the target word verification result output is failed.

4. The content-aware PDF image fast retrieval system as claimed in claim 2, characterized in that: When the target word verification result is failed, generating a target word warning signal; Sending the target word warning signal to the target user to obtain user feedback data; The retrieval target word is optimized according to the user feedback data.

5. The content-aware PDF image fast retrieval system according to claim 1, characterized in that: The content-aware retrieval module is used to perform content-related retrieval on the PDF file based on the content registration constraint and in combination with the retrieval keyword list to obtain a plurality of related image blocks, including: Perform multiple authentications according to the PDF file to obtain a PDF authentication result; When the PDF authentication result is passed, splitting the PDF file to obtain multiple PDF images; Extracting the nth search keyword according to the search keyword list, where n is a positive integer; Based on the content registration constraint, content association retrieval is performed on the multiple PDF images according to the nth search keyword, an nth associated image block is established, and the nth associated image block is added to the multiple associated image blocks.

6. The content-aware PDF image fast retrieval system according to claim 5, characterized in that: Based on the content registration constraint, performing content association retrieval on the plurality of PDF images according to the nth search keyword to establish an nth associated image block includes: Traversing the multiple PDF images, extracting a first PDF image; Performing content perception according to the first PDF image to obtain content feature information of the first image; Performing an association evaluation on the first image content feature information according to the nth search keyword to obtain a first image content registration coefficient; Determining whether the first image content registration coefficient satisfies the content registration constraint; If the first image content registration coefficient satisfies the content registration constraint, the first PDF image is set as a first associated image, and the first associated image is added to the nth associated image block.

7. The content-aware PDF image fast retrieval system according to claim 5, characterized in that: The deep search module is used to perform a deep search on the multiple associated image blocks according to the predetermined search window constraint and the deep feature twin constraint in combination with the search keyword list to obtain a PDF image search result, including: sending the predetermined search window constraint to the target user to obtain a selected search window, wherein the predetermined search window constraint includes a plurality of predetermined search windows; Dividing each associated image in the nth associated image block according to the selected retrieval window to obtain a plurality of window associated images; According to the nth search keyword, deep feature twin recognition is performed on the multiple window-associated images to obtain multiple deep feature twin coefficients; Determine whether the multiple deep feature twin coefficients satisfy the deep feature twin constraints, and generate a winning deep feature twin distribution; According to the superior deep feature twin distribution, the multiple window-associated images are sorted out to obtain an nth keyword image retrieval result, and the nth keyword image retrieval result is added to the PDF image retrieval result.

8. The content-aware PDF image fast retrieval system according to claim 7, characterized in that: According to the nth search keyword, deep feature twin recognition is performed on the multiple window-associated images to obtain multiple deep feature twin coefficients, including: Traversing the plurality of window-associated images, and extracting a first window-associated image; Performing a multi-feature expectation analysis based on the nth search keyword to determine the nth expected underlying image feature and the nth expected deep semantic feature; Perform multi-feature recognition based on the first window associated image to obtain the first window bottom-layer image features and the first window deep-layer semantic features; Performing twin recognition according to the nth expected underlying image feature and the first window underlying image feature to obtain a first underlying image feature twin coefficient; Performing twin recognition according to the nth expected deep semantic feature and the first window deep semantic feature to obtain a first deep semantic feature twin coefficient; According to the deep feature twin weight constraint, the first underlying image feature twin coefficient and the first deep semantic feature twin coefficient are weightedly calculated to obtain the first deep feature twin coefficient, and the first deep feature twin coefficient is added to the multiple deep feature twin coefficients.

Images