A method for querying archival data based on image retrieval

Abstract

The present invention discloses a method for querying archival data based on picture retrieval, which calculates the similarity measure between the sketch features and the data in the archival image feature database; displays the archival images with a similarity higher than the first threshold in the interface according to the sorting; obtains the associated information of the archival image based on the second operation of the user; this method enables users to retrieve information in a convenient way, diversifies the information retrieval methods, and improves the accuracy and speed of sketch retrieval at the same time.

Description

Technical Field

[0001] The present invention belongs to the field of data query, and in particular relates to an archive data query method based on image retrieval. Background Art

[0002] The existing image retrieval technologies mainly include the following three methods: keyword-based image retrieval, content-based image retrieval and sketch-based image retrieval.

[0003] When using keyword-based image retrieval, users only need to enter keywords to search for images that meet the conditions. This method is based on understanding the image, adding manually annotated keywords to all images in the image library, and then storing them in a traditional database and establishing corresponding indexes for retrieval. This method has the advantages of simplicity and fast retrieval speed, but its disadvantages are also obvious. Pure keyword-based image search requires a lot of manpower for manual annotation, and in some scenarios, it is difficult for users to describe the image in words. In response to these situations, content-based image retrieval technology has been significantly improved. Its basic principle is to extract image features, including underlying visual information such as color and texture, as well as some higher-level semantic information, and then use a set of vectors to represent the image. These vectors are usually stored in a database, and then a corresponding index structure is established for retrieval. Content-based image retrieval is more in line with the user's visual cognition than keyword-based image retrieval. In some scenarios, it is difficult to accurately describe the characteristics of the target object with text. Sketch retrieval came into being; with the increasing popularity of touch-screen devices such as mobile phones, tablets, and notebooks, inputting hand-drawn information can be completed at the fingertips, making the collection of sketch information quick and easy, bringing broad development prospects for sketch retrieval.

[0004] However, sketch retrieval deals with the inherent ambiguity of sketches compared to natural images. The main reasons for the ambiguity are the following three points: sketches are often abstract descriptions of original objects, which are statistically different from original images; when people draw sketches, they often cannot refer to real images and objects, which leads to great differences in structure and appearance; due to painting skills and personal influences, sketches often have great intra-class differences, and the drawing effects often vary from person to person. It is precisely for these reasons that sketch retrieval is more difficult than content-based image retrieval. Archival data is usually stored in a tree structure. How to implement the application of hand-drawn patterns in archival data and improve the efficiency of data retrieval has become one of the technical problems that need to be solved in this field. Summary of the invention

[0005] In view of the current defects of slow data extraction efficiency and poor robustness in existing hand-drawn patterns, the present invention provides a method for querying archival data based on image retrieval. Through sketch features and other descriptive features, similarity measurement calculations are performed on the sketch features and the data in the archival image feature database, enabling sketch retrieval to be applied in archival data, allowing users to retrieve information in a convenient manner, diversifying information retrieval methods, and simultaneously improving the accuracy and speed of sketch retrieval.

[0006] The technical solution adopted by the present invention to solve the above technical problems is: A method for querying archival data based on image retrieval, including the following steps:

[0007] S1. Obtain a sketch according to the user's first operation.

[0008] S2. After performing a first preprocessing on the sketch, perform first feature extraction to obtain sketch features.

[0009] S3. After performing a second preprocessing on the images in the original archival database, perform second feature extraction to obtain an archival image feature set, and construct an archival image feature database according to the archival image feature set.

[0010] S4. Perform similarity measurement calculations on the sketch features and the data in the archival image feature database.

[0011] S5. Display the archival images with a similarity higher than the first threshold in the interface according to the ranking.

[0012] S6. Obtain the associated information of the archival image based on the user's second operation.

[0013] Further, the first operation includes: the user draws according to a graphics tablet or the user selects from a device.

[0014] Further, in the step of performing first feature extraction to obtain sketch features after performing a first preprocessing on the sketch, the preprocessing includes eliminating redundant strokes, clustering points, and closing and filling to reduce sketch noise; the first feature extraction includes edge histogram and shape tracking.

[0015] Further, the shape tracking includes: edge detection and contour tracking; Canny operator is used for edge detection, and 4-connected tracking is used for contour tracking.

[0016] Further, in the step of performing second feature extraction to obtain an archival image feature set after performing a second preprocessing on the images in the original archival database, the second preprocessing includes Gaussian filtering; the second feature extraction includes text features and visual features, and the visual features include color features, texture features, shape features, and contour features.

[0017] Further, in the calculation of the similarity measure between the sketch feature and the data in the archive image feature database, specifically, the similarity measure between the sketch feature and the contour feature of the archive image feature is calculated first to obtain the first similarity.

[0018] Further, between S4 and S5, the user can also input a description message, and calculate the second similarity by performing a similarity measure calculation on the description message and one or more of the text feature, color feature, texture feature, and contour feature of the archive image feature.

[0019] Further, a target image is obtained according to the first similarity and the second similarity.

[0020] Further, in the obtaining of the associated information of the archive image based on the second operation of the user, the second operation is a click, touch, gesture input, or voice input.

[0021] Further, the similarity measure calculation includes calculating the Euclidean distance and the cosine distance.

[0022] The beneficial effects of the present invention are as follows:

[0023] The shape feature of the image can be quickly obtained through edge detection and contour tracking. By calculating the similarity measure between the sketch feature and the data in the archive image feature database, sketch retrieval can be applied to archive data. At the same time, through the combined use of the sketch feature and other description features, the user can retrieve information in a convenient way, making the information retrieval method diversified, and improving the accuracy and speed of sketch retrieval.

[0024] The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above description and other purposes, features, and advantages of the present invention more obvious and understandable, preferred embodiments are specifically given and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0026] Figure 1 is a flowchart of an archive data query method based on picture retrieval DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0028] In the description of the present invention, unless otherwise clearly defined and limited, terms such as "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, it can be a connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] Embodiment 1

[0030] A method for querying archival data based on image retrieval includes the following steps:

[0031] S1. Obtain a sketch according to the user's first operation.

[0032] S2. After performing a first preprocessing on the sketch, perform a first feature extraction to obtain sketch features.

[0033] S3. After performing a second preprocessing on the images in the original archival database, perform a second feature extraction to obtain a set of archival image features, and construct an archival image feature database according to the set of archival image features.

[0034] S4. Perform a similarity measurement calculation on the sketch features and the data in the archival image feature database.

[0035] S5. Display the archival images with a similarity higher than the first threshold in the interface according to the sorting.

[0036] S6. Obtain the associated information of the archival image based on the user's second operation.

[0037] Further, the first operation includes: the user draws according to a graphics tablet or the user selects from a device.

[0038] Further, in the step of performing a first preprocessing on the sketch and then performing a first feature extraction to obtain sketch features, the preprocessing includes grayscale conversion, binarization, elimination of redundant strokes, clustering points, and closed filling to reduce sketch noise; the first feature extraction includes edge histogram.

[0039] Further, the shape tracking includes: edge detection and contour tracking; the Canny operator is used for edge detection, and 4-connected tracking or 8-connected tracking is used for contour tracking.

[0040] Further, in the process of performing second feature extraction on the images in the original file database after second preprocessing to obtain the file image feature set, the second preprocessing includes Gaussian filtering; the second feature extraction includes text features and visual features, and the visual features include color features, texture features, shape features, and contour features.

[0041] Further, in the process of performing similarity measurement calculation on the sketch features and the data in the file image feature database, specifically, first, similarity measurement calculation is performed on the sketch features and the contour features of the file image features to obtain the first similarity.

[0042] Further, between S4 and S5, the user can also input descriptive information, and perform similarity measurement calculation on the descriptive information and one or more of the text features, color features, texture features, and contour features of the file image features to obtain the second similarity.

[0043] Further, a target image is obtained according to the first similarity and the second similarity.

[0044] Further, in the process of obtaining the association information of the file image based on the second operation of the user, the second operation is click touch or gesture input or voice input.

[0045] Further, the similarity measurement calculation includes calculating the Euclidean distance and the cosine distance.

[0046] Further, the steps of using the Canny operator to extract shape features are as follows: use a Gaussian filter to smooth and filter the shape to eliminate noise; for each pixel, calculate the gradient magnitude and direction using the first-order partial derivative finite difference in the horizontal and vertical directions; perform non-maximum suppression on the gradient magnitude; use a double-threshold algorithm to detect and connect edges.

[0047] Further, the steps of the shape tracking algorithm are as follows: S101. Search in the image in the order from top to bottom and from left to right, and take the first point with a pixel value of 1 found as the starting point, denoted as A; S102. Find the boundary point among the adjacent points in the 8-neighborhood direction in the counterclockwise order. If there is a pixel value of 1 and there is a pixel value of 0 in the 4-neighborhood direction of this point, then this point is marked as the boundary point B; S103. If B is point A, it means that a full circle has been searched and the program ends; S104. Otherwise, continue to search from point B until point A is found.

[0048] Further, the edge histogram includes the mean value:

[0049]

[0050] Among them, L is the number of gray levels of the image, N(z i ) represents the number of pixels with gray value z i , and M is the total number of pixels of the image.

[0051] Furthermore, the edge mean histogram can be combined with the pyramid gradient direction histogram to obtain the feature vector K; the method for obtaining the pyramid gradient direction histogram is: (1) segment the sketch image by pyramid, with 2 or 3 levels; (2) extract the edge features of the segmented image; (3) calculate the HOG features of each level of the pyramid sub-sketch image; (4) concatenate the gradient direction histograms of each level of the image pyramid to form the PHOG feature vector; the total length of the feature vector can be expressed as: where L is the level and K is the length of the HOG vector; the expression of the combined feature vector is (m, c).

[0052] Construct the combined vector (m', c') of the archival image in the archival database in the same way. Calculate the similarity Sim1 between the two.

[0053] Embodiment 2

[0054] Based on S1-S4 of Embodiment 1, after obtaining the retrieval result of the sketch, in order to further improve the retrieval accuracy, the user can also input descriptive information, and perform similarity measurement calculation on the descriptive information and one or more of the text features, color features, texture features, and contour features of the archival image features to obtain the second similarity Sim2; obtain the comprehensive similarity SimK according to the first similarity Sim1 and the second similarity Sim2: SimK = w 1 Sim1 + w 2 Sim2, where w 1 + w 2 = 1, w 1 , w 2 represents the weight and can be set according to actual needs.

[0055] Furthermore, the similarity calculation can adopt the Euclidean distance: x = (m, c), y = (m', c').

[0056] The advantages of the present invention are:

[0057] The shape features of an image can be quickly obtained through edge detection and contour tracking. By performing similarity measurement calculations on the sketch features and the data in the archive image feature database, sketch retrieval can be applied to archive data. At the same time, through the combined use of sketch features and other descriptive features, users can retrieve information in a convenient manner, diversifying the information retrieval methods and improving the accuracy and speed of sketch retrieval.

[0058] As mentioned above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for querying archival data based on image retrieval, characterized in that, S1. Obtain a sketch according to the user's first operation, S2. After performing a first preprocessing on the sketch, perform a first feature extraction to obtain sketch features. In the process of performing a first preprocessing on the sketch and then performing a first feature extraction to obtain sketch features, the preprocessing includes eliminating redundant strokes, clustering points, and closed filling to reduce sketch noise; the first feature extraction includes edge histogram and shape tracking; S3. After performing a second preprocessing on the images in the original archival database, perform a second feature extraction to obtain an archival image feature set, and construct an archival image feature database according to the archival image feature set; S4. Perform a similarity measurement calculation on the sketch features and the data in the archival image feature database; First, perform a similarity measurement calculation on the sketch features and the contour features of the archival image features to obtain a first similarity; The user can also input description information, and perform a similarity measurement calculation on the description information and one or more of the text features, color features, texture features, and contour features of the archival image features to obtain a second similarity Sim2; obtain a target image according to the first similarity and the second similarity; S5. Display the archival images with a similarity higher than the first threshold in the interface according to the sorting; S6. Obtain the associated information of the archival image based on the user's second operation; Among them, the edge histogram includes the mean value: Among them, L is the number of gray levels of the image, N(z i ) represents the number of pixels with gray value z i , and M is the total number of pixels in the image; Among them, the edge mean histogram and the pyramid gradient orientation histogram are combined to obtain the feature vector K; the method for obtaining the pyramid gradient orientation histogram is as follows: (1) The sketch image is segmented by a pyramid, and the number of levels is 2 or 3; (2) The edge features of the segmented image are extracted; (3) The HOG features of each level of the pyramid sub-sketch image are calculated; (4) The gradient orientation histograms of each level of the image pyramid are concatenated to form the PHOG feature vector; the total length of the feature vector can be expressed as: where L is the number of levels and K is the length of the HOG vector; the expression of the combined feature vector is (m, c); Construct the joint vector (m', c') of the archival images in the archival database in the same way; calculate the similarity Sim1 between the two; The obtaining the target image according to the first similarity and the second similarity includes: Obtain the comprehensive similarity SimK based on the first similarity Sim1 and the second similarity Sim2: SimK = w 1 Sim1 + w 2 Sim2, where w 1 + w 2 = 1, w 1 , w 2 represents the weight.

2. The method for querying archival data based on image retrieval according to claim 1, characterized in that: The first operation includes: the user draws according to a graphics tablet or the user selects from a device.

3. The method for querying archival data based on image retrieval according to claim 1, characterized in that: The shape tracking includes: edge detection and contour tracking; Canny operator is used for edge detection, and 4-connected tracking is used for contour tracking.

4. The method for querying archival data based on image retrieval according to claim 1, characterized in that: In the process of performing a second preprocessing on the images in the original archival database and then performing a second feature extraction to obtain an archival image feature set, the second preprocessing includes Gaussian filtering; the second feature extraction includes text features and visual features, and the visual features include color features, texture features, shape features, and contour features.

5. The method for querying archival data based on image retrieval according to claim 1, characterized in that: In the obtaining the associated information of the archival image based on the user's second operation, the second operation is click touch or gesture input or voice input.

6. The method for querying archival data based on image retrieval according to claim 1, characterized in that: The similarity measurement calculation includes calculating Euclidean distance and cosine distance.

Images