[0042] See figure 1 , The present invention is a panoramic image retrieval and display method, which is characterized in that it mainly includes an index building step A and an image query step B. The index building step A extracts the characteristics of image files in the gallery to generate an index file; user C queries the image Step B uploads the image to be retrieved, the image query step B extracts the feature of the image to be retrieved, the image uploaded by the user C is matched with the feature of the index file in the indexing step A, and the matching result list is provided to the user C.
[0043] The operating system of the present invention (the system mentioned below is the present invention and operating system) is mainly a server and a user terminal. The gallery is set in the server, and the user terminal includes a smart phone and various PCs. The present invention mainly includes three major functions: input function, search function, and query function. The input function is mainly to extract features from the panoramic image resource library, establish an index, and then save the index file. The search function is mainly to search and filter index files. The final query function is mainly to retrieve panoramic images similar to the pictures uploaded by the user based on the user's feedback. The process of the invention is as follows: first, the system sequentially extracts the features of the panoramic images in the gallery, generates an index file, and saves the index file in the server for retrieval. Then the user logs in to the server through the user terminal to upload the image, extracts the characteristics of the image submitted by the user and performs similarity matching with the index file, and returns the image to the user in the order of the similarity value (such as figure 1 Shown).
[0044] The core process of the present invention will be described in detail below, mainly including image feature extraction, index creation and image query.
[0045] 1. Image retrieval process:
[0046] (1) Image feature extraction:
[0047] The core of the present invention includes two major functions, index creation and query. The purpose of index creation is to extract image features from a large number of image resources and build an index library for storage, eliminating the cumbersome operation of traversing the library to extract features each time during retrieval, greatly improving retrieval efficiency. The query operation also needs to extract image feature analysis and compare with the image resource information in the index library, and finally obtain the result list.
[0048] From this point of view, in the entire image retrieval process, image feature extraction has played a very important role. The first step of the present invention-image feature extraction will be described below. When it comes to images, the most intuitive reflection to us is the color, which means that color is the most intuitive visual feature. Technically, it is the most direct and simplest conventional method to express the color information of an image through a color histogram. However, an image is retrieved only by color information, and the accuracy rate is very low. The image feature information must also be added to the factors to be queried during retrieval.
[0049] Therefore, in the process of feature extraction, the texture information of the image is also one of the factors that must be considered. The texture feature histogram can divide the texture information of the image into multiple categories, and at the same time can combine the color information well. Color and Edge Directivity Descriptor (CEDD) is a description method that combines image color and texture. So in the system development, we used the image feature extraction method based on Lucene Image Retrieval (Lucene Image Retrieval, LIRE).
[0050] The object of feature extraction is the panoramic image uploaded by system maintenance personnel. The specific steps are:
[0051] Step1: Obtain panoramic image information uploaded by system maintenance personnel.
[0052] Step2: Traverse image resources and cut each picture into 40*40=1600 small picture blocks.
[0053] Step3: For each image block, use the CEDD tool to convert the RGB model to the HSV model (this is a conventional technology, see the document "Tang Guangyan. The realization of the conversion between RGB color model and HSV color model in VB. Science and Technology Information, 2009(2) :457-458").
[0054] Step4: Through 10-bins fuzzy filtering, output the obtained HSV information to a 10-dimensional histogram.
[0055] Step5: Input the 10-dimensional histogram and S (saturation) and V (luminance) components in Step4 into the 24-bins blur filter to obtain a 24-dimensional histogram of the small block.
[0056] Step6: Convert each small image block from RGB model to YIQ model.
[0057] Step7: Then divide each small square into four equal parts, corresponding to the digital filter structure used in the texture information extraction, and calculate the gray value of each block and normalize it.
[0058] Step8: Plan the obtained texture information of each piece into a 6-dimensional edge histogram.
[0059] Step9: Combine the two part histograms obtained in step5 and step8 to obtain a 144-dimensional histogram to complete the image extraction work. The specific methods of the above steps are conventional techniques (see the literature "Fu Zhicheng, Li Xiaoqiang, Li Fufeng. Tongue image segmentation based on radial edge detection and Snake model. Chinese Journal of Image and Graphics, 2009, 14(4):688-693; Basil G. Mertzios, Konstantinos D. Tsirikolias. Coordinate logic filters: theory and applications in image analysis. Nonlinear image processing.2001:331-354; Wei Zhen, Zhu Minchen. Adaptive skin color detection algorithm based on YIQ color space. Journal of Fuzhou University (Natural Science Edition), 2008, 36(3):336-340; Guan Qianning, Qin Tuanfa, Shuai Qin, etc. Image retrieval method integrating texture and color features in MPEG-7. Application Research of Computers, 2008, 25(3 ):957-960.").
[0060] The core code for generating feature information is given below:
[0061] Fuzzy10BinResultTable = Fuzzy10.ApplyFilter(HSV[0], HSV[1], HSV[2], 2);
[0062] Fuzzy24BinResultTable = Fuzzy24.ApplyFilter(HSV[0], HSV[1], HSV[2],Fuzzy10BinResultTable, 2);
[0063] for (inti = 0; i
[0064] for (intj = 0; j <24; j++) {
[0065] if (Fuzzy24BinResultTable[j]> 0) CEDD[24 *Edges[i] + j] +=Fuzzy24BinResultTable[j];
[0066] }
[0067] }
[0068] (2) Create an index:
[0069] In section (1) above, we get the color and texture characteristics of the image. In order to reduce the amount of calculation during query and improve query efficiency, an inverted index is constructed for characteristic information. The index creation process is as follows figure 2 Shown.
[0070] The principle is that in the index library, the feature information of each picture corresponds to a unique ID number. When the index catalog is generated, the corresponding relationship between the feature value and its appearance in a record is saved. For example, the feature Feature1 is saved as Feature1-> in the image feature information with IDs 1 and 3 1,3. Inverted index schematic diagram such as image 3 Shown.
[0071] The detailed steps for creating an index are given below:
[0072] Step1: Create a DocumentBuilder (CEDD.class) object, and integrate the image feature information into the builder.
[0073] Step2: Use the document builder to add image feature information to the constructed document Document.
[0074] Step3: Define the index document write object IndexWrite.
[0075] Step4: Finally, write the index document Document to the server according to the configured address by writing the object.
[0076] The core code for creating an index document is as follows:
[0077] for (Iterator it = images.iterator(); it.hasNext();) {
[0078] String imageFilePath = it.next();
[0079] BufferedImage img = ImageIO. read (new FileInputStream(imageFilePath));
[0080] Document document = globalDocumentBuilder.createDocument(img,imageFilePath);
[0081] iw.addDocument(document);
[0082] }
[0083] The created index directory structure is as follows Figure 4 Shown.
[0084] (3) Image query:
[0085] Through the work of (1) and (2), we have obtained the features of the image and created the index file. The process of panoramic image query will be introduced below. First, give the specific flowchart of image query, such as Figure 5 Shown.
[0086] The search module function is mainly implemented by the Searcher class. First, the GenericFastImageSearcher class is called, the number of returned results is set, and the method for extracting feature values is added. code show as below:
[0087] ImageSearcher searcher = new GenericFastImageSearcher(30, CEDD.class);
[0088] Secondly, call the search(img, ir) method of the ImageSearchHits class, where img is the path of the image to be searched, and ir is the index file information read. In the search method, by comparing the distance between the queried image and the feature value in the index (maxDistance=findSimilar(reader, globalFeature)), the result that meets a specific threshold is returned.
[0089] The detailed steps are given below:
[0090] Step1: Get the image uploaded by the user.
[0091] Step2: Use extract() to extract the feature value of the input image.
[0092] Step3: According to the extracted feature value, use findSimilar() to find similar pictures.
[0093] Step4: Create a new ImageSearchHits to store the search results.
[0094] Step5: Traverse ImageSearchHits to obtain the score, path, name and other information of the corresponding or similar panorama respectively.
[0095] 2. Panoramic roaming display:
[0096] Through the panoramic retrieval process in section 1 above, we will get a list of panoramic images similar to the user's query, with the purpose of displaying 360° in all directions. After getting the list, you first need to change the xml file that configures the panorama to make the path information in the file point to the result of this query. Then the panoramic engine loads the file to realize the panoramic display of the query results. The process of xml file modification is as Image 6 Shown.
[0097] Next, we will do a panoramic roaming operation on the panoramic image. The Krpano method has been given above. The main implementation of this method is to load the Krpano html5 panorama engine through the html file, read the panorama image, and display it to the browser. The implementation process is as follows Figure 7 Shown.
[0098] The specific steps of Krpano framework to achieve panoramic roaming are given below:
[0099] Step1: The browser loads the html file, which is used for the entry configuration of panoramic roaming, including setting the panoramic roaming xml file.
[0100] Step2: Load the Krpano html5 panorama engine on the html page.
[0101] Step3: The panorama engine loads the main xml file corresponding to the panorama, which configures the result list obtained by the image retrieval process, and reads the panorama resource.
[0102] Step4: Add the read panoramic resource to the container to display it in the browser.
[0103] Through the above 4 steps, a panoramic display can be realized. At the end of the panorama search module, the panorama clicked by the user will be displayed, and the rest of the panorama will be displayed in the form of thumbnails at the bottom of the browser. The user can click on the thumbnails to switch arbitrarily, which is convenient for users to view.
[0104] In summary, the detailed process from panoramic image retrieval to panoramic display is realized, which can well solve the problems of poor interactivity, low efficiency, and inconvenience in use in the prior art.
[0105] The functions of panoramic retrieval and display of the present invention are described as follows:
[0106] First, click the upload panoramic image button under panoramic image management on the background homepage to jump to the upload page.
[0107] After uploading all the panoramic images, the administrator clicks the Create Index button under the panoramic image management on the back-end homepage to execute the index creation work. The index file generated by index creation is stored on the server, and the file structure of the index has been given above.
[0108] After completing the above work, visit the system homepage in the browser. The browsers supported by the system include Chrome32 and above, Firefox, Cheetah browser and other browsers with Chrome as the core.
[0109] The system implements the image retrieval function. On the system home page, the user clicks the picture retrieval button to jump to the user upload search picture interface.
[0110] The present invention uses Image Retrieval (CBIR, Content Based ImageRetrieval) based on image content. The method extracts image features, builds an index database, and finally completes image retrieval by matching features. This method not only greatly improves the accuracy of retrieval Rate and efficiency, and the process is all completed automatically, without manual intervention.
[0111] The specific implementation method of each step in the present invention is conventional technology.
