Image retrieval method based on SpringBoot and related device

Abstract

The invention provides an image retrieval method based on SpringBoot and a related device, and the method comprises the steps: reading picture data under a specified resource directory or in a database, and loading the picture data into a memory; calculating the picture data loaded into the memory through a color moment algorithm and an ORB algorithm to obtain a feature vector of the image; clustering the feature vectors of all the picture data in the memory through an ISODATA algorithm, and storing clustering result data into an Elasticsearch database so as to establish an index of an image feature clustering result; and receiving a picture or text query request of a user, and obtaining the metadata information of the corresponding picture from the Elasticsearch database according to the ID list of the picture queried by the user, so as to retrieve and obtain a target image. According to the method provided by the invention, quick retrieval and matching of similar pictures are realized by using image feature extraction and image clustering modes, so that the development efficiency and the user experience are improved.

Description

Technical Field

[0001] The present application belongs to the field of image retrieval technology, and in particular, relates to an image retrieval method and related devices based on SpringBoot. Background Art

[0002] Image retrieval refers to the technology of retrieving images that match user needs from a massive image database based on user needs. Image retrieval methods are mainly divided into two categories: content-based image retrieval (CBIR) and text-based image retrieval (TBIR). Among them, content-based image retrieval refers to matching and sorting images based on image content features, such as color, texture, shape, etc.; text-based image retrieval refers to matching and sorting images based on image text descriptions, such as titles, tags, comments, etc.

[0003] The advantage of content-based image retrieval is that it can directly reflect the visual information of the image, does not rely on manual text annotation, and is suitable for image databases with no annotations or inaccurate annotations. The disadvantage of content-based image retrieval is that it is difficult to capture the semantic information of the image, and there is a semantic gap problem, that is, the difference between user needs and the content characteristics of the image. The advantage of text-based image retrieval is that it can utilize the semantic information of the image, which is more in line with the user's natural language query and is suitable for image databases with annotations or accurate annotations. The disadvantage of text-based image retrieval is that it relies on manual text annotations, has problems with insufficient or inconsistent annotations, and is difficult to handle polysemous or synonymous words.

[0004] In order to overcome the shortcomings of the above two methods, some researchers have proposed image retrieval methods based on deep learning, which use deep neural networks to extract high-level features of images and achieve the fusion of image content and semantics. However, image retrieval methods based on deep learning have obvious advantages and disadvantages. The advantage is that they can automatically learn the features of images and improve the accuracy and efficiency of image retrieval; the disadvantage is that they require a large amount of annotated data and computing resources, and it is difficult to explain the principles and processes of image retrieval. Summary of the invention

[0005] In view of this, the present application aims to propose an image retrieval method and related devices based on SpringBoot to solve the problem that the image retrieval method based on deep learning requires a large amount of annotated data and computing resources, and it is difficult to explain the principles and processes of image retrieval.

[0006] To achieve the above purpose, the technical solution of this application is implemented as follows:

[0007] In a first aspect, the present application provides an image retrieval method based on SpringBoot, comprising:

[0008] Read the image data in the specified resource directory or database and load it into memory;

[0009] The image data loaded into the memory is calculated using the color moment algorithm and the ORB algorithm to obtain the feature vector of the image;

[0010] The feature vectors of all image data in memory are clustered using the ISODATA algorithm, and the clustering result data is stored in the Elasticsearch database to establish an index for the image feature clustering results;

[0011] Receive the user's image or text query request, and obtain the metadata information of the corresponding image from the Elasticsearch database according to the ID list of the image obtained by the user's query to retrieve the target image.

[0012] In the second aspect, based on the same inventive concept, the present application also provides an image retrieval device based on SpringBoot, comprising:

[0013] The initialization module is configured to read the image data in the specified resource directory or database and load it into the memory;

[0014] The feature extraction module is configured to calculate the feature vector of the image by using the color moment algorithm and the ORB algorithm to obtain the image data loaded into the memory;

[0015] The clustering module is configured to cluster the feature vectors of all the image data in the memory by using the ISODATA algorithm, and store the clustering result data in the Elasticsearch database to establish an index of the image feature clustering results;

[0016] The retrieval module is configured to receive a user's image or text query request, and obtain metadata information of the corresponding image from the Elasticsearch database according to the ID list of the image obtained by the user's query, so as to retrieve the target image.

[0017] In a third aspect, based on the same inventive concept, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0018] In a fourth aspect, based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0019] Compared with the prior art, the SpringBoot-based image retrieval method and related device described in this application have the following beneficial effects:

[0020] The SpringBoot-based image retrieval method and related devices described in this application are based on the characteristics of SpringBoot, and can quickly build and run a content-based image retrieval system without complicated configuration and deployment. Upon receiving the user's image data query request, image feature extraction and image clustering are used to achieve rapid retrieval and matching of similar images, thereby improving development efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 This is a flow chart of the image retrieval method based on SpringBoot described in an embodiment of the present application;

[0023] Figure 2 This is a specific implementation flow chart of the SpringBoot-based image retrieval method described in the embodiment of this application;

[0024] Figure 3 This is a cluster hot update flow chart described in an embodiment of the present application;

[0025] Figure 4 This is a flowchart of the picture retrieval described in the embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of the structure of the image retrieval device based on SpringBoot described in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the hardware structure of the electronic device described in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0029] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0030] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0031] SpringBoot is a Java application development tool based on the Spring framework, which provides a fast, easy and flexible way to create and run Web-based applications. The advantage of SpringBoot is that it can automatically configure Spring and third-party libraries, and provides a wealth of functional modules, such as security, caching, monitoring, etc. At present, there are relatively few studies and applications on the design and implementation of image retrieval methods based on SpringBoot, and mature solutions and standards have not yet been formed. Therefore, this application aims to provide a design and implementation of an image retrieval method based on SpringBoot, taking advantage of SpringBoot, combined with image algorithms and clustering technologies, to build an efficient, accurate, and easy-to-use image retrieval system.

[0032] See also Figure 1 and Figure 2 As shown, this embodiment provides an image retrieval method based on SpringBoot, which specifically includes the following steps:

[0033] Step S101, read the picture data in the specified resource directory or database, and load it into the memory.

[0034] Specifically, in this embodiment, the main process of this step is the initialization loading and preprocessing process. The initialization program uses a multi-threaded approach to read the image data in the specified resource directory or database and loads it into the memory. The module needs to support loading multiple types of image formats and can preprocess the image to improve the image quality and consistency.

[0035] Among them, multi-threading is used to increase the image loading speed and reduce the waiting time of serial loading. Batch processing is used to improve concurrency capabilities so that other concurrent processing operations can be performed while loading images, thereby improving overall processing efficiency.

[0036] In some implementations, the application is started and a thread pool is automatically created to concurrently execute the loading and preprocessing tasks;

[0037] The main task of the thread traverses all the image files in the resource directory or database, reads the image file list, pre-processes the image data in the list, and stores the pre-processed image data in the memory;

[0038] Wait for all threads to complete their tasks and release thread pool resources.

[0039] Specifically, in this embodiment, when the application is started, a thread pool is created. In this embodiment, the size of the thread pool is set to 10, which is used to concurrently execute loading and preprocessing tasks;

[0040] The thread main task will traverse all the image files in the resource directory or database. A thread can be assigned to each batch of files. The following operations are performed:

[0041] 1.1. Read the image file list and use the ImageIO class to support various types of image formats, such as JPEG, PNG, GIF, etc.

[0042] 1.2. Preprocess the image data and use the OpenCV library to perform image preprocessing operations, including resizing, cropping, rotating, grayscale, etc., to improve the quality and consistency of the image.

[0043] 1.3. Store the preprocessed image data in memory to prepare for subsequent feature extraction and clustering.

[0044] Wait for all threads to complete their tasks, release thread pool resources, and end the initialization loading process.

[0045] Step S102: The image data loaded into the memory is calculated by a color moment algorithm and an ORB algorithm (Oriented Fast and Rotated Brief, image feature algorithm) to obtain a feature vector of the image.

[0046] Specifically, in this embodiment, the main process of this step is to calculate the feature distribution of the image, that is, the corresponding feature vector, by using the color moment algorithm and the ORB algorithm for the image data loaded into the memory. In this embodiment, the color moment algorithm is used to comprehensively consider the color distribution of the image, and the ORB algorithm is used to describe the texture and local features of the image. The ORB algorithm used in this embodiment is a conventional technical means for image processing by those skilled in the art, and this application does not improve it, so it will not be further described here.

[0047] It should be noted that since the feature vectors obtained by the color moment algorithm and the ORB algorithm are of fixed length, in order to improve the image retrieval speed, a large number of feature vectors generated by the combination of the two algorithms need to be clustered. The clustering process is described in step S103.

[0048] In some embodiments, the color moment algorithm is used to consider the color distribution of the image, and the ORB algorithm is used to describe the texture and local features of the image, wherein the color moment algorithm includes:

[0049] 2.1. Read the image, convert the image from RGB color space to HSV color space, and resize the image to a uniform size;

[0050] 2.2. Separate the three channels of the HSV image, calculate the color moment features for each channel, calculate the mean and standard deviation based on the color moment features obtained for each channel, and at the same time, calculate the skewness of each channel;

[0051] In the HSV color space, an image is divided into three channels: hue, saturation, and value. This embodiment separates these channels through an image processing library (such as OpenCV in Python).

[0052] For each channel, the average value of all pixel values ​​is calculated, which is obtained by summing all pixel values ​​in the channel and dividing by the total number of pixels; the average value of the square of the difference between each pixel value and the mean is calculated, and the standard deviation is the square root of the variance. The mean and standard deviation involved in this embodiment are conventional technical means adopted by those skilled in the art, and will not be described in detail here.

[0053] The skewness value is calculated as follows:

[0054]

[0055] Where I(i, j) represents the pixel value of the image at position (i, j), μ represents the mean, σ represents the standard deviation, and M×N represents the total number of pixels in the image.

[0056] 2.3. Convert the calculation result to double type and return an array containing color moment features.

[0057] Some core codes are as follows:

[0058]

[0059] Imgproc.cvtColor(srcImage,srcImage,Imgproc.COLOR_BGR2HSV);

[0060] / / Same size

[0061] Size scaleSize=new Size(256, 256);

[0062] Imgproc.resize(srcImage, srcImage, scaleSize, 0, 0);

[0063] / / Separate the image into three channels

[0064] ArrayList <mat>channels=new ArrayList<>(3);

[0065] Core.split(srcImage, channels);

[0066] Mat H = channels.get(0);

[0067] Mat S = channels.get(1);

[0068] Mat V = channels.get(2);

[0069] / / Calculate color moment features

[0070] MatOfDouble H_mean=new MatOfDouble(), H_stddev=new MatOfDouble();

[0071] MatOfDouble S_mean=new MatOfDouble(), S_stddev=new MatOfDouble();

[0072] MatOfDouble V_mean=new MatOfDouble(), V_stddev=new MatOfDouble();

[0073] / / Calculate mean and standard deviation

[0074] Core.meanStdDev(H, H_mean, H_stddev);

[0075] Core.meanStdDev(S, S_mean, S_stddev);

[0076] Core.meanStdDev(V, V_mean, V_stddev);

[0077] / / Calculate skewness

[0078] double H_skewness=calculateSkewness(H, H_mean);

[0079] double S_skewness=calculateSkewness(S, S_mean);

[0080] double V_skewness = calculateSkewness(V, V_mean);

[0081] / / Convert to double type

[0082] double[] H_mean_double = H_mean.get(0, 0); double[] S_mean_double = S_mean.get(0, 0);

[0083] double[] V_mean_double = V_mean.get(0, 0); double[] H_stddev_double = H_stddev.get(0, 0); double[] S_stddev_double = S_stddev.get(0, 0); double[] V_stddev_double = V_stddev.get(0, 0); / / Release resources

[0084] H.release();

[0085] S.release();

[0086] V.release();

[0087] H_mean.release();

[0088] S_mean.release();

[0089] V_mean.release();

[0090] H_stddev.release();

[0091] S_stddev.release();

[0092] V_stddev.release();

[0093] / / Return color moment features

[0094] return new double[]{

[0095] H_mean_double[0],

[0096] S_mean_double[0],

[0097] V_mean_double[0],

[0098] H_stddev_double[0],

[0099] S_stddev_double[0],

[0100] V_stddev_double[0],

[0101]

[0102] Step S103: cluster the feature vectors of all the image data in the memory using the ISODATA algorithm, and store the clustering result data in the Elasticsearch database to establish an index of the image feature clustering results.

[0103] Specifically, in this embodiment, the Spring Boot program is used to implement the ISODATA algorithm that supports the generation of clustering results, and the feature vectors generated by the color moment and ORB algorithms are clustered to improve the retrieval speed. ISODATA is an unsupervised clustering algorithm, which is mainly used to classify data sets. The algorithm can automatically adjust the number of clusters and cluster centers to adapt to categories of different shapes and sizes, and then cluster and classify the feature vectors of the two algorithms, and store the obtained clustering results in the database Elasticsearch (Elasticsearch is a distributed search and analysis engine that can quickly index and retrieve a large amount of data), and establish an index for the image feature clustering results.

[0104] The ISODATA algorithm used in this embodiment is a conventional algorithm in the art, and this application does not improve it, so it will not be further described here.

[0105] In some embodiments, Figure 3 As shown, it also includes:

[0106] Clustering data hot update, real-time update of clustering result data generated by images. In response to the addition of new image data in the resource directory, the backend listener listens to the event and sends the message to the scheduler through the message queue to process the message regularly.

[0107] Add the new feature vector extracted from the new image data to the cluster, and synchronize the updated cluster result data to the database Elasticsearch;

[0108] In response to updates exceeding a certain number limit, the scheduler automatically recalculates the center of the cluster, re-splits and merges dense and sparse points and automatically adjusts them, continuously updates in an iterative manner, and stores the updated clustering result data in the Elasticsearch database.

[0109] Specifically, in this embodiment, clustering data hot update is to update the clustering result data generated by the image in real time. When new image data is added to the resource directory, the backend listener will listen to the event and send the message to the scheduler through the message queue. The scheduler will process the message regularly. Every once in a while, the scheduler will check whether there is a new file addition message from the listener from the message queue. If there is a new file added, the new feature vector extracted from these images will be added to the cluster, and the clustering result data will be stored in the database Elasticsearch.

[0110] Furthermore, the clustering result data generated by the image is updated in real time. The specific steps are as follows:

[0111] 3.1. Create a backend listener to monitor whether there is new image data added to the resource directory. If so, the new image data detected will be sent to a message queue through information and wait for the scheduler to process it.

[0112] 3.2. Create a scheduler to periodically obtain information about new files from the message queue, extract new feature vectors from new image data, and add them to the cluster. Use the ISODATA algorithm to re-cluster the feature vectors of all image data in memory, update the number and center of clusters to adapt to categories of different shapes and sizes, and synchronize the updated clustering result data to the database Elasticsearch.

[0113] When the number of file updates exceeds a certain limit, the scheduler will automatically recalculate the center of the cluster, and then automatically re-split and merge the dense and sparse points and automatically adjust them. The clustering results are continuously updated through iteration to make them more accurate and stable.

[0114] Specifically, in this embodiment, if the number of new files exceeds a certain limit, such as 1000, the following operations are performed:

[0115] 3.3 Recalculate the center of the cluster, and then automatically split and merge the dense and sparse points again, and continuously update them in an iterative manner to make the clustering results more accurate and stable.

[0116] 3.4. Store the updated clustering result data in the Elasticsearch database and create an index for the image feature clustering results for quick query.

[0117] / **

[0118] * Clustering

[0119] * /

[0120] public List <Pair<Point,String> >processClustering(){

[0121]

[0122]

[0123] Step S104: receiving a picture or text query request from a user, and obtaining metadata information of corresponding pictures from the Elasticsearch database according to the picture ID list obtained by the user query, so as to retrieve a target image.

[0124] Specifically, in this embodiment, Figure 4 As shown in the figure, for image retrieval, the front-end web page uploads the image and sends a request to the back-end. The back-end program calls the color moment and ORB algorithms to calculate the feature vectors of the image under the two algorithms, and then finds its nearest cluster center through the ISODATA algorithm, calculates the similarity, and returns the matching image file data in the database in descending order of matching degree.

[0125] To further explain, the front end receives the user's image or text query request through the web page and sends the request to the back end program. The back end program performs different processing according to the user's query method:

[0126] 4.1. If the user's query method is based on an image, that is, the user uploads an image to indicate the user's search intent, such as a picture of a cat, a picture of a landscape, etc., then the following operations are performed:

[0127] Based on the image, the nearest cluster center is found to obtain the category to which it belongs. The feature vectors of all images in the category are queried from the Elasticsearch database, and the similarity is calculated with the feature vectors of the images uploaded by the user to obtain the ID list of the images most similar to the user query.

[0128] 4.2. If the user's query method is a text-based query, that is, the user enters a keyword or a sentence to express the user's search intention, such as "cat", "scenery", etc., then the following operations are performed:

[0129] 4.2.1. Use natural language processing techniques, such as Word2Vec and BERT, to convert the user’s query into a semantic vector that represents the user’s search intent.

[0130] 4.2.2. Query the feature vectors of all images from the Elasticsearch database, calculate the similarity with the user's semantic vector, and obtain the ID list of the images that best match the user's query.

[0131] 4.3. According to the ID list of the pictures obtained by the user's query, the metadata information of the corresponding pictures, such as path, size, format, etc., is obtained from the Elasticsearch database. The search results are returned to the user in the form of pictures through the Web interface, and the relevant information of the pictures, such as similarity, source, etc., are displayed at the same time.

[0132] The method described in this application is based on the characteristics of SpringBoot, and can quickly build and run a content-based image retrieval system without complex configuration and deployment. Upon receiving a user's image data query request, it uses image feature extraction and image clustering to achieve rapid retrieval and matching of similar images, thereby improving development efficiency and user experience.

[0133] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the embodiment of the present application also provides an image retrieval device based on SpringBoot.

[0135] like Figure 5 As shown, the image retrieval device based on SpringBoot includes:

[0136] Initialization module 11 is configured to read the image data in the specified resource directory or database and load it into the memory;

[0137] The feature extraction module 12 is configured to calculate the feature vector of the image by using the color moment algorithm and the ORB algorithm to obtain the image data loaded into the memory;

[0138] The clustering module 13 is configured to cluster the feature vectors of all the image data in the memory by using the ISODATA algorithm, and store the clustering result data in the Elasticsearch database to establish an index of the image feature clustering result;

[0139] The retrieval module 14 is configured to receive a user's image or text query request, and obtain metadata information of the corresponding image from the Elasticsearch database according to the ID list of the image obtained by the user's query, so as to retrieve the target image.

[0140] For the convenience of description, the above devices are described in terms of functions and are divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0141] The device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0142] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0143] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0144] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0145] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0146] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0147] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0148] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0149] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0150] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0151] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0152] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0153] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0154] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0155] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0156] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0157] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.< / mat>

Claims

1. A SpringBoot-based image retrieval method, characterized in that: include: Read the image data in the specified resource directory or database and load it into memory; The image data loaded into the memory is calculated using the color moment algorithm and the ORB algorithm to obtain the feature vector of the image; The feature vectors of all image data in memory are clustered using the ISODATA algorithm, and the clustering result data is stored in the Elasticsearch database to establish an index for the image feature clustering results; Receive the user's image or text query request, and obtain the metadata information of the corresponding image from the Elasticsearch database according to the ID list of the image obtained by the user's query to retrieve the target image.

2. The method according to claim 1, characterized in that The step of reading the image data in the specified resource directory or in the database and loading it into the memory includes: Start the application and automatically create a thread pool to concurrently execute loading and preprocessing tasks; The main task of the thread traverses all the image files in the resource directory or database, reads the image file list, pre-processes the image data in the list, and stores the pre-processed image data in the memory; Wait for all threads to complete their tasks and release thread pool resources.

3. The method according to claim 1, characterized in that The color moment algorithm is used to consider the color distribution of the image, and the ORB algorithm is used to describe the texture and local features of the image, wherein the color moment algorithm includes: Read the image, convert the image from RGB color space to HSV color space, and resize the image to a uniform size; Separate the three channels of the HSV image, calculate the color moment features for each channel, calculate the mean and standard deviation based on the color moment features obtained for each channel, and at the same time, calculate the skewness of each channel; Convert the calculation result to double type and return an array containing the color moment characteristics.

4. The method according to claim 3, characterized in that: Where I(i, j) represents the pixel value of the image at position (i, j), μ represents the mean, σ represents the standard deviation, and M×N represents the total number of pixels in the image.

5. The method according to claim 1, characterized in that Also includes: Clustering data hot update, real-time update of clustering result data generated by images. In response to the addition of new image data in the resource directory, the backend listener listens to the event and sends the message to the scheduler through the message queue to process the message regularly. The new feature vector extracted from the new image data is added to the cluster, and the updated clustering result data is synchronously updated to the database Elasticsearch.

6. The method according to claim 5, characterized in that: In response to updates exceeding a certain number limit, the scheduler automatically recalculates the center of the cluster, re-splits and merges dense and sparse points and automatically adjusts them, continuously updates in an iterative manner, and stores the updated clustering result data in the Elasticsearch database.

7. The method according to claim 1, characterized in that The receiving of a user's image or text query request, and obtaining metadata information of corresponding images from the Elasticsearch database according to the ID list of images obtained by the user's query, so as to retrieve a target image, includes: In response to the user's query method being an image query, the feature vectors of all images in the category are queried from the Elasticsearch database, and the similarity is compared with the feature vectors of the images uploaded by the user to obtain a list of IDs of images that are most similar to the user's query; In response to the user's query method being a text query, the user's query is converted into a semantic vector, the feature vectors of all images are queried from the Elasticsearch database, and the similarity is compared with the user's semantic vector to obtain an ID list of images with the highest matching degree with the user's query; According to the ID list of the pictures obtained by the user's query, the metadata information of the corresponding pictures is obtained from the Elasticsearch database, and the search results are returned to the user in the form of pictures, while displaying the relevant information of the pictures.

8. An image retrieval device based on SpringBoot, characterized in that: include: The initialization module is configured to read the image data in the specified resource directory or database and load it into the memory; The feature extraction module is configured to calculate the feature vector of the image by using the color moment algorithm and the ORB algorithm to obtain the image data loaded into the memory; The clustering module is configured to cluster the feature vectors of all the image data in the memory by using the ISODATA algorithm, and store the clustering result data in the Elasticsearch database to establish an index of the image feature clustering results; The retrieval module is configured to receive a user's image or text query request, and obtain metadata information of the corresponding image from the Elasticsearch database according to the ID list of the image obtained by the user's query, so as to retrieve the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: in, The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 7.

Images