Image clustering method and electronic equipment

Abstract

The embodiment of the invention provides an image clustering method and electronic equipment, and the method comprises the steps: determining a plurality of images, determining the structural similarity between the images, and generating a structural similarity matrix; inputting the structural similarity matrix into a self-organizing mapping network, and enabling the self-organizing mapping network to search for an optimal matching neuron node for a first image included in each image pair according to the structural similarity matrix based on a competitive learning rule, and mapping a second image which is included in each image pair and has structural similarity with the first image to topological adjacent neuron nodes to generate a mapping result of the plurality of image pairs. And performing multi-stage clustering processing based on the mapping result to obtain a fine clustering result. Therefore, based on the combination of the structural similarity and the self-organizing mapping network, the visual sensitive similarity measurement and hierarchical clustering division are realized, the clustering fineness is improved, and the operation and maintenance cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image clustering method and an electronic device. BACKGROUND

[0002] Image clustering is a key technology in computer vision, and the core is to realize image grouping or classification by calculating the similarity between images. The image clustering method in the prior art is usually based on pixel value or feature vector to calculate similarity (such as Euclidean distance or cosine similarity), and usually similarity is calculated directly in the global feature space. Based on this method, using pixel value or low-dimensional feature vector to calculate similarity, there is a problem of insufficient image clustering precision, using high-dimensional feature vector to calculate similarity, there is a problem of high cost, and calculating similarity in the global feature space also has a problem of insufficient image clustering precision.

[0003] Therefore, the image clustering method in the prior art has the problems of insufficient image clustering precision and high cost. SUMMARY

[0004] The embodiments of the present application provide an image clustering method and an electronic device, which can solve the problems of insufficient image clustering precision and high cost in the prior art.

[0005] To solve the above technical problems, in a first aspect, the embodiments of the present application provide an image clustering method, which comprises: determining a plurality of images, and determining the structural similarity between each image to obtain the structural similarity corresponding to each image pair, and generating a structural similarity matrix according to a plurality of structural similarities, each image pair comprising a first image and a second image, the first image and the second image being two images with structural similarity; inputting the structural similarity matrix into a self-organizing mapping network, and making the self-organizing mapping network determine a first neuron node matched with the first image in each image pair based on a competitive learning rule according to the structural similarity matrix, mapping the first image to the first neuron node, and mapping the second image similar to the first image in each image pair to a second neuron node adjacent to the first neuron node to obtain the mapping result of each image pair, so as to obtain the mapping result of a plurality of image pairs; performing multi-level clustering processing according to the mapping result of a plurality of image pairs to obtain the clustering result corresponding to a plurality of image pairs.

[0006] By adopting the technical solution, the structural similarity matrix is obtained by determining the structural similarity between each image, and the structural similarity matrix is input into the self-organizing mapping network, so that the self-organizing mapping is based on the competitive learning rule, and the similar image pairs (for example, the first image and the second image) are matched with the adjacent neuron nodes according to the structural similarity matrix, the mapping result of each image pair corresponding to the neuron node is obtained, the mapping result of the multiple image pairs is obtained, and then the mapping result is processed by multi-level clustering according to the mapping result, and the clustering result is obtained. In this way, in the clustering stage, the multiple images are processed by multi-level clustering based on the self-organizing mapping according to the structural similarity matrix, and the multi-level clustering result is obtained, the hierarchical clustering division of the similar images is realized, the clustering of the similar images is more detailed, and based on the neuron nodes, the similar images are mapped. Because the neuron nodes can retain the semantic information such as the similarity reason of each similar image pair, the interpretability of each similar image is enhanced, the user can know the reason why two images are similar images, and the clustering of the images is more detailed.

[0007] Further, the structural similarity index is combined with the self-organizing mapping network, the high labor cost and time cost required for data labeling are reduced, the dependence on manual work is reduced through the automatic process of multi-level clustering, and the operation and maintenance cost is reduced. Moreover, compared with using high-dimensional feature vectors for similarity calculation, the operation and maintenance cost is also reduced.

[0008] In a possible implementation of the first aspect, the self-organizing mapping network is a two-dimensional mapping network or a three-dimensional mapping network, the self-organizing mapping network is composed of multiple neuron nodes, each neuron node has a corresponding weight vector, and the input of the structural similarity matrix into the self-organizing mapping network enables the self-organizing mapping network to determine the first neuron node matched with the first image in each image pair based on the competitive learning rule according to the structural similarity matrix, including: inputting the structural similarity matrix into the self-organizing mapping network, enabling the self-organizing mapping network to obtain the first image, the second image included in each image pair, and the structural similarity corresponding to each image pair from the structural similarity matrix based on the competitive learning rule; calculating first distance information of the first image and the weight vector corresponding to each neuron node; and selecting the neuron node corresponding to the smallest first distance information as the first neuron node matched with the first image.

[0009] By adopting the technical solution, the self-organizing mapping network autonomously performs distance calculation of the image and the weight vector of the neuron node based on the competitive learning rule, and can autonomously perform best matching unit matching of the image, thereby realizing efficient matching and mapping of high-dimensional image similarity data in a low-dimensional space. In this way, the correspondence between image similarity and topological structure is established, which lays a foundation for subsequent image clustering, thereby reducing the computational complexity, improving the processing efficiency, and saving the computing resource cost.

[0010] In a possible implementation of the first aspect, the weight vector corresponding to each neuron node is obtained according to a mapping result of a previous image pair on which the first distance information is calculated.

[0011] Further, the weight vector corresponding to each neuron node is obtained according to a mapping result of a previous image pair on which the first distance information is calculated, including: determining a neighborhood radius of the best matching unit corresponding to the image pair according to the mapping result of the previous image pair on which the first distance information is calculated; and determining a neuron node range according to the neighborhood radius with the best matching unit as the center, and updating the weight vector of each neuron node in the neuron node range.

[0012] According to the above technical solution, the neighborhood radius and the weight vector of the neuron node are iteratively updated, so that the neighborhood radius and the weight vector are determined every time the neuron node matching is performed, to ensure that the information of the neuron node is updated in real time, and the accuracy of the next neuron node matching is ensured. Moreover, the self-learning and optimization of the topology structure related to the mapping result are realized through autonomous updating, the dependence on manual parameter adjustment is reduced, and the maintenance cost is reduced.

[0013] In a possible implementation of the first aspect, the mapping result of the plurality of image pairs is used to perform multi-level clustering processing to obtain a clustering result, including: determining second distance information of adjacent neuron nodes according to the mapping result of the plurality of image pairs and the weight vector of each neuron node, to generate a distance matrix; and performing multi-level clustering processing on each neuron node according to the distance matrix to obtain the clustering result.

[0014] According to the above technical solution, the distance matrix between the neuron nodes is generated according to the mapping result and the weight vector, and the image clustering analysis is performed based on the distance matrix, so that the complex image clustering problem is converted into a structured node clustering problem. In this way, the complexity of the calculation is reduced, the efficiency of the image clustering is improved, and the accuracy of the image clustering result is further ensured.

[0015] In a possible implementation of the first aspect, the multi-level clustering processing is a three-level clustering processing, and the multi-level clustering processing is performed on each neuron node according to the distance matrix to obtain a clustering result, including: merging neuron nodes with similar structures into the same class according to the distance matrix to obtain a plurality of first-level clusters, to obtain a first-level clustering result, the neuron nodes with similar structures being adjacent neuron nodes with second distance information less than a preset first threshold; merging neuron nodes with similar textures in each first-level cluster into the same class according to the distance matrix to obtain a plurality of second-level clusters corresponding to each first-level cluster, to obtain a second-level clustering result, the neuron nodes with similar textures being adjacent neuron nodes with second distance information less than a preset second threshold, the second threshold being less than the first threshold; and merging neuron nodes with similar objects in each second-level cluster into the same class according to the distance matrix to obtain a plurality of third-level clusters corresponding to each second-level cluster, to obtain a third-level clustering result, the neuron nodes with similar objects being adjacent neuron nodes with second distance information greater than a preset third threshold, the third threshold being less than the second threshold.

[0016] By using the above technical solution, the three-level clustering strategy from coarse to fine is adopted, and progressive clustering is performed from the structure, texture and object levels, so that multi-level fine analysis of image content is realized. In this way, the accuracy of the clustering result is improved, and through the hierarchical processing manner, the allocation of computing resources is optimized, the processing efficiency is improved, and the overall cost is reduced.

[0017] In a possible implementation of the first aspect, the image clustering method further includes determining the first threshold, the second threshold and the third threshold by: determining a distribution histogram of the distance matrix, determining the first threshold according to the distribution histogram; determining a first similarity distribution graph of images corresponding to neuron nodes included in the first-level clustering result, determining the second threshold according to the first similarity distribution graph; and determining a second similarity distribution graph of images corresponding to neuron nodes included in the second-level clustering result, determining the third threshold according to the second similarity distribution graph.

[0018] By using the above technical solution, the thresholds of each level of clustering are adaptively determined according to the data distribution histogram and the similarity distribution graph, so that the threshold setting is objective. In this way, the tedious manual adjustment of the threshold depending on artificial experience is avoided, the human resources and time cost are saved, the problem of re-clustering due to improper threshold setting can be effectively avoided, and the accuracy of image clustering can be better improved by dynamically adjusting the threshold.

[0019] In a possible implementation of the first aspect, the determining of the structural similarity between the images comprises: determining a brightness similarity, a contrast similarity and a structural similarity between the images; and obtaining the structural similarity between the images according to the brightness similarity, the contrast similarity and the structural similarity between the images, and the structural similarity is obtained by the following manner:

[0020]

[0021] wherein A is the first image, B is the second image, is the structural similarity between the first image and the second image, is the brightness similarity between the first image and the second image, is the contrast similarity between the first image and the second image, is the structural similarity between the first image and the second image, 、 、 is a corresponding weighting coefficient.

[0022] By using the structural similarity index of the brightness similarity, the contrast similarity and the structural similarity conforming to the human visual perception characteristics instead of using a simple pixel-level difference to measure the image similarity, the similarity between the images is referenced more, the similarity accuracy is higher, and thus the accuracy of the image clustering is higher. In this way, the accuracy of the image content understanding is improved, the error clustering caused by inaccurate similarity measurement is reduced, and the additional cost required for subsequent correction of the error result is reduced.

[0023] In a possible implementation of the first aspect, the image clustering method further comprises: mapping the clustering result into a topology structure of a self-organizing mapping network for visual display.

[0024] By mapping the clustering result back to the topology structure of the self-organizing mapping network for visual display, the user is provided with an intuitive result display. In this way, the user is facilitated to verify, analyze and understand the clustering result, the result interpretation time is shortened, the user experience is improved, and thus the overall time cost is saved.

[0025] In a possible implementation of the first aspect, the image is a preprocessed image block.

[0026] By preprocessing the image and dividing the image into image blocks for processing, the analysis focus is concentrated on the local features. In this way, the capturing ability of the detailed features is enhanced, the clustering precision is improved, and the calculation cost is reduced.

[0027] In a second aspect, the implementation of the present application further discloses an image retrieval method, comprising determining a to-be-retrieved image, feature information of the to-be-retrieved image, and a clustering result of an image library, wherein the clustering result of the image library is obtained based on the image clustering method provided in any one of the implementations of the first aspect, similar neuron nodes are matched from the clustering result according to the feature information of the to-be-retrieved image, corresponding similar images and semantic information of the similar images are obtained according to the matched neuron nodes, and the similar images and the semantic information are presented to a user.

[0028] In a third aspect, the implementation of the present application further discloses an electronic device, comprising a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the electronic device implements the image clustering method provided in any one of the implementations of the first aspect.

[0029] In a fourth aspect, the implementation of the present application further discloses a computer readable storage medium, which stores a computer program, the computer program can be executed by a computer cluster to implement the inference method based on the neural network model provided in any one of the implementations of the first aspect, and / or the image clustering method provided in any one of the implementations of the second aspect.

[0030] In a fifth aspect, the implementation of the present application further discloses a computer program product, comprising a computer program, which, when executed by a computer cluster, implements the inference method based on the neural network model provided in any one of the implementations of the first aspect, and / or the image clustering method provided in any one of the implementations of the second aspect.

[0031] The related beneficial effects of the third aspect to the fifth aspect can be referred to the related description in the first aspect or the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the implementation description will be briefly introduced as follows.

[0033] Figure 1 A flowchart of an image clustering method provided by an embodiment of the present application;

[0034] Figure 2 A flowchart of a method for determining a structural similarity matrix provided by an embodiment of the present application;

[0035] Figure 3 A flowchart of a method for determining a mapping result of a pair of images provided by an embodiment of the present application;

[0036] Figure 4A flowchart of one of the methods for determining a clustering result provided by an embodiment of the present application is shown in FIG. 1.

[0037] Figure 5 A flowchart of another method for determining a clustering result provided by an embodiment of the present application is shown in FIG. 2.

[0038] Figure 6 A flowchart of another image clustering method provided by an embodiment of the present application is shown in FIG. 3.

[0039] Figure 7 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0040] As mentioned above, in the field of computer vision and image analysis, image clustering is a basic and key task, which aims to automatically group a set of unlabeled images or image blocks according to the similarity of visual content. Existing image clustering methods first extract feature representations from images, rely on distance measurements (such as Euclidean distance, cosine similarity, etc.) of feature representations (such as pixel values or feature vectors) to directly calculate similarity in the global feature space, and finally complete clustering. However, such traditional processing methods have the following shortcomings when dealing with image data rich in structural information (such as texture, shape consistency).

[0041] Insufficient structure sensitivity, most existing methods rely on similarity measurements based on pixel values or low-dimensional embedded features. Such measurement methods are difficult to accurately capture and describe the real similarity of images in high-level semantic structures (such as texture patterns, geometric shapes), and cannot effectively express the structural consistency perceived by the human visual system, resulting in limited accuracy of similarity judgment.

[0042] Single clustering granularity, existing methods often directly cluster in the global feature space, lacking a multi-level analysis perspective. This single-granularity clustering method ignores the hierarchical organization relationship within the image (for example, from low-level edge texture, to medium-level local pattern, to high-level semantic object), cannot achieve fine-grained clustering division, and is difficult to meet the higher requirement for the level of detail of the clustering result.

[0043] Contradiction between computational overhead and clustering accuracy, in order to improve accuracy, some existing methods use complex deep neural networks to extract high-dimensional deep features. Although the performance is improved to some extent, the huge computational complexity, storage overhead and energy consumption seriously limit its practical deployment and application in resource-constrained environments (such as mobile devices, edge computing scenarios), resulting in a contradiction between computational efficiency and clustering accuracy.

[0044] Therefore, there is an urgent need for an image clustering method that can achieve higher precision and lower cost.

[0045] To solve the above technical problems, the embodiment of the present application provides an image clustering method, which combines structural similarity and clustering capability of self-organizing mapping to realize hierarchical clustering (i.e., hierarchical clustering) of images at a visual level, so as to make the image clustering more refined while reducing the clustering cost.

[0046] As shown in Figure 1 The image clustering method provided by the embodiment of the present application specifically includes the following steps.

[0047] S100, a plurality of images are determined, and structural similarities between the images are determined to obtain structural similarities corresponding to a plurality of image pairs, and a structural similarity matrix is generated according to the plurality of structural similarities, each image pair includes a first image and a second image, and the first image and the second image are two images with structural similarities.

[0048] S200, the structural similarity matrix is input into a self-organizing mapping network, the self-organizing mapping network determines a first neuron node matched with the first image in each image pair according to the structural similarity matrix based on a competitive learning rule, maps the first image to the first neuron node, and maps a second image similar to the first image in each image pair to a second neuron node adjacent to the first neuron node to obtain a mapping result of each image pair, so as to obtain mapping results of a plurality of image pairs.

[0049] S300, a multi-level clustering process is performed according to the mapping results of the plurality of image pairs to obtain a clustering result corresponding to the plurality of images.

[0050] By using the above technical solution, the structural similarity matrix is obtained by determining the structural similarities between the images, and the structural similarity matrix is input into the self-organizing mapping network, so that the self-organizing mapping network matches the adjacent neuron nodes according to the structural similarity matrix for similar image pairs (e.g., the first image and the second image) based on the competitive learning rule, to obtain the mapping result of each image pair corresponding to the neuron node, so as to obtain the mapping result of the plurality of image pairs, and then the mapping result is subjected to the multi-level clustering process according to the mapping result to obtain the clustering result. In this way, in the clustering stage, the multi-level clustering result is obtained by performing the multi-level clustering process on the plurality of images based on the self-organizing mapping according to the structural similarity matrix, so as to realize the hierarchical clustering division of the similar images, making the clustering of the similar images more refined. Moreover, the similar images are mapped based on the neuron nodes, and the neuron nodes can retain the semantic information such as the similarity reasons of each similar image pair, so as to enhance the interpretability of each similar image, facilitate the user to know the reasons why two images are similar images, and improve the user experience.

[0051] Further, the structural similarity index is combined with a self-organizing mapping network to reduce the high labor cost and time cost required for data labeling, and the automation process of multi-level clustering reduces the dependence on manual work and further reduces the operation and maintenance cost.

[0052] Firstly, step S100 is performed to determine a plurality of images and determine the structural similarity (SSIM) between each image, obtain the structural similarity corresponding to each image pair, and generate a structural similarity matrix according to the plurality of structural similarities.

[0053] In an implementation of the present application, each image pair includes a first image and a second image, and the first image and the second image are two images with determined structural similarity.

[0054] In the implementation of the present application, the image can be a complete image or an image block.

[0055] Further, in an implementation of the present application, the image is a preprocessed image or image block.

[0056] The preprocessing includes standardizing the size and brightness of each image, so that the size of each image or image block is uniform and the brightness is within a preset brightness range, thereby reducing the influence of illumination difference between images.

[0057] In an implementation of the present application, as shown in Figure 2 The structural similarity matrix includes the following steps.

[0058] S110, determine the brightness similarity, contrast similarity and structural similarity between each image.

[0059] For example, taking the first image as image A and the second image as image B, the brightness similarity between the two images is calculated.

[0060] In the implementation of the present application, the brightness similarity between each image is calculated by the following method:

[0061]

[0062]

[0063] wherein, is the brightness similarity between each image, is the brightness mean of image A, is the brightness mean of image B, is the pixel value dynamic range of image A and image B, is a constant, and the default value is 0.01.

[0064] wherein, if image A and image B are 8-bit images, L = 255.

[0065] Further, the contrast similarity between images is calculated by:

[0066]

[0067]

[0068] wherein, is the contrast similarity between images, is the contrast standard deviation of image A, is the contrast standard deviation of image B, is a constant, and by default is 0.03.

[0069] Further, the structure similarity between images is calculated by:

[0070]

[0071]

[0072] wherein, is the structure similarity between images, is the covariance of image A and image B.

[0073] S120, according to the brightness similarity, the contrast similarity and the structure similarity between images, obtains the structural similarity between images.

[0074] For example, the weighting coefficients corresponding to the brightness similarity, the contrast similarity and the structure similarity are determined, and the structural similarity between images is obtained according to the brightness similarity, the contrast similarity, the structure similarity and the corresponding weighting coefficients between images.

[0075] wherein, the structural similarity between images is obtained by:

[0076]

[0077] wherein, A is the first image, B is the second image, is the structural similarity between the first image and the second image, is the brightness similarity between the first image and the second image, is the contrast similarity between the first image and the second image, is the structure similarity between the first image and the second image, , , corresponding weight coefficient.

[0078] wherein the weight coefficient can be pre-set, .

[0079] S130, obtaining structure similarities of the plurality of image pairs according to structure similarities between the images, and generating a structure similarity matrix according to the plurality of structure similarities.

[0080] Illustratively, the structure similarities of the image pairs are obtained according to the structure similarities between the first image and the second image, and then the structure similarity matrix is generated.

[0081] wherein the structure similarity matrix is a symmetric matrix SSIM reflecting structure similarity relationships between the images.

[0082] In the implementation of the present application, the structure similarity matrix SSIM is as follows:

[0083] wherein, , …, = 1, = .

[0084] In the implementation of the present application, the structure similarity index of the luminance similarity, the contrast similarity and the structure similarity conforming to the human visual perception characteristics is used to calculate the similarity between the images or image blocks, so as to retain the consistency of the texture, luminance and structure information, instead of using the simple pixel-level difference to measure the image similarity, so that the image similarity measurement is more comprehensive and more accurate. The accuracy of the image content similarity judgment is improved, the error clustering caused by the inaccurate similarity measurement is reduced, and the additional cost required for subsequent correction of the error result is reduced.

[0085] Next, step S200 is performed, the structure similarity matrix is input into a self-organizing map network (SOM), the self-organizing map network determines a first neuron node matching the first image in each image pair based on a competitive learning rule according to the structure similarity matrix, maps the first image to the first neuron node, and maps a second image similar to the first image in each image pair to a second neuron node adjacent to the first neuron node, to obtain a mapping result of each image pair, so as to obtain mapping results of the plurality of image pairs.

[0086] Illustratively, the structure similarity matrix is input into the self-organizing map network, and through competitive learning and topology preservation, images or image blocks that are visually similar are mapped to adjacent neuron nodes, so as to realize low-dimensional topology expression of high-level similarity information.

[0087] It should be noted that the self-organizing mapping network (SOM) is an artificial neural network based on unsupervised learning method, which maps high-dimensional data to low-dimensional space (two-dimensional or three-dimensional) through a competitive learning mechanism, and automatically forms a topological structure mapping of input data.

[0088] The self-organizing mapping network includes an input layer and an output layer, and the structural similarity matrix is input to the input layer, and the output neuron grid, each neuron corresponds to a node in the low-dimensional space, that is, a neuron node.

[0089] The competitive learning rule includes a competition phase and a cooperation phase, wherein the competition phase is to compare the input similarity matrix with the neuron weight, and select the most similar neuron node as the best matching unit (BMU), and the cooperation phase is to adjust the BMU and its field weight according to the distance of each neuron node, so that similar images are gathered in topologically adjacent positions. Each image pair is selected by the competitive learning rule to select the best matching unit, and self-organization is realized by adjusting the field weight.

[0090] Therefore, in one possible implementation of the present application, the self-organizing mapping network is a two-dimensional mapping network or a three-dimensional mapping network, and the self-organizing mapping network is composed of a plurality of neuron nodes, each neuron node having a corresponding weight vector, also known as an initial weight vector.

[0091] In the implementation of the present application, as shown in Figure 3 , the structural similarity matrix is input to the self-organizing mapping network, so that the self-organizing mapping network determines the first neuron node matching the first image in each image pair based on the competitive learning rule according to the structural similarity matrix, including the following steps.

[0092] S210, input the structural similarity matrix to the self-organizing mapping network, so that the self-organizing mapping network obtains the first image and the second image included in each image pair and the structural similarity corresponding to each image pair from the structural similarity matrix based on the competitive learning rule.

[0093] For example, a two-dimensional or three-dimensional SOM network is constructed in advance, and the weight vector of each neuron node is initially randomly distributed as a random value, wherein the random value can be realized in a uniform distribution or a small range Gaussian distribution, and the dimension of the weight vector of the neuron node is the same as the dimension of the image included in each image pair in the structural similarity.

[0094] Further, the image pair in the input structural similarity matrix is extracted from the structural similarity matrix to extract the image pair (including the first image and the second image ) and its structural similarity value .

[0095] S220, calculating first distance information of the first image and the weight vector corresponding to each neuron node.

[0096] For example, using a competitive learning rule, the first image is calculated with the Euclidean distance of the weight vector of all neuron nodes (as an example of the first distance information):

[0097]

[0098] wherein, is the first distance information of the first image and the weight vector of the kth neuron node , , M is the number of neuron nodes.

[0099] S230, selecting the neuron node corresponding to the smallest first distance information as the first neuron node matched with the first image.

[0100] For example, the neuron node c with the smallest distance (as an example of the first neuron node) is selected as the best matching unit (BMU):

[0101]

[0102] In this way, the neuron node matching of the first image in all image pairs is completed.

[0103] S240, mapping the second image similar to the first image in each image pair to the second neuron node adjacent to the first neuron node to obtain the mapping result of each image pair, so as to obtain the mapping result of multiple image pairs.

[0104] For example, the second image corresponding to the first image in each image pair is matched to the second neuron node adjacent to the first neuron node, so that the mapping result of each image pair can be obtained, and the mapping result of all image pairs is the final mapping result.

[0105] In this way, through the self-organizing mapping network, the distance between the image and the neuron weight vector is calculated based on the competitive learning rule to determine the best matching unit, realizing efficient matching and mapping of high-dimensional image similarity data in low-dimensional space. In this way, the corresponding relationship between image similarity and topological structure is established, laying a foundation for subsequent image clustering, thereby reducing the computational complexity, improving the processing efficiency, and saving the computing resource cost. Moreover, based on the topological mapping of the self-organizing mapping network, the similar semantic information between images and similar images can be mapped to the corresponding neuron nodes, thereby maximizing the preservation of the similar semantic information between images and enhancing the similarity interpretability between images.

[0106] It should be noted that the explainability of the image is to explain the similar reasons, similar features and other semantic information of the image and other similar images.

[0107] Further, in the implementation of the present application, the weight vector information of the neuron node related to the best matching unit is updated every time the mapping is performed, so that similar images or image blocks are gathered in topologically adjacent network positions, that is, after the present mapping is completed, the weight vectors of each neuron node are saved for the mapping processing of the next pair of images. Therefore, the weight vector corresponding to each neuron node is obtained according to the mapping result of the image pair in which the first distance information is calculated last time.

[0108] In the implementation of the present application, the weight vector corresponding to each neuron node is obtained according to the mapping result of the image pair in which the first distance information is calculated last time, comprising: determining the neighborhood radius of the best matching unit corresponding to the image pair according to the mapping result of the image pair in which the first distance information is calculated last time; determining the range of neuron nodes according to the neighborhood radius with the best matching unit as the center, and updating the weight vectors of each neuron node in the range of neuron nodes.

[0109] For example, the best matching unit is taken as the center, and the range of neuron nodes is determined according to the current neighborhood radius The range of affected neuron nodes is determined based on the Gaussian neighborhood function, and the weight vector is updated.

[0110] For example, the weight vector of the first neuron node corresponding to the first image is updated by the following method:

[0111]

[0112]

[0113] wherein, is the updated weight vector of the first neuron node, is the weight vector of the first neuron node for the last mapping, is a preset decay learning rate, is a neighborhood decay function, is the feature information corresponding to the first image, is the grid coordinate of the best matching unit, is the grid coordinate of the first neuron node.

[0114] Further, the weight vector updating of each neuron node is repeated until all image pairs are mapped, or until the maximum number of iterations is reached, or the weight vector converges.

[0115] In the implementation of the application, the neighborhood radius and weight vector of the neuron node are iteratively updated, so that the neighborhood radius and weight vector are determined every time the neuron node is matched, to ensure real-time updating of the information of the neuron node and the accuracy of the next neuron node matching. Moreover, autonomous learning and optimization of the topology related to the mapping result are achieved through autonomous updating, reducing the dependence on manual parameter adjustment and reducing maintenance costs.

[0116] Next, step S300 is performed, and multi-level clustering processing is performed according to the mapping results of the plurality of image pairs to obtain a clustering result.

[0117] For example, a merging algorithm based on the distance between adjacent neuron nodes is used on the SOM mapping result, and multi-level clustering processing is performed from top to bottom to obtain a multi-level clustering result.

[0118] In the implementation of the application, as shown in Figure 4 the mapping result of the plurality of image pairs is subjected to multi-level clustering processing to obtain a clustering result, including the following steps.

[0119] S310, according to the mapping result of the plurality of image pairs and the weight vector of each neuron node, determining the second distance information of adjacent neuron nodes, and generating a distance matrix.

[0120] For example, after the self-organizing mapping is completed, the weight vector of each neuron node and the coordinates of the self-organizing mapping grid are recorded, the best matching unit ownership relationship of all images is extracted, and an image-neuron node mapping table is generated.

[0121] For all adjacent neuron nodes in the self-organizing mapping grid, the Euclidean distance of the weight vector (as an example of the second distance information) is calculated:

[0122]

[0123] wherein, is the second distance information of the kth neuron node and the mth neuron node adjacent thereto, is the weight vector of the kth neuron node, is the weight vector of the mth neuron node.

[0124] In this way, the second distance information of all adjacent neuron nodes is obtained, and a symmetric distance matrix is further obtained.

[0125] wherein, the distance matrix is in the following form:

[0126]

[0127] wherein, , …, , = .

[0128] S320, performing multi-level clustering processing on each neuron node according to the distance matrix to obtain a clustering result.

[0129] For example, based on the upper triangular information of the distance matrix, top-down hierarchical clustering processing is performed to perform multi-level clustering division.

[0130] In an implementation manner of the present application, the multi-level clustering processing can be three-level clustering processing, that is, structure category division, local texture subdivision, object feature subdivision, etc.

[0131] In the implementation manner of the present application, as shown in Figure 5 , the multi-level clustering processing is performed on each neuron node according to the distance matrix to obtain a clustering result, including the following steps.

[0132] S321, according to the distance matrix, merging the neuron nodes similar in structure into the same class to obtain a plurality of first-level clusters to obtain a first-level clustering result, the neuron nodes similar in structure being adjacent neuron nodes with second distance information less than a preset first threshold value.

[0133] For example, each neuron node in the distance matrix is regarded as an initial cluster. The second distance information of all adjacent neuron nodes is traversed, and if (as an example of the preset first threshold value), the adjacent neuron nodes meeting the condition are merged into the same cluster. In this way, a plurality of structure clusters (as an example of the first-level clusters) can be obtained, each first-level cluster containing a plurality of topologically connected neuron nodes, that is, images similar in overall structure (for example, all being sky, ground, building, etc.) are clustered to realize merging the neuron nodes corresponding to the images similar in structure into the corresponding first-level cluster to obtain the first-level clustering result.

[0134] S322, according to the distance matrix, merging the neuron nodes similar in texture in each first-level cluster into the same class to obtain a plurality of second-level clusters corresponding to each first-level cluster to obtain a second-level clustering result, the neuron nodes similar in texture being adjacent neuron nodes with second distance information less than a preset second threshold value, the second threshold value being less than the first threshold value.

[0135] For example, each neuron node in each first-level cluster included in the first-level clustering result is regarded as an independent subset. For each first-level cluster, the second distance information of each adjacent neuron node in the first-level cluster is determined. ​(As an example of the preset second threshold) adjacent neuron nodes are merged into the same cluster to obtain multiple texture clusters (as an example of the second-level cluster). Each second-level cluster contains several topologically connected neuron nodes, that is, clustering images with similar textures (such as clouds, grass, etc.) to merge the neuron nodes corresponding to images with similar textures into the corresponding second-level clusters to obtain the second-level clustering results.

[0136] S323, Based on the distance matrix, merge the neuron nodes with similar objects in each second-level cluster into the same class to obtain multiple third-level clusters corresponding to each second-level cluster, so as to obtain the third-level clustering result. The neuron nodes with similar objects are the adjacent neuron nodes whose second distance information is greater than the preset third threshold, and the third threshold is less than the second threshold.

[0137] For example, each neuron node in each second-level cluster included in the second-level clustering results is taken as an independent subset, and for each second-level cluster, the following is determined: (As an example of a preset third threshold) adjacent neuron nodes are merged into the same cluster to obtain multiple object clusters (as an example of a third-level cluster). Each third-level cluster contains several topologically connected neuron nodes, that is, clustering images with similar objects (such as a single cloud, a leaf, the same face, etc.) to merge the neuron nodes corresponding to images with similar objects into the corresponding third-level cluster.

[0138] Furthermore, for The adjacent neuron nodes are split into separate object clusters, thus obtaining multiple third-level clusters to obtain the third-level clustering results.

[0139] In this implementation, based on the SOM clustering results, a distance matrix is ​​used to first perform structural partitioning according to a first threshold, resulting in multiple first-level clusters. This preserves the large-scale visual structure of the first-level clusters. Then, a second threshold, less than the first threshold, is used to perform texture partitioning on each first-level cluster, resulting in multiple second-level clusters. This refines the second-level clustering results down to local texture features. Finally, a third threshold, less than the second threshold, is used to perform object partitioning on each second-level cluster, resulting in multiple third-level clusters. This refines the third-level clustering results down to individual object features. This top-down hierarchical clustering achieves three levels of clustering from coarse to fine, progressively dividing the image into multi-level visual similarity sets. This not only enables multi-level, refined analysis of image content but also further improves the accuracy of the clustering results, supporting retrieval and analysis at different levels of intensity.

[0140] Furthermore, in one implementation of this application, the first threshold, the second threshold, and the third threshold can be preset according to requirements.

[0141] In another implementation, the threshold is dynamically adjusted according to a similarity distribution curve of the images within the clustering level, so that the clustering neither disperses too much nor loses important differences.

[0142] Therefore, in the implementation of the present application, the first threshold, the second threshold and the third threshold are determined by the following manner: a distribution histogram of the distance matrix is determined, the first threshold is determined according to the distribution histogram; a first similarity distribution graph of the images corresponding to the neuron nodes included in the first-level clustering result is determined, the second threshold is determined according to the first similarity distribution graph; a second similarity distribution graph of the images corresponding to the neuron nodes included in the second-level clustering result is determined, and the third threshold is determined according to the second similarity distribution graph.

[0143] For example, a distribution histogram of the distance matrix is drawn, a value at a preset proportion range (for example, 70%) of the distribution histogram is taken as the first threshold, the first-level clustering is performed based on the first threshold, and the first-level clustering result is obtained.

[0144] Further, a first similarity distribution graph of the structural similarity of the images corresponding to the neuron nodes included in the first-level clustering result is drawn, a value at a preset proportion range (for example, 40%) of the first similarity distribution graph is taken as the second threshold, the second-level clustering is performed based on the second threshold, and the second-level clustering result is obtained.

[0145] Further, a second similarity distribution graph of the structural similarity of the images corresponding to the neuron nodes included in the second-level clustering result is drawn, a value at a preset proportion range (for example, 20%) of the second similarity distribution graph is taken as the third threshold, the third-level clustering is performed based on the third threshold, and the third-level clustering result is obtained.

[0146] It should be noted that for each level of cluster, a corresponding threshold can be determined, that is, the second thresholds corresponding to the first-level clusters can be the same or different, and the third thresholds corresponding to the second-level clusters can be the same or different.

[0147] Further, in another implementation of the present application, the number of levels of the multi-level clustering processing is also determined based on the distance matrix histogram.

[0148] For example, the level division is dynamically performed according to the distance matrix histogram.

[0149] In the implementation of the present application, the threshold and the merging strategy of the clustering are dynamically adjusted according to the clustering level and the visual similarity distribution, so that the threshold setting is objective, and the robustness and interpretability of the clustering are improved.

[0150] Further, in the implementation of the present application, the clustering result is also mapped into the topology structure of the self-organizing mapping network for visual display.

[0151] For example, the clustering result is mapped back to the topology of the self-organizing mapping network for visual display, so as to provide an intuitive result display for the user and realize visual analysis. In this way, the user can conveniently verify, analyze and understand the clustering result, the result interpretation time is shortened, the user experience is improved, and the overall time cost is saved.

[0152] The image clustering method provided by the implementation manner of the present application is a visual hierarchical clustering calculation method based on structural similarity (SSIM) and a self-organizing mapping (SOM) network, as shown in Figure 6 The visual hierarchical clustering calculation method based on SSIM and SOM includes the following steps:

[0153] S101, data preparation and preprocessing, inputting a source image or an image block and performing standardization preprocessing on the source image or the image block.

[0154] The standardization processing includes: size normalization, scaling all input images or image blocks to a uniform size; and brightness normalization, standardizing pixel values to eliminate the influence of light difference.

[0155] S102, SSIM calculation, for any two images or image blocks, calculating their brightness similarity, contrast similarity and structural similarity, and obtaining comprehensive structural similarity according to a preset weight fusion of the three pieces of information, traversing all image pairs to generate a structural similarity matrix.

[0156] S103, SOM clustering, inputting the structural similarity matrix into a pre-initialized two-dimensional SOM or three-dimensional SOM, using a competitive learning rule to find a best matching unit (BMU) for each first image included in the image pair, and dynamically updating the weight vector of the BMU and the neurons in the neighborhood of the BMU, so that similar images are clustered in the topological space, so that all images are mapped to the corresponding neuron nodes of the SOM network, forming a low-dimensional representation with topological preservation characteristics.

[0157] S104, hierarchical clustering, after obtaining the SOM mapping result, calculating the distance between the weight vectors of adjacent neuron nodes in the SOM network, and using a top-down strategy to perform multi-level clustering, so that the first-level clustering result retains the visual structure of the large class, the second-level clustering result is subdivided to the local texture feature, the third-level clustering result is refined to the single object feature, and a hierarchical fine clustering result is obtained.

[0158] S105, result visualization and output, mapping the multi-level clustering result back to the topology of the SOM network for visual display, outputting the final cluster label corresponding to each image and a complete hierarchical clustering tree, and providing an intuitive and interpretable analysis result for the user.

[0159] The multi-level clustering result is mapped back to the topology of the SOM network for visual display, different clusters are usually marked with different colors, and the boundaries of the clusters can be drawn. The user can intuitively understand the clustering result and the hierarchical relationship between different categories through the visual interface.

[0160] By introducing the multi-level clustering framework combining SSIM and SOM network, the above method solves the problems of insufficient structure sensitivity, single clustering granularity, and difficulty in balancing computational cost and accuracy of existing visual clustering methods. That is, using SSIM instead of traditional pixel-level metrics can capture brightness, contrast, and structural information, making the clustering result more consistent with human visual perception. By mapping high-dimensional feature data to low-dimensional space through SOM, not only the complexity of high-dimensional feature space is reduced, but also the topological relationship between data is maintained, providing intuitive visual support for subsequent branches. Different levels of clustering strength are supported, from coarse to fine multi-level clustering, breaking the limitation of single granularity, realizing hierarchical and fine visual information division, and facilitating flexible use in image retrieval, target detection, pattern discovery, and other application scenarios. The entire process has high interpretability. Compared with traditional distance metrics, SSIM can capture brightness, contrast, and structural information, making the clustering result more consistent with human visual perception. SOM not only reduces the complexity of high-dimensional feature space, but also maintains the topological relationship between data, providing intuitive visual support for subsequent analysis, effectively enhancing the credibility and usability of the results. This method avoids complex deep feature extraction, uses computationally efficient SSIM and SOM mechanisms, reduces computational resource consumption and deployment costs. At the same time, the dynamic threshold adjustment mechanism reduces human intervention and improves the adaptability of the method. This method is versatile and can be widely applied to image library management, video frame similarity detection, satellite remote sensing image classification, medical image clustering, and other fields, providing reliable technical support for efficient image clustering.

[0161] Further, the present application also provides an image retrieval method, comprising determining a to-be-retrieved image, feature information of the to-be-retrieved image, and a clustering result of an image library, wherein the clustering result of the image library is obtained based on the image clustering method provided in any one of the implementation manners of the first aspect, similar neuron nodes are matched from the clustering result according to the feature information of the to-be-retrieved image, corresponding similar images and semantic information of the similar images are obtained according to the matched neuron nodes, and the similar images and the semantic information are presented to a user.

[0162] For example, when image retrieval is needed, the feature information of the image to be retrieved is directly matched with each cluster from the clustering result, and then matched with the neuron nodes included in the matched cluster, to obtain the image corresponding to the matched neuron node as a similar image, output to the user, and output to the user the reason why the image is a similar image (has similar structure / has similar texture / has similar object).

[0163] The image clustering method and the image retrieval method provided by the implementation manner of the present application can be applied to an electronic device.

[0164] Figure 7 As shown in the structural schematic diagram of the electronic device provided by the embodiment of the present application. As shown in the structural schematic diagram of the electronic device provided by the embodiment of the present application. Figure 7 The electronic device can include a transceiver 121, a processor 122, and a memory 123.

[0165] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the technical solutions of the image clustering method in the above embodiments. The processor 122 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0166] The memory 123 is connected with the processor 122 through a system bus and completes mutual communication, and the memory 123 is used for storing computer program instructions.

[0167] By way of example, and not limitation, the memory 123 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Storage memory 123, where appropriate, can be removable or non-removable (or fixed) media. Storage memory 123, where appropriate, can be internal or external to the integrated gateway device. In a particular embodiment, the storage memory 123 is non-volatile solid-state memory. In a particular embodiment, the storage memory 123 includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these. The transceiver 121 can be used to obtain tasks to be run and configuration information of the tasks to be run.

[0168] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries and read-only libraries). The memory can include random access memory (RAM), and can also include non-volatile memory.

[0169] Further, the electronic device may, for example, be a computer, a mobile phone, a server, or the like.

[0170] It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Although the application has been described with reference to the preferred embodiments, persons skilled in the art will recognize that changes can be made in form and detail without departing from the spirit and the scope of the application. Those skilled in the art will recognize or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments described herein. It is the following claims, including any amendments thereto, that define the scope of the application.

[0171] It is to be understood that the same may be employed in the same or other embodiments without departing from the scope of the application. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0172] It should be noted that the terms "first", "second", and so on do not necessarily indicate any relative importance.

[0173] It should be noted that in the drawings, some structural or methodical features can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order can not be required. Rather, in some embodiments, these features can be arranged in a different manner and / or order than shown in the illustrative drawings. Additionally, inclusion of structural or methodical features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, these features can not be included or can be combined with other features.

[0174] While the application has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only the preferred embodiments have been shown and described and that all changes and modifications that come within the spirit of the application are desired to be protected. There are many alternative ways of implementing the application. The disclosed embodiments are illustrative of the application and not restrictive.

Claims

1. An image clustering method characterized by, The method comprises: determining a plurality of images, determining structural similarities between each of the images, obtaining a plurality of structural similarities corresponding to each of the image pairs, and generating a structural similarity matrix according to the plurality of structural similarities, each of the image pairs comprising a first image and a second image, the first image and the second image being two of the images for which the structural similarity is determined; inputting the structural similarity matrix into a self-organizing mapping network, causing the self-organizing mapping network to determine, based on a competitive learning rule, a first neuron node matching the first image in each of the image pairs according to the structural similarity matrix, map the first image to the first neuron node, and map a second image similar to the first image in each of the image pairs to a second neuron node adjacent to the first neuron node, to obtain a mapping result of each of the image pairs, so as to obtain mapping results of the plurality of image pairs; performing multi-level clustering processing according to the mapping results of the plurality of image pairs to obtain a clustering result corresponding to the plurality of image pairs.

2. The image clustering method of claim 1, wherein, The self-organizing mapping network is a two-dimensional mapping network or a three-dimensional mapping network, the self-organizing mapping network is composed of a plurality of neuron nodes, each of the neuron nodes has a corresponding weight vector, and the inputting of the structural similarity matrix into the self-organizing mapping network, causing the self-organizing mapping network to determine, based on a competitive learning rule, a first neuron node matching the first image in each of the image pairs according to the structural similarity matrix comprises: inputting the structural similarity matrix into the self-organizing mapping network, causing the self-organizing mapping network to obtain, based on a competitive learning rule, the first image, the second image included in each of the image pairs, and the structural similarity corresponding to each of the image pairs from the structural similarity matrix; calculating first distance information of the first image and the weight vector corresponding to each of the neuron nodes; selecting the neuron node corresponding to the smallest first distance information as the first neuron node matching the first image.

3. The image clustering method of claim 2, wherein, The weight vector corresponding to each of the neuron nodes is obtained according to the mapping result of the image pair in which the first distance information is calculated last time.

4. The image clustering method of claim 3, wherein, According to the mapping results of the plurality of image pairs, performing multi-level clustering processing to obtain a clustering result, comprising: determining second distance information of adjacent neuron nodes according to the mapping results of the plurality of image pairs and the weight vectors of each of the neuron nodes to generate a distance matrix; performing multi-level clustering processing on each of the neuron nodes according to the distance matrix to obtain the clustering result.

5. The image clustering method of claim 4, wherein, The multi-level clustering processing is three-level clustering processing, and performing multi-level clustering processing on each of the neuron nodes according to the distance matrix to obtain the clustering result comprises: combining the neuron nodes with structural similarity into the same class according to the distance matrix to obtain a plurality of first-level clusters, to obtain a first-level clustering result, the neuron nodes with structural similarity being adjacent neuron nodes with the second distance information less than a preset first threshold value; According to the distance matrix, the neuron nodes with similar textures in each first-level cluster are merged into the same class to obtain a plurality of second-level clusters corresponding to each first-level cluster, so as to obtain a second-level clustering result, the neuron nodes with similar textures are adjacent neuron nodes with the second distance information less than a preset second threshold, and the second threshold is less than the first threshold; According to the distance matrix, the neuron nodes with similar objects in each second-level cluster are merged into the same class to obtain a plurality of third-level clusters corresponding to each second-level cluster, so as to obtain a third-level clustering result, the neuron nodes with similar objects are adjacent neuron nodes with the second distance information greater than a preset third threshold, and the third threshold is less than the second threshold.

6. The image clustering method of claim 5, wherein, The method further comprises determining the first threshold, the second threshold and the third threshold by the following manner: determining a distribution histogram of the distance matrix, and determining the first threshold according to the distribution histogram; determining a first similarity distribution diagram of the images corresponding to the neuron nodes included in the first-level clustering result, and determining the second threshold according to the first similarity distribution diagram; determining a second similarity distribution diagram of the images corresponding to the neuron nodes included in the second-level clustering result, and determining the third threshold according to the second similarity distribution diagram.

7. The image clustering method according to any one of claims 1 to 6, characterized in that, determining the structural similarity between the images, comprising: determining the brightness similarity, the contrast similarity and the structural similarity between the images; obtaining the structural similarity between the images according to the brightness similarity, the contrast similarity and the structural similarity between the images, and the structural similarity is obtained by the following manner: wherein A is the first image, B is the second image, is the structural similarity between the first image and the second image, is the brightness similarity between the first image and the second image, is the contrast similarity between the first image and the second image, is the structural similarity between the first image and the second image, , , is a corresponding weighting factor.

8. The image clustering method according to any one of claims 1 to 6, characterized in that, The method further comprises: mapping the clustering result into the topology structure of the self-organizing mapping network for visual display.

9. The image clustering method according to any one of claims 1 to 6, characterized in that, The images are preprocessed image blocks.

10. An electronic device, comprising: comprising: a processor, and a memory connected to the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to execute the image clustering method according to any one of claims 1-9.

Images