Image clustering method and electronic device

Abstract

The embodiment of the application provides an image clustering method and an electronic device, the method comprises the following steps: determining a plurality of images, and determining the structural similarity between the images to generate a structural similarity matrix. The structural similarity matrix is input into a self-organizing mapping network, and the self-organizing mapping network finds the best matching neuron node for a first image included in each image pair based on a competitive learning rule according to the structural similarity matrix, and maps a second image included in each image pair and having structural similarity with the first image to a topologically adjacent neuron node to generate a mapping result of the plurality of image pairs. Based on the mapping result, a multi-level clustering process is performed to obtain a fine clustering result. In this way, based on the combination of structural similarity and the self-organizing mapping network, visual-sensitive similarity measurement and hierarchical clustering division are realized, the clustering fineness is improved, and the operation and maintenance cost is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image clustering method and electronic device. Background Technology

[0002] Image clustering is a key technology in computer vision, its core being the calculation of similarity between images to achieve image grouping or classification. Current image clustering methods typically calculate similarity based on pixel values ​​or feature vectors (such as Euclidean distance or cosine similarity), and usually directly in the global feature space. This approach suffers from insufficient clustering precision when using pixel values ​​or low-dimensional feature vectors, and high computational cost when using high-dimensional feature vectors. Furthermore, calculating similarity in the global feature space also suffers from insufficient clustering precision.

[0003] Therefore, existing image clustering methods suffer from insufficient image clustering precision and high cost. Summary of the Invention

[0004] This application provides an image clustering method and electronic device that can solve the problems of insufficient image clustering precision and high cost in the prior art.

[0005] To address the aforementioned technical problems, in a first aspect, embodiments of this application provide an image clustering method. This method includes: determining multiple images and determining the structural similarity between each image to obtain structural similarity corresponding to multiple image pairs; generating a structural similarity matrix based on the multiple structural similarities, wherein each image pair includes a first image and a second image, and the first image and the second image are two images determined to have structural similarity; inputting the structural similarity matrix into a self-organizing map network (SOM), causing the SOM to determine, based on a competitive learning rule, a first neuron node matching the first image in each image pair according to the structural similarity matrix; mapping the first image to the first neuron node; and mapping a second image similar to the first image in each image pair to a second neuron node adjacent to the first neuron node, thereby obtaining mapping results for each image pair, thus obtaining mapping results for multiple image pairs; and performing multi-level clustering processing based on the mapping results of the multiple image pairs to obtain clustering results corresponding to the multiple images.

[0006] By employing the above technical solution, a structural similarity matrix is ​​obtained by determining the structural similarity between images. This matrix is ​​then input into a self-organizing map (SOM) network. Based on competitive learning rules, the SOM matches adjacent neuron nodes of similar image pairs (e.g., the first and second images) according to the structural similarity matrix, resulting in mapping results for each image pair's corresponding neuron nodes. This yields mapping results for multiple image pairs. Furthermore, based on these mapping results, multi-level clustering is performed to obtain the final clustering result. Thus, in the clustering stage, multi-level clustering of multiple images based on the structural similarity matrix using the SOM achieves hierarchical clustering of similar images, making the clustering of similar images more refined. Moreover, mapping similar images based on neuron nodes preserves semantic information such as the reasons for the similarity between each image pair, enhancing the interpretability of each similar image and making it easier for users to understand why two images are similar, further refining the image clustering.

[0007] Furthermore, by combining structural similarity metrics with self-organizing mapping networks, the high labor and time costs required for data annotation are reduced. The automated multi-level clustering process reduces reliance on manual labor, lowering operation and maintenance costs. Moreover, compared to using high-dimensional feature vectors for similarity calculation, this also reduces operation and maintenance costs.

[0008] In one possible implementation of the first aspect described above, the self-organizing map network is a two-dimensional or three-dimensional mapping network, composed of multiple neuron nodes, each with a corresponding weight vector. Then, the structural similarity matrix is ​​input into the self-organizing map network, enabling it to determine the first neuron node matching the first image in each image pair based on a competitive learning rule. This includes: inputting the structural similarity matrix into the self-organizing map network, enabling it to obtain the first image, the second image, and the corresponding structural similarity of each image pair from the structural similarity matrix based on a competitive learning rule; calculating the first distance information between the first image and the weight vectors corresponding to each neuron node; and selecting the neuron node corresponding to the smallest first distance information as the first neuron node matching the first image.

[0009] By employing the aforementioned technical solution, a self-organizing map network (SOMR) based on competitive learning rules autonomously calculates the distance between the weight vectors of images and neuron nodes, and autonomously matches the best matching unit for each image. This achieves efficient matching and mapping of high-dimensional image similarity data in a low-dimensional space. In this way, a correspondence between image similarity and topological structure is established, laying the foundation for subsequent image clustering, thereby reducing computational complexity, improving processing efficiency, and saving computational resource costs.

[0010] In one possible implementation of the first aspect described above, the weight vector corresponding to each neuron node is obtained based on the mapping result of the image pair for the previous calculation of the first distance information.

[0011] Further, the weight vectors corresponding to each neuron node are obtained based on the mapping results of the image pairs for the previous calculation of the first distance information, including: determining the neighborhood radius of the best matching unit corresponding to the image pair based on the mapping results of the image pairs for the previous calculation of the first distance information; determining the range of neuron nodes based on the neighborhood radius with the best matching unit as the center, and updating the weight vectors of each neuron node within the range of the neuron node.

[0012] By employing the above technical solution, the neighborhood radius and weight vector of each neuron node are iteratively updated. This ensures that the neighborhood radius and weight vector are determined after each neuron node matching, guaranteeing real-time updates to neuron node information and ensuring the accuracy of subsequent neuron node matching. Furthermore, autonomous updating enables the autonomous learning and optimization of the topological structure related to the mapping results, reducing reliance on manual parameter tuning and lowering maintenance costs.

[0013] In one possible implementation of the first aspect above, multi-level clustering processing is performed based on the mapping results of multiple image pairs to obtain clustering results, including: determining the second distance information of each adjacent neuron node based on the mapping results of multiple image pairs and the weight vector of each neuron node, and generating a distance matrix; and performing multi-level clustering processing on each neuron node based on the distance matrix to obtain clustering results.

[0014] By employing the above technical solution, a distance matrix between neuron nodes is generated based on the mapping results and weight vectors. Image clustering analysis is then performed based on this distance matrix, transforming the complex image clustering problem into a structured node clustering problem. This reduces computational complexity, improves the efficiency of image clustering, and further ensures the accuracy of the image clustering results.

[0015] In one possible implementation of the first aspect above, the multi-level clustering process is a three-level clustering process. Based on the distance matrix, multi-level clustering is performed on each neuron node to obtain clustering results, including: merging structurally similar neuron nodes into the same class to obtain multiple first-level clusters, thus obtaining a first-level clustering result; structurally similar neuron nodes are adjacent neuron nodes whose second distance information is less than a preset first threshold; merging texture-similar neuron nodes in each first-level cluster into the same class to obtain multiple second-level clusters corresponding to each first-level cluster, thus obtaining a second-level clustering result; texture-similar neuron nodes are adjacent neuron nodes whose second distance information is less than a preset second threshold, where the second threshold is less than the first threshold; and merging object-similar neuron nodes in each second-level cluster into the same class to obtain multiple third-level clusters corresponding to each second-level cluster, thus obtaining a third-level clustering result; object-similar neuron nodes are adjacent neuron nodes whose second distance information is greater than a preset third threshold, where the third threshold is less than the second threshold.

[0016] By employing the above technical solution, a three-level clustering strategy, progressing from coarse to fine, is used to cluster data at the structural, texture, and object levels, achieving multi-level, refined analysis of image content. This not only improves the accuracy of the clustering results but also optimizes the allocation of computing resources, increases processing efficiency, and reduces overall costs through hierarchical processing.

[0017] In one possible implementation of the first aspect described above, the image clustering method further includes determining a first threshold, a second threshold, and a third threshold by: determining a distribution histogram of the distance matrix and determining the first threshold based on the distribution histogram; determining a first similarity distribution map of the images corresponding to the neuron nodes included in the first-level clustering results and determining the second threshold based on the first similarity distribution map; determining a second similarity distribution map of the images corresponding to the neuron nodes included in the second-level clustering results and determining the third threshold based on the second similarity distribution map.

[0018] By adopting the above technical solution, the clustering thresholds at each level are adaptively determined based on the data distribution histogram and similarity distribution map, ensuring the objectivity of the threshold setting. This avoids the tedious process of manually adjusting thresholds based on experience, saving human resources and time. It effectively prevents the need for re-clustering due to improper threshold settings, and the dynamic threshold adjustment method further improves the accuracy of image clustering.

[0019] In one possible implementation of the first aspect described above, determining the structural similarity between images includes: determining the brightness similarity, contrast similarity, and structural similarity between images; obtaining the structural similarity between images based on the brightness similarity, contrast similarity, and structural similarity, and the structural similarity is obtained in the following manner:

[0020]

[0021] Where A is the first image and B is the second image. For the structural similarity between the first image and the second image, The brightness similarity between the first and second images. The contrast similarity between the first and second images. The structural similarity between the first image and the second image. , , These are the corresponding weighting coefficients.

[0022] By employing the aforementioned technical solution, structural similarity indicators such as brightness similarity, contrast similarity, and structural similarity, which align with human visual perception characteristics, are used instead of simple pixel-level differences to measure image similarity. This provides more similarity references between images, resulting in higher similarity accuracy and consequently, higher image clustering accuracy. Consequently, the accuracy of image content understanding is improved, erroneous clustering caused by inaccurate similarity measurements is reduced, and the additional costs required for subsequent error correction are lowered.

[0023] In one possible implementation of the first aspect described above, the image clustering method further includes mapping the clustering results to the topology of a self-organizing map network for visualization.

[0024] By employing the above technical solution, the clustering results are visualized by mapping them back to the topology of the self-organizing map network, providing users with an intuitive display of the results. This facilitates the verification, analysis, and understanding of the clustering results, shortens the interpretation time, improves the user experience, and ultimately saves overall time costs.

[0025] In one possible implementation of the first aspect described above, the image is a preprocessed image block.

[0026] By employing the above technical solution, the image is preprocessed and segmented into image patches for further processing, focusing the analysis on local features. This enhances the ability to capture detailed features, improves the precision of clustering, and reduces computational costs.

[0027] Secondly, this application also discloses an image retrieval method, including determining the image to be retrieved, the feature information of the image to be retrieved, and the clustering results of the image library. The clustering results of the image library are obtained based on the image clustering method provided by any of the implementation methods of the first aspect. Similar neuron nodes are matched from the clustering results according to the feature information of the image to be retrieved, so as to obtain the corresponding similar images and semantic information of the similar images according to the matched neuron nodes, and the similar images and semantic information are presented to the user.

[0028] Thirdly, this application also discloses an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to enable the electronic device to implement the image clustering method provided by any of the implementations of the first aspect above.

[0029] Fourthly, this application also discloses a computer-readable storage medium storing a computer program that can be executed by a computer cluster to implement the inference method based on a neural network model provided by any of the implementations of the first aspect above, and / or the image clustering method provided by any of the implementations of the second aspect above.

[0030] Fifthly, this application also discloses a computer program product, including a computer program that, when executed by a computer cluster, implements the inference method based on a neural network model provided by any of the implementations of the first aspect above, and / or the image clustering method provided by any of the implementations of the second aspect above.

[0031] The relevant beneficial effects of the third to fifth aspects mentioned above can be found in the relevant descriptions in the first or second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0032] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0033] Figure 1 This is a schematic flowchart of an image clustering method provided in an embodiment of this application;

[0034] Figure 2 A flowchart illustrating a method for determining a structural similarity matrix provided in an embodiment of this application;

[0035] Figure 3 A flowchart illustrating a method for determining the mapping result of an image pair according to an embodiment of this application;

[0036] Figure 4A flowchart illustrating a method for determining clustering results provided in an embodiment of this application;

[0037] Figure 5 Another flowchart illustrating the method for determining clustering results provided in this application embodiment;

[0038] Figure 6 Another flowchart illustrating the image clustering method provided in this application embodiment;

[0039] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0040] As mentioned earlier, image clustering is a fundamental and crucial task in the field of computer vision and image analysis. Its goal is to automatically group a set of unlabeled images or image patches based on the similarity of their visual content. Existing image clustering methods first extract feature representations from the images, and then use distance metrics (such as Euclidean distance, cosine similarity, etc.) that depend on these feature representations (e.g., pixel values ​​or feature vectors) to directly calculate similarity in the global feature space, ultimately completing the clustering. However, these traditional processing methods have the following shortcomings when processing image data rich in structural information (such as texture and shape consistency).

[0041] Due to insufficient structural sensitivity, most existing methods rely on similarity measures based on pixel values ​​or low-dimensional embedded features. These measures struggle to accurately capture and characterize the true similarity of images in high-level semantic structures (such as texture patterns and geometric shapes), and cannot effectively express the structural consistency perceived by the human visual system, thus limiting the accuracy of similarity judgments.

[0042] The current clustering methods often perform clustering directly in the global feature space, lacking a multi-level analytical perspective. This single-granularity clustering approach ignores the hierarchical organization within the image (e.g., from low-level edge textures to mid-level local patterns and then to high-level semantic objects), making it impossible to achieve fine-grained clustering and failing to meet higher requirements for the level of detail in the clustering results.

[0043] The trade-off between computational overhead and clustering accuracy is a significant challenge. To improve accuracy, some existing methods employ complex deep neural networks to extract high-dimensional deep features. While this improves performance to some extent, the enormous computational complexity, storage overhead, and energy consumption severely limit their practical deployment and application in resource-constrained environments (such as mobile devices and edge computing scenarios), leading to a trade-off between computational efficiency and clustering accuracy.

[0044] Therefore, there is an urgent need for an image clustering method that can achieve higher precision and lower cost.

[0045] To address the aforementioned technical problems, the embodiments of this application provide an image clustering method that combines structural similarity and the clustering capability of self-organizing maps to achieve hierarchical clustering (i.e., graded clustering) of images at the visual level. This method can reduce clustering costs while making image clustering more refined.

[0046] like Figure 1 As shown, the image clustering method provided in this application includes the following steps.

[0047] S100, determine multiple images and determine the structural similarity between each image to obtain the structural similarity corresponding to multiple image pairs, and generate a structural similarity matrix based on the multiple structural similarities. Each image pair includes a first image and a second image, and the first image and the second image are two images that are determined to have structural similarity.

[0048] S200, the structural similarity matrix is ​​input into the self-organizing map network, so that the self-organizing map network determines the first neuron node that matches the first image in each image pair according to the structural similarity matrix based on the competitive learning rule, maps the first image to the first neuron node, and maps the second image in each image pair that is similar to the first image to the second neuron node adjacent to the first neuron node, so as to obtain the mapping results of each image pair, and thus obtain the mapping results of multiple image pairs.

[0049] S300: Based on the mapping results of multiple image pairs, perform multi-level clustering processing to obtain the clustering results corresponding to multiple images.

[0050] By employing the aforementioned technical solution, a structural similarity matrix is ​​obtained by determining the structural similarity between images. This matrix is ​​then input into a self-organizing map (SOM) network. Based on competitive learning rules, the SOM matches adjacent neuron nodes of similar image pairs (e.g., the first and second images) according to the structural similarity matrix, resulting in mapping results for each image pair's corresponding neuron nodes. This yields mapping results for multiple image pairs. Furthermore, based on these mapping results, multi-level clustering is performed to obtain the final clustering result. Thus, in the clustering stage, multi-level clustering of multiple images based on the structural similarity matrix using the SOM achieves hierarchical clustering of similar images, making the clustering more refined. Moreover, mapping similar images based on neuron nodes preserves semantic information such as the reasons for the similarity between image pairs, enhancing the interpretability of each similar image and making it easier for users to understand why two images are similar, thereby improving the user experience.

[0051] Furthermore, by combining structural similarity indicators with self-organizing mapping networks, the high manpower and time costs required for data annotation are reduced; the automated process of multi-level clustering reduces reliance on manual labor, further lowering operation and maintenance costs.

[0052] First, step S100 is executed to determine multiple images and the structural similarity index (SSIM) between each image, thereby obtaining the structural similarity of multiple image pairs and generating a structural similarity matrix based on the multiple structural similarities.

[0053] In one implementation of this application, each image pair includes a first image and a second image, wherein the first image and the second image are two images that are determined to have structural similarity.

[0054] In the implementation of this application, the image can be a complete image or an image block.

[0055] Furthermore, in one implementation of this application, the image is a preprocessed image or image block.

[0056] The preprocessing includes standardizing the size and brightness of each image to ensure that the size of each image or image block is uniform and the brightness is within a preset range, thus reducing the impact of lighting differences between images.

[0057] In one implementation of this application, such as Figure 2 As shown, determining the structural similarity matrix involves the following steps.

[0058] S110, determine the brightness similarity, contrast similarity, and structural similarity among the images.

[0059] For example, taking the first image as image A and the second image as image B, the brightness similarity between the two images is calculated.

[0060] In this application, the brightness similarity between images is calculated as follows:

[0061]

[0062]

[0063] in, The brightness similarity between the images. Let be the average brightness value of image A. Let be the average brightness value of image B. Let A and B be the dynamic range of pixel values. This is a constant, with a default value of 0.01.

[0064] If image A and image B are 8-bit images, then L = 255.

[0065] Furthermore, the contrast similarity between the images is calculated as follows:

[0066]

[0067]

[0068] in, The contrast similarity between the images. Let A be the standard deviation of the contrast of image A. Let B be the standard deviation of the contrast. This is a constant, with a default value of 0.03.

[0069] Furthermore, the structural similarity between the images is calculated as follows:

[0070]

[0071]

[0072] in, The structural similarity between the images, Let be the covariance of image A and image B.

[0073] S120: Based on the brightness similarity, contrast similarity, and structural similarity among the images, the structural similarity among the images is obtained.

[0074] For example, weighting coefficients corresponding to brightness similarity, contrast similarity, and structural similarity are determined, and the structural similarity between images is obtained based on the brightness similarity, contrast similarity, structural similarity, and corresponding weighting coefficients between each image.

[0075] The structural similarity between the images was obtained in the following way:

[0076]

[0077] Where A is the first image and B is the second image. For the structural similarity between the first image and the second image, The brightness similarity between the first and second images. The contrast similarity between the first and second images. The structural similarity between the first image and the second image. , , These are the corresponding weighting coefficients.

[0078] The weighting coefficients can be preset. .

[0079] S130: Based on the structural similarity between images, obtain the structural similarity of multiple image pairs, and generate a structural similarity matrix based on the multiple structural similarities.

[0080] For example, the structural similarity of each image pair is obtained based on the structural similarity between the first image and the second image, and then a structural similarity matrix is ​​generated.

[0081] The structural similarity matrix (SSIM) is a symmetric matrix that reflects the structural similarity relationship between images.

[0082] In the implementation of this application, the structural similarity matrix SSIM is as follows:

[0083] in, ... =1, = .

[0084] This application achieves similarity between images or image blocks by employing structural similarity indices such as brightness similarity, contrast similarity, and structural similarity, which conform to human visual perception characteristics. This preserves the consistency of texture, brightness, and structural information, rather than using simple pixel-level differences to measure image similarity. This makes image similarity measurement more comprehensive and accurate. It improves the accuracy of judging image content similarity, reduces erroneous clustering caused by inaccurate similarity measurement, and lowers the additional cost required for subsequent correction of erroneous results.

[0085] Next, step S200 is executed, where the structural similarity matrix is ​​input into the Self-Organizing Map (SOM) network. Based on the competitive learning rules, the SOM network determines the first neuron node that matches the first image in each image pair according to the structural similarity matrix, maps the first image to the first neuron node, and maps the second image in each image pair that is similar to the first image to the second neuron node adjacent to the first neuron node, thus obtaining the mapping results of each image pair, and thus obtaining the mapping results of multiple image pairs.

[0086] For example, the structural similarity matrix is ​​input into a self-organizing map network, and through competitive learning and topology preservation, visually similar images or image patches are mapped to adjacent neuron nodes, so as to realize the low-dimensional topological representation of high-dimensional similarity information.

[0087] It should be noted that Self-Organizing Maps (SOM) are artificial neural networks based on unsupervised learning methods. They use a competitive learning mechanism to map high-dimensional data to a low-dimensional space (two-dimensional or three-dimensional), automatically forming a topological mapping of the input data.

[0088] The self-organizing map network consists of an input layer and an output layer. The structural similarity matrix is ​​input to the input layer, and the output is a neuron grid. Each neuron corresponds to a node in the low-dimensional space, which is also called a neuron node.

[0089] The competitive learning rule includes a competition phase and a cooperation phase. In the competition phase, the input similarity matrix is ​​compared with the neuron weights, and the most similar neuron node is selected as the Best Matching Unit (BMU). In the cooperation phase, the BMU and its neighborhood weights are adjusted based on the distance between each neuron node, causing similar images to cluster in topological proximity. Each image pair selects its best matching unit through the competitive learning rule and achieves self-organization by adjusting its neighborhood weights.

[0090] Therefore, in one possible implementation of this application, the self-organizing mapping network is a two-dimensional mapping network or a three-dimensional mapping network. The self-organizing mapping network consists of multiple neuron nodes, and each neuron node has a corresponding weight vector, also known as the initial weight vector.

[0091] In the implementation method of this application, such as Figure 3 As shown, the structural similarity matrix is ​​input into the self-organizing map network, which then determines the first neuron node that matches the first image in each image pair based on the structural similarity matrix according to the competitive learning rule. This includes the following steps.

[0092] S210, the structural similarity matrix is ​​input into the self-organizing map network, so that the self-organizing map network can obtain the first image, the second image, and the structural similarity of each image pair from the structural similarity matrix based on the competitive learning rule.

[0093] For example, a two-dimensional or three-dimensional SOM network is pre-constructed, and the weight vectors of each neuron node are initially set to random distribution values. The random distribution values ​​can be implemented using a uniform distribution or a small-range Gaussian distribution. The dimension of the weight vectors of the neuron nodes is the same as the dimension of the images in each image pair included in the structural similarity.

[0094] Furthermore, image pairs are input from the structural similarity matrix to extract image pairs (including the first image) from the structural similarity matrix. Second image ) and its structural similarity value .

[0095] S220, calculate the first distance information between the first image and the weight vectors corresponding to each neuron node.

[0096] For example, using a competitive learning rule, the first image is computed. The weight vector of all neuron nodes Euclidean distance (as an example of first distance information):

[0097]

[0098] in, For the first image The weight vector of the k-th neuron node First distance information, , M represents the number of neuron nodes.

[0099] S230, select the neuron node corresponding to the smallest first distance information as the first neuron node that matches the first image.

[0100] For example, the neuron node c with the smallest distance (as an example of the first neuron node) is selected as the best matching unit (BMU):

[0101]

[0102] In this way, the neuron node matching of the first image in all image pairs is completed.

[0103] S240, map the second image in each image pair that is similar to the first image to the second neuron node adjacent to the first neuron node to obtain the mapping result of each image pair, so as to obtain the mapping result of multiple image pairs.

[0104] For example, the second image corresponding to the first image in each image pair is matched to the second neuron node adjacent to the first neuron node, thus obtaining the mapping result of each image pair. The mapping result of all image pairs is the final mapping result.

[0105] Thus, by using a self-organizing map network and calculating the distance between the image and the neuron weight vector based on competitive learning rules to determine the optimal matching unit, efficient matching and mapping of high-dimensional image similarity data in a low-dimensional space is achieved. This establishes a correspondence between image similarity and topological structure, laying the foundation for subsequent image clustering, thereby reducing computational complexity, improving processing efficiency, and saving computational resources. Furthermore, topological mapping based on the self-organizing map network can map the similar semantic information between images and similar images to corresponding neuron nodes, preserving the similar semantic information between images to the greatest extent, thus enhancing the interpretability of image similarity.

[0106] It should be noted that the interpretability of an image refers to the ability to explain the semantic information such as the reasons for the similarity and similar features between the image and other similar images.

[0107] Furthermore, in this implementation, the weight vector information of the neuron nodes associated with the best matching unit is updated after each mapping, so that similar images or image patches are clustered in topologically adjacent network positions. That is, after the current mapping is completed, the weight vectors of each neuron node are saved for the mapping processing of the next pair of images. Therefore, the weight vectors corresponding to each neuron node are obtained based on the mapping results of the image pairs for which the first distance information was calculated in the previous step.

[0108] In the implementation of this application, the weight vector corresponding to each neuron node is obtained based on the mapping result of the image pair for which the first distance information was calculated in the previous step. This includes: determining the neighborhood radius of the best matching unit corresponding to the image pair based on the mapping result of the image pair for which the first distance information was calculated in the previous step; determining the range of the neuron node based on the neighborhood radius with the best matching unit as the center, and updating the weight vector of each neuron node within the range of the neuron node.

[0109] For example, centered on the best matching unit, based on the current neighborhood radius The range of affected neuron nodes is determined based on the Gaussian neighborhood function, and the weight vector is updated.

[0110] Taking image A as the first image as an example, the weight vector of the first neuron node corresponding to the first image is updated in the following way:

[0111]

[0112]

[0113] in, This is the updated weight vector of the first neuron node. This is the weight vector from the previous mapping of the first neuron node. The preset decay learning rate, Let be the neighborhood decay function. The feature information corresponding to the first image. The grid coordinates of the best matching unit. These are the grid coordinates of the first neuron node.

[0114] Furthermore, the weight vector updates for each neuron node are repeated until all image pairs are mapped, or until the maximum number of iterations is reached, or until the weight vector changes converge.

[0115] In this implementation, the neighborhood radius and weight vector of each neuron node are iteratively updated. This ensures that the neighborhood radius and weight vector are determined after each neuron node matching, guaranteeing real-time updates to the neuron node information and ensuring the accuracy of the next neuron node matching. Furthermore, autonomous updating enables the autonomous learning and optimization of the topological structure related to the mapping result, reducing reliance on manual parameter tuning and lowering maintenance costs.

[0116] Next, step S300 is executed, and multi-level clustering is performed based on the mapping results of multiple image pairs to obtain the clustering results.

[0117] For example, on the SOM mapping results, a merging algorithm based on the distance between adjacent neuron nodes is used to perform multi-level clustering from top to bottom to obtain multi-level clustering results.

[0118] In the implementation method of this application, such as Figure 4 As shown, based on the mapping results of multiple image pairs, multi-level clustering is performed to obtain the clustering results, including the following steps.

[0119] S310: Based on the mapping results of multiple image pairs and the weight vector of each neuron node, determine the second distance information of each adjacent neuron node and generate a distance matrix.

[0120] For example, after completing the self-organizing map, the weight vector of each neuron node is recorded. and its coordinates in the self-organizing map mesh Extract the best matching unit affiliation relationship from all images and generate an image-neuron node mapping table.

[0121] For all adjacent neuron nodes in the self-organizing map mesh, calculate the Euclidean distance of the weight vectors (as an example of second distance information):

[0122]

[0123] in, This represents the second distance information between the k-th neuron node and its neighboring m-th neuron node. Let be the weight vector of the k-th neuron node. Let be the weight vector of the m-th neuron node.

[0124] In this way, the second distance information of all adjacent neuron nodes is obtained, and thus a symmetric distance matrix is ​​obtained.

[0125] The distance matrix has the following form:

[0126]

[0127] in, ... , = .

[0128] S320: Based on the distance matrix, perform multi-level clustering on each neuron node to obtain the clustering result.

[0129] For example, top-down hierarchical clustering is performed based on the upper triangular information of the distance matrix to achieve multi-level clustering.

[0130] In one implementation of this application, the multi-level clustering process can specifically be a three-level clustering process, namely, performing structural category division, local texture subdivision, and object feature subdivision.

[0131] In the implementation method of this application, such as Figure 5 As shown, based on the distance matrix, multi-level clustering is performed on each neuron node to obtain the clustering results, including the following steps.

[0132] S321, Based on the distance matrix, structurally similar neuron nodes are merged into the same class to obtain multiple first-level clusters, thus obtaining the first-level clustering result. Structurally similar neuron nodes are adjacent neuron nodes whose second distance information is less than a preset first threshold.

[0133] For example, each neuron node in the distance matrix is ​​considered as an initial cluster. The second distance information of all neighboring neurons is traversed, if... (As an example of the preset first threshold), adjacent neuron nodes that meet the conditions are merged into the same cluster. In this way, multiple structural clusters can be obtained (as an example of the first-level cluster). Each first-level cluster contains several topologically connected neuron nodes. That is, clustering images with similar overall structures (e.g., all are sky, ground, buildings, etc.) is performed to merge the neuron nodes corresponding to images with similar structures into the corresponding first-level clusters to obtain the first-level clustering results.

[0134] S322, based on the distance matrix, merge the neuron nodes with similar textures in each first-level cluster into the same class to obtain multiple second-level clusters corresponding to each first-level cluster, so as to obtain the second-level clustering result. The neuron nodes with similar textures are the adjacent neuron nodes whose second distance information is less than a preset second threshold, and the second threshold is less than the first threshold.

[0135] For example, each neuron node in each first-level cluster included in the first-level clustering results is taken as an independent subset. For each first-level cluster, determine... (As an example of the preset second threshold) adjacent neuron nodes are merged into the same cluster to obtain multiple texture clusters (as an example of the second-level cluster). Each second-level cluster contains several topologically connected neuron nodes, that is, clustering images with similar textures (such as clouds, grass, etc.) to merge the neuron nodes corresponding to images with similar textures into the corresponding second-level clusters to obtain the second-level clustering results.

[0136] S323, Based on the distance matrix, merge the neuron nodes with similar objects in each second-level cluster into the same class to obtain multiple third-level clusters corresponding to each second-level cluster, so as to obtain the third-level clustering result. The neuron nodes with similar objects are the adjacent neuron nodes whose second distance information is greater than the preset third threshold, and the third threshold is less than the second threshold.

[0137] For example, each neuron node in each second-level cluster included in the second-level clustering results is taken as an independent subset, and for each second-level cluster, the following is determined: (As an example of a preset third threshold) adjacent neuron nodes are merged into the same cluster to obtain multiple object clusters (as an example of a third-level cluster). Each third-level cluster contains several topologically connected neuron nodes, that is, clustering images with similar objects (such as a single cloud, a leaf, the same face, etc.) to merge the neuron nodes corresponding to images with similar objects into the corresponding third-level cluster.

[0138] Furthermore, for The adjacent neuron nodes are split into separate object clusters, thus obtaining multiple third-level clusters to obtain the third-level clustering results.

[0139] In this implementation, based on the SOM clustering results, a distance matrix is ​​used to first perform structural partitioning according to a first threshold, resulting in multiple first-level clusters. This preserves the large-scale visual structure of the first-level clusters. Then, a second threshold, less than the first threshold, is used to perform texture partitioning on each first-level cluster, resulting in multiple second-level clusters. This refines the second-level clustering results down to local texture features. Finally, a third threshold, less than the second threshold, is used to perform object partitioning on each second-level cluster, resulting in multiple third-level clusters. This refines the third-level clustering results down to individual object features. This top-down hierarchical clustering achieves three levels of clustering from coarse to fine, progressively dividing the image into multi-level visual similarity sets. This not only enables multi-level, refined analysis of image content but also further improves the accuracy of the clustering results, supporting retrieval and analysis at different levels of intensity.

[0140] Furthermore, in one implementation of this application, the first threshold, the second threshold, and the third threshold can be preset according to requirements.

[0141] In another implementation, the threshold is dynamically adjusted based on the similarity distribution curve of images within the cluster level, so that the clustering is neither too scattered nor loses important differences.

[0142] Therefore, in the implementation of this application, the first threshold, the second threshold, and the third threshold are determined in the following ways: determining the distribution histogram of the distance matrix, and determining the first threshold based on the distribution histogram; determining the first similarity distribution map of the images corresponding to the neuron nodes included in the first-level clustering results, and determining the second threshold based on the first similarity distribution map; determining the second similarity distribution map of the images corresponding to the neuron nodes included in the second-level clustering results, and determining the third threshold based on the second similarity distribution map.

[0143] For example, a distribution histogram is plotted on the distance matrix, and the first threshold is taken from the distribution histogram at a preset proportion range (e.g., 70%), so as to perform first-level clustering based on the first threshold and obtain the first-level clustering result.

[0144] Furthermore, the structural similarity of the images corresponding to the neuron nodes included in the first-level clustering results is plotted to obtain a first similarity distribution map. Based on the first similarity distribution map, a second threshold is set at a preset proportion range (e.g., 40%). Second-level clustering is then performed based on the second threshold to obtain the second-level clustering results.

[0145] Furthermore, the structural similarity of the images corresponding to the neuron nodes included in the second-level clustering results is plotted to obtain a second similarity distribution map. Based on the second similarity distribution map, a third threshold is set at a preset proportion range (e.g., 20%). Third-level clustering is then performed based on the third threshold to obtain the third-level clustering results.

[0146] It should be noted that for each level of cluster, a corresponding threshold can be determined. That is, the second threshold corresponding to each first-level cluster can be the same or different, and the third threshold corresponding to each second-level cluster can be the same or different.

[0147] Furthermore, in another implementation of this application, the number of levels in the multi-level clustering process is determined based on the distance matrix histogram.

[0148] For example, the hierarchy is dynamically divided based on the distance matrix histogram.

[0149] In the implementation method of this application, the clustering threshold and merging strategy are dynamically adjusted according to the clustering level and visual similarity distribution, so that the threshold setting is objective and the robustness and interpretability of clustering are improved.

[0150] Furthermore, in the implementation of this application, the clustering results are also mapped to the topology of the self-organizing map network for visualization.

[0151] For example, the clustering results are mapped back to the topology of the self-organizing network for visualization, providing users with an intuitive display of results and enabling visual analysis. This facilitates the verification, analysis, and understanding of the clustering results, shortens the interpretation time, improves the user experience, and ultimately saves overall time costs.

[0152] The image clustering method provided in this application is a computational method for visual hierarchical clustering based on structural similarity (SSIM) and self-organizing map (SOM) networks, such as... Figure 6 As shown, the computational method for visual hierarchical clustering based on SSIM and SOM includes the following steps:

[0153] S101, Data Preparation and Preprocessing: Input source image or image patch and perform standardized preprocessing on it.

[0154] The standardization process includes: size normalization, which scales all input images or image blocks to a uniform size; and brightness normalization, which standardizes pixel values ​​to eliminate the influence of lighting differences.

[0155] S102, SSIM calculation, calculates the brightness similarity, contrast similarity and structural similarity of any two images or image blocks, and fuses these three pieces of information according to preset weights to obtain a comprehensive structural similarity. It iterates through all image pairs to generate a structural similarity matrix.

[0156] S103, SOM clustering, inputs the structural similarity matrix into a pre-initialized two-dimensional or three-dimensional SOM, and adopts a competitive learning rule to find the best matching unit (BMU) for the first image included in each image pair, and dynamically updates the weight vectors of the BMU and its neighborhood neurons, so that similar images are clustered in the topological space, so that all images are mapped to the corresponding neurons of the SOM network, forming a low-dimensional representation with topology preservation properties.

[0157] S104, hierarchical clustering: After obtaining the SOM mapping result, the distance between the weight vectors of adjacent neuron nodes in the SOM network is calculated. A top-down strategy is used to perform multi-level clustering so that the first-level clustering result retains the visual structure of the major categories, the second-level clustering result is subdivided into local texture features, and the third-level clustering result is refined into individual object features, resulting in hierarchical fine clustering results.

[0158] S105, Results Visualization and Output, maps the multi-level clustering results back to the topology of the SOM network for visualization, outputs the final cluster label and complete hierarchical clustering tree corresponding to each image, and provides users with intuitive and interpretable analysis results.

[0159] This method maps the multi-level clustering results back to the topology of the SOM network for visualization. Different clusters are typically marked with different colors, and cluster boundaries can be drawn. Users can intuitively understand the clustering results and the hierarchical relationships between different categories through this visualization interface.

[0160] By employing the above method and introducing a multi-level clustering framework combining SSIM and SOM networks, the problems of insufficient structural sensitivity, limited clustering granularity, and difficulty in balancing computational cost and accuracy in existing visual clustering methods are addressed. Specifically, SSIM replaces traditional pixel-level measurements, capturing brightness, contrast, and structural information, making the clustering results more consistent with human visual perception. SOM maps high-dimensional feature data to a low-dimensional space, reducing the complexity of the high-dimensional feature space while preserving the topological relationships between data, providing intuitive visualization support for subsequent branches. It supports different levels of clustering strength, from coarse to fine multi-level clustering, breaking through the limitations of single-granularity clustering and achieving hierarchical and refined visual information segmentation, which is convenient for flexible use in application scenarios such as image retrieval, object detection, and pattern discovery. The entire process is highly interpretable. Compared with traditional distance metrics, SSIM can capture brightness, contrast, and structural information, making the clustering results more consistent with human-like facial visual perception. SOM not only reduces the complexity of high-dimensional feature space but also maintains the topological relationship between data, providing intuitive visualization support for subsequent analysis and effectively enhancing the credibility and usability of the results. This method avoids complex deep feature extraction and adopts the computationally efficient SSIM and SOM mechanisms, reducing computational resource consumption and deployment costs. At the same time, the dynamic threshold adjustment mechanism reduces manual intervention and improves the method's adaptability. This method is highly versatile and can be widely applied to multiple fields such as image library management, video frame similarity detection, satellite remote sensing image classification, and medical image clustering, providing reliable technical support for achieving efficient image clustering.

[0161] Furthermore, this application also provides an image retrieval method, including determining an image to be retrieved, feature information of the image to be retrieved, and clustering results of an image library, wherein the clustering results of the image library are obtained based on an image clustering method provided by any implementation of the first aspect, matching similar neuron nodes from the clustering results according to the feature information of the image to be retrieved, obtaining corresponding similar images and semantic information of the similar images according to the matched neuron nodes, and presenting the similar images and semantic information to the user.

[0162] For example, when image retrieval is required, the image to be retrieved is directly matched with each cluster based on the feature information of the clustering results. Then, it is matched with the neuron nodes included in the matched clusters to obtain the image corresponding to the matched neuron node. This image is then output to the user as a similar image, and the reason why the image is a similar image is also output to the user (it has similar structure / it has similar texture / it has similar objects).

[0163] The image clustering and image retrieval methods provided in this application can be applied to electronic devices.

[0164] Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0165] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the technical solution of the image clustering method in the above embodiments. Processor 122 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital data processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0166] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0167] For example, and not as a limitation, memory 123 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 123 may include removable or non-removable (or fixed) media. Where appropriate, memory 123 may be internal or external to the integrated gateway device. In a particular embodiment, memory 123 is non-volatile solid-state memory. In a particular embodiment, memory 123 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only ROM (PROM), an erasable programmable read-only ROM (EPROM), an electrically erasable programmable read-only ROM (EEPROM), an electrically alterable read-only ROM (EAROM), or flash memory, or a combination of two or more of these. Transceiver 121 can be used to obtain the task to be run and its configuration information.

[0168] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0169] Furthermore, the electronic device can be, for example, a computer, a mobile phone, a server, or other electronic devices.

[0170] It should be noted that, in addition to the specific embodiments described above, those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Although the description of this application is presented in conjunction with preferred embodiments, this does not mean that the features of this invention are limited to this implementation. On the contrary, the purpose of describing the invention in conjunction with the implementation is to cover other options or modifications that may be derived from this application. To provide a thorough understanding of this application, many specific details are included in the above description, and this application may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of this application, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0171] It should be noted that in this specification, similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0172] It should be noted that the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0173] It should be noted that some structural or methodological features may be shown in the accompanying drawings in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0174] Although this application has been illustrated and described with reference to certain preferred embodiments, those skilled in the art should understand that the above description is a further detailed explanation of the application in conjunction with specific implementations, and should not be construed as limiting the specific implementation of the application to these descriptions. Those skilled in the art can make various changes in form and detail, including some simple deductions or substitutions, without departing from the spirit and scope of this application.

Claims

1. An image clustering method, characterized in that, The method includes: Multiple images are identified, and the structural similarity between each image is determined to obtain the structural similarity corresponding to multiple image pairs. A structural similarity matrix is ​​generated based on the multiple structural similarities. Each image pair includes a first image and a second image, wherein the first image and the second image are two images that are determined to have the structural similarity. The structural similarity matrix is ​​input into a self-organizing map network, which, based on a competitive learning rule, determines a first neuron node that matches the first image in each image pair according to the structural similarity matrix, maps the first image to the first neuron node, and maps a second image in each image pair that is similar to the first image to a second neuron node adjacent to the first neuron node, thereby obtaining the mapping results of each image pair, and thus obtaining the mapping results of the multiple image pairs. Based on the mapping results of the multiple image pairs, multi-level clustering processing is performed to obtain the clustering results corresponding to the multiple images.

2. The image clustering method according to claim 1, characterized in that, The self-organizing mapping network is a two-dimensional or three-dimensional mapping network, composed of multiple neuron nodes, each with a corresponding weight vector. The structural similarity matrix is ​​input into the self-organizing mapping network, enabling it to determine, based on a competitive learning rule and the structural similarity matrix, the first neuron node matching the first image in each image pair, including: The structural similarity matrix is ​​input into a self-organizing map network, which, based on a competitive learning rule, obtains the first image, the second image, and the structural similarity corresponding to each image pair from the structural similarity matrix. Calculate the first distance information between the first image and the weight vector corresponding to each neuron node; The neuron node corresponding to the smallest first distance information is selected as the first neuron node that matches the first image.

3. The image clustering method according to claim 2, characterized in that, The weight vector corresponding to each neuron node is obtained based on the mapping result of the image pair in the previous calculation of the first distance information.

4. The image clustering method according to claim 3, characterized in that, Based on the mapping results of the multiple image pairs, multi-level clustering processing is performed to obtain the clustering results, including: Based on the mapping results of the multiple image pairs and the weight vector of each neuron node, the second distance information of each adjacent neuron node is determined, and a distance matrix is ​​generated; Based on the distance matrix, multi-level clustering is performed on each neuron node to obtain the clustering result.

5. The image clustering method according to claim 4, characterized in that, The multi-level clustering process is a three-level clustering process. Based on the distance matrix, multi-level clustering is performed on each of the neuron nodes to obtain the clustering results, including: Based on the distance matrix, the structurally similar neuron nodes are merged into the same class to obtain multiple first-level clusters, thereby obtaining the first-level clustering result. The structurally similar neuron nodes are the adjacent neuron nodes whose second distance information is less than a preset first threshold. Based on the distance matrix, the neuron nodes with similar textures in each first-level cluster are merged into the same class to obtain multiple second-level clusters corresponding to each first-level cluster, so as to obtain the second-level clustering result. The neuron nodes with similar textures are the adjacent neuron nodes whose second distance information is less than a preset second threshold, and the second threshold is less than the first threshold. Based on the distance matrix, the neuron nodes with similar objects in each second-level cluster are merged into the same class to obtain multiple third-level clusters corresponding to each second-level cluster, so as to obtain the third-level clustering result. The neuron nodes with similar objects are the adjacent neuron nodes whose second distance information is greater than a preset third threshold, and the third threshold is less than the second threshold.

6. The image clustering method according to claim 5, characterized in that, The method further includes determining the first threshold, the second threshold, and the third threshold in the following manner: Determine the distribution histogram of the distance matrix, and determine the first threshold based on the distribution histogram; A first similarity distribution map of the images corresponding to the neuron nodes included in the first-level clustering results is determined, and a second threshold is determined based on the first similarity distribution map; Determine the second similarity distribution map of the image corresponding to the neuron nodes included in the second-level clustering result, and determine the third threshold based on the second similarity distribution map.

7. The image clustering method according to any one of claims 1-6, characterized in that, Determining the structural similarity between the images includes: Determine the brightness similarity, contrast similarity, and structural similarity among the images; The structural similarity between the images is obtained based on the brightness similarity, contrast similarity, and structural similarity among the images, and the structural similarity is obtained in the following manner: Wherein, A is the first image, and B is the second image. The structural similarity between the first image and the second image, The brightness similarity between the first image and the second image. The contrast similarity between the first image and the second image. The structural similarity between the first image and the second image. , , These are the corresponding weighting coefficients.

8. The image clustering method according to any one of claims 1-6, characterized in that, The method further includes: The clustering results are mapped to the topology of the self-organizing map network for visualization.

9. The image clustering method according to any one of claims 1-6, characterized in that, The image is a pre-processed image block.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory to perform the image clustering method as described in any one of claims 1-9.

Images