Image clustering method, electronic equipment and computer readable storage medium

Abstract

The invention discloses an image clustering method, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining feature similarity of at least two to-be-clustered images on at least one feature dimension; performing statistical processing on the obtained historical image clustering result to obtain a feature similarity interval on at least one feature dimension and a clustering probability value of each feature similarity interval; determining a target clustering probability value of the at least two to-be-clustered images according to a size relationship between the feature similarity and the feature similarity interval; and performing clustering processing on the to-be-clustered images according to the target clustering probability value to obtain an image clustering result. Therefore, the clustering accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image clustering method, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Image clustering processing refers to classifying images according to the similarity between images. In actual applications, images of various angles of the same object can be aggregated in the same class to provide good support for subsequent applications.

[0003] Current image clustering is mainly based on the feature similarity of each image for clustering. The commonly used way is to cluster images with feature similarity greater than a preset similarity into a class. However, this way depends on the feature extraction quality of the images to be clustered, and it is difficult to process images to be clustered with poor feature extraction quality. SUMMARY

[0004] The technical problem solved by the present application is to provide an image clustering method, an electronic device and a computer readable storage medium, which can improve the accuracy of image clustering.

[0005] To solve the above technical problem, one technical solution adopted by the present application is to provide an image clustering method, which comprises: obtaining the feature similarity of at least two images to be clustered in at least one feature dimension; statistically processing the obtained historical image clustering results to obtain the feature similarity interval in at least one feature dimension and the clustering probability value of each feature similarity interval; determining the target clustering probability value of the at least two images to be clustered according to the size relationship between the feature similarity and the feature similarity interval; and clustering each image to be clustered according to the target clustering probability value to obtain an image clustering result.

[0006] To solve the above technical problem, another technical solution adopted by the present application is to provide an electronic device comprising a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above image clustering method.

[0007] To solve the above technical problem, another technical solution adopted by the present application is to provide a computer readable storage medium comprising program data, wherein the program data is executed by a processor to implement the above image clustering method.

[0008] Different from the prior art, the image clustering method provided in the application obtains feature similarity of at least two images to be clustered in at least one feature dimension; performs statistical processing on the obtained historical image clustering result to obtain feature similarity intervals in the at least one feature dimension and clustering probability values of each feature similarity interval; determines a target clustering probability value of the at least two images to be clustered according to the size relationship between the feature similarity and the feature similarity interval; and performs clustering processing on each image to be clustered according to the target clustering probability value to obtain an image clustering result. Thus, the clustering probability values in different situations are determined according to the historical image clustering result, and the clustering probability values are used to assist the clustering processing of the current images to be clustered, so that the clustering accuracy can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can be obtained from these drawings without creative labor.

[0010] Figure 1 is a flow diagram of an exemplary embodiment of the image clustering method shown in the present application;

[0011] Figure 2 is a flow diagram of another exemplary embodiment of the image clustering method shown in the present application;

[0012] Figure 3 is a structural diagram of an exemplary embodiment of the image clustering device shown in the present application;

[0013] Figure 4 is a structural diagram of an embodiment of the electronic device provided in the present application;

[0014] Figure 5 is a structural diagram of an embodiment of the computer readable storage medium provided in the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely in the present application with reference to the drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0016] First of all, it needs to be pointed out that the current image clustering is mostly dependent on a single feature dimension, such as facial features or body features, and the image clustering result is obtained by clustering in a single feature dimension. For image clustering dependent on multiple feature dimensions, it is usually assumed that the features of each feature dimension are complete and available, and it is difficult to deal with the problem of feature missing.

[0017] Based on this, the embodiment of the present application proposes an image clustering method, an electronic device and a computer readable storage medium, which can assist the clustering of the to-be-clustered images in combination with the historical image clustering results to improve the clustering effect. For details, please refer to Figure 1 , Figure 1 is a flow diagram of an exemplary embodiment of the image clustering method shown in the present application.

[0018] The execution subject of the image clustering method can be a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (User Equipment, UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The execution subject of the image clustering method can also be an image clustering apparatus. In some possible implementation manners, the image clustering method can be realized by a processor calling computer readable instructions stored in a memory.

[0019] Specifically, the image clustering method of the embodiment includes the following steps:

[0020] S110: Obtain the feature similarity of at least two to-be-clustered images in at least one feature dimension.

[0021] The to-be-clustered image refers to an image that needs to be clustered. Exemplarily, the to-be-clustered image can be an image selected from an image set, for example, it can be a randomly selected image in the image set, or an image with the top N image quality in the image set, etc. The to-be-clustered image can also be an image collected by an image collection device, or a video frame extracted from a video stream collected by a video collection device. The to-be-clustered image includes one or more target objects to be clustered, at this time the image clustering is actually clustering the target objects in the to-be-clustered image, and the to-be-clustered images containing the same target object are clustered into a class. The at least two to-be-clustered images can be two or more to-be-clustered images, and the embodiment takes two to-be-clustered images as an example. The image clustering apparatus acquires multiple collected images; combines the multiple collected images in pairs to obtain multiple collected image groups; and randomly selects a collected image group from the multiple collected image groups as the to-be-clustered images.

[0022] The feature dimension can also be referred to as a feature view, which represents different image features extracted from different perspectives. The at least one feature dimension includes one or more feature dimensions. As an example, the feature dimensions can include, but are not limited to, facial features, body features, attribute features, etc., that is, features of three feature dimensions of facial features, body features, and attribute features of each image to be clustered are extracted. Among them, the image clustering device can use deep learning models in different modal scenarios to extract features of each feature dimension, for example, using a facial feature extraction model to extract d1-dimensional facial features f in each image to be clustered, using a body feature extraction model to extract d2-dimensional body features q in each image to be clustered, and using an attribute feature extraction model to extract s attribute features g in each image to be clustered, which can be represented as 1 ,g 2 ,…,g s .

[0023] The feature similarity can be the similarity between the features of the plurality of images to be clustered in at least one feature dimension. Due to the independence between the features, the extracted multi-dimensional features cannot be compared with each other, for example, the feature similarity between facial features and body features cannot be calculated with each other in general, so after the image clustering device extracts the features of the images to be clustered from different feature dimensions, it needs to calculate the feature similarity between each image to be clustered in the same feature dimension. Among them, when calculating the feature similarity between each image to be clustered, the feature similarity between each pair of images to be clustered can be calculated.

[0024] After obtaining at least two images to be clustered, the image clustering device extracts features of each image to be clustered from different feature dimensions to obtain features of each image to be clustered in at least one feature dimension; then the feature similarity between each image to be clustered in the same feature dimension is calculated respectively. It should be noted that the feature dimensions of each image to be clustered can be missing, and it is not required that each image to be clustered can extract features from each feature dimension. For example, facial features can be missing due to the target object wearing a mask or long-distance shooting, body features can be incomplete due to motion blur or occlusion, and attribute features can not be accurately extracted due to image resolution or annotation noise.

[0025] S120: statistically processing the obtained historical image clustering results to obtain a feature similarity interval in at least one feature dimension and a clustering probability value of each feature similarity interval.

[0026] The historical image clustering result is an image clustering result earlier in time than the image to be clustered. The historical image clustering result can include the feature similarity of each historical image in at least one feature dimension and the result of whether each historical image is clustered into a class. For example, the historical image clustering result can be obtained from a preset image clustering library. The preset image clustering library can include pre-labeled historical image clustering results and / or image clustering results obtained by the image clustering method of the present embodiment. As an example, the features of N historical images are obtained, and the features of the i-th historical image are represented as When the at least one feature dimension includes three feature dimensions, and the second feature dimension is an attribute feature dimension including a plurality of sub-attribute feature dimensions, the feature similarity of the i-th historical image and the j-th historical image in the at least one feature dimension can be represented as:

[0027]

[0028] wherein F i,j represents the feature similarity of the i-th historical image and the j-th historical image in the first feature dimension, f i represents the feature of the i-th historical image in the first feature dimension, f j represents the feature of the j-th historical image in the first feature dimension; represents the feature similarity of the i-th historical image and the j-th historical image in the k-th sub-attribute feature dimension of the second feature dimension, represents the feature of the i-th historical image in the k-th sub-attribute feature dimension of the second feature dimension, represents the feature of the j-th historical image in the k-th sub-attribute feature dimension of the second feature dimension; q i,j represents the feature similarity of the i-th historical image and the j-th historical image in the third feature dimension, q i represents the feature of the i-th historical image in the third feature dimension, q j represents the feature of the j-th historical image in the third feature dimension; Lut f and Lut q represent mapping operations on the similarity, depending on whether the respective algorithm team is to be processed. And it is known whether any pair of historical images in the N historical images belongs to the same class: a i,j = 1 indicates that the i-th historical image and the j-th historical image belong to the same class, a i,j = 0 indicates that the i-th historical image and the j-th historical image do not belong to the same class.

[0029] For N historical images, N*(N-1) / 2 historical image pairs can be formed, and the historical image pair in the embodiment includes any two historical images. In some embodiments, the clustering result of the historical images can be counted only once to obtain the historical image clustering result, and the historical image clustering result can be permanently retained in the database and indexed for repeated and rapid acquisition. Generally, the historical image clustering result will not change, and the historical image clustering result will be updated only in the case of change of the feature extraction model resulting in change of the feature, or increase of the number of historical images.

[0030] The feature similarity interval can be a range of feature similarity. In some embodiments, the historical image clustering result can be statistically processed to obtain the feature similarity interval in at least one feature dimension. In other embodiments, the feature similarity interval in at least one feature dimension can also be obtained by predefining the interval division of the feature similarity from 0-1.

[0031] The clustering probability value refers to the probability of clustering the historical image pair into a class in the corresponding feature similarity interval. In some embodiments, the proportion of clustering the historical image pair into a class in each feature similarity interval can be counted to obtain the clustering probability value. In other embodiments, the ratio between the number of clustering the historical image pair into a class in a feature similarity interval and the total number of clustering the historical image pair into a class in all feature similarity intervals can also be counted to obtain the clustering probability value of the corresponding feature similarity interval.

[0032] S130: determining the target clustering probability value of the at least two images to be clustered according to the size relationship between the feature similarity and the feature similarity interval.

[0033] The size relationship is used to represent whether the feature similarity is in the feature similarity interval. Exemplarily, the relationship between the feature similarity and the interval minimum value and the interval maximum value of the feature similarity interval can be judged to confirm the size relationship between the feature similarity and the feature similarity interval. Specifically, when the feature similarity is less than the interval minimum value, it means that the feature similarity is less than the feature similarity interval; when the feature similarity is greater than the interval maximum value, it means that the feature similarity is greater than the feature similarity interval; and when the feature similarity is greater than or equal to the interval minimum value and less than or equal to the interval maximum value, it means that the feature similarity is in the feature similarity interval.

[0034] The target clustering probability value refers to the probability of clustering the at least two images to be clustered into a class. Exemplarily, the image clustering device selects the clustering probability value satisfied by the images to be clustered from the clustering probability values of each feature similarity interval according to the size relationship between the feature similarity and the feature similarity interval; and then determines the target clustering probability value based on the satisfied clustering probability value.

[0035] S140: clustering processing is performed on each of the to-be-clustered images according to the target clustering probability value, to obtain an image clustering result.

[0036] The image clustering result includes a result of whether each of the to-be-clustered images is clustered into a class. In some embodiments, the image clustering result of each of the to-be-clustered images can be determined according to a size relationship between the target clustering probability value and a preset clustering probability threshold. In other embodiments, a plurality of acquisition images are combined two by two to obtain acquisition image groups, for example, the acquisition image groups can be obtained by matrix combination, and the acquisition images in each of the acquisition image groups are determined as to-be-clustered images; at this time, a difference value between the preset value and the target clustering probability value can be determined as a distance value of each of the acquisition image groups, and after the distance values of all the acquisition image groups are obtained, a clustering algorithm is used to cluster the distance values of the plurality of to-be-clustered image groups, so as to obtain the clustering result of each of the to-be-clustered images. The preset value can be 1, because the higher the target clustering probability value is, the higher the possibility that the to-be-clustered images are clustered into a class is, and therefore the distance between each of the to-be-clustered images should be smaller, so the difference value between the preset value and the target clustering probability value is used as the distance value between each of the to-be-clustered images. The clustering algorithm can be a DBSCAN clustering algorithm (Density-Based Spatial Clustering of Applications with Noise) or a K-means clustering algorithm.

[0037] After the image clustering result is obtained, the historical image clustering result can be updated based on the image clustering result, and the updated historical image clustering result can be used to assist image clustering of a next to-be-clustered image.

[0038] It can be seen that the image clustering method in the embodiments of the present application obtains feature similarity of at least two to-be-clustered images in at least one feature dimension; performs statistical processing on the obtained historical image clustering result, to obtain a feature similarity interval in the at least one feature dimension and a clustering probability value of each feature similarity interval; determines a target clustering probability value of the at least two to-be-clustered images according to a size relationship between the feature similarity and the feature similarity interval; and performs clustering processing on each of the to-be-clustered images according to the target clustering probability value, to obtain an image clustering result. Thus, the clustering probability value in different situations is determined based on the historical image clustering result, and the clustering probability value is used to assist clustering processing of the current to-be-clustered image, so that the clustering accuracy can be improved.

[0039] Further, the historical image clustering result includes feature similarity of each historical image pair and clustering state of each historical image pair, and the step S120 of performing statistical processing on the obtained historical image clustering result to obtain feature similarity interval in at least one feature dimension and clustering probability value of each feature similarity interval can include: dividing the feature similarity of each historical image pair in at least one feature dimension to obtain at least one feature similarity interval, wherein the historical image pair includes at least two historical images; and determining the clustering probability value of each feature similarity interval according to the clustering state of each historical image pair in the feature similarity interval. Thus, the clustering probability values of different feature similarity intervals in at least one feature dimension are calculated, and more comprehensive and detailed clustering probability values can be obtained.

[0040] The historical image pair includes at least two historical images, for example, two or more historical images. In this embodiment, the historical image pair includes two historical images as an example. For example, the image clustering device can obtain N historical image pairs, and each historical image pair includes two historical images.

[0041] In some embodiments, after obtaining the feature similarity of each historical image pair, the image clustering device divides the feature similarity of each historical image pair in each feature dimension to obtain at least one feature similarity interval. For example, the feature dimensions include a first feature dimension, a second feature dimension, and a third feature dimension, the feature similarity of each historical image pair in the first feature dimension is divided into a feature similarity interval in the first feature dimension, the feature similarity of each historical image pair in the second feature dimension is divided into a feature similarity interval in the second feature dimension, and the feature similarity of each historical image pair in the third feature dimension is divided into a feature similarity interval in the third feature dimension. The interval division process can be to obtain the maximum value and the minimum value of the feature similarity in each feature dimension, and to comprehensively divide according to the distribution of the feature similarity in the interval between the minimum value and the maximum value to obtain multiple feature similarity intervals in each feature dimension.

[0042] In other embodiments, after obtaining the feature similarity of each historical image pair, the image clustering device can first divide the feature similarity of each historical image pair in each feature dimension to obtain the initial feature similarity interval in each feature dimension, and then combine the initial feature similarity intervals in each feature dimension to obtain at least one feature similarity interval.

[0043] As an example, the at least one feature dimension includes a first feature dimension and a second feature dimension, the feature similarity of each historical image pair in the first feature dimension is divided into intervals to obtain at least one first initial feature similarity interval in the first feature dimension; the feature similarity of each historical image pair in the second feature dimension is divided into intervals to obtain at least one second initial feature similarity interval in the second feature dimension; and the first initial feature similarity interval and the second initial feature similarity interval are combined to obtain at least one feature similarity interval. In this way, the feature similarity in different feature dimensions is combined to strengthen the correlation between feature dimensions, which is more practical than a single feature dimension.

[0044] In the above embodiment, the first initial feature similarity interval and the second initial feature similarity interval can be combined in pairs to obtain at least one feature similarity interval. Specifically, given a first initial feature similarity interval, each second initial feature similarity interval is combined with the given first initial feature similarity interval to obtain at least one feature similarity interval.

[0045] In other embodiments, the second feature dimension can also include at least two sub-feature dimensions, and the image clustering device divides the feature similarity of each historical image pair into intervals for each sub-feature dimension to obtain at least one sub-feature similarity interval in each sub-feature dimension; and at least one sub-feature similarity interval in different sub-feature dimensions is combined to obtain at least one second initial feature similarity interval. In this way, more detailed feature similarity intervals can be obtained to provide more accurate clustering probability values.

[0046] The second feature dimension can be an attribute feature, and the sub-feature dimension can be a sub-attribute feature in the attribute feature. For example, the sub-attribute feature of the attribute feature includes but is not limited to a clothing attribute feature, an attribute feature of whether to wear a mask, an attribute feature of whether to wear eyes, etc. As an example, when the second feature dimension includes t important sub-feature dimensions, the value of t can be determined according to the importance of the sub-attribute feature, and 1-4 important sub-attribute features can be selected as important sub-feature dimensions, and the feature similarity of the jth sub-feature dimension is divided into k j sub-feature similarity intervals, which can be represented as The sub-feature similarity intervals of the t important sub-feature dimensions are subjected to Cartesian product operation to form grid intervals, which can obtain k 1 *k 2 *…*k t second initial feature similarity intervals, each of which is composed of the sub-feature similarity intervals of the t important sub-feature dimensions.

[0047] In the embodiment, after obtaining the second initial feature similarity interval, the image clustering apparatus combines the first initial feature similarity interval and the second initial feature similarity interval to obtain at least one feature similarity interval. As an example, the first feature dimension is divided into k first initial feature similarity intervals, and the first initial feature similarity interval and the second initial feature similarity interval are subjected to Cartesian product operation to form grid intervals, and k*k 1 *k 2 *…*k t grid intervals can be obtained, each of which represents a feature similarity interval and is composed of one initial feature similarity interval and t sub-feature similarity intervals.

[0048] As another example, the at least one feature dimension includes a first feature dimension, a second feature dimension and a third feature dimension, the feature similarity of each historical image pair in the first feature dimension is subjected to interval division to obtain at least one first initial feature similarity interval in the first feature dimension, the feature similarity of each historical image pair in the second feature dimension is subjected to interval division to obtain at least one second initial feature similarity interval in the second feature dimension, the feature similarity of each historical image pair in the third feature dimension is subjected to interval division to obtain at least one third initial feature similarity interval in the third feature dimension, and the first initial feature similarity interval, the second initial feature similarity interval and the third initial feature similarity interval are combined to obtain at least one feature similarity interval.

[0049] The combination manner includes combining the first initial feature similarity interval and the second initial feature similarity interval, combining the first initial feature similarity interval and the third initial feature similarity interval, combining the second initial feature similarity interval and the third initial feature similarity interval, and combining the first initial feature similarity interval, the second initial feature similarity interval and the third initial feature similarity interval.

[0050] In some embodiments, the image clustering apparatus can determine the initial feature similarity intervals in each feature dimension and the combinations of the initial feature similarity intervals as the feature similarity intervals. As an example, the feature similarity intervals include at least one feature similarity interval in the first feature dimension, at least one feature similarity interval in the second feature dimension, at least one feature similarity interval in the third feature dimension, a combination of the first feature similarity interval and the second feature similarity interval, a combination of the first feature similarity interval and the third feature similarity interval, and a combination of the second feature similarity interval and the third feature similarity interval. In actual applications, the first feature dimension can be a facial feature, the second feature dimension can be an attribute feature, and the third feature dimension can be a body feature. The facial feature is more reliable than the body feature, and the attribute feature is usually matched with the facial feature, for example, when the attribute feature is matched with the facial feature in the same image, the constraint on the attribute feature similarity is relaxed, so that the images are more reasonably clustered. Thus, the feature similarity intervals can include the feature similarity intervals of the facial feature, the feature similarity intervals of the body feature, the feature similarity intervals of the attribute feature, and the combination of the feature similarity intervals of the facial feature and the feature similarity intervals of the attribute feature.

[0051] In addition, in order to simplify the interval division, when each feature dimension is divided, the feature similarity intervals can be divided in the 0-1 interval according to a preset division manner. For example, for the facial feature, the feature similarity intervals can be divided in the 0-1 interval, which can be represented as 0-b1, b1-b2, …, b k-1 -1. The number of feature similarity intervals can be 4-6, and the interval minimum value and the interval maximum value of the feature similarity interval can be selected according to the distribution of the feature similarity or can be equally divided.

[0052] After the image clustering apparatus obtains the feature similarity intervals, the clustering probability values of the feature similarity intervals are determined according to the clustering states of each historical image pair in each feature similarity interval.

[0053] Specifically, the image clustering device counts the total number of image pairs successfully clustered according to the clustering states of the historical image pairs in each feature similarity interval, counts the number of interval successful image pairs according to the clustering states of the historical image pairs in the same feature similarity interval, and counts the total number of image pairs in the same feature similarity interval; determines the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval as the first sub-clustering probability value of the corresponding feature similarity interval; and / or, respectively determines the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval as the second sub-clustering probability value of the corresponding feature similarity interval. The first sub-clustering probability value refers to the probability of successful clustering of the historical image pairs in each feature similarity interval, and the second sub-clustering probability value refers to the probability of the successfully clustered historical image pairs falling in each feature similarity interval.

[0054] In some embodiments, the clustering probability value includes the first sub-clustering probability value, the image clustering device counts the number of interval successful image pairs according to the clustering states of the historical image pairs in the same feature similarity interval, and counts the total number of image pairs in the same feature similarity interval; determines the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval as the first sub-clustering probability value of the corresponding feature similarity interval; and then determines the clustering probability value of the at least two images to be clustered based on the first sub-clustering probability value.

[0055] In other embodiments, the clustering probability value includes the second sub-clustering probability value, the image clustering device counts the total number of image pairs successfully clustered according to the clustering states of the historical image pairs in each feature similarity interval, counts the number of interval successful image pairs according to the clustering states of the historical image pairs in the same feature similarity interval; respectively determines the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval as the second sub-clustering probability value of the corresponding feature similarity interval; and then determines the clustering probability value of the at least two images to be clustered based on the first sub-clustering probability value.

[0056] In some embodiments, the clustering probability value comprises a first sub-clustering probability value and a second sub-clustering probability value, the image clustering device counts the total number of image pairs successfully clustered according to the clustering states of the historical image pairs in each feature similarity interval, counts the number of interval successful image pairs according to the clustering states of the historical image pairs in the same feature similarity interval, and counts the total number of image pairs in the same feature similarity interval; the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval is determined as the first sub-clustering probability value of the corresponding feature similarity interval; and the ratio between the number of interval successful image pairs and the total number of image pairs in each feature similarity interval is determined as the second sub-clustering probability value of the corresponding feature similarity interval.

[0057] As an example, the image clustering device can calculate the first sub-clustering probability value and the second sub-clustering probability value of each feature similarity interval for all feature similarity intervals. As another example, the image clustering device can determine the clustering probability value calculation method of each feature similarity interval according to the feature dimensions of the feature similarity interval. For example, when the feature similarity interval includes at least one feature similarity interval in the first feature dimension, at least one feature similarity interval in the second feature dimension, at least one feature similarity interval in the third feature dimension, and the combination of the first feature dimension and the second feature dimension forms a feature similarity interval, for the feature similarity interval in the first feature dimension, the number of interval successful image pairs and the total number of image pairs in the corresponding interval in the first feature dimension are counted to obtain the first sub-clustering probability value of each feature similarity interval; the first sub-clustering probability value is the clustering prior value of each feature similarity interval, which can be expressed as The meaning is that when the feature similarity of each image to be clustered in the first feature dimension is located in the feature similarity interval [b j ,b j+1 ], the probability of belonging to the same class is

[0058] For the feature similarity interval in the second feature dimension, the ratio between the number of interval successful image pairs and the total number of image pairs successfully clustered according to the clustering states of the historical image pairs in each feature similarity interval is obtained. The number of interval successful image pairs refers to the number of historical image pairs successfully clustered in a single feature similarity interval, and the total number of image pairs refers to the number of historical image pairs successfully clustered in all feature similarity intervals. The second feature dimension can include multiple sub-feature dimensions, and the meaning is that on the second feature dimension, when the historical image pairs belong to the same class, the feature similarity is located in the corresponding feature similarity interval.

[0059] For the feature similarity interval of the third feature dimension, the ratio between the number of interval successful image pairs of each historical image pair in the same feature similarity interval and the total number of image pairs in which the clustering state indicates clustering success is calculated to obtain the second sub-clustering probability value of each feature similarity interval. The second sub-clustering probability value of the corresponding feature similarity interval on the third feature dimension can be expressed as The meaning is that when the historical image pair belongs to the same class, the feature similarity is located in the corresponding feature similarity interval on the third feature dimension. For the feature similarity interval formed by the combination of the first feature dimension and the second feature dimension, the number of interval successful image pairs of each feature similarity interval under the combination of the first feature dimension and the second feature dimension and the corresponding interval total image pair number are counted to obtain the first sub-clustering probability value of each feature similarity interval. The meaning is that when the feature similarity of each image to be clustered in the first feature dimension is located in the corresponding feature similarity interval, and the feature similarity in the second feature dimension is located in the corresponding feature similarity interval, the probability of belonging to the same class.

[0060] In some embodiments, the top t important sub-feature dimensions can be selected from the s sub-feature dimensions of the second feature dimension according to the importance of each sub-feature dimension, combined with the first feature dimension to form a feature similarity interval, and the first sub-clustering probability value of each feature similarity interval can be calculated, which can be expressed as For other sub-feature dimensions in the second feature dimension except the important sub-feature dimensions, the corresponding second sub-clustering probability value is calculated, which can be expressed as

[0061] It should be noted that the acquisition of the first sub-clustering probability value of the first feature dimension, the acquisition of the second sub-clustering probability value of the second feature dimension and the third feature dimension, and the acquisition of the first sub-clustering probability value of the combination of the first feature dimension and the second feature dimension are only examples, and the corresponding clustering probability value calculation method can be selected according to actual needs in actual application.

[0062] The image clustering device determines target clustering probability values of the to-be-clustered images according to the size relationship between the feature similarity of each to-be-clustered image and the feature similarity interval after obtaining the clustering probability values of each feature similarity interval. Specifically, for the same feature dimension, it is determined whether the feature similarity of at least two to-be-clustered images is located in the feature similarity interval; if yes, the clustering probability value corresponding to the feature similarity interval is determined as an initial clustering probability value; and the target clustering probability values of the at least two to-be-clustered images are determined based on the initial clustering probability values under different feature dimensions. Thus, the initial clustering probability value is selected for the same feature dimension, and it is not required that the feature dimension of the to-be-clustered image is completely the same as the feature dimension of the historical image, that is, the number of feature dimensions of the to-be-clustered image is less than or equal to the number of feature dimensions of the historical image, so as to effectively solve the problem of feature missing by multi-dimensional matching.

[0063] The image clustering device extracts features of each to-be-clustered image according to the feature dimensions in the historical image clustering result, and obtains features of at least one feature dimension. It should be noted that due to the complexity of the real environment, the image clustering device may not be able to extract the features of the complete feature dimension, for example, the features of the first feature dimension may be missing, and the feature missing further causes the feature similarity to be missing. Based on this, the image clustering device only selects the clustering probability value corresponding to the feature similarity interval under the same feature dimension as the initial clustering probability value; and determines the target clustering probability values of the to-be-clustered images based on the initial clustering probability values.

[0064] After obtaining the initial clustering probability values, the initial clustering probability values can also be screened according to the feature dimensions of the feature similarity intervals corresponding to the initial clustering probability values, to obtain screened initial clustering probability values; and the screened initial clustering probability values are subjected to probability calculation to obtain the target clustering probability values of the at least two to-be-clustered images. Thus, it can be prevented that the initial clustering probability values are redundant and affect the accuracy of the final target clustering probability values. Of course, the initial clustering probability values can also be directly subjected to probability calculation to obtain the target clustering probability values of the at least two to-be-clustered images.

[0065] In some embodiments, the feature dimensions of the feature similarity intervals corresponding to each initial clustering probability value are unioned to obtain multiple feature dimensions. If there are duplicate feature dimensions among the multiple feature dimensions, the initial clustering probability values ​​are deduplicated according to the duplicate feature dimensions to obtain the filtered initial clustering probability values. As an example, the initial clustering probability values ​​include the clustering probability values ​​of a certain feature similarity interval under the first feature dimension, the clustering probability values ​​of a certain feature similarity interval under the second feature dimension, the clustering probability values ​​of a certain feature similarity interval under the third feature dimension, and the clustering probability values ​​of a certain feature similarity interval under the combination of the first and second feature dimensions. In this case, the initial clustering probability values ​​need to be obtained by deleting the clustering probability values ​​of a certain feature similarity interval under the first feature dimension to obtain the filtered initial clustering probability values. When the second feature dimension also includes multiple sub-feature dimensions, only the clustering probability values ​​of a certain feature similarity interval under the first feature dimension are deleted to obtain the filtered initial clustering probability values.

[0066] In other embodiments, the feature dimensions of the feature similarity intervals corresponding to each initial clustering probability value can be matched with preset feature dimensions to obtain matching results. Based on the matching results, the corresponding filtering objects are determined from a preset mapping table, which stores the correspondence between preset matching results and preset filtering objects. As an example, if the matching results indicate that all feature dimensions are successfully matched, it means that the feature dimensions are complete. Then, the preset filtering object corresponding to the successfully matched result is queried from the preset mapping table. For example, the preset filtering object can be the clustering probability value of a certain feature similarity interval under the first feature dimension.

[0067] After obtaining the initial clustering probability values ​​after filtering, the image clustering device can acquire the product between the initial clustering probability values ​​after filtering under different feature dimensions; the product is then normalized to obtain the target clustering probability values ​​for at least two images to be clustered. Thus, through the two steps of productization and normalization, it can be ensured that the target clustering probability value satisfies the probability value rule, that is, that the target clustering probability value is between 0 and 1.

[0068] In other embodiments, the image clustering device can pre-construct a posterior probability pre-expression. For example, for image i and image j to be clustered, the posterior probability pre-expression can be:

[0069]

[0070] in, This represents the clustering probability value for a certain feature similarity interval under the first feature dimension. This represents the clustering probability value under the combination of the first feature similarity interval of the first feature dimension and the second feature similarity interval of the second feature dimension of the important sub-feature dimensions. This represents the clustering probability value for a certain feature similarity interval under the third feature dimension. This represents the clustering probability value of a certain feature similarity interval under the sub-feature dimension corresponding to the second feature dimension.

[0071] Pre-expression of normalized coefficients:

[0072]

[0073] Then, the pre-expression is modified according to the feature dimensions of each image to be clustered. When the features are complete, P′ is... i,j Multiplication of α Delete it to avoid calculating an extra cluster probability value under the first feature dimension; when the first feature dimension is missing, P′ i,j Multiplication terms in α Sum of terms Delete, when a feature of a sub-feature dimension in the second feature dimension is missing, P′ i,j The corresponding multiplication terms involved in α should be deleted. If features of important sub-feature dimensions in the second feature dimension are missing, the multiplication terms also need to be removed. Delete; when the third feature dimension is missing, P′ i,j Multiplication terms in α Delete it.

[0074] After the pre-expression is corrected, the target clustering probability value can be the ratio between the corrected posterior probability and the normalized coefficient, and its formula can be expressed as:

[0075] P i,j =Lut3(P′) i,j / α)

[0076] Among them, P i,j P′ represents the target clustering probability value. i,j Let represent the posterior probability, and α represent the normalization coefficient.

[0077] To elaborate on the image clustering method of this application, Figure 2 The framework diagram shown below provides further explanation, as detailed below:

[0078] Multi-dimensional feature extraction: The image clustering device extracts features of each image to be clustered under the first feature dimension, the second feature dimension, and the third feature dimension, respectively. The second feature dimension may include multiple sub-feature dimensions. Features of the same image to be clustered under different feature dimensions are correlated.

[0079] Multi-dimensional feature similarity extraction: After the image clustering device extracts the features of each image to be clustered under different feature dimensions, it obtains the feature similarity of each image to be clustered under at least one feature dimension based on actual needs. For example, the feature similarity of each image to be clustered under the first feature dimension, the second feature dimension, and the third feature dimension can be obtained separately.

[0080] Historical image clustering results statistics: The image clustering device performs statistical analysis on the historical image clustering results of historical image pairs. For example, it can calculate the clustering probability value for each feature similarity interval under the first feature dimension. Specifically, the feature similarity under the first feature dimension can be divided into intervals to obtain multiple feature similarity intervals. Then, the clustering probability value for each feature similarity interval is calculated as the ratio between the number of successfully clustered image pairs and the total number of historical image pairs. Given a feature similarity interval for the first feature dimension, we can statistically analyze the clustering probabilities of the t important sub-feature dimensions under the second feature dimension across different feature similarity intervals. Specifically, we can divide the feature similarity of the first feature dimension into k feature similarity intervals, and divide the feature similarity of the j-th important sub-feature dimension under the second feature dimension into k... j The similarity intervals of the first feature dimension and the similarity intervals of the t important sub-feature dimensions are then subjected to a Cartesian product operation to obtain k*k. 1 *k 2 *…*k t There are 3 feature similarity intervals, each consisting of a feature similarity interval under a first feature dimension and t sub-feature similarity intervals under important sub-feature dimensions. Then, the clustering probability value for each feature similarity interval is obtained by calculating the ratio between the number of successfully clustered image pairs and the total number of historical image pairs under each feature similarity interval. The clustering probability value for each feature similarity interval under the third feature dimension is calculated. Specifically, the feature similarity under the third feature dimension is divided into intervals, resulting in multiple feature similarity intervals. Then, the ratio of the number of successfully clustered image pairs (representing clustering success) in a single feature similarity interval to the total number of successfully clustered image pairs across all feature similarity intervals is calculated. This ratio yields the clustering probability value for each feature similarity interval. The clustering probability value of each feature similarity interval under the second feature dimension is calculated. Specifically, the feature similarity of other sub-feature dimensions (excluding the t important sub-feature dimensions) under the second feature dimension is divided into intervals, resulting in multiple feature similarity intervals for each sub-feature dimension. Then, the ratio of the number of successfully clustered image pairs (representing clustering success) in a single feature similarity interval to the total number of successfully clustered image pairs across all feature similarity intervals is calculated. This ratio yields the clustering probability value for each feature similarity interval under each sub-feature dimension.

[0081] Target clustering probability calculation: Construct a posterior probability pre-expression and a normalization coefficient pre-expression based on the images to be clustered, i and j. Refer to the formulas above; they will not be repeated here. Correct the posterior probability pre-expression and the normalization coefficient pre-expression according to the feature dimensions of each image to be clustered, obtaining the corrected posterior probability pre-expression and the corrected normalization coefficient pre-expression. Based on the relationship between the feature similarity of each image to be clustered and the feature similarity intervals in the historical image clustering results statistics, select the required clustering probability values ​​from the clustering probability values ​​corresponding to each feature similarity interval in the corrected posterior probability pre-expression and the corrected normalization coefficient pre-expression, and substitute them into the two pre-expressions to obtain the calculated posterior probability and normalization coefficient. The ratio between the posterior probability and the normalization coefficient is determined as the target clustering probability value for each image to be clustered.

[0082] Clustering based on target clustering probability values: After obtaining the target clustering probability values, the distance between each image to be clustered is determined by the preset value and the target clustering probability value; and the clustering algorithm is used to perform clustering processing on each image to obtain the image clustering results.

[0083] The above scheme, through Bayesian inference theory, uses historical image clustering results to construct the prior probability of image clustering and integrates multiple feature dimensions to determine the posterior probability of each image to be clustered. It can adapt well to scenarios with missing features, greatly increasing the scope of application and lowering the threshold for use.

[0084] It should be noted that there is no sequential relationship between the above-mentioned historical image clustering results statistics, multi-dimensional feature extraction, and multi-dimensional feature similarity extraction.

[0085] Please see Figure 3 , Figure 3This is a schematic diagram of an exemplary embodiment of the image clustering apparatus shown in this application. The terminal device 300 includes an acquisition module 310, a statistics module 320, a determination module 330, and a clustering module 340. The acquisition module 310 is used to acquire the feature similarity of at least two images to be clustered in at least one feature dimension. The statistics module 320 is used to perform statistical processing on the acquired historical image clustering results to obtain feature similarity intervals in at least one feature dimension, and clustering probability values ​​for each feature similarity interval. The determination module 330 determines the target clustering probability values ​​for at least two images to be clustered based on the relationship between feature similarity and feature similarity intervals. The clustering module 340 performs clustering processing on each image to be clustered based on the target clustering probability values ​​to obtain image clustering results.

[0086] The above scheme involves an image clustering device acquiring the feature similarity of at least two images to be clustered in at least one feature dimension; statistically processing the acquired historical image clustering results to obtain feature similarity intervals in at least one feature dimension, and clustering probability values ​​for each feature similarity interval; determining target clustering probability values ​​for at least two images to be clustered based on the relationship between feature similarity and feature similarity intervals; and performing clustering processing on each image to be clustered based on the target clustering probability values ​​to obtain the image clustering result. Thus, by determining clustering probability values ​​under different conditions using historical image clustering results, and using these clustering probability values ​​to assist in the clustering processing of the current images to be clustered, clustering accuracy can be improved.

[0087] The functions of each module can be found in the implementation examples of the image clustering method, and will not be repeated here.

[0088] To implement the image clustering method of the above embodiments, this application proposes another electronic device, please refer to [link / reference needed]. Figure 4 , Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0089] Electronic device 400 includes memory 410 and processor 420, wherein memory 410 and processor 420 are coupled together.

[0090] The memory 410 is used to store program data, and the processor 420 is used to execute the program data to implement the image clustering method of the above embodiment.

[0091] In this embodiment, processor 420 can also be referred to as a CPU (Central Processing Unit). Processor 420 may be an integrated circuit chip with signal processing capabilities. Processor 420 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 420 can be any conventional processor.

[0092] This application also provides a computer-readable storage medium, such as Figure 5 As shown, the computer-readable storage medium 500 is used to store program data 510, which, when executed by a processor, is used to implement the image clustering method as described in the method embodiments of this application.

[0093] The methods involved in the image clustering method embodiments of this application, when implemented as software functional units and sold or used as independent products, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image clustering method characterized by, The image clustering method comprises: obtaining feature similarity of at least two images to be clustered in at least one feature dimension; statistically processing the obtained historical image clustering result to obtain feature similarity intervals in at least one feature dimension and clustering probability values of each feature similarity interval; determining a target clustering probability value of the at least two images to be clustered according to a size relationship between the feature similarity and the feature similarity interval; performing clustering processing on each image to be clustered according to the target clustering probability value to obtain an image clustering result.

2. The image clustering method of claim 1, wherein, The historical image clustering result comprises feature similarity of each historical image pair and clustering state of each historical image pair, and the step of statistically processing the obtained historical image clustering result to obtain feature similarity intervals in at least one feature dimension and clustering probability values of each feature similarity interval comprises: dividing the feature similarity of each historical image pair into intervals for at least one feature dimension to obtain at least one feature similarity interval, wherein the historical image pair comprises at least two historical images; determining the clustering probability value of each feature similarity interval according to the clustering state of each historical image pair in the feature similarity interval.

3. The image clustering method of claim 2, wherein, The clustering probability value comprises a first sub-clustering probability value and / or a second sub-clustering probability value, and the step of determining the clustering probability value of each feature similarity interval according to the clustering state of each historical image pair in the feature similarity interval comprises: counting the total number of image pairs successfully clustered according to the clustering state of each historical image pair in each feature similarity interval, the interval success image pair number of each historical image pair in the same feature similarity interval, and the interval total image pair number of each historical image pair in the same feature similarity interval; determining the first sub-clustering probability value of the corresponding feature similarity interval as the ratio between the interval success image pair number and the interval total image pair number of each feature similarity interval; and / or respectively determining the second sub-clustering probability value of the corresponding feature similarity interval as the ratio between the interval success image pair number and the total image pair number of each feature similarity interval.

4. The image clustering method of claim 2, wherein, The at least one feature dimension comprises a first feature dimension and a second feature dimension, and the step of dividing the feature similarity of each historical image pair into intervals for at least one feature dimension to obtain at least one feature similarity interval comprises: dividing the feature similarity of each historical image pair in the first feature dimension into intervals to obtain at least one first initial feature similarity interval in the first feature dimension; dividing the feature similarity of each historical image pair in the second feature dimension into intervals to obtain at least one second initial feature similarity interval in the second feature dimension; combining the first initial feature similarity interval and the second initial feature similarity interval to obtain at least one feature similarity interval.

5. The image clustering method of claim 4, wherein, The second feature dimension comprises at least two sub-feature dimensions, and the step of interval dividing the feature similarity of each historical image pair in the second feature dimension to obtain at least one second initial feature similarity interval in the second feature dimension comprises: For each sub-feature dimension, interval dividing the feature similarity of each historical image pair to obtain at least one sub-feature similarity interval in each sub-feature dimension; Combining at least one sub-feature similarity interval in different sub-feature dimensions to obtain at least one second initial feature similarity interval.

6. The image clustering method of claim 1, wherein, The step of determining the target clustering probability value of the at least two images to be clustered according to the size relationship between the feature similarity and the feature similarity interval comprises: For the same feature dimension, determining whether the feature similarity of the at least two images to be clustered is located in the feature similarity interval; If yes, determining the clustering probability value corresponding to the feature similarity interval as an initial clustering probability value; Determining the target clustering probability value of the at least two images to be clustered based on the initial clustering probability values in different feature dimensions.

7. The image clustering method of claim 6, wherein, The step of determining the target clustering probability value of the at least two images to be clustered based on the initial clustering probability values in different feature dimensions comprises: Filtering each initial clustering probability value according to the feature dimension of the feature similarity interval corresponding to the initial clustering probability value to obtain a filtered initial clustering probability value; Performing probability calculation on the filtered initial clustering probability value to obtain the target clustering probability value of the at least two images to be clustered.

8. The image clustering method of claim 7, wherein, The step of performing probability calculation on the filtered initial clustering probability value to obtain the target clustering probability value of the at least two images to be clustered comprises: Obtaining the product between the filtered initial clustering probability values in different feature dimensions; Performing normalization processing on the product to obtain the target clustering probability value of the at least two images to be clustered.

9. An electronic device, comprising: It comprises: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It comprises: Storing program data, which is used to implement the method of any one of claims 1-8 when executed by a processor.

Images