Image clustering method, device, equipment and computer-readable storage medium

Abstract

The present application is applicable to the field of image processing technology and provides an image clustering method, apparatus, device and computer-readable storage medium. The method includes: obtaining a plurality of frames of images and a connection graph corresponding to the plurality of frames of images, wherein the plurality of frames of images include images of multiple categories; constructing an objective function based on the connection graph; obtaining the minimum value of the objective function, and determining the number of categories of the plurality of frames of images and the images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest; dividing the images corresponding to each category into one category to obtain a clustering result of the plurality of frames of images. Thus, the present application uses a method of constructing an objective function through a connection graph and then solving the minimum value of the objective function to accurately classify a large number of multi-frame images of different categories and obtain an accurate clustering result.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to an image clustering method, apparatus, device, and computer-readable storage medium. Background Art

[0002] Data mining is a widely used technology for analyzing and processing data. It uses methods such as association analysis, cluster analysis, and classification to extract useful knowledge from massive amounts of data. Image clustering is a typical method in image data mining. Image clustering involves grouping images belonging to the same category. For example, clustering multiple facial images can be used to group facial images belonging to the same user into the same category.

[0003] However, due to reasons such as lighting, angle or occlusion, the similarity between images belonging to the same category is relatively low, which may cause images of the same category to be classified into other categories during the image clustering process, resulting in clustering errors. Summary of the Invention

[0004] The present application provides an image clustering method, apparatus, device, and computer-readable storage medium, which avoid clustering errors caused by classifying images of the same category into other categories and ensure high clustering accuracy.

[0005] In a first aspect, the present application provides an image clustering method, comprising:

[0006] Acquire multiple frames of images and connection graphs corresponding to the multiple frames of images, wherein the multiple frames of images include images of multiple categories, and the connection graph is used to estimate images of different categories;

[0007] constructing an objective function according to the connection graph, wherein the objective function is used to indicate the difference between images of the same category and the difference between images of different categories;

[0008] Obtaining a minimum value of the objective function, and determining the number of categories of the multiple frames of images and images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest;

[0009] The images corresponding to each category are divided into one category to obtain a clustering result of the multiple frames of images.

[0010] This application first obtains a connection graph corresponding to multiple frames of images, then constructs an objective function based on the connection graph, obtains the minimum value of the objective function, and based on the minimum value of the objective function, determines the number of categories of the multiple frames of images and the images corresponding to each category. Finally, the images corresponding to each category are classified into one category to obtain the clustering results of the multiple frames of images. Thus, by constructing the objective function and then solving the minimum value of the objective function, a large number of multi-frame images of different categories can be accurately classified, avoiding the clustering error caused by classifying images of the same category into other categories, and ensuring the accuracy of the clustering results.

[0011] In a second aspect, the present application provides an image clustering device, which is configured to perform the method of the first aspect or any possible implementation of the first aspect. Specifically, the device may include:

[0012] an acquisition module, configured to acquire a plurality of frames of images and a connection graph corresponding to the plurality of frames of images, wherein the plurality of frames of images include images of multiple categories, and the connection graph is used to estimate images of different categories;

[0013] A construction module, configured to construct an objective function based on the connection graph, wherein the objective function is used to indicate the difference between images of the same category and the difference between images of different categories;

[0014] a determination module, configured to obtain a minimum value of the objective function and determine the number of categories of the multiple frames of images and the images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest;

[0015] The clustering module is used to classify the images corresponding to each category into one category to obtain the clustering results of the multiple frames of images.

[0016] In a third aspect, the present application provides an electronic device comprising a memory and a processor. The memory is used to store instructions; the processor executes the instructions stored in the memory, causing the device to perform the image clustering method of the first aspect or any possible implementation of the first aspect.

[0017] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the image clustering method in the first aspect or any possible implementation of the first aspect.

[0018] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a device, causes the device to execute the image clustering method in the first aspect or any possible implementation of the first aspect.

[0019] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1a This is a flow chart of an image clustering method provided in one embodiment of the present application;

[0022] Figure 1b is a schematic diagram of multiple frames of images before and after clustering provided by an embodiment of the present application;

[0023] Figure 2 1 is a flow chart of an image clustering method provided in one embodiment of the present application;

[0024] Figure 3 This is a flow chart of an image clustering method provided in one embodiment of the present application;

[0025] Figure 4 1 is a flow chart of an image clustering method provided in one embodiment of the present application;

[0026] Figure 5 1 is a flow chart of an image clustering method provided in one embodiment of the present application;

[0027] Figure 6 1 is a flow chart of an image clustering method provided in one embodiment of the present application;

[0028] Figure 7 Schematic diagram of the structure of an image clustering device provided in one embodiment of the present application;

[0029] Figure 8 It is a structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0031] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0032] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0033] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0034] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0036] The present application provides an image clustering method, apparatus, device, and computer-readable storage medium. The image clustering method is applied to scenarios such as shopping mall customer flow analysis that require clustering of massive images.

[0037] The multiple frames of images include images of multiple categories.

[0038] For example, when multiple frames of images are all facial images, images of the same category are facial images belonging to the same user.

[0039] The multi-frame images may be other types of images in addition to face images. The embodiment of the present application does not impose any restrictions on the specific types of the multi-frame images.

[0040] Of course, in addition to facial images of the same user, images of the same category may also be facial images of people of the same gender. The embodiment of the present application does not impose any restriction on the specific types of images of the same category.

[0041] The above-mentioned image clustering method can be implemented by an electronic device. The electronic device can execute the image clustering method by a clustering system.

[0042] The electronic device may be a smart phone, a tablet computer, a desktop computer, a laptop computer, a handheld device, a server, etc. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

[0043] Before introducing the image clustering method provided in the embodiment of the present application, an example of a scenario in which the image clustering method is applicable is given:

[0044] Taking the shopping mall customer flow analysis scenario as an example, after the electronic device obtains multiple frames of facial images of three users A, B, and C moving in the mall through a surveillance camera that communicates with the electronic device, the multiple frames of facial images can be clustered, thereby accurately distinguishing the multiple frames of facial images of the three users A, B, and C, so that the facial images of the same user are divided into one category, to obtain the clustering results of the facial images of the three users A, B, and C.

[0045] The facial images of the same user are classified into one category to facilitate obtaining the number of users in the mall, or the distribution of customer flow in each store in the mall over time (the number of users in each store in the mall over time).

[0046] Among them, the facial images of the same user are divided into one category, specifically: the facial images corresponding to user A are divided into one category, the facial images corresponding to user B are divided into one category, and the facial images corresponding to user C are divided into one category.

[0047] Based on the above scenario description, below, this application takes electronic equipment as an example, combined with the accompanying drawings and application scenarios, to explain in detail the image clustering method provided in the embodiment of this application.

[0048] See also Figure 1a , Figure 1a A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0049] like Figure 1a As shown, the image clustering method provided by this application includes:

[0050] S101: Acquire multiple frames of images and connection graphs corresponding to the multiple frames of images.

[0051] The multiple frames of images include images of multiple categories, and the connection graph is used to estimate images of different categories.

[0052] In some embodiments, the multiple frames of images are images captured by a surveillance camera.

[0053] The surveillance camera can be connected to the electronic device for communication. After capturing an image, the surveillance camera can send the image to the electronic device, which then obtains a connection diagram corresponding to the multiple frames of image.

[0054] S102: Construct an objective function according to the connection graph.

[0055] The objective function is used to indicate the difference between images of the same category and the difference between images of different categories.

[0056] Since the connection map can estimate images of different categories, that is, the connection map can roughly reflect the different categories of multiple frame images, the connection map can be regarded as a rough division of different categories of multiple frame images.

[0057] In some embodiments, the objective function reflects the differences between images of the same category and the differences between images of different categories.

[0058] It can be understood that by reflecting the differences between images of the same category and the differences between images of different categories, data preparation can be provided for the electronic device to obtain the minimum value of the objective function corresponding to the minimum difference between images of the same category and the maximum difference between images of different categories in S103.

[0059] Furthermore, the electronic device can finely divide the different categories of the multi-frame images according to the minimum value of the objective function.

[0060] For example, when the multiple frames of images are 7 frames of facial images, the electronic device first obtains a connection graph corresponding to the 7 frames of facial images based on the 7 frames of facial images, and roughly divides the 7 frames of facial images into categories.

[0061] The electronic device then constructs an objective function based on the connection graph to reflect the differences between facial images of the same category and the differences between facial images of different categories, so as to further fine-tune the different categories of the 7 frames of facial images.

[0062] S103 : Obtain a minimum value of the objective function, and determine the number of categories of the multiple frames of images and the images corresponding to each category according to the minimum value of the objective function.

[0063] The minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest.

[0064] For example, when the multiple frames of images are 7 frames of facial images, the electronic device first obtains a connection graph corresponding to the 7 frames of facial images based on the 7 frames of facial images, and roughly divides the 7 frames of facial images into categories.

[0065] The electronic device then constructs an objective function based on the connection graph to reflect the differences between images of the same category and the differences between images of different categories. The minimum value of the objective function can be obtained when the difference between facial images of the same category is minimized and the difference between facial images of different categories is maximized.

[0066] Finally, the electronic device can determine the number of categories of the 7 frames of face images and the face images corresponding to each category according to the minimum value of the objective function, and perform fine division of the different categories of the 7 frames of face images.

[0067] S104: Classify the images corresponding to each category into one category to obtain a clustering result of the multiple frames of images.

[0068] In some embodiments, when multiple frames of images are all facial images, images of the same category are facial images belonging to the same user, and images corresponding to each category are divided into one category, that is, facial images of the same user are divided into one category.

[0069] For example, when the multiple frames of images are 7 frames of facial images and include facial images of 3 categories, the electronic device may classify the facial images corresponding to each of the 3 categories into one category.

[0070] like Figure 1b As shown in Figure 2, the left frame shows multiple face images before clustering, and the right frame shows multiple face images after clustering. In the left and right frames, the diagonal frame, black frame, solid line frame, and dotted line frame represent face images of different categories respectively.

[0071] The electronic device can cluster multiple frames of facial images with an uncertain number of categories and images corresponding to each category through an image clustering method, so that the facial images corresponding to each category are divided into one category.

[0072] It can be seen that:

[0073] Before clustering, there are 16 diagonal frames, black frames, solid line frames, and dotted line frames, indicating a total of 16 frames of face images.

[0074] After clustering, there are 5 diagonal boxes, indicating that there are 5 frames of face images of this category; there are 4 black boxes, indicating that there are 4 frames of face images of this category; there are 4 solid line boxes, indicating that there are 4 frames of face images of this category; there are 3 dotted line boxes, indicating that there are 3 frames of face images of this category.

[0075] The image clustering method provided in the present application obtains a connection graph corresponding to multiple frames of images through an electronic device, then constructs an objective function for indicating the differences between images of the same category and the differences between images of different categories based on the connection graph, then solves the minimum value of the objective function corresponding to the minimum difference between images of the same category and the maximum difference between images of different categories, and determines the number of categories of the multiple frames of images and the images corresponding to each category based on the minimum value of the objective function, and finally divides the images corresponding to each category into one category to obtain the clustering results of the multiple frames of images. Thus, by constructing the objective function and solving the minimum value of the objective function to determine the number of categories of the multiple frames of images and the images corresponding to each category, the clustering error caused by dividing images of the same category into other categories can be avoided. At the same time, the clustering error caused by the imbalance of the number of massive images of different categories can be avoided, thereby ensuring the high accuracy of clustering and greatly improving the scene adaptability of the image clustering method.

[0076] Based on the above Figure 1a In the description of the illustrated embodiment, the electronic device may construct the objective function based on the connection graph in a variety of ways, for example, by constructing the objective function using inter-class entropy and intra-class entropy.

[0077] Next, combine Figure 2 , which introduces in detail the specific implementation process of the image clustering method of this application.

[0078] The electronic device obtains the inter-class entropy and intra-class entropy corresponding to the connection graph, and then uses the sum of the inter-class entropy and the intra-class entropy as the objective function.

[0079] Among them, the inter-class entropy is used to indicate the difference between images of different categories, and the intra-class entropy is used to indicate the difference between images of the same category.

[0080] It should be noted that the entropy mentioned above refers to information entropy. Information entropy is a basic concept in information theory. It is used to describe the uncertainty of possible events in an information source.

[0081] See also Figure 2 , Figure 2 A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0082] like Figure 2 As shown, the image clustering method provided by this application may include:

[0083] S201: Convert the connection graph into a transition probability matrix.

[0084] Each data in the transition probability matrix is ​​used to represent the transition probability between any two frames of images in the multiple frames of images.

[0085] Optionally, the electronic device may obtain a similarity matrix based on the connection graph, and then convert the similarity matrix into a transition probability matrix.

[0086] Each data in the similarity matrix is ​​used to represent the similarity between any two frames of images in the multiple frames of images.

[0087] It should be noted that the similarity between any two frames of images corresponds to the transition probability between any two frames of images, and they are in direct proportion. The greater the similarity, the higher the transition probability.

[0088] For example, when the multiple frames of images are 7 frames of facial images, the electronic device first obtains the similarity between any one frame of facial images in the 7 frames of facial images and other facial images except the any one frame of facial images based on the 7 frames of facial images, forms a similarity matrix, and then converts the similarity matrix into a transition probability matrix.

[0089] Among them, in the similarity matrix, the first column is the similarity between each face image in the multiple face images and the first face image.

[0090] In the transition probability matrix, the first column is the transition probability between each face image in multiple frames of face images and the first face image.

[0091] S202: Divide each frame of the multiple frames into a category to obtain a first category.

[0092] In some embodiments, each frame of the multiple frames of images is divided into a category, that is, each frame of the image corresponds to a category, and there are as many categories as there are frames of the image.

[0093] For example, when the multiple frames of images are 7 frames of facial images, the electronic device first divides each of the 7 frames of facial images into a category, obtaining 7 categories, and marks the first frame of facial image as category 1, the second frame of facial image as category 2, and the seventh frame of facial image as category 7.

[0094] S203 : Determine a first probability, a second probability, and a third probability according to the transition probability matrix, the number of the first categories, and the image corresponding to each category in the first category.

[0095] The first probability includes the sum of the probabilities that each frame image does not belong to each category in the second category, the second probability includes the probability that each frame image belongs to each category in the second category, and the third probability includes the probability that each frame image does not belong to each category in the second category.

[0096] The second category is a category generated by adjusting the quantity of the first category.

[0097] In some embodiments, the calculation formula of the first probability is:

[0098]

[0099] in, represents the first probability, that is, the sum of the M third probabilities, that is, the sum of the probabilities that each frame image does not belong to each category in the second category, It represents the third probability, that is, the probability that each frame image does not belong to the kth category in the second category. During initialization, the value of M is equal to the number of the first category. In the process of solving the minimum value of the objective function, the value of M is the number of the second category. When the minimum value of the objective function is obtained, the value of M is the actual number of categories of the multi-frame images. k represents the sequence number (k ranges from 1 to M).

[0100] It can be understood that the maximum value of M is the value corresponding to the number of the first category at initialization; the minimum value of M is the value corresponding to the actual number of categories of the multi-frame images when solving the minimum value of the objective function. At this time, the number of the second category is equal to the actual number of categories of the multi-frame images.

[0101] For example, if the multi-frame image is 7 frames of facial images, then the initial value of M is 7. If the 7 frames of facial images include facial images of 3 categories, the value of M may be 6, 5, 4, or 3 when solving the minimum value of the objective function. When the minimum value of the objective function is obtained, the corresponding value of M is 3.

[0102] In some embodiments, the second probability is calculated as:

[0103]

[0104] in, represents the second probability, that is, the probability that each frame image belongs to the kth category in the second category, represents the third probability, α represents any frame image in multiple frames, p α Represents the transition probability between each frame image and any frame image in the multi-frame image, and α∈k indicates that any frame image α in the multi-frame image belongs to the kth category in the second category.

[0105] In some embodiments, the third probability is calculated as:

[0106]

[0107] in, Represents the third probability, G represents the connection graph, and G(α, β) represents the transition probability between the images corresponding to any two nodes in the connection graph.

[0108] S204: Determine inter-class entropy and intra-class entropy according to the first probability, the second probability, and the third probability.

[0109] In some embodiments, the inter-class entropy is expressed as:

[0110]

[0111] In some embodiments, the intra-class entropy is expressed as:

[0112]

[0113] S205: Determine the sum of the intra-class entropy and the inter-class entropy as the objective function.

[0114] In some embodiments, the objective function is expressed as:

[0115]

[0116] Among them, min represents the minimum value of the objective function, and Y represents the clustering label result (which category each frame image belongs to).

[0117] In this application, the electronic device converts the connection graph into a transition probability matrix, and then divides each frame of the multi-frame image into a category to obtain a first category. According to the transition probability matrix and the number of the first categories and the images corresponding to each category in the first category, the first probability, the second probability and the third probability are determined. Then, according to the first probability, the second probability and the third probability, the intra-class entropy and the inter-class entropy are determined. Finally, the sum of the intra-class entropy and the inter-class entropy is determined as the objective function. The objective function is constructed with the help of the inter-class entropy and the intra-class entropy, and the number of categories of the corresponding multi-frame images and the images corresponding to each category are determined to provide accurate data preparation when the minimum value of the objective function is obtained.

[0118] Based on the above Figure 1a In the description of the illustrated embodiment, the electronic device determines the minimum value of the objective function and, based on the minimum value of the objective function, determines the number of categories of the multiple frames and the images corresponding to each category. This can be done in a variety of ways. For example, the electronic device can use a greedy algorithm to solve the minimum value of the objective function.

[0119] Next, combine Figure 3 , which introduces in detail the specific implementation process of the image clustering method of this application.

[0120] A greedy algorithm, also known as a greedy algorithm, always makes the best choice at the moment when solving a problem. In other words, it does not consider the overall optimal solution and only obtains a local optimal solution in some sense.

[0121] The greedy algorithm is used to solve the objective function, which can quickly obtain the minimum value of the objective function.

[0122] See also Figure 3 , Figure 3 A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0123] like Figure 3 As shown, the image clustering method provided by this application may include:

[0124] S301. Adjust the number of the second categories and the images corresponding to each category in the second category in the between-class entropy and the intra-class entropy, and obtain the minimum value of the objective function when the number of the second categories and the images corresponding to each category in the second category meet preset conditions.

[0125] The preset condition is that the difference between images of the same category is the smallest and the difference between images of different categories is the largest.

[0126] In some embodiments, a greedy algorithm is used to solve the minimum value of the objective function to adjust the number of second categories in the inter-class entropy and the intra-class entropy and the images corresponding to each category in the second category.

[0127] Specifically, a simulated annealing algorithm is used to solve the minimum value of the objective function to adjust the number of second categories in the inter-class entropy and the intra-class entropy and the images corresponding to each category in the second category.

[0128] The simulated annealing algorithm can be stored in a storage device. The storage device can communicate with the electronic device, allowing the electronic device to retrieve the simulated annealing algorithm from the storage device. This application does not limit the storage method and specific type of the storage device.

[0129] For example, after obtaining the connection graph corresponding to 7 frames of facial images, the electronic device converts the connection graph into a transition probability matrix, and then calls the objective function from the storage device. According to the transition probability in the transition probability matrix, the simulated annealing algorithm is used to solve the objective function to adjust the number of second categories in the inter-class entropy and intra-class entropy and the facial images corresponding to each category in the second category.

[0130] S302: Obtain the number of categories of the multiple frames of images corresponding to the minimum value of the objective function and the images corresponding to each category.

[0131] For example, when the multiple frames of images are 7 frames of face images, the 7 frames of face images include face images of 3 categories, that is, face image 1, face image 2 and face image 3 belong to the same category, face image 4 and face image 5 belong to the same category, and face image 6 and face image 7 belong to the same category.

[0132] The minimum value of the objective function finally obtained by the electronic device corresponds to 7 frames of face images including 3 categories of face images, face image 1, face image 2 and face image 3 belong to the same category, face image 4 and face image 5 belong to the same category, face image 6 and face image 7 belong to the same category.

[0133] In the present application, the electronic device adjusts the number of second categories in the inter-class entropy and the intra-class entropy and the images corresponding to each category in the second category, and obtains the minimum value of the objective function when the number of second categories and the images corresponding to each category in the second category meet preset conditions. The electronic device adjusts the number of second categories in the inter-class entropy and the intra-class entropy and the images corresponding to each category in the second category to obtain the minimum value of the objective function to accurately determine the number of categories in the multi-frame image and the images corresponding to each category, thereby ensuring the accuracy of image clustering.

[0134] Based on the above Figure 1a In the description of the illustrated embodiment, the electronic device may obtain a connection graph and perform clustering according to the connection graph.

[0135] Next, combine Figure 4 , which introduces in detail the specific implementation process of the image clustering method of this application.

[0136] The electronic device adaptively obtains the similarity threshold corresponding to each frame of image by means of multivariate analysis processing, and then constructs a connection graph based on the similarity threshold corresponding to each frame of image.

[0137] Multivariate analysis processing, also known as z-score standardization processing, is to input data into the z-score model for processing.

[0138] See also Figure 4 , Figure 4 A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0139] like Figure 4 As shown, the image clustering method provided by this application may include:

[0140] S401: Acquire multiple similarity values ​​corresponding to each frame of the multiple frames of images.

[0141] The multiple similarity values ​​corresponding to any one frame image are used to represent the similarity between the any one frame image and other frame images in the multiple frames image except the any one frame image.

[0142] In some embodiments, the multiple frames of images are images captured by a surveillance camera.

[0143] The surveillance camera can be connected to the electronic device for communication. After capturing an image, the surveillance camera can send the image to the electronic device, and the electronic device obtains multiple similarity values ​​corresponding to each frame of the multiple frames.

[0144] In some embodiments, after acquiring multiple frames of images from a surveillance camera, the electronic device extracts features of each frame of image through a deep learning model to form a feature set, and obtains multiple similarity values ​​corresponding to each feature in the feature set based on the similarity between any feature in the feature set and other features in the feature set except any feature in the feature set.

[0145] Among them, the feature set can be expressed as C = {x i}, i=1…N.

[0146] The formula for calculating the similarity value is:

[0147]

[0148] Among them, x i Represents any feature in the feature set, x j Represents one of the features in the feature set except any one feature.

[0149] In addition, the above-mentioned deep learning model can be a residual network (ResNet) or a Transformer model.

[0150] The deep learning model can be stored in a storage device. The storage device can communicate with the electronic device, allowing the electronic device to retrieve the deep learning model from the storage device. This application does not limit the storage method and specific type of the storage device.

[0151] For example, when the multiple facial image frames are 7 facial image frames, the electronic device may perform the following steps:

[0152] After the electronic device obtains 7 frames of face images, it extracts the features of each frame of face image through the residual network (ResNet), and obtains the feature set C = {x i}, i = 1…7;

[0153] Get the similarity value between any feature x2 in the feature set and the six features other than any other feature.

[0154] Determine the set of features corresponding to the 6 features that do not contain any feature x2 as C i =\x2, that is, the similarity value between any feature x7 and features x1, x2, x3, x4, x5, x6 can be expressed as a1, a2, a3, a4, a5, a6.

[0155] S402 : Perform multivariate analysis on the multiple similarity values ​​corresponding to each frame of image to obtain multiple analysis data.

[0156] The analysis data is used to indicate the difference between similarity values ​​corresponding to images belonging to the same category as each frame of image and similarity values ​​corresponding to images not belonging to the same category as each frame of image.

[0157] In some embodiments, a multivariate analysis process is performed on the multiple similarity values ​​corresponding to each frame of image, that is, the multiple similarity values ​​corresponding to each frame of image are input into a z-score model for processing.

[0158] The electronic device inputs multiple similarity values ​​corresponding to each frame of image into the z-score model for processing and can output multiple analysis data.

[0159] It can be understood that the similarity values ​​corresponding to images belonging to the same category as each frame of image are relatively close and have relatively large similarity values, while the similarity values ​​corresponding to images not belonging to the same category as each frame of image are relatively small.

[0160] The electronic device obtains multiple analysis data according to multiple similarity values ​​corresponding to each frame of image through multivariate analysis, and can provide data preparation for obtaining the similarity threshold corresponding to each frame of image.

[0161] For example, the similarity values ​​corresponding to facial images belonging to the same category as facial image 1 may be 0.95, 0.93, and 0.92, while the similarity values ​​corresponding to facial images not belonging to the same category as facial image 1 may be 0.45 and 0.32.

[0162] Obviously, there is a large difference between the similarity values ​​corresponding to face images belonging to the same category as face image 1 and the similarity values ​​corresponding to face images not belonging to the same category as face image 1. This difference can be detected through multivariate analysis.

[0163] S403 : Determine a similarity threshold corresponding to each frame of image from multiple similarity values ​​corresponding to each frame of image according to the maximum analysis data among the multiple analysis data.

[0164] The similarity threshold is a decomposition value between a similarity value corresponding to an image belonging to the same category as each frame of image and a similarity value corresponding to an image not belonging to the same category as each frame of image.

[0165] When the electron undergoes multivariate analysis, the maximum analysis data among the multiple analysis data obtained is used to indicate that the difference between the similarity value corresponding to the image belonging to the same category as each frame image and the similarity value corresponding to the image not belonging to the same category as each frame image is the largest.

[0166] For example, the similarity values ​​corresponding to facial images belonging to the same category as facial image 1 may be 0.95, 0.93, and 0.92, while the similarity values ​​corresponding to facial images not belonging to the same category as facial image 1 may be 0.45 and 0.32.

[0167] Obviously, the similarity value corresponding to the maximum difference between the similarity values ​​corresponding to the face images belonging to the same category as the face image 1 and the similarity values ​​corresponding to the face images not belonging to the same category as the face image 1 is 0.92.

[0168] Therefore, it can be inferred that the dividing value between the similarity values ​​corresponding to facial images belonging to the same category as facial image 1 and the similarity values ​​corresponding to facial images not belonging to the same category as facial image 1 is 0.92, that is, the similarity threshold is 0.92.

[0169] In some embodiments, the similarity threshold can be expressed as a p .

[0170] S404: Construct a connection graph according to the similarity threshold corresponding to each frame of image.

[0171] In some embodiments, the electronic device may perform the following steps to construct a connection map:

[0172] Determining a similarity value that is greater than or equal to the similarity threshold from the multiple similarity values ​​corresponding to each frame of image;

[0173] Determining images corresponding to similarity values ​​greater than or equal to the similarity threshold as the subset corresponding to each frame of image;

[0174] Each image in the subset corresponding to each frame of image is used as a node, and the nodes are connected to each other using connecting edges to obtain the connection graph.

[0175] In some embodiments, the subset corresponding to each frame of image can be represented as B i ,in That is, subset B i The elements must meet the following conditions: s(x i ,x j )≥a p .

[0176] For multiple frames of images, after each frame of image has a corresponding subset, all subsets are combined into a set A, that is, A={B1,…B N}.

[0177] Furthermore, each image in each subset of set A is taken as a node, and the nodes are connected to each other using connecting edges to obtain a connection graph.

[0178] It should be understood that in the connection graph, one node is used to represent one frame of image, and the connection edge between two nodes is used to represent the similarity between the two frames of image.

[0179] For example, when the multi-frame image is 7 frames of facial images, the electronic device can obtain the similarity value between any feature x7 and features x1, x2, x3, x4, x5, and x6, which can be expressed as a1, a2, a3, a4, a5, and a6. If the similarity threshold is determined to be a3, the facial images corresponding to a1, a2, and a3 are determined as the subset B corresponding to the facial image corresponding to any feature x7. i ,in, That is, subset B i The elements must meet the following conditions: s(x i ,x j )≥a3.

[0180] In this application, an electronic device obtains multiple similarity values ​​corresponding to each frame of multiple images, performs multivariate analysis on the multiple similarity values ​​corresponding to each frame of image, obtains multiple analysis data, and then determines a similarity threshold corresponding to each frame of image from the multiple similarity values ​​corresponding to each frame of image based on the maximum analysis data among the multiple analysis data. Finally, a connection graph is constructed based on the similarity threshold corresponding to each frame of image. With the help of the multivariate analysis processing method, it is possible to obtain a demarcation value corresponding to the maximum difference between the similarity values ​​corresponding to images belonging to the same category as each frame of image and the similarity values ​​corresponding to images not belonging to the same category as each frame of image, thereby adaptively obtaining an accurate similarity threshold corresponding to each category, thereby improving the quality of the connection graph.

[0181] Based on the above Figure 4 In the description of the illustrated embodiment, in order to obtain the accuracy of analysis data based on multivariate analysis, the electronic device introduces differential processing and performs differential processing on multiple similarity values ​​to obtain stable data, thereby ensuring the accuracy of multivariate analysis.

[0182] Next, combine Figure 5 , which introduces in detail the specific implementation process of the image clustering method of this application.

[0183] See also Figure 5 , Figure 5A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0184] like Figure 5 As shown, the image clustering method provided by this application may include:

[0185] S501 : Sort the multiple similarity values ​​in descending order to obtain a first sequence.

[0186] Optionally, the multiple similarity values ​​are sorted in descending order to obtain a similarity sequence, which can be expressed as rs(C i ).

[0187] Furthermore, the first K similarity values ​​are taken from the similarity sequence, and the first sequence is generated according to the K similarity values, where K is a positive integer.

[0188] It should be understood that taking the first K similarity values ​​from the similarity sequence and then performing multivariate analysis based on the K similarity values ​​can reduce the amount of data calculation and facilitate faster determination of the similarity threshold.

[0189] Among them, the first sequence can be expressed as rs(C i ,K),rs(C i ,K)=[a1,…,a K ]. The first sequence rs(C i ,K) means the set C i Neutralize image x i The K most similar images.

[0190] For example, if the multi-frame image contains seven facial images and K is 5, the electronic device can obtain the similarity value between any feature x7 and features x1, x2, x3, x4, x5, and x6, which can be expressed as a1, a2, a3, a4, a5, and a6. After sorting in descending order, the resulting similarity sequence is expressed as [a1, a2, a3, a4, a5, and a6]. The corresponding first sequence is expressed as [a1, a2, a3, a4, and a5].

[0191] S502: Perform differential processing on the similarity values ​​in the first sequence to obtain a second sequence.

[0192] When performing multivariate analysis on the first sequence, the electronic device may first perform differential processing on the similarity values ​​in the first sequence to make the data tend to be stable.

[0193] In some embodiments, first-order difference processing is performed on the similarity values ​​in the first sequence to obtain a second sequence.

[0194] Among them, the first-order difference refers to the difference between two consecutive adjacent terms in a discrete function.

[0195] It is understood that the purpose of performing first-order difference processing on the similarity values ​​in the first sequence is to remove noise from the data and make the data more stable. Therefore, in addition to performing first-order difference processing on the similarity values ​​in the first sequence, second-order difference processing or multi-order difference processing can also be performed on the similarity values ​​in the first sequence, without limitation herein.

[0196] In some embodiments, when the first sequence is represented as [a1,…,a K ], the second sequence can be expressed as drs(C i ,K),drs(C i ,K)=[a1-a2,a2-a3,…,a K-1 -a K ].

[0197] Since the similarity values ​​in the first sequence are arranged in descending order, a i -a i+1 ≥0.

[0198] For example, when the first sequence is expressed as [a1, a2, a3, a4, a5], the corresponding second sequence is expressed as [a1-a2, a2-a3, a3-a4, a4-a5].

[0199] S503: Determine the mean of any differential data in the second sequence and other differential data except the any differential data, to obtain multiple means.

[0200] For example, when the second sequence is expressed as [a1-a2, a2-a3, a3-a4, a4-a5], the first mean of the first differential data in the second sequence and the second differential data, third differential data, and fourth differential data excluding the first differential data is expressed as:

[0201]

[0202] S504: Determine, based on the multiple means, a standard deviation between any one differential data in the second sequence and other differential data except the any one differential data, to obtain multiple standard deviations.

[0203] For example, when the second sequence is expressed as [a1-a2, a2-a3, a3-a4, a4-a5], the first standard deviation of the first differential data in the second sequence and the second differential data, third differential data, and fourth differential data excluding the first differential data is expressed as:

[0204]

[0205] S505. Determine the plurality of analysis data according to the plurality of differential data, the plurality of means, and the plurality of standard deviations in the second sequence.

[0206] In some embodiments, the formula for calculating the analytical data is:

[0207]

[0208] Among them, a i -a i+1 represents the i-th differential data, μ i represents the mean of the ith differential data in the second sequence and the other differential data except the ith differential data, σ i Represents the standard deviation of the i-th differential data in the second sequence and the other differential data except the i-th differential data.

[0209] For example, when the second sequence is expressed as [a1-a2, a2-a3, a3-a4, a4-a5], the first analysis data can be expressed as:

[0210]

[0211] Among them, b1 represents the first analysis data, a1-a2 represents the first difference data, σ1 represents the first standard deviation, and μ1 represents the first mean.

[0212] In the present application, the electronic device sorts the multiple similarity values ​​in descending order to obtain a first sequence, performs differential processing on the similarity values ​​in the first sequence to obtain a second sequence, determines the mean and standard deviation of any differential data in the second sequence and other differential data except the any differential data, to obtain multiple means and multiple standard deviations, and determines multiple analysis data based on the multiple differential data, multiple means, and multiple standard deviations. By differential processing, the data is made to tend to be stationary, the accuracy of multivariate analysis is guaranteed, and accurate analysis data is obtained. Therefore, a more accurate similarity threshold can be obtained through accurate analysis data, and thus accurate clustering results can be obtained.

[0213] Based on the above Figure 5 In the description of the illustrated embodiment, after acquiring multiple analysis data, the electronic device can determine a similarity threshold corresponding to each frame of image from multiple similarity values ​​corresponding to each frame of image according to the maximum analysis data among the multiple analysis data.

[0214] Next, combine Figure 6 , which introduces in detail the specific implementation process of the image clustering method of this application.

[0215] See also Figure 6 , Figure 6 A flow chart of an image clustering method provided in an embodiment of the present application is shown.

[0216] like Figure 6 As shown, the image clustering method provided by this application may include:

[0217] S601 : Sort the plurality of analysis data in the order of obtaining corresponding analysis data from the first differential data to the last differential data in the second sequence to obtain a third sequence.

[0218] In some embodiments, the third sequence can be represented as zdrs(C i ,K)=[b1,…,b K-1 ].

[0219] S602: Determine the arrangement position P of the maximum analysis data in the third sequence, where P is a positive integer.

[0220] In some embodiments, the electronic device may record the arrangement position P of the maximum analysis data in the third sequence in the third sequence.

[0221] It should be understood that the above-mentioned arrangement position P refers to the Pth one.

[0222] For example, if the maximum analysis data is arranged at position 3 in the third sequence, the maximum analysis data is the third in the third sequence.

[0223] For example, when the third sequence is represented as [b1, b2, b3, b4], when the electronic device determines that the maximum analysis data is b3, it can determine that the arrangement position of the maximum analysis data in the third sequence is the third.

[0224] S603: Determine the Pth differential data in the second sequence according to the arrangement position P.

[0225] In some embodiments, the first sequence is represented as [a1,…,a K ], the second sequence is expressed as [a1-a2,a2-a3,…,a K-1 -a K ], the third sequence is expressed as [b1,b2,…,b K-1 ].

[0226] From the arrangement of the above sequences, it can be seen that the order of the analysis data in the third sequence corresponds to the order of the differential data in the second sequence and the order of the similarity values ​​in the first sequence.

[0227] That is, the arrangements of the first sequence, the second sequence and the third sequence correspond one to one. When the maximum analysis data is determined to be the Pth in the third sequence, the Pth differential data in the second sequence can be correspondingly determined.

[0228] When determining the arrangement position P of the maximum analysis data in the third sequence, the electronic device can determine the position P in the second sequence, that is, the Pth differential data.

[0229] For example, when the first sequence is expressed as [a1, a2, a3, a4, a5] and the second sequence is expressed as [a1-a2, a2-a3, a3-a4, a4-a5], the third sequence can be expressed as [b1, b2, b3, b4].

[0230] From the arrangement of the above sequences, it can be seen that the first analysis data b1 in the third sequence corresponds to the first differential data a1 - a2 in the second sequence and the first similarity value a1 in the first sequence.

[0231] Therefore, when it is determined that the arrangement position of the maximum analysis data in the third sequence is the third, the third differential data in the second sequence [a1-a2, a2-a3, a3-a4, a4-a5] can be determined.

[0232] S604: Determine a P-th similarity value in the first sequence according to the P-th differential data.

[0233] In some embodiments, when the maximum analysis data in the third sequence is arranged at position P in the third sequence, the electronic device can determine position P in the second sequence, that is, the Pth differential data, and then, the electronic device can determine position P in the first sequence, that is, the Pth similarity value.

[0234] For example, the maximum analysis data is arranged in the third sequence at position 3. The electronic device can determine the third differential data in the second sequence [a1-a2, a2-a3, a3-a4, a4-a5], and further determine the third similarity value in the first sequence [a1, a2, a3, a4, a5].

[0235] S605: Determine the Pth similarity value in the first sequence as the similarity threshold.

[0236] In some embodiments, the Pth similarity value in the first sequence can be determined as the similarity threshold a p .

[0237] For example, when it is determined that the maximum analysis data is arranged in the third position in the third sequence, the third similarity value in the first sequence [a1, a2, a3, a4, a5] can be determined as the similarity threshold.

[0238] In the present application, the electronic device can determine the Pth differential data in the second sequence by determining the arrangement position P of the maximum analysis data in the third sequence. Furthermore, based on the Pth differential data, the Pth similarity value in the first sequence is determined, and the Pth similarity value in the first sequence is determined as the similarity threshold. Because the first, second, and third sequences have a one-to-one correspondence, after obtaining the maximum analysis data in the third sequence, the electronic device can determine the position of the maximum analysis data in the third sequence, thereby inferring the position of the corresponding similarity value in the first sequence. The similarity value corresponding to this position in the first sequence is used as the similarity threshold, thereby ensuring the accuracy of the determined similarity threshold.

[0239] Corresponding to the above Figure 1a The embodiment shown describes an image clustering method, and the present application also provides an image clustering device.

[0240] Next, combine Figure 7 , an image clustering device provided in an embodiment of the present application is described in detail.

[0241] See also Figure 7 , Figure 7 A schematic block diagram of an image clustering device provided in an embodiment of the present application is shown.

[0242] like Figure 7 As shown, an image clustering device provided by an embodiment of the present application includes an acquisition module 701 , a construction module 702 , a determination module 703 and a clustering module 704 .

[0243] An acquisition module 701 is configured to acquire multiple frames of images and a connection graph corresponding to the multiple frames of images, wherein the multiple frames of images include images of multiple categories, and the connection graph is used to estimate images of different categories;

[0244] A construction module 702 is configured to construct an objective function based on the connection graph, wherein the objective function is used to indicate differences between images of the same category and differences between images of different categories;

[0245] a determination module 703 configured to obtain a minimum value of the objective function and determine the number of categories of the multi-frame image and the images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function indicates that the difference between images of the same category is minimized and the difference between images of different categories is maximized;

[0246] The clustering module 704 is configured to classify the images corresponding to each category into one category, and obtain clustering results of the multiple frames of images.

[0247] In some embodiments, the construction module 702 is specifically configured to:

[0248] Converting the connection graph into a transition probability matrix, wherein each data in the transition probability matrix is ​​used to represent a transition probability between any two frames of images in the multiple frames of images;

[0249] Classifying each frame of the multiple frames of images into a category to obtain a first category;

[0250] determining, according to the transition probability matrix, the number of the first categories, and the images corresponding to each category in the first category, a first probability, a second probability, and a third probability, wherein the first probability includes the sum of probabilities that each frame of image does not belong to each category in the second category, the second probability includes the probability that each frame of image belongs to each category in the second category, and the third probability includes the probability that each frame of image does not belong to each category in the second category, wherein the second category is a category generated by adjusting the number of the first categories;

[0251] Determine, according to the first probability, the second probability, and the third probability, an intra-class entropy and an inter-class entropy, wherein the intra-class entropy is used to indicate the difference between images of the same category, and the inter-class entropy is used to indicate the difference between images of different categories;

[0252] The sum of the intra-class entropy and the inter-class entropy is determined as the objective function.

[0253] In some embodiments, the determination module 703 is specifically configured to:

[0254] Adjusting the number of the second categories and the images corresponding to each category in the second category in the inter-class entropy and the intra-class entropy, and obtaining a minimum value of the objective function when the number of the second categories and the images corresponding to each category in the second category meet a preset condition, wherein the preset condition is that the difference between images of the same category is minimized and the difference between images of different categories is maximized;

[0255] The number of categories of the multiple frames of images corresponding to the minimum value of the objective function and the images corresponding to each category are obtained.

[0256] In some embodiments, the acquisition module 701 is specifically configured to:

[0257] Obtaining multiple similarity values ​​corresponding to each frame of the multiple frames of images;

[0258] Performing multivariate analysis on multiple similarity values ​​corresponding to each frame of image to obtain multiple analysis data;

[0259] determining a similarity threshold corresponding to each frame of image from a plurality of similarity values ​​corresponding to each frame of image according to a maximum analysis data among the plurality of analysis data;

[0260] According to the similarity threshold corresponding to each frame image, a connection graph is constructed.

[0261] In some embodiments, the acquisition module 701 is specifically configured to:

[0262] Sorting the multiple similarity values ​​in descending order to obtain a first sequence;

[0263] performing differential processing on the similarity values ​​in the first sequence to obtain a second sequence;

[0264] determining a mean of any differential data in the second sequence and other differential data except the any differential data to obtain a plurality of means; determining a standard deviation of any differential data in the second sequence and other differential data except the any differential data to obtain a plurality of standard deviations;

[0265] The plurality of analysis data are determined based on the plurality of differential data in the second sequence, the plurality of means and the plurality of standard deviations.

[0266] In some embodiments, the acquisition module 701 is specifically configured to:

[0267] Sorting the multiple similarity values ​​in descending order to obtain a similarity sequence;

[0268] Taking the first K similarity values ​​from the similarity sequence, where K is a positive integer;

[0269] The first sequence is generated according to the K similarity values.

[0270] In some embodiments, the acquisition module 701 is specifically configured to:

[0271] sorting the plurality of analysis data in the order of obtaining corresponding analysis data from the first differential data to the last differential data in the second sequence to obtain a third sequence;

[0272] Determine an arrangement position P of the maximum analysis data in the third sequence, where P is a positive integer;

[0273] Determining the Pth differential data in the second sequence according to the arrangement position P;

[0274] Determining a P-th similarity value in the first sequence according to the P-th differential data;

[0275] The Pth similarity value in the first sequence is determined as the similarity threshold.

[0276] It should be understood that the image clustering device 700 of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figure 1a The image clustering method shown, when implemented by software Figure 1a When the image clustering method is shown, the apparatus 700 and its various modules may also be software modules.

[0277] Figure 8 This is a schematic diagram of the structure of an electronic device provided by this application. Figure 8 As shown, the specific implementation of the electronic device can refer to the description of the above electronic device, which can execute Figure 1a The image clustering method shown.

[0278] The electronic device 800 includes a processor 801, a memory 802, a communication interface 803, and a bus 804. The processor 801, the memory 802, and the communication interface 803 communicate via the bus 804, and may also communicate via other means such as wireless transmission. The memory 802 is used to store instructions, and the processor 801 is used to execute the instructions stored in the memory 802. The memory 802 stores program code 8021, and the processor 801 can call the program code 8021 stored in the memory 802 to execute Figure 1a The image clustering method shown.

[0279] It should be understood that in the present application, the processor 801 may be a CPU, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0280] The memory 802 may include a read-only memory and a random access memory, and provides instructions and data to the processor 801. The memory 802 may also include a non-volatile random access memory. The memory 802 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0281] In addition to the data bus, the bus 804 may also include a power bus, a control bus, and a status signal bus. Figure 8 Various buses are labeled as bus 804.

[0282] It should be understood that the device 800 according to the present application may correspond to the apparatus 700 in the present application, and may correspond to the apparatus 700 in the present application. Figure 2 In the method shown, when the device 800 corresponds to Figure 1a When the device is used in the method shown, the above and other operations and / or functions of each module in the device 800 are respectively to implement Figure 1a For the sake of brevity, the operating steps of the method executed by the device are not repeated here.

[0283] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps in the above-mentioned various method embodiments.

[0284] The present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0285] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0286] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0287] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0288] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0289] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0290] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the above modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0291] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0292] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An image clustering method, characterized in that: include: Acquire multiple similarity values ​​corresponding to each frame of an image in a plurality of frames of images, wherein the plurality of frames of images include images of multiple categories; Performing multivariate analysis on multiple similarity values ​​corresponding to each frame of image to obtain multiple analysis data; determining a similarity threshold corresponding to each frame of image from a plurality of similarity values ​​corresponding to each frame of image according to a maximum analysis data among the plurality of analysis data; Constructing a connection graph based on a similarity threshold corresponding to each frame of image, wherein the connection graph is used to estimate images of different categories; constructing an objective function according to the connection graph, wherein the objective function is used to indicate the difference between images of the same category and the difference between images of different categories; Obtaining a minimum value of the objective function, and determining the number of categories of the multiple frames of images and images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest; The images corresponding to each category are divided into one category to obtain a clustering result of the multiple frames of images.

2. The method according to claim 1, wherein The objective function is constructed according to the connection graph, comprising: Converting the connection graph into a transition probability matrix, wherein each data in the transition probability matrix is ​​used to represent a transition probability between any two frames of images in the multiple frames of images; Classifying each frame of the multiple frames of images into a category to obtain a first category; determining, according to the transition probability matrix, the number of the first categories, and the images corresponding to each category in the first category, a first probability, a second probability, and a third probability, wherein the first probability includes the sum of probabilities that each frame of image does not belong to each category in the second category, the second probability includes the probability that each frame of image belongs to each category in the second category, and the third probability includes the probability that each frame of image does not belong to each category in the second category, wherein the second category is a category generated by adjusting the number of the first categories; Determine, according to the first probability, the second probability, and the third probability, an intra-class entropy and an inter-class entropy, wherein the intra-class entropy is used to indicate the difference between images of the same category, and the inter-class entropy is used to indicate the difference between images of different categories; The sum of the intra-class entropy and the inter-class entropy is determined as the objective function.

3. The method according to claim 2, wherein Obtaining the minimum value of the objective function and determining the number of categories of the multiple frames of images and the images corresponding to each category according to the minimum value of the objective function includes: Adjusting the number of the second categories and the images corresponding to each category in the second category in the inter-class entropy and the intra-class entropy, and obtaining a minimum value of the objective function when the number of the second categories and the images corresponding to each category in the second category meet a preset condition, wherein the preset condition is that the difference between images of the same category is minimized and the difference between images of different categories is maximized; The number of categories of the multiple frames of images corresponding to the minimum value of the objective function and the images corresponding to each category are obtained.

4. The method according to claim 1, wherein The multivariate analysis is performed on the multiple similarity values ​​corresponding to each frame of image to obtain multiple analysis data, including: Sorting the multiple similarity values ​​in descending order to obtain a first sequence; performing differential processing on the similarity values ​​in the first sequence to obtain a second sequence; determining a mean of any differential data in the second sequence and other differential data except the any differential data, to obtain a plurality of means; determining a standard deviation between any one differential data in the second sequence and other differential data except the any one differential data, to obtain a plurality of standard deviations; The plurality of analysis data are determined based on the plurality of differential data in the second sequence, the plurality of means and the plurality of standard deviations.

5. The method according to claim 4, wherein The plurality of similarity values ​​are sorted in descending order to obtain a first sequence, comprising: Sorting the multiple similarity values ​​in descending order to obtain a similarity sequence; Taking the first K similarity values ​​from the similarity sequence, where K is a positive integer; The first sequence is generated according to the K similarity values.

6. The method according to claim 4 or 5, characterized in that The determining, based on the maximum analysis data among the plurality of analysis data, a similarity threshold corresponding to each frame of image from a plurality of similarity values ​​corresponding to each frame of image, comprises: sorting the plurality of analysis data in the order of obtaining corresponding analysis data from the first differential data to the last differential data in the second sequence to obtain a third sequence; Determine an arrangement position P of the maximum analysis data in the third sequence, where P is a positive integer; Determining the Pth differential data in the second sequence according to the arrangement position P; Determining a P-th similarity value in the first sequence according to the P-th differential data; The Pth similarity value in the first sequence is determined as the similarity threshold.

7. An image clustering method and device, characterized in that: include: an acquisition module, configured to acquire a plurality of similarity values ​​corresponding to each frame of a plurality of frames of images, wherein the plurality of frames of images include images of a plurality of categories; Performing multivariate analysis on multiple similarity values ​​corresponding to each frame of image to obtain multiple analysis data; determining a similarity threshold corresponding to each frame of image from a plurality of similarity values ​​corresponding to each frame of image according to a maximum analysis data among the plurality of analysis data; Constructing a connection graph based on a similarity threshold corresponding to each frame of image, wherein the connection graph is used to estimate images of different categories; A construction module, configured to construct an objective function based on the connection graph, wherein the objective function is used to indicate the difference between images of the same category and the difference between images of different categories; a determination module, configured to obtain a minimum value of the objective function and determine the number of categories of the multiple frames of images and the images corresponding to each category based on the minimum value of the objective function, wherein the minimum value of the objective function is used to indicate that the difference between images of the same category is the smallest and the difference between images of different categories is the largest; The clustering module is used to classify the images corresponding to each category into one category to obtain the clustering results of the multiple frames of images.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images