Image clustering method and device, equipment, storage medium and program product
By constructing and fusing the hypergraph of image features, the problem of difficulty in describing multivariate relationships and error accumulation in the prior art is solved, and a more accurate image clustering effect is achieved.
Patent Information
- Application Number
- CN202411783879.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-05
- Publication Date
- 2025-03-28
AI Technical Summary
The existing image clustering method based on graph model is difficult to describe the multivariate relationship between images, and the error accumulation during multi-feature fusion affects the image clustering effect.
By determining multiple feature matrices based on multiple image features of the set of images to be clustered, and a feature hypergraph is constructed based on these feature matrices, and fused into a target hypergraph. Then the target hypergraph is cut to obtain a vector representation of the image and clustering is performed based on this.
It realizes adaptive learning of each hypergraph weight and completes multi-hypergraph fusion, solving the error accumulation problem during multi-feature fusion, and improving the accuracy of image clustering.
Smart Images
Figure CN119723133A_ABST
Abstract
Description
Technical Field
[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for image clustering. Background Art
[0002] In current Internet businesses, image clustering is often required. Existing image clustering methods based on graph models can usually only describe binary relationships between samples, but are difficult to describe multivariate relationships between images. However, since images have multiple feature representations, clustering methods based on graph models build a graph based on each feature and then perform image fusion, which is prone to error accumulation and affects the image clustering effect. Summary of the invention
[0003] In a first aspect of the present disclosure, a method for image clustering is provided. The method comprises: determining multiple feature matrices according to multiple image features of a set of images to be clustered, each feature matrix in the multiple feature matrices corresponding to an image feature of the set of images to be clustered; determining a target hypergraph according to feature hypergraphs corresponding to the multiple feature matrices, the target hypergraph being a hypergraph after the fusion of multiple image features; cutting the target hypergraph to determine a vector representation of the set of images to be clustered; and clustering multiple images in the set of images to be clustered based on the vector representation of the set of images to be clustered and a predetermined number of clustering clusters.
[0004] In a second aspect of the present disclosure, a device for image clustering is provided. The device includes: a first determination module, configured to determine multiple feature matrices according to multiple image features of a set of images to be clustered, each of the multiple feature matrices corresponding to an image feature of multiple images to be clustered; a second determination module, configured to determine a target hypergraph according to feature hypergraphs corresponding to the multiple feature matrices; the target hypergraph is a hypergraph after the fusion of multiple image features; a cutting module, configured to determine a vector representation of the set of images to be clustered by cutting the target hypergraph; and a clustering module, configured to cluster multiple images in the set of images to be clustered based on the vector representation of the set of images to be clustered and a predetermined number of clustering clusters.
[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect.
[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.
[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes a computer program, wherein when the computer program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.
[0008] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;
[0011] Figure 2A A schematic diagram of an image clustering system according to some embodiments of the present disclosure is shown;
[0012] Figure 2B A schematic diagram showing an example hypergraph according to some embodiments of the present disclosure is shown;
[0013] Figure 2C A schematic diagram showing an example correlation matrix according to some embodiments of the present disclosure;
[0014] Figure 3 A schematic diagram showing a hypergraph fusion process in a learning model according to some embodiments of the present disclosure is shown;
[0015] Figure 4 A schematic diagram showing an image clustering process according to some embodiments of the present disclosure is shown;
[0016] Figure 5 A schematic structural block diagram of an apparatus for image clustering according to some embodiments of the present disclosure is shown; and
[0017] Figure 6 A block diagram of a computing device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION
[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0020] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0021] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0022] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.
[0023] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0024] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation to the embodiments of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the embodiments of the present disclosure.
[0025] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.
[0026] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes input from the previous layer.
[0027] The term "hypergraph" is a general term for a graph whose edges can connect any number of nodes. Specifically, a hypergraph is a pair consisting of a vertex set and a hyperedge set. Each hyperedge is a subset of the vertex set and can contain any number of vertices. A vertex may also be in multiple hyperedges. In general, a hypergraph is a high-dimensional graphical representation of data, specifically used to describe systems with paired combinations of relationships (vertices and hyperedges).
[0028] In some scenarios, an interface for image clustering can be provided, and the user can input the image to be clustered and the number of clusters into the interface for image clustering to obtain the clustering result of the corresponding number of clusters. Generally, the clustering method mainly includes two steps: first, based on each feature of the image to be clustered, a feature hypergraph corresponding to the feature is constructed, and then multiple feature hypergraphs are fused to obtain the fused target hypergraph. In such a clustering process, completing the construction of the target hypergraph in two steps is prone to error accumulation, which affects the clustering effect.
[0029] The embodiment of the present disclosure provides a scheme for image clustering. According to the scheme, multiple feature matrices can be determined based on multiple image features of the image set to be clustered, each of the multiple feature matrices corresponds to an image feature of the image set to be clustered; a target hypergraph can be determined based on the feature hypergraphs corresponding to the multiple feature matrices, and the target hypergraph is a hypergraph after the fusion of multiple image features; the target hypergraph is cut to determine the vector representation of the image set to be clustered; and multiple images in the image set to be clustered are clustered based on the vector representation of the image set to be clustered and a predetermined number of clustering clusters.
[0030] Therefore, the embodiments of the present disclosure can provide an image clustering solution for self-weighted multi-hypergraph fusion, which solves the error accumulation during multi-feature fusion by adaptively learning the weights of each hypergraph and completing multi-hypergraph fusion.
[0031] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, the example environment 100 may include an electronic device 110. The electronic device 110 may run an application 120 that supports image clustering. A user 140 may interact with the application 120 via the electronic device 110 and / or its attached devices.
[0032] In the exemplary environment 100, if the application 120 is active, the electronic device 110 may present an interface 150 for supporting image clustering through the application 120. The electronic device 110 may input the image to be clustered into the application 120, and present the result of image clustering on the interface 150.
[0033] In some embodiments, the electronic device 110 communicates with the server 130 to provide services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a handheld computer, a portable game terminal, a VR / AR device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an e-book device, a game device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.).
[0034] The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 130 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc. The server 130 may provide background services for the application 120 that supports content presentation in the electronic device 110.
[0035] A communication connection may be established between the server 130 and the electronic device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 130 and the electronic device 110 may implement signaling interaction through the communication connection between the two.
[0036] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.
[0037] Figure 2A 1 shows a schematic diagram of an image clustering system 200 according to some embodiments of the present disclosure. The image clustering system 200 may be run at the electronic device 110 to implement the image clustering function of the application 120. For ease of discussion, reference will be made to Figure 1 The system 200 is described with reference to the environment 100 of FIG.
[0038] like Figure 2A As shown, the information acquisition module 210 is configured to acquire image features of the image to be clustered. The image features include, for example, local binary patterns, 576-dimensional directional gradient histograms, 512-dimensional spatial envelope features, 254-dimensional Centrist features, and 24-dimensional color matrices. Based on the image features, a feature matrix is constructed respectively.
[0039] In some embodiments, based on the local binary pattern, a feature matrix is constructed Each row represents the feature representation of an image.
[0040] like Figure 2A As shown, the hypergraph generation module 220 is configured to generate a feature hypergraph corresponding to the feature matrix based on the feature matrix constructed by the information acquisition module 210.
[0041] like Figure 2B As shown in Figure 1, a hypergraph is a special graph structure that allows multiple connections between nodes. A data set is represented by a hypergraph G = (V, ε, H), where V represents the vertex set, i.e., each image; ε represents the hyperedge set, and each hyperedge can connect multiple vertices. In an example, a hypergraph includes v i vertices, where i is a positive integer less than or equal to 10. Each hyperedge can contain any number of vertices. For example, hyperedge e2 connects vertices v5v6v7; Figure 2C As shown, H represents the correlation matrix of the hypergraph, and the elements in the matrix [H] ij Represents the connection strength between vertex i and hyperedge j.
[0042] like Figure 2A As shown, the learning model 230 is configured to learn and obtain a target hypergraph based on the feature hypergraphs corresponding to multiple image features. In the learning model 230, in order to avoid the error accumulation caused by obtaining the target hypergraph in two steps, a method of adaptively learning the weights of each feature hypergraph is selected to complete the fusion of multiple feature hypergraphs. The learning model 230 first sets a correlation matrix of a target hypergraph, and then uses the set correlation matrix of the target hypergraph and the correlation matrix of multiple feature hypergraphs for learning. First, according to the correlation matrix of the target hypergraph, the feature weights of each feature hypergraph are updated. Then, based on the updated correlation matrix of each feature hypergraph, the correlation matrix of the target hypergraph is updated. Then, according to the updated correlation matrix of the target hypergraph, the feature weights of each feature hypergraph are updated again. In such a learning process, a more accurate representation of the target hypergraph can be obtained.
[0043] In some embodiments, in the learning model 230, for each feature hypergraph correlation matrix H (t) And the target hypergraph correlation matrix H is learned using formula (1):
[0044]
[0045] In the formula [H] ij ≥0, where m represents m features, n represents n samples, and D ij represents the distance between vertex i and hyperedge j, α is a custom parameter; H (t) and In the figure, t represents the serial number of the feature matrix, and argmin is used to find the H corresponding to the minimum value in the brackets. (t) and the value of H, ∥HH (t) ∥ F Indicates the search for H (t) The norm of the difference between and H.
[0046] For the correlation matrix, we use the adaptive nearest neighbor method to obtain it. The farther the distance between the vertex and the hyperedge, the smaller the connection strength. The constraint condition means that the sum of the connection strength of each vertex relative to each hyperedge in the feature hypergraph or target hypergraph is 1, and the connection strength is greater than or equal to 0. Furthermore, the Lagrange multiplier method is used to solve equation (1) and we get:
[0047]
[0048] in is the Lagrange multiplier, μ i 'for The derivative of , max is used to find the maximum value in the brackets, t in the formula is the serial number of the corresponding feature matrix, and the other symbols have the same meaning as those in formula (1).
[0049] like Figure 2A As shown, the cutting module 240 is configured to solve the minimum value of the hypergraph Laplacian matrix corresponding to the indicator matrix of the target hypergraph. Each row in the indicator matrix represents a sample, which is a vector representation of the sample in the target hypergraph. In the cutting module 240, the purpose is to find the closely related vertices in the target hypergraph, corresponding to the hyperedges with the lowest connection strength, that is, to find the minimum value of the hypergraph Laplacian matrix.
[0050] In some embodiments, by solving equation (4), the indicator matrix F can be obtained:
[0051]
[0052] Where L is the hypergraph Laplacian matrix, I is the unit matrix, W, D v , D e are all diagonal matrices, W is the weight of the hyperedge, W is usually set to the unit matrix, where T represents the matrix transpose, D v is the degree of the vertex D e is the degree of the hyperedge v i is the vertex, e j is a hyperedge, w(e j ) is the weight of the corresponding hyperedge.
[0053] like Figure 2A As shown, the image clustering module 250 is configured to perform clustering according to the indicator matrix F and the preset number of clustering clusters to obtain a clustering result corresponding to the preset number of clustering clusters. The image clustering module 250 can cluster the indicator matrix F using, for example, a k-means algorithm to complete image clustering.
[0054] In some embodiments, the image clustering module 250 divides the 240 images of faces, cars, bicycles, buildings, trees, etc. into 8 non-overlapping clusters based on the indicator matrix obtained from the 240 images of 8 categories.
[0055] Figure 3 A schematic diagram of a hypergraph fusion process 300 in a learning model according to some embodiments of the present disclosure is shown.
[0056] Reference Figure 3 In block 310, the electronic device 110 first initializes the correlation matrix of the target hypergraph and uses the correlation matrix of the target hypergraph as a known quantity at this time. At this time, it can be seen from the above formula (2) that if the correlation matrix H of the target hypergraph is known, then the correlation matrix H of the multi-feature hypergraph (t) Can be solved.
[0057] In block 320, the electronic device 110 updates the correlation matrix corresponding to each feature hypergraph based on the correlation matrix of the initialized target hypergraph based on the above formula (2). Then, the correlation matrix corresponding to the updated feature hypergraph is used as a known quantity to update the correlation matrix of the target hypergraph. At this time, it can be seen from the above formula (2) that if the correlation matrix H of multiple feature hypergraphs (t) If , then the correlation matrix H of the target hypergraph can be solved.
[0058] In block 330, the electronic device 110 updates the correlation matrix H corresponding to the feature hypergraph. (t) As a known quantity, the correlation matrix of the target hypergraph is updated based on formula (3).
[0059] In block 340, the electronic device 110 solves the feature weights of the feature hypergraph corresponding to the updated correlation matrix of the target hypergraph based on the correlation matrix of the target hypergraph, the correlation matrix of the feature hypergraph, and the above formula (1). When solving the correlation matrices of multiple feature hypergraphs and the correlation matrix of the target hypergraph, the weights of each hypergraph can be adaptively learned to complete the fusion of multiple hypergraphs.
[0060] In box 350, the electronic device 110 determines whether the learning model has converged. If it has converged, it means that the correlation matrix of the target hypergraph is already the optimal solution under the current calculation method, and the target hypergraph can be directly output. If it has not converged and the preset number of iterations has not been reached, the steps of box 320 to box 340 are repeated until the learning model converges or the maximum number of iterations is reached.
[0061] Figure 4 FIG. 4 is a schematic diagram of an image clustering process 400 according to some embodiments of the present disclosure. The process 400 may be implemented as or included in the electronic device 110 .
[0062] In block 410 , the electronic device 110 determines a plurality of feature matrices according to a plurality of image features of the image set to be clustered, wherein each feature matrix of the plurality of feature matrices corresponds to an image feature of the image set to be clustered.
[0063] In box 420, the electronic device 110 determines a target hypergraph based on the feature hypergraphs corresponding to the multiple feature matrices, where the target hypergraph is a hypergraph obtained by fusing multiple image features.
[0064] In block 430 , the electronic device 110 determines a vector representation of the image set to be clustered by cutting the target hypergraph.
[0065] In block 440 , the electronic device 110 clusters the plurality of images in the to-be-clustered image set based on the vector representation of the to-be-clustered image set and a predetermined number of clustering clusters.
[0066] In some embodiments, determining a plurality of feature matrices includes: establishing a feature matrix according to an image feature among a plurality of image features, wherein each row of the feature matrix corresponds to a feature representation of an image to be clustered in the set of images to be clustered.
[0067] In some embodiments, determining the target hypergraph includes: generating multiple feature hypergraphs corresponding to the multiple feature matrices, and determining the correlation matrix corresponding to each feature hypergraph in the multiple feature hypergraphs and the correlation matrix of the target hypergraph based on the multiple feature hypergraphs, the feature weights corresponding to the multiple feature hypergraphs, and a preset learning model.
[0068] In some embodiments, determining the correlation matrix of the plurality of feature hypergraphs and the correlation matrix of the target hypergraph includes: initializing the correlation matrix of the target hypergraph, updating the correlation matrix of the plurality of feature hypergraphs according to the correlation matrix of the initialized target hypergraph and the learning model, and updating the correlation matrix of the target hypergraph according to the plurality of correlation matrices of the updated plurality of feature hypergraphs and the learning model.
[0069] In some embodiments, process 400 further includes: determining the distance between each to-be-clustered image and the hyperedge in a plurality of feature hypergraphs, and determining a plurality of correlation matrices of the plurality of feature hypergraphs.
[0070] In some embodiments, process 400 further includes: initializing a correlation matrix of the target hypergraph, and updating a feature weight corresponding to each feature matrix in the plurality of feature matrices according to the learning model.
[0071] In some embodiments, process 400 further includes: after determining that the result of the learning model converges or reaches a maximum number of iterations, cutting the target hypergraph.
[0072] In some embodiments, determining the vector representation of the image set to be clustered includes: obtaining a Laplacian matrix of a target hypergraph, and determining the vector representation of the image set to be clustered according to the Laplacian matrix of the target hypergraph.
[0073] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an apparatus 500 for image clustering according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware or any combination thereof.
[0074] like Figure 5 As shown, the device 500 includes: a first determination module 510, configured to determine a plurality of feature matrices according to a plurality of image features of the image set to be clustered, each of the plurality of feature matrices corresponding to an image feature of a plurality of images to be clustered; a second determination module 520, configured to determine a target hypergraph according to feature hypergraphs corresponding to the plurality of feature matrices; the target hypergraph is a hypergraph after a plurality of image features are fused; a cutting module 530, configured to determine a vector representation of the image set to be clustered by cutting the target hypergraph; and a clustering module 540, configured to cluster a plurality of images in the image set to be clustered based on the vector representation of the image set to be clustered and a predetermined number of clustering clusters.
[0075] In some embodiments, the apparatus 500 is further configured to establish a feature matrix according to one of the plurality of image features, wherein each row of the feature matrix corresponds to a feature representation of an image to be clustered in the set of images to be clustered.
[0076] In some embodiments, the apparatus 500 is further configured to generate multiple feature hypergraphs corresponding to multiple feature matrices, and determine the correlation matrix corresponding to each feature hypergraph in the multiple feature hypergraphs and the correlation matrix of the target hypergraph based on the multiple feature hypergraphs, the feature weights corresponding to the multiple feature hypergraphs, and the preset learning model.
[0077] In some embodiments, the apparatus 500 is further configured to initialize a correlation matrix of the target hypergraph, update correlation matrices of multiple feature hypergraphs based on the correlation matrix of the initialized target hypergraph and the learning model, and update the correlation matrix of the target hypergraph based on the multiple correlation matrices of the updated multiple feature hypergraphs and the learning model.
[0078] In some embodiments, the apparatus 500 is further configured to determine the distance between each image to be clustered and the hyperedge in a plurality of feature hypergraphs, and determine a plurality of correlation matrices of the plurality of feature hypergraphs.
[0079] In some embodiments, the apparatus 500 is further configured to initialize a correlation matrix of the target hypergraph, and update a feature weight corresponding to each feature matrix in the plurality of feature matrices according to the learning model.
[0080] In some embodiments, the apparatus 500 is further configured to cut the target hypergraph after determining that the result of the learning model converges or reaches a maximum number of iterations.
[0081] In some embodiments, the apparatus 500 is further configured to obtain a Laplacian matrix of a target hypergraph, and determine a vector representation of the image set to be clustered according to the Laplacian matrix of the target hypergraph.
[0082] The units and / or modules included in the device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 500 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0083] It should be understood that one or more steps in the above method can be performed by a suitable electronic device or a combination of electronic devices. Such an electronic device or a combination of electronic devices may include, for example, Figure 1 The electronic device 110 in.
[0084] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 Electronic device 110 or Figure 5 Device 500.
[0085] like Figure 6As shown, the electronic device 600 is in the form of a general electronic device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 600.
[0086] The electronic device 600 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 630 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 600.
[0087] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.
[0088] The communication unit 640 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
[0089] The input device 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 660 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 600, or communicate with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).
[0090] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
[0091] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0092] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0093] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0094] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0095] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.
Claims
1. An image clustering method, comprising: Determine a plurality of feature matrices according to a plurality of image features of the image set to be clustered, wherein each feature matrix of the plurality of feature matrices corresponds to an image feature of the image set to be clustered; Determine a target hypergraph according to the feature hypergraphs corresponding to the multiple feature matrices, wherein the target hypergraph is a hypergraph after the multiple image features are fused; Determining the vector representation of the image set to be clustered by cutting the target hypergraph; as well as Clustering is performed on a plurality of images in the image set to be clustered based on the vector representation of the image set to be clustered and a predetermined number of clustering clusters.
2. The method of claim 1, wherein determining a plurality of feature matrices comprises: A feature matrix is established according to an image feature among the multiple image features, and each row of the feature matrix corresponds to a feature representation of an image to be clustered in the set of images to be clustered.
3. The method according to claim 1, wherein determining the target hypergraph comprises: Generating a plurality of feature hypergraphs corresponding to the plurality of feature matrices; as well as According to the multiple feature hypergraphs, the feature weights corresponding to the multiple feature hypergraphs and a preset learning model, a correlation matrix corresponding to each feature hypergraph in the multiple feature hypergraphs and a correlation matrix of the target hypergraph are determined.
4. The method according to claim 3, wherein determining the correlation matrix of the plurality of feature hypergraphs and the correlation matrix of the target hypergraph comprises: Initializing a correlation matrix of the target hypergraph; According to the initialized correlation matrix of the target hypergraph and the learning model, updating the correlation matrices of the multiple feature hypergraphs; as well as The correlation matrix of the target hypergraph is updated according to the updated correlation matrices of the multiple feature hypergraphs and the learning model.
5. The method according to claim 4, further comprising: The distance between each image to be clustered and the hyperedge in the multiple feature hypergraphs is determined, and multiple correlation matrices of the multiple feature hypergraphs are determined.
6. The method according to claim 3, further comprising: Initializing a correlation matrix of the target hypergraph; as well as The feature weight corresponding to each feature matrix in the multiple feature matrices is updated according to the learning model.
7. The method according to claim 6, further comprising: After determining that the result of the learning model converges or reaches a maximum number of iterations, the target hypergraph is cut.
8. The method according to claim 1, wherein determining the vector representation of the set of images to be clustered comprises: Obtaining the Laplacian matrix of the target hypergraph; as well as According to the Laplacian matrix of the target hypergraph, a vector representation of the set of images to be clustered is determined.
9. A device for image clustering, comprising: A first determination module is configured to determine a plurality of feature matrices according to a plurality of image features of the set of images to be clustered, wherein each feature matrix of the plurality of feature matrices corresponds to an image feature of the plurality of images to be clustered; A second determination module is configured to determine a target hypergraph according to the feature hypergraphs corresponding to the multiple feature matrices; the target hypergraph is a hypergraph after the multiple image features are fused; A cutting module is configured to determine a vector representation of a set of images to be clustered by cutting the target hypergraph; as well as The clustering module is configured to cluster the multiple images in the image set to be clustered based on the vector representation of the image set to be clustered and a predetermined number of clustering clusters.
10. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.
11. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 8.
12. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 8.