Image processing method and device and computer equipment

Abstract

The invention discloses an image processing method and device and computer equipment. The method comprises the following steps: acquiring an image set; the images in the image set are clustered to obtain a plurality of initial image clusters, the plurality of initial image clusters are respectively used for representing sets of the images of different objects, and the same initial image cluster represents a set of the images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set. According to the image clustering method and device, the technical problem that the clustering effect is poor when images are clustered based on a conventional clustering method in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to an image processing method, apparatus and computer equipment. Background Art

[0002] In the urban archive clustering scenario, archive clustering is the basic supporting data for many upper-level practical applications, so its reliability and stability are very important. However, when clustering images with existing information, the clustering results may be incomplete due to missing information or the generalization ability of statistical learning features.

[0003] Therefore, when clustering is performed based on conventional clustering methods, there is often a problem of category splitting: the number of clusters actually divided is greater than the number of true categories, and there is a certain split ratio. The clusters obtained based on clustering are often of high purity, and separating one or two abnormal samples from such a cluster will only slightly improve the clustering.

[0004] Therefore, when the related art performs image clustering on images based on conventional clustering methods, there is a problem of poor clustering effect.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide an image processing method, apparatus and computer device to at least solve the technical problem that when the related art clusters images based on conventional clustering methods, the clustering effect is poor.

[0007] According to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: acquiring an image set; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0008] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining a leadership image cluster having a leadership category from the multiple initial image clusters; and determining the at least two intermediate image clusters from leadership image clusters having the same leadership category.

[0009] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: respectively determining the purity of the multiple initial image clusters; and determining the at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0010] Optionally, determining the at least two intermediate image clusters from the initial image clusters having purity greater than a predetermined purity threshold comprises: respectively determining image representatives of the initial image clusters having purity greater than the predetermined purity threshold; determining the similarity between the image representatives of the initial image clusters having purity greater than the predetermined purity threshold; and determining that the initial image clusters corresponding to the target image representatives having similarity greater than the similarity threshold are the at least two intermediate image clusters.

[0011] Optionally, the method of separately determining image representatives of initial image clusters whose purity is greater than the predetermined purity threshold includes: for a target initial image cluster whose purity is greater than the predetermined purity threshold, acquiring images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; determining an average similarity of any image in the target initial image cluster relative to other images; determining an image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; and based on a method for determining image representatives of target initial image clusters, separately determining image representatives of initial image clusters whose purity is greater than the predetermined purity threshold.

[0012] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: respectively determining sizes of the multiple initial image clusters; and determining the at least two intermediate image clusters from initial images whose sizes are greater than a predetermined size threshold.

[0013] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from the multiple initial image clusters; dividing the miscellaneous clusters to obtain a leadership image cluster with a leadership category; and determining the at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster obtained by dividing the miscellaneous clusters.

[0014] Optionally, clustering the images in the image set to obtain a plurality of initial image clusters includes: obtaining similar information on whether image pairs in the image set belong to the same object; and clustering the images in the image set based on the similar information to obtain the plurality of initial image clusters.

[0015] Optionally, the fusing the at least two intermediate image clusters to obtain the image clustering result of the image set includes: fusing the at least two intermediate image clusters to obtain a fused image cluster; acquiring other image clusters except the at least two intermediate image clusters from the multiple initial image clusters; and determining that the clustering result including the fused image cluster and the other image clusters is the image clustering result of the image set.

[0016] According to another aspect of an embodiment of the present invention, there is also provided an image processing method, comprising: receiving a search request, wherein the search request carries product information of a target product to be searched; acquiring an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search result of the target product based on the image clustering result.

[0017] According to another aspect of an embodiment of the present invention, there is also provided an image processing method, comprising: receiving a query request, wherein the query request carries information of a target traffic participant to be queried; acquiring a traffic video in response to the query request; performing video frame splitting on the traffic video to obtain an image set including a plurality of video frames; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the query result of the target traffic participant based on the image clustering result.

[0018] According to one aspect of an embodiment of the present invention, there is provided an image processing apparatus, comprising: an acquisition module for acquiring an image set; a clustering module for clustering images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; a determination module for determining at least two intermediate image clusters from the plurality of initial image clusters; and a fusion module for fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0019] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image processing methods.

[0020] According to another aspect of an embodiment of the present invention, a computer device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the above-mentioned image processing methods when running.

[0021] In an embodiment of the present invention, the images in the image set are clustered to obtain a plurality of initial image clusters; at least two intermediate image clusters are determined from the plurality of initial image clusters; and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on the image as only a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when the related technology clusters images based on conventional clustering methods.

[0022] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0024] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown;

[0025] Figure 2 is a flowchart of an image processing method 1 provided according to an embodiment of the present invention;

[0026] Figure 3 is a flowchart of a second image processing method provided according to an embodiment of the present invention;

[0027] Figure 4 is a flowchart of a second image processing method provided according to an embodiment of the present invention;

[0028] Figure 5 is a structural block diagram of an image processing device 1 provided according to an embodiment of the present invention;

[0029] Figure 6 is a structural block diagram of an image processing device 2 provided according to an embodiment of the present invention;

[0030] Figure 7 is a structural block diagram of an image processing device 3 provided according to an embodiment of the present invention;

[0031] Figure 8 It is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] First, some nouns or terms that appear in the process of describing the embodiments of the present invention are subject to the following explanations:

[0035] Pure class: a class in the clustering result, that is, the objects that actually belong to the same class account for more than 50%, otherwise it is called miscellaneous class;

[0036] Clustering: The process of dividing a collection of physical or abstract objects into multiple classes consisting of similar objects is called clustering. The clusters generated by clustering are a set of data objects that are similar to the objects in the same cluster and different from the objects in other clusters.

[0037] Active clustering: semi-supervised clustering, that is, clustering when some samples have real labels, is also called constrained clustering (CC). In many scenarios, it is expensive to accurately label samples, and only experts can do it (for example, the classification of animals and plants, and sample labeling when the label set is very large). Therefore, active clustering generally considers introducing pairwise constency information: must-link (two samples are of the same type) and cannot-link (two samples are of different types). This labeling task is relatively simple (it only needs to determine whether two samples are of the same nature), and non-expert labelers can also complete it well.

[0038] Euclidean distance: refers to the distance between two points in Euclidean space, which is used to calculate the distance between the eigenvectors of a certain mode between two objects.

[0039] Feature vector: A one-dimensional array calculated for an image. Usually, the similarity between two images can be obtained by calculating the Euclidean distance between their feature vectors.

[0040] Cosine similarity: Evaluates the similarity between two vectors by calculating the cosine of the angle between them. For example, it can be used to calculate the similarity between the feature data of a certain modality between two objects.

[0041] Matching probability: the probability that a pair (two) objects belong to the same cluster category.

[0042] Indicator function: The letter I represents the indicator function, I (True) = 1, I (False) = 0.

[0043] Cluster and class: For the sake of distinction, in this embodiment, “cluster” refers to the sample division obtained by clustering, and “class” refers to the actual category of the sample.

[0044] Leading class and messy cluster: If there is a certain type of sample in a cluster obtained by clustering, and its number is not less than half of the cluster size, then this category is called the leading category of this cluster. Otherwise, this cluster has no leading category and is called a messy cluster.

[0045] Purity: For each cluster obtained by clustering, its purity is the proportion of samples of the class with the highest proportion in this cluster. The total purity of the cluster is the sum of the number of samples of the class with the highest proportion in each cluster divided by the total number of samples.

[0046] Class fusion (Merge): If two clusters both have leader categories and the leader categories are the same, the operation of merging the two clusters into one cluster is called class fusion.

[0047] Purify: If a cluster is a mixed cluster, the operation of splitting it into several small clusters with purity 1 is called class purification.

[0048] Query State and State Matrix: Query State Set = {-1: different class; 1: same class; 0: unlabeled}, state matrix S∈{-1,0,1}^(N×N), S_ij represents the query state of sample pair (i,j).

[0049] Example 1

[0050] According to an embodiment of the present invention, a method embodiment of an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] The method embodiment provided in the first embodiment of the present invention may be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0052] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0053] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the image processing method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned image processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0055] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0056] Under the above operating environment, this application provides Figure 2 The image processing method shown. Figure 2 1 is a flow chart of an image processing method 1 provided according to an embodiment of the present invention. The image processing method 1 comprises the following steps:

[0057] Step S202, obtaining an image set;

[0058] As an optional embodiment, the execution subject of the method of this embodiment can be a terminal or server for image processing. For example, when applied to a terminal for performing image processing, it can easily implement image processing in simple image scenes; for another example, when applied to a server, it can call on the server's rich computing resources, or a relatively larger and more accurate image model, thereby more accurately recognizing the image.

[0059] It should be noted that the above-mentioned terminal can be of various types, for example, it can be a mobile terminal with certain computing capabilities, or it can be a fixed computer device with identification capabilities, etc. The above-mentioned server can also be of various types, for example, it can be a local server or a virtual cloud server. The server can be a single computer device or a computer cluster in which multiple computer devices are integrated according to computing capabilities.

[0060] As an optional embodiment, the above-mentioned image set may be a collection of atlases in a database. For example, in an e-commerce scenario, the image set may be a collection of images in an e-commerce image library. The image set includes multiple images of at least one commodity, and the image set may include images of multiple commodities. In a traffic scenario, the image set may be a collection of multi-frame images obtained by splitting a video. The multi-frame image set includes at least one traffic participant, and the multi-frame image set includes multiple traffic participants. Among them, the above-mentioned traffic participants can only be any objects involved in traffic, such as people, vehicles, some traffic facilities, traffic signs, etc.

[0061] Step S204, clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object;

[0062] As an optional embodiment, when clustering the images in the image set to obtain multiple initial image clusters, multiple methods can be used. For example, different model algorithms can be used for clustering, or a combination of multiple clustering methods can be used for clustering. It should be noted that any of the multiple initial image clusters obtained by clustering is a collection of images belonging to the same object obtained based on clustering. For example, for the collection of images included in the image library of an e-commerce company, the products included in the image library include clothes, pants, shoes, etc. Moreover, clothes, pants, and shoes have multiple brands, styles, colors, etc. In an optional embodiment of the present invention, when clustering is performed based on objects, that is, clustering is performed based on products, the product corresponding to the initial image cluster obtained by clustering can refer to the one product, not a category of products. That is, the images in an initial image cluster that can be obtained by clustering all have the product. It should be noted that the clustering operation refers to finding a collection that belongs to the same product in the image collection, and treating the images including the same product as a category.

[0063] Step S206, determining at least two intermediate image clusters from the multiple initial image clusters;

[0064] As an optional embodiment, at least two intermediate image clusters are determined from multiple initial image clusters for subsequent fusion of the at least two intermediate image clusters. There are multiple ways to determine at least two intermediate image clusters from multiple initial image clusters, which are described below with examples.

[0065] For example, to determine at least two intermediate image clusters from multiple initial image clusters, the following processing method may be used: determining a leadership image cluster having a leadership category from multiple initial image clusters; and determining at least two intermediate image clusters from leadership image clusters having the same leadership category.

[0066] The leading category refers to a cluster in which the number of images belonging to the same category (for example, for commodities in an e-commerce scenario, the number of images belonging to the same commodity) is greater than a predetermined number threshold (for example, represented by a ratio, which may be 50% of the total number of clusters). Of course, in an optional embodiment of the present invention, the threshold may also be flexibly modified as needed.

[0067] By determining the at least two intermediate image clusters from the leader image clusters with the same leader category, since the purity of the images belonging to the same object in the leader image cluster is higher than a certain threshold, the at least two intermediate image clusters can be determined from the leader image cluster, so that at least two intermediate image clusters can be quickly determined. Moreover, the purity of the leader image cluster is higher than a certain threshold, so that when clustering is performed based on the fusion operation later, not only the clustering effect but also the clustering efficiency can be guaranteed.

[0068] For another example, in order to further improve the clustering effect, when determining at least two intermediate image clusters from multiple initial image clusters, the following processing method can be used: determine the purity of multiple initial image clusters respectively; determine at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold. That is, directly based on the purity of multiple initial image clusters, at least two intermediate image clusters are determined based on the initial image clusters whose purity is greater than a predetermined purity threshold. The predetermined purity threshold can be flexibly adjusted based on needs. For example, when the clustering accuracy is not high, the predetermined purity threshold can be set lower; when the clustering accuracy requirement is relatively high, the predetermined purity threshold can be set relatively higher. By accurately determining at least two intermediate image clusters based on purity, the clustering effect obtained by subsequent fusion based on the at least two intermediate image clusters can be accurately guaranteed.

[0069] Among them, based on the initial image cluster whose purity is greater than the predetermined purity threshold, at least two intermediate image clusters are determined, and the following processing method can be adopted: respectively determine the image representatives of the initial image cluster whose purity is greater than the predetermined purity threshold; determine the similarity between the image representatives of the initial image cluster whose purity is greater than the predetermined purity threshold; determine the initial image cluster corresponding to the target image representative whose similarity is greater than the similarity threshold as at least two intermediate image clusters. According to the similarity between the image representatives of the initial image cluster, the initial image cluster corresponding to the target image representative whose similarity is greater than the similarity threshold is determined as the at least two intermediate image clusters. Since the image representatives can represent the corresponding initial image cluster to a certain extent, the similarity between the image representatives is used to represent the overall similarity between the initial image clusters. Since it is not necessary to calculate the similarity for all the images in the initial image cluster, the efficiency of determining at least two intermediate image clusters can be improved.

[0070] Wherein, respectively determining image representatives of initial image clusters whose purity is greater than a predetermined purity threshold comprises: for a target initial image cluster whose purity is greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; determining an average similarity of any image in the target initial image cluster relative to other images; determining an image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; and based on a method for determining image representatives of the target initial image cluster, respectively determining image representatives of the initial image clusters whose purity is greater than a predetermined purity threshold.

[0071] By adopting the above processing, i.e. based on the average similarity of any image in the initial image cluster with respect to other images in the initial image cluster, and taking the image with the highest average similarity in the initial image cluster as the image representative, i.e. adopting a statistical method, the image that best represents the initial image cluster is obtained. It should be noted that the similarity between images here can be the similarity probability that the product in the image is the same product as the product in other images. The similarity probability can be determined based on the Euclidean distance or cosine distance between image features.

[0072] For another example, to determine at least two intermediate image clusters from multiple initial image clusters, the following processing may be used: respectively determine the sizes of the multiple initial image clusters; and determine at least two intermediate image clusters from the initial images whose sizes are greater than a predetermined size threshold. At least two intermediate image clusters are determined based on the sizes of the initial image clusters. Since the larger the clusters are, the more obvious the clustering effect will be. Therefore, clustering is effectively improved by fusing the clusters based on the sizes of the initial image clusters.

[0073] It should be noted that the above method of determining at least two intermediate image clusters from multiple initial image clusters is not a single operation, and multiple methods can also be considered in combination. For example, the above at least two intermediate image clusters can be determined based on the purity of multiple initial image clusters and the size of multiple initial image clusters. When the above at least two intermediate image clusters are determined based on multiple factors, a better clustering effect can be obtained more efficiently.

[0074] In addition, when at least two intermediate image clusters are determined from multiple initial image clusters, the following processing method can also be adopted: determine a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from multiple initial image clusters; divide the miscellaneous cluster to obtain a leadership image cluster with a leadership category; and determine at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster obtained by dividing the miscellaneous cluster.

[0075] By adopting the above processing, not only the leading image clusters among the multiple initial image clusters are considered, but also the miscellaneous clusters without the leading category are divided and processed, and then the leading image clusters are obtained from the clusters obtained after the division. Since the leading image clusters are obtained from the clusters obtained after the division, the purification of the initial image clusters is achieved. Fusion based on the purified image clusters can make the purity of the clustering results after clustering higher, and effectively improve the clustering effect. It should be noted that the better the clustering effect mentioned in the embodiment of the present invention means that the images belonging to the same object are classified into one category as much as possible.

[0076] As an optional embodiment, clustering the images in the image set to obtain a plurality of initial image clusters includes: obtaining similar information of whether the image pairs in the image set belong to the same object; based on the similar information, clustering the images in the image set to obtain a plurality of initial image clusters. Based on the similar information of whether the images belong to the same object, the efficiency of clustering the images in the image set can be improved. For example, when the first image and the second image belong to the same category, and the second image and the third image belong to the same category, then the first image and the third image also belong to the same category. Based on the similar information, it is deduced whether two objects belong to the same category, which avoids the problem of large amount of calculation caused by directly calculating whether they are of the same category based on the images separately, and uses the implicit relationship between the above categories to effectively improve the clustering efficiency of clustering images. It should be noted that the same category here can refer to the same object.

[0077] Step S208: fuse at least two intermediate image clusters to obtain an image clustering result of the image set.

[0078] Optionally, fusing at least two intermediate image clusters to obtain an image clustering result of the image set includes: fusing at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters except at least two intermediate image clusters from multiple initial image clusters; and determining the clustering result including the fused image cluster and other image clusters as the image clustering result of the image set. Through the above processing, the image clustering result of the image set obtained includes the fused image cluster obtained by fusing the at least two intermediate image clusters, and also includes other image clusters except at least two intermediate image clusters from multiple initial image clusters.

[0079] Through the above-mentioned embodiments, the images in the image set are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; and the at least two intermediate image clusters are fused to obtain the image clustering results of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on the image as only a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when the related technology clusters images based on conventional clustering methods.

[0080] Figure 3 is a flowchart of a second image processing method according to an embodiment of the present invention. Figure 3 As shown, the second image processing method includes the following steps:

[0081] Step S302, receiving a search request, wherein the search request carries product information of a target product to be searched;

[0082] As an optional embodiment, the target products may be various products that can be presented on the e-commerce website, for example, products that are classified based on multiple classification standards, such as various categories of products included in clothing, various categories of products included in electrical appliances, and various categories of products included in electronic products, etc.

[0083] Step S304, in response to the search request, an image set is obtained; images in the image set are clustered to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different commodities, and the same initial image cluster represents a set of images of the same commodity; at least two intermediate image clusters are determined from the plurality of initial image clusters, and the at least two intermediate image clusters are merged to obtain an image clustering result of the image set;

[0084] Step S306: Feedback search results of the target product based on the image clustering results.

[0085] As an optional embodiment, when the search results of the target product are fed back based on the image clustering results, the target image cluster corresponding to the target product is determined from the image clustering results, and the product included in the target image cluster is fed back as the search result. It should be noted that when the product images included in the target image cluster are fed back as the search results, the product images included in the target image cluster can be processed and the processed product images can be fed back. The above processing of the products can be to sort the product images based on the degree to which the images represent the target products, and the sorted product images can be fed back.

[0086] Through the above-mentioned embodiments, based on the search request for searching for the target product, the images in the image set are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; at least two intermediate image clusters are fused to obtain the image clustering results of the image set, and the search results of the target product are fed back based on the image clustering results. By fusing at least two intermediate image clusters, the purpose of fast clustering of products based on cluster fusion is achieved. Compared with clustering separately based on the image as a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem that the related technology has poor clustering effect when clustering images based on conventional clustering methods. Moreover, when applied to the search scenario, since the clustering effect of the target product is clearer, more accurate and more efficient, the search results fed back to the user can also be more accurate, which effectively improves the user's search experience.

[0087] It should be noted that when the product images in the product library are clustered based on the search request, the search results obtained are also related to the product information of the target product to be searched. For example, when the product information of the target product to be searched is relatively accurate and unique, the corresponding search results obtained are also more accurate.

[0088] Figure 4 is a flowchart of a third image processing method according to an embodiment of the present invention. Figure 4 As shown, the image processing method three includes the following steps:

[0089] Step S402, receiving a query request, wherein the query request carries information of a target traffic participant to be queried;

[0090] As an optional embodiment, the target traffic participant may be a variety of objects, such as a person, a vehicle, a traffic facility, a traffic sign, etc.

[0091] The target traffic participant information may be information used to describe the traffic participant, for example, if the traffic participant is a person, it may be a pedestrian, a traffic policeman, or a sanitation worker, etc. If the traffic participant is a vehicle, it may be the model of the vehicle, the brand of the vehicle, the color of the vehicle, and the appearance of the vehicle, etc. The query request may be used to request a query of the traffic participant of interest, for example, it may be a test vehicle used for testing, etc.

[0092] Step S404, in response to the query request, obtaining traffic video;

[0093] As an optional embodiment, the above-mentioned traffic video can be a video collected by traffic collection equipment on the road, or a video stored in a traffic database, or a video recorded by a driving recorder of a vehicle on the road, or a traffic-related video provided by other personnel, etc.

[0094] Step S406, splitting the traffic video into video frames to obtain an image set including multiple video frames;

[0095] As an optional embodiment, when splitting traffic videos, multiple splitting methods can be used. For example, a predetermined software can be used for splitting. By flexibly configuring the splitting time interval, flexible splitting of the traffic video can be achieved to obtain an image set including multiple video frames that meets the requirements.

[0096] Step S408, clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant;

[0097] As an optional embodiment, multiple initial image clusters are respectively used to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant. For example, one of the multiple initial image clusters may be a set of images representing the same traffic toll booth, and one of the multiple initial image clusters may be a set of images representing the same car with a flamingo pattern painted on its body, etc.

[0098] Step S410, determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set;

[0099] Step S412: Based on the image clustering result, the query result of the target traffic participant is fed back.

[0100] As an optional embodiment, when the query result of the target traffic participant is fed back based on the image clustering result, the image cluster corresponding to the target traffic participant is found from the image clustering result, and the images included in the image cluster are fed back as the query result. It should be noted that when the images included in the image cluster are fed back as the query result, the images included in the image cluster can also be processed and the processed images can be fed back. For example, the location of the target traffic participant in the image included in the image cluster is marked, and the marked image is displayed.

[0101] Through the above-mentioned embodiments, based on the query request for querying the target traffic participants, the images in the image set obtained by splitting the traffic video frames are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; at least two intermediate image clusters are fused to obtain the image clustering results of the image set, and the query results of the target traffic participants are fed back based on the image clustering results. By fusing at least two intermediate image clusters, the purpose of fast clustering of the target traffic participants based on cluster fusion is achieved. Compared with clustering the images separately as a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem that the related technology has poor clustering effect when clustering images based on conventional clustering methods. Moreover, when applied to the target traffic participant scenario, since the clustering effect of the target traffic participant is more clear, accurate and efficient, the query results fed back to the user can also be more accurate, which effectively improves the user's query experience.

[0102] Based on the above embodiments and optional embodiments, an optional implementation manner is provided.

[0103] Given that the above-mentioned related technologies have poor clustering effects when clustering products in images based on conventional clustering methods, in real application scenarios, human-machine collaboration to complete some key tasks can significantly improve the effect. It is particularly important to maximize the clustering effect through a small amount of manual introduction, or to minimize the amount of manual annotation to achieve the required effect.

[0104] In related technologies, two methods are mainly used to achieve active clustering: Active Constraint Acquisition and Active Clustering with Constraints. The difference lies in whether manual labeling and clustering adjustment are performed simultaneously. Some algorithms consider first determining all the pairs that need to be labeled before performing constrained clustering, while others continuously use existing labels to adjust the clustering results, and then select new pairs based on this. Furthermore, there are also two ideas for selecting samples in active clustering: pair-based and sample-based. Pair-based refers to the independent selection of sample pairs, while sample-based is to build the skeleton of clustering through human labeling: the relationship between representative samples of each category is labeled, and the division of the remaining samples depends on the clustering skeleton. Since the complexity of the pair-based method is O(N^2), N is the number of samples, it is often only applicable to data with a small number of categories and samples.

[0105] Probabilistic clustering / conventional clustering often has the problem of category splitting: the number of clusters actually divided is greater than the number of true categories, that is, there is a certain split ratio. The clusters obtained by probabilistic clustering are often of high purity, and separating one or two abnormal samples from such a cluster will only slightly improve the clustering.

[0106] That is, in an optional embodiment of the present invention, the clustering effect is iteratively optimized by quantitatively evaluating the improvement in clustering effect brought about by manual annotation, so that a large improvement in effect can be obtained through the smallest possible amount of manual annotation. In an optional embodiment of the present invention, a probability metric is introduced to automatically judge the miscellaneous classes, and the miscellaneous classes are converted into pure classes through maximum likelihood probability clustering, and the expected Normalized Mutual Information (NMI) index improvement of the combination of different pure classes is calculated, and representative samples of the best class pairs are selected for annotation. The optional embodiment of the present invention can be applied to the clustering of large-scale urban data, and a significant improvement in effect can be obtained with a small amount of manual intervention.

[0107] In an optional embodiment of the present invention, the goals of active probabilistic clustering are: 1) to find the clusters corresponding to the classes that were split in the clustering process and merge them; 2) to purify the found mixed clusters.

[0108] To achieve the above goals, the following approaches can be adopted.

[0109] 1. Cluster fusion based on cluster purity and size

[0110] Assume that the maximum likelihood solution obtained by probabilistic clustering of N samples is Ω = {w1 ,w 2 ,…,w k}, the real clustering is divided into C = {c 1 ,c 2 ,…,c J}, the NMI value of Ω is n 1 For any two clusters, let w 1 ,w 2 , if they all have leading categories, their purities are t 1 ,t 2 , the cluster sizes are p and q respectively. Then there exists t∈(min(t 1 ,t 2 ),max(t 1 ,t 2 )), when w 1 ,w 2 When merging into a cluster, if n 1 ≥2×(1-t), then the new clustering partition Ω ′ ={w 1,2 ,w 3 ,…,w k}'s NMI value satisfies n 2 ≥n 1 .

[0111] Based on the above-mentioned class fusion operation, clusters can be fused based on the purity and size of the clusters. For example, clusters with a larger purity (not less than 50%) should be selected for fusion. For mixed clusters, samples of mixed categories need to be separated before fusion.

[0112] 2. Clustering based on latent label reasoning

[0113] The pair-based active clustering query strategy has an inefficient utilization problem, that is, after manually annotating several sample pairs, some unannotated sample pairs between them may have been determined (called potential known sample pairs): for example, 1 With x 2 Same type, x 2 With x 3 are also of the same type, then x 1 and x 3 The traditional active learning algorithm does not utilize this potential information, so in this optional implementation, an efficient annotation information reasoning algorithm is proposed to solve this problem.

[0114] Each time a pair of samples {i, j} is manually labeled, the algorithm is used to update the state value. If there is no potential known sample pair related to {i, j} (ml or cl between i or j), the state value is 0.

[0115] Assume that the sample sets with i and j as ml are G i ,G j (including i, j), and the sample sets where i, j are cl are G i ′ ,G j ′ .

[0116] 1. When i=j:

[0117] Update the state of sample i: G after updating ml of i i =G i +j, which also leads to G j =G j +G i , and after i's cl is updated G j ′ =G j ′ +G i ′ , note that at this time only j and G i ,G i ′ The status between samples is updated, but the status between other samples is not updated.

[0118] Update the state of sample j: j's ml is updated after G i =G i +G j , at this time G i ,G j The state values ​​of all sample pairs become 1. After the cl of j is updated, G i ′ =G i ′ +G j ′ , at this time {G i ,G j} and {G i ′ ,G j ′} all state values ​​become -1, that is, all potential known sample pairs are updated.

[0119] 2. When i≠j, the update of ml will not cause the state value to change

[0120] Update the cl related state of sample i: G j ′ =G j ′ +G i . Update the cl-related state of sample j: G i ′=G i ′ +G j , at this time G i ,G j The state values ​​between them all become -1, that is, all potential known sample pairs are updated.

[0121] 3. Recall of clusters of the same category

[0122] When considering the manual labeling sample pairs, we give priority to clusters that are likely to be of the same type, determine whether they have a leading category, and then ask the labeler to determine whether their leading categories are the same. If they are the same, we force the two clusters to be merged into one cluster. The most critical part of this process is to estimate the probability that two clusters can be merged.

[0123] When selecting cluster-pairs, the cluster-pairs that are expected to improve NMI more are given priority. From the above theory, we can see that the change in NMI after fusion is determined by the purity and size of the two clusters, and the clusters obtained by probabilistic clustering are often of higher purity. Therefore, we can assume that the purity is 1, and the entropy value before fusion H(Ω) to the entropy value after fusion H(Ω) ′ ) The larger the change Δh, the greater the NMI improvement. There are two conclusions about Δh: 1. When p+q is fixed, the smaller |pq| is, the larger Δh is; 2. The larger p, q is, the larger Δh is.

[0124]

[0125] Finally, the following expected return is used to recall the cluster-pair.

[0126]

[0127] The leading category and representative samples can be determined based on the greedy selection algorithm: through at most two manual annotations, it is determined whether the purity of a cluster meets the subsequent fusion requirements. Here, the purity selection is randomly sampled from a uniform distribution of [0.5, 0.7] to avoid the small probability event (exactly encountering an outlier) that occurs when the lower limit of purity is fixed to 0.5. Optionally, this process can determine the purity of a class through Gibbs sampling.

[0128] It should be noted that the active probability clustering algorithm can only recall cluster-pairs with a fusion probability greater than a preset threshold (the fusion probability of the other massive cluster-pairs is too low, and they are basically not of the same type, so it is not worth labeling). In addition, in the above algorithm, each time a pair of samples is manually labeled, the algorithm is immediately called to update the status.

[0129] Based on the above optional implementation method, starting from the granularity of the category, the complexity of the calculation and judgment required for active clustering is greatly reduced. And from the statistical perspective of probability, the effect gain obtained by all category judgments is quantified, and the clustering indicators are optimized in a direction. By quantitatively analyzing the improvement in clustering effect that may be brought about by the discrimination of category pairs, the best sample pairs are selected for annotation, which greatly reduces the amount of manual annotation required to achieve certain indicators, and maximizes the clustering effect through a small amount of manual annotation; this solution can be generally applied to the clustering results obtained by various clustering algorithms, and with the addition of a matching probability function, the class purity can be judged, the miscellaneous classes can be purified, and the pure classes with the highest expected improvement can be continuously selected for manual judgment.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0131] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0133] Example 2

[0134] According to an embodiment of the present invention, a device for implementing the above-mentioned image processing method 1 is also provided. Figure 5 is a structural block diagram of an image processing device 1 provided according to an embodiment of the present invention, such as Figure 5As shown, the device includes: a first acquisition module 52, a first clustering module 54, a first determination module 56 and a first fusion module 58. The device is described below.

[0135] A first acquisition module 52 is used to acquire an image set; a first clustering module 54 is connected to the first acquisition module 52 and is used to cluster the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; a first determination module 56 is connected to the first clustering module 54 and is used to determine at least two intermediate image clusters from the plurality of initial image clusters; a first fusion module 58 is connected to the first determination module 56 and is used to fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0136] It should be noted that the first acquisition module 52, the first clustering module 54, the first determination module 56 and the first fusion module 58 correspond to steps S202 to S208 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above modules can also be run in the computer terminal 10 provided in Example 1 as part of the device.

[0137] According to an embodiment of the present invention, a device for implementing the above-mentioned second image processing method is also provided. Figure 6 is a structural block diagram of an image processing device 2 provided according to an embodiment of the present invention, such as Figure 6 As shown, the device includes: a first receiving module 62, a first processing module 64 and a first feedback module 66. The device is described below.

[0138] A first receiving module 62 is used to receive a search request, wherein the search request carries product information of a target product to be searched; a first processing module 64 is connected to the first receiving module 62, and is used to obtain an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; a first feedback module 66 is connected to the first processing module 64, and is used to feed back search results of the target product based on the image clustering result.

[0139] It should be noted that the first receiving module 62, the first processing module 64 and the first feedback module 66 correspond to steps S302 to S306 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules or units may be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above modules may also be part of the device and run in the computer terminal 10 provided in Example 1.

[0140] According to an embodiment of the present invention, a device for implementing the above-mentioned image processing method 3 is also provided. Figure 7 is a structural block diagram of an image processing device 3 provided according to an embodiment of the present invention, such as Figure 7 As shown, the device includes: a second receiving module 72, a second acquiring module 74, a splitting module 76, a second clustering module 78, a second determining module 710 and a second feedback module 712. The device is described below.

[0141] A second receiving module 72 is used to receive a query request, wherein the query request carries information of a target traffic participant to be queried; a second acquisition module 74 is connected to the second receiving module 72, and is used to acquire a traffic video in response to the query request; a splitting module 76 is connected to the second acquisition module 74, and is used to split the traffic video into video frames to obtain an image set including a plurality of video frames; a second clustering module 78 is connected to the splitting module 76, and is used to cluster the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are used to represent sets of images of different traffic participants, respectively, and the same initial image cluster represents a set of images of the same traffic participant; a second determination module 710 is connected to the second clustering module 78, and is used to determine at least two intermediate image clusters from the plurality of initial image clusters, and fuse the at least two intermediate image clusters to obtain an image clustering result of the image set; a second feedback module 712 is connected to the second determination module 710, and is used to feed back the query result of the target traffic participant based on the image clustering result.

[0142] It should be noted that the second receiving module 72, the second acquisition module 74, the splitting module 76, the second clustering module 78, the second determination module 710 and the second feedback module 712 correspond to steps S402 to S412 in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above modules can also be run in the computer terminal 10 provided in Example 1 as part of the device.

[0143] Example 3

[0144] The embodiment of the present application may provide a computer terminal (or computer device), which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0145] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0146] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method: obtaining an image set; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the multiple initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0147] Optionally, Figure 8 is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 8 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 82, a memory 84, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0148] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the image processing method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain an image set; cluster the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determine at least two intermediate image clusters from the multiple initial image clusters; and fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0150] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining a leadership image cluster having a leadership category from multiple initial image clusters; and determining at least two intermediate image clusters from leadership image clusters having the same leadership category.

[0151] Optionally, the processor may also execute program code of the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: respectively determining the purity of the multiple initial image clusters; and determining at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0152] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from an initial image cluster whose purity is greater than a predetermined purity threshold, including: respectively determining image representatives of the initial image cluster whose purity is greater than a predetermined purity threshold; determining the similarity between image representatives of the initial image cluster whose purity is greater than a predetermined purity threshold; and determining that the initial image cluster corresponding to the target image representative whose similarity is greater than the similarity threshold is at least two intermediate image clusters.

[0153] Optionally, the processor may also execute program code for the following steps: respectively determining image representatives of initial image clusters having purity greater than a predetermined purity threshold, including: for a target initial image cluster having purity greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster having purity greater than the predetermined purity threshold; determining an average similarity of any image in the target initial image cluster relative to other images; determining an image with the highest average similarity in the target initial image cluster as an image representative of the target initial image cluster; and based on a method for determining image representatives of the target initial image cluster, respectively determining image representatives of initial image clusters having purity greater than a predetermined purity threshold.

[0154] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: respectively determining the sizes of the plurality of initial image clusters; and determining at least two intermediate image clusters from initial images whose sizes are greater than a predetermined size threshold.

[0155] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from a plurality of initial image clusters; dividing the miscellaneous cluster to obtain a leadership image cluster with a leadership category; and determining at least two intermediate image clusters based on the leadership image cluster determined from the plurality of initial image clusters and the leadership image cluster obtained by dividing the miscellaneous cluster.

[0156] Optionally, the processor may also execute program code for the following steps: clustering images in an image set to obtain a plurality of initial image clusters, including: obtaining similar information on whether image pairs in the image set belong to the same object; and clustering images in the image set based on similar information to obtain a plurality of initial image clusters.

[0157] Optionally, the processor may also execute program code for the following steps: fusing at least two intermediate image clusters to obtain an image clustering result of the image set, including: fusing at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters except at least two intermediate image clusters from multiple initial image clusters; and determining that the clustering result including the fused image cluster and other image clusters is the image clustering result of the image set.

[0158] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: receiving a search request, wherein the search request carries product information of a target product to be searched; obtaining an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search result of the target product based on the image clustering result.

[0159] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: receiving a query request, wherein the query request carries information of a target traffic participant to be queried; obtaining a traffic video in response to the query request; performing video frame segmentation on the traffic video to obtain an image set including multiple video frames; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively used to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the query result of the target traffic participant based on the image clustering result.

[0160] According to the embodiment of the present invention, images in an image set are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on an image as only a sample, the technical effect of improving clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when the related art clusters images based on conventional clustering methods.

[0161] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 8 It does not limit the structure of the above electronic device. For example, the computer terminal may also include Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 8Different configurations shown.

[0162] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0163] Example 4

[0164] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the image processing method provided in the above embodiment 1.

[0165] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0166] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: acquiring an image set; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0167] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for executing the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining a leadership image cluster with a leadership category from multiple initial image clusters; determining at least two intermediate image clusters from leadership image clusters with the same leadership category.

[0168] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for performing the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: respectively determining the purity of the multiple initial image clusters; determining at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0169] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for executing the following steps: determining at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold, including: separately determining image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold; determining the similarity between the image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold; determining that the initial image clusters corresponding to the target image representatives whose similarity is greater than the similarity threshold are at least two intermediate image clusters.

[0170] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for executing the following steps: respectively determining image representatives of initial image clusters having a purity greater than a predetermined purity threshold, including: for a target initial image cluster having a purity greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster having a purity greater than the predetermined purity threshold; determining an average similarity of any image in the target initial image cluster relative to other images; determining an image with the highest average similarity in the target initial image cluster as an image representative of the target initial image cluster; and based on a method for determining image representatives of the target initial image cluster, respectively determining image representatives of the initial image clusters having a purity greater than a predetermined purity threshold.

[0171] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for performing the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: respectively determining the sizes of the multiple initial image clusters; and determining at least two intermediate image clusters from the initial images whose sizes are greater than a predetermined size threshold.

[0172] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code for executing the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from the multiple initial image clusters; dividing the miscellaneous clusters to obtain a leadership image cluster with a leadership category; and determining at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster obtained by dividing the miscellaneous clusters.

[0173] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code for executing the following steps: clustering images in an image set to obtain multiple initial image clusters, including: obtaining similar information on whether image pairs in the image set belong to the same object; based on the similar information, clustering the images in the image set to obtain multiple initial image clusters.

[0174] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program codes for executing the following steps: fusing at least two intermediate image clusters to obtain an image clustering result of the image set, including: fusing at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters except at least two intermediate image clusters from multiple initial image clusters; and determining that the clustering result including the fused image cluster and other image clusters is the image clustering result of the image set.

[0175] Optionally, in this embodiment, a computer-readable storage medium is configured to store program codes for executing the following steps: receiving a search request, wherein the search request carries product information of a target product to be searched; obtaining an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search results of the target product based on the image clustering result.

[0176] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for executing the following steps: receiving a query request, wherein the query request carries information of a target traffic participant to be queried; acquiring a traffic video in response to the query request; performing video frame splitting on the traffic video to obtain an image set including multiple video frames; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively used to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the query result of the target traffic participant based on the image clustering result.

[0177] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0178] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0179] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0183] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Get the image set; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; The at least two intermediate image clusters are fused to obtain an image clustering result of the image set.

2. The method according to claim 1, characterized in that The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: determining a leadership image cluster having a leadership category from the plurality of initial image clusters; The at least two intermediate image clusters are determined from leader image clusters having the same leader category.

3. The method according to claim 1, characterized in that The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: respectively determining the purity of the plurality of initial image clusters; The at least two intermediate image clusters are determined from the initial image clusters whose purity is greater than a predetermined purity threshold.

4. The method according to claim 3, characterized in that The step of determining the at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold comprises: respectively determining image representatives of the initial image clusters having purity greater than the predetermined purity threshold; determining similarities between image representatives of the initial image cluster having a purity greater than the predetermined purity threshold; It is determined that the target images with similarities greater than a similarity threshold represent corresponding initial image clusters, which are the at least two intermediate image clusters.

5. The method according to claim 4, characterized in that The step of respectively determining image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold comprises: For a target initial image cluster whose purity is greater than the predetermined purity threshold, acquiring images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; Determine the average similarity of any image in the target initial image cluster relative to other images; Determine an image with the highest average similarity in the target initial image cluster as an image representative of the target initial image cluster; Based on the method for determining the image representatives of the target initial image cluster, image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold are respectively determined.

6. The method according to claim 1, characterized in that Determining at least two intermediate image clusters from the plurality of initial image clusters comprises: respectively determining the sizes of the plurality of initial image clusters; The at least two intermediate image clusters are determined from initial images having a size greater than a predetermined size threshold.

7. The method according to claim 1, characterized in that The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: Determine a leading image cluster having a leading category and a miscellaneous cluster without a leading category from the plurality of initial image clusters; dividing the miscellaneous clusters to obtain a leadership image cluster having a leadership category; The at least two intermediate image clusters are determined based on the leading image cluster determined from the plurality of initial image clusters and the leading image cluster obtained by dividing the miscellaneous clusters.

8. The method according to claim 1, characterized in that The clustering of the images in the image set to obtain a plurality of initial image clusters comprises: Obtaining information of the same type as to whether the image pairs in the image set belong to the same object; Based on the similar information, the images in the image set are clustered to obtain the multiple initial image clusters.

9. The method according to any one of claims 1 to 8, characterized in that The fusing the at least two intermediate image clusters to obtain the image clustering result of the image set includes: fusing the at least two intermediate image clusters to obtain a fused image cluster; Acquire other image clusters except the at least two intermediate image clusters from the plurality of initial image clusters; A clustering result including the fused image cluster and the other image clusters is determined as the image clustering result of the image set.

10. An image processing method, characterized in that: include: Receiving a search request, wherein the search request carries product information of a target product to be searched; In response to the search request, acquiring an image set; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different commodities, and the same initial image cluster represents a set of images of the same commodity; Determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; Based on the image clustering result, the search result of the target product is fed back.

11. An image processing method, characterized in that: include: Receiving a query request, wherein the query request carries information of a target traffic participant to be queried; In response to the query request, acquiring traffic video; Splitting the traffic video into video frames to obtain an image set including a plurality of video frames; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; Determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; Based on the image clustering result, the query result of the target traffic participant is fed back.

12. An image processing device, characterized in that: include: An acquisition module, used to acquire an image set; A clustering module, used for clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively used to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; A determination module, configured to determine at least two intermediate image clusters from the plurality of initial image clusters; The fusion module is used to fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 11.

14. A computer device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the image processing method described in any one of claims 1 to 11 when running.

Images