Image processing method and apparatus, and computer device

Abstract

Disclosed in the present disclosure are an image processing method and apparatus, and a computer device. The method comprises: acquiring an image set; clustering images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively configured to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from among the multiple initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

Description

Image processing method, device and computer equipment

[0001] Cross-reference

[0002] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on August 14, 2023, application number 202311020705.5, application name “Image processing method, device and computer equipment”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to the field of image processing, and in particular to an image processing method, apparatus, and computer equipment. Background Art

[0004] In urban archive clustering scenarios, archive clustering serves as fundamental supporting data for many higher-level practical applications, so its reliability and stability are crucial. However, when clustering images based on existing information, the clustering results can be incomplete due to missing information or the generalization capabilities of statistical learning features.

[0005] Therefore, when clustering using conventional clustering methods, there is often a problem of class splitting: the number of clusters actually divided is greater than the actual number of categories, resulting in a certain split ratio. The clusters obtained based on clustering are often relatively pure, and separating one or two outliers from such a cluster will only slightly improve the clustering performance.

[0006] Therefore, when related technologies perform image clustering on images based on conventional clustering methods, there is a problem of poor clustering effect.

[0007] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0008] Summary of the Invention

[0009] The embodiments of the present invention provide an image processing method, apparatus, and computer device to at least solve the technical problem in related arts of poor clustering effect when clustering images based on conventional clustering methods.

[0010] According to one aspect of an embodiment of the present invention, there is provided an image processing method, comprising: acquiring an image set; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0011] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining a leadership image cluster having a leadership category from the multiple initial image clusters; and determining the at least two intermediate image clusters from the leadership image clusters having the same leadership category.

[0012] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining the purity of the multiple initial image clusters respectively; and determining the at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0013] Optionally, determining the at least two intermediate image clusters from the initial image cluster having a purity greater than a predetermined purity threshold includes: separately determining image representatives of the initial image cluster having a purity greater than the predetermined purity threshold; determining the similarity between the image representatives of the initial image cluster having a purity greater than the predetermined purity threshold; and determining that the initial image clusters corresponding to the target image representatives having a similarity greater than the similarity threshold are the at least two intermediate image clusters.

[0014] Optionally, the method of separately determining the image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold includes: for a target initial image cluster whose purity is greater than the predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; determining the average similarity of any image in the target initial image cluster relative to other images; determining the image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; and based on the method of determining the image representatives of the target initial image cluster, separately determining the image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold.

[0015] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining sizes of the multiple initial image clusters respectively; and determining the at least two intermediate image clusters from initial images having sizes greater than a predetermined size threshold.

[0016] Optionally, determining at least two intermediate image clusters from the multiple initial image clusters includes: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from the multiple initial image clusters; dividing the miscellaneous cluster to obtain a leadership image cluster with a leadership category; and determining the at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster obtained by dividing the miscellaneous cluster.

[0017] Optionally, clustering the images in the image set to obtain a plurality of initial image clusters includes: obtaining similar information on whether image pairs in the image set belong to the same object; and clustering the images in the image set based on the similar information to obtain the plurality of initial image clusters.

[0018] Optionally, the fusing of the at least two intermediate image clusters to obtain the image clustering result of the image set includes: fusing the at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters other than the at least two intermediate image clusters from the multiple initial image clusters; and determining that the clustering result including the fused image cluster and the other image clusters is the image clustering result of the image set.

[0019] According to another aspect of an embodiment of the present invention, an image processing method is also provided, including: receiving a search request, wherein the search request carries product information of a target product to be searched; obtaining an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to be sets of images representing different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search results of the target product based on the image clustering result.

[0020] According to another aspect of an embodiment of the present invention, an image processing method is provided, comprising: receiving a query request, wherein the query request carries information of a target traffic participant to be queried; obtaining a traffic video in response to the query request; performing video frame splitting on the traffic video to obtain an image set including multiple video frames; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back a query result of the target traffic participant based on the image clustering result.

[0021] According to one aspect of an embodiment of the present invention, there is provided an image processing device, comprising: an acquisition module, configured to acquire an image set; a clustering module, configured to cluster images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively configured to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; a determination module, configured to determine at least two intermediate image clusters from the plurality of initial image clusters; and a fusion module, configured to fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0022] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image processing methods.

[0023] According to another aspect of an embodiment of the present invention, a computer device is provided, comprising: a memory storing an executable program; and a processor configured to run the program, wherein the program executes any one of the above-mentioned image processing methods when running.

[0024] In an embodiment of the present invention, the images in the image set are clustered to obtain a plurality of initial image clusters; at least two intermediate image clusters are determined from the plurality of initial image clusters; and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on an image as a sample, the technical effect of improving clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when clustering images based on conventional clustering methods in related technologies.

[0025] It is easy to note that the above general description and the following detailed description are for the purpose of exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0027] FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) configured to implement an image processing method;

[0028] FIG2 is a flowchart of an image processing method 1 provided according to an embodiment of the present invention;

[0029] FIG3 is a flowchart of a second image processing method provided according to an embodiment of the present invention;

[0030] FIG4 is a flowchart of a second image processing method provided according to an embodiment of the present invention;

[0031] FIG5 is a structural block diagram of an image processing device 1 provided according to an embodiment of the present invention;

[0032] FIG6 is a structural block diagram of a second image processing apparatus provided according to an embodiment of the present invention;

[0033] FIG7 is a structural block diagram of a third image processing apparatus according to an embodiment of the present invention;

[0034] FIG8 is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] First, some nouns or terms that appear in the description of the embodiments of the present invention are subject to the following explanations:

[0038] Pure class: A category in the clustering result, that is, the objects that actually belong to the same category account for more than 50%, otherwise it is called miscellaneous class;

[0039] Clustering: The process of dividing a collection of physical or abstract objects into clusters of similar objects is called clustering. The clusters generated by clustering are a collection of data objects that are similar to objects in the same cluster and different from objects in other clusters.

[0040] Active clustering: Semi-supervised clustering, also known as constrained clustering (CC), involves clustering some samples with true labels. Accurately labeling samples is expensive in many scenarios and only experts can perform this task (for example, in the classification of plants and animals, or when labeling samples with very large collections of labels). Therefore, active clustering generally considers introducing pairwise consistency information: must-link (two samples are of the same type) and cannot-link (two samples are of different types). This labeling task is relatively simple (it only requires determining whether two samples are of the same nature) and can be performed well by non-expert annotators.

[0041] Euclidean distance: refers to the distance between two points in Euclidean space, which is set to calculate the distance between the eigenvectors of a certain mode between two objects.

[0042] Eigenvector: A one-dimensional array calculated for an image. Usually, the similarity between two images can be obtained by calculating the Euclidean distance between their eigenvectors.

[0043] Cosine similarity: Evaluates the similarity between two vectors by calculating the cosine of the angle between them. For example, it can be set to calculate the similarity between the feature data of a certain modality between two objects.

[0044] Matching probability: the probability that a pair (two) objects belong to the same cluster category.

[0045] Indicator function: The letter I represents the indicator function, I(True)=1, I(False)=0.

[0046] Cluster and Class: For ease of distinction, in this embodiment, “cluster” refers to the sample division obtained by clustering, and “class” refers to the actual category of the sample.

[0047] Leading class and Mess cluster: If there is a certain type of sample in a cluster obtained by clustering, and its number is not less than half of the cluster size, then this class is called the leading class of this cluster. Otherwise, this cluster has no leading class and is called a mess cluster.

[0048] Purity: For each cluster obtained by clustering, its purity is the proportion of samples of the class with the highest proportion in this cluster. The total purity of the cluster is the sum of the number of samples of the class with the highest proportion in each cluster divided by the total number of samples.

[0049] Class fusion (Merge): If two clusters both have a leader category and the leader categories are the same, the operation of merging the two clusters into one cluster is called class fusion.

[0050] Purify: If a cluster is a mixed cluster, the operation of splitting it into several small clusters with purity 1 is called class purification.

[0051] Query State and State Matrix: Query State Set = {-1: different class; 1: same class; 0: unlabeled}, state matrix S∈{-1,0,1}^(N×N), S_ij represents the query state of sample pair (i,j).

[0052] Example 1

[0053] According to an embodiment of the present invention, a method embodiment of an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0054] The method embodiment provided in the first embodiment of the present invention can be executed in a mobile terminal, a computer terminal, or a similar computing device. Figure 1 shows a hardware block diagram of a computer terminal (or mobile device) configured to implement the image processing method. As shown in Figure 1, the computer terminal 10 (or mobile device) may include one or more processors 102 (illustrated as 102a, 102b, ..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 configured to store data, and a transmission device 106 configured to perform communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 1 is for illustration only and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in Figure 1, or have a configuration different from that shown in Figure 1.

[0055] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present invention, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0056] The memory 104 can be configured to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the image processing method in the embodiment of the present invention. The processor 102 executes the software programs and modules stored in the memory 104 to perform various functional applications and data processing, that is, to implement the above-mentioned image processing method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0057] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module configured to communicate with the Internet wirelessly.

[0058] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0059] Under the above operating environment, the present disclosure provides an image processing method as shown in FIG2. FIG2 is a flow chart of an image processing method 1 provided according to an embodiment of the present invention. The image processing method 1 includes the following steps:

[0060] Step S202, obtaining an image set;

[0061] As an optional embodiment, the method of this embodiment can be executed by a terminal or server configured to perform image processing. For example, when applied to a terminal configured to perform image processing, it can easily implement image processing in simple image scenarios. For another example, when applied to a server, it can utilize the server's abundant computing resources or a relatively larger and more accurate image model, thereby enabling more accurate image recognition.

[0062] It should be noted that the aforementioned terminals can be of various types, such as mobile terminals with certain computing capabilities or fixed computers with identification capabilities. The aforementioned servers can also be of various types, such as local servers or virtual cloud servers. Depending on the computing power, the servers can be single computers or computer clusters integrating multiple computers.

[0063] As an optional embodiment, the above-mentioned image set can be a collection of atlases in a database. For example, in an e-commerce scenario, the image set can be a collection of images in an e-commerce image library. The image set includes multiple images of at least one product, and the image set can include images of multiple products. In a traffic scenario, the image set can be a collection of multi-frame images obtained by splitting a video. The collection of multi-frame images includes at least one traffic participant, and the collection of multi-frame images includes multiple traffic participants. The above-mentioned traffic participants can only be any objects involved in traffic, such as people, vehicles, some traffic facilities, traffic signs, etc.

[0064] Step S204: clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object;

[0065] As an optional embodiment, when clustering images in an image collection to obtain multiple initial image clusters, various methods can be used. For example, clustering can be performed using different model algorithms, or a combination of multiple clustering methods. It should be noted that any of the multiple initial image clusters obtained through clustering is a collection of images belonging to the same object, obtained through clustering. For example, consider the collection of images included in an e-commerce image library, where the products included in the image library include clothing, pants, shoes, etc. Furthermore, clothing, pants, and shoes each have a variety of brands, styles, colors, etc. In an optional embodiment of the present invention, when clustering is performed based on objects, that is, clustering is performed based on products, the product corresponding to the initial image cluster obtained through clustering can refer to that individual product, not a category of products. That is, all images in an initial image cluster obtained through clustering contain that product. It should be noted that the clustering operation refers to finding a collection of images belonging to the same product in the image collection, and treating the images containing that same product as a category.

[0066] Step S206 , determining at least two intermediate image clusters from the multiple initial image clusters;

[0067] As an optional embodiment, at least two intermediate image clusters are determined from the multiple initial image clusters, and the at least two intermediate image clusters are subsequently fused. Determining the at least two intermediate image clusters from the multiple initial image clusters can be done in a variety of ways, each of which is described below with examples.

[0068] For example, to determine at least two intermediate image clusters from multiple initial image clusters, the following processing method can be used: determine a leadership image cluster with a leadership category from multiple initial image clusters; and determine at least two intermediate image clusters from the leadership image clusters with the same leadership category.

[0069] A leading category refers to a cluster in which the number of images belonging to the same category (e.g., the same product in an e-commerce scenario) exceeds a predetermined threshold (e.g., expressed as a ratio, which may be 50% of the total number of clusters). Of course, in alternative embodiments of the present invention, this threshold may also be flexibly modified as needed.

[0070] By determining the at least two intermediate image clusters from leader image clusters of the same leader category, and because the purity of images belonging to the same subject in the leader image clusters exceeds a certain threshold, the at least two intermediate image clusters can be quickly determined. Furthermore, since the purity of the leader image clusters exceeds the certain threshold, subsequent clustering based on a fusion operation can ensure clustering efficiency without guaranteeing clustering effectiveness.

[0071] For another example, to further improve the clustering effect, when determining at least two intermediate image clusters from multiple initial image clusters, the following processing method can be used: determine the purity of each of the multiple initial image clusters; and determine at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold. That is, directly based on the purity of the multiple initial image clusters, at least two intermediate image clusters are determined based on the initial image clusters whose purity is greater than the predetermined purity threshold. The predetermined purity threshold can be flexibly adjusted based on needs. For example, when the clustering accuracy is not high, the predetermined purity threshold can be set lower; when the clustering accuracy requirement is relatively high, the predetermined purity threshold can be set relatively higher. By accurately determining at least two intermediate image clusters based on purity, the clustering effect obtained by subsequent fusion based on the at least two intermediate image clusters can be accurately guaranteed.

[0072] Determining at least two intermediate image clusters based on initial image clusters with purities greater than a predetermined purity threshold can employ the following processing: determining image representatives for each initial image cluster with purities greater than the predetermined purity threshold; determining the similarity between image representatives for each initial image cluster with purities greater than the predetermined purity threshold; and determining that the initial image clusters corresponding to target image representatives with similarities greater than the similarity threshold are at least two intermediate image clusters. Based on the similarities between the image representatives for the initial image clusters, the initial image clusters corresponding to the target image representatives with similarities greater than the similarity threshold are determined as the at least two intermediate image clusters. Because the image representatives can represent the corresponding initial image clusters to a certain extent, the similarity between the image representatives is used to represent the overall similarity between the initial image clusters. This improves the efficiency of determining the at least two intermediate image clusters, as similarities do not need to be calculated for all images in the initial image cluster.

[0073] Wherein, respectively determining image representatives of initial image clusters whose purity is greater than a predetermined purity threshold comprises: for a target initial image cluster whose purity is greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; determining an average similarity of any image in the target initial image cluster relative to other images; determining an image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; and based on a method for determining image representatives of the target initial image cluster, respectively determining image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold.

[0074] By employing the above-described process, i.e., based on the average similarity of any image in the initial image cluster relative to other images in the cluster, and selecting the image with the highest average similarity within the cluster as the representative image, a statistical approach is employed to obtain the image that best represents the initial image cluster. It should be noted that image-to-image similarity here can refer to the probability that the product in an image is the same product as products in other images. The similarity probability can be determined based on the Euclidean distance or cosine distance between image features.

[0075] For another example, to determine at least two intermediate image clusters from multiple initial image clusters, the following process can be employed: the sizes of the multiple initial image clusters are determined; and at least two intermediate image clusters are determined from the initial images whose sizes are greater than a predetermined size threshold. Determining at least two intermediate image clusters based on the sizes of the initial image clusters results in more pronounced clustering effects, as larger clusters are fused. Therefore, fusion of clusters based on the sizes of the initial image clusters effectively improves clustering effectiveness and efficiency.

[0076] It should be noted that the above method of determining at least two intermediate image clusters from multiple initial image clusters is not a standalone operation; multiple methods can be combined. For example, the at least two intermediate image clusters can be determined based on a combination of the purity and size of the multiple initial image clusters. Determining the at least two intermediate image clusters based on multiple factors can achieve more efficient and optimal clustering results.

[0077] In addition, when determining at least two intermediate image clusters from multiple initial image clusters, the following processing method can also be used: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from the multiple initial image clusters; dividing the miscellaneous cluster to obtain a leadership image cluster with a leadership category; and determining at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster divided from the miscellaneous cluster.

[0078] The above process disregards the leading image clusters among the multiple initial image clusters, and instead partitions the miscellaneous clusters without a leading category. The leading image clusters are then derived from the resulting partitioned clusters. By deriving the leading image clusters from the partitioned clusters, the initial image clusters are purified. Fusion based on the purified image clusters enhances the purity of the clustering results, effectively improving the clustering effect. It should be noted that the better clustering effect mentioned in the embodiments of the present invention means that images belonging to the same object are more likely to be grouped into the same category.

[0079] As an optional embodiment, clustering the images in the image set to obtain multiple initial image clusters includes: obtaining similar information on whether the image pairs in the image set belong to the same object; and clustering the images in the image set based on the similar information to obtain multiple initial image clusters. Based on the similar information on whether the images belong to the same object, the efficiency of clustering the images in the image set can be improved. For example, when the first image and the second image belong to the same category, and the second image and the third image belong to the same category, then the first image and the third image also belong to the same category. Based on the similar information, it is deduced whether two objects belong to the same category, which avoids the problem of large amount of calculation caused by directly calculating whether they are of the same category based on the images separately. By utilizing the implicit relationship between the above categories, the efficiency of clustering images is effectively improved. It should be noted that the similar category here can refer to the same object.

[0080] Step S208: Fusing at least two intermediate image clusters to obtain an image clustering result of the image set.

[0081] Optionally, fusing the at least two intermediate image clusters to obtain an image clustering result for the image set includes: fusing the at least two intermediate image clusters to obtain a fused image cluster; obtaining image clusters other than the at least two intermediate image clusters from the plurality of initial image clusters; and determining a clustering result including the fused image cluster and the other image clusters as the image clustering result for the image set. Through the above processing, the image clustering result for the image set obtained includes the fused image cluster obtained by fusing the at least two intermediate image clusters and the other image clusters other than the at least two intermediate image clusters from the plurality of initial image clusters.

[0082] Through the above embodiments, the images in the image set are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; and the at least two intermediate image clusters are fused to obtain the image clustering results of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on the image as a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when the related technology clusters images based on conventional clustering methods.

[0083] FIG3 is a flow chart of a second image processing method according to an embodiment of the present invention. As shown in FIG3 , the second image processing method includes the following steps:

[0084] Step S302: receiving a search request, wherein the search request carries product information of a target product to be searched;

[0085] As an optional embodiment, the target products may be various products available on an e-commerce platform, for example, products classified based on multiple classification criteria. For example, the target products may be products of various categories of clothing, various categories of electrical appliances, various categories of electronic products, etc.

[0086] Step S304: In response to the search request, an image set is obtained; the images in the image set are clustered to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent sets of images of different products, and the same initial image cluster represents a set of images of the same product; at least two intermediate image clusters are determined from the plurality of initial image clusters, and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set;

[0087] Step S306: Feedback the search results of the target product based on the image clustering results.

[0088] As an optional embodiment, when providing feedback on search results for a target product based on image clustering results, a target image cluster corresponding to the target product is determined from the image clustering results, and the products included in the target image cluster are provided as feedback on the search results. It should be noted that when providing feedback on the product images included in the target image cluster as search results, the product images included in the target image cluster may be processed and the processed product images provided as feedback on the search results. The processing of the products may include ranking the product images based on the degree to which the images represent the target product, and providing feedback on the ranked product images.

[0089] Through the above-mentioned embodiments, based on a search request for a target product, the images in the image set are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; the at least two intermediate image clusters are fused to obtain an image clustering result of the image set, and the search results of the target product are fed back based on the image clustering result. By fusing the at least two intermediate image clusters, the purpose of rapidly clustering products based on cluster fusion is achieved. Compared with clustering images separately as a sample, the technical effect of improving clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when clustering images based on conventional clustering methods in related technologies. Moreover, when applied to search scenarios, since the clustering effect of the target product is clearer, more accurate, and more efficient, the search results fed back to the user can also be more accurate, effectively improving the user's search experience.

[0090] It should be noted that when clustering product images in the product library based on the search request, the search results obtained are also related to the product information of the target product to be searched. For example, when the product information of the target product to be searched is more accurate and unique, the corresponding search results obtained are also more accurate.

[0091] FIG4 is a flow chart of a third image processing method according to an embodiment of the present invention. As shown in FIG4 , the third image processing method includes the following steps:

[0092] Step S402: receiving a query request, wherein the query request carries information of a target traffic participant to be queried;

[0093] As an optional embodiment, the target traffic participant may be a variety of objects, such as a person, a vehicle, a traffic facility, or a traffic sign.

[0094] The target traffic participant information can be configured to describe the traffic participant, such as a pedestrian, traffic police officer, or sanitation worker if the traffic participant is a person. If the traffic participant is a vehicle, the information can include the vehicle model, brand, color, and appearance. The query request can be configured to request a specific traffic participant, such as a test vehicle.

[0095] Step S404, in response to the query request, obtaining traffic video;

[0096] As an optional embodiment, the above-mentioned traffic video can be a video collected by traffic collection equipment on the road, a video stored in a traffic database, a video recorded by a driving recorder of a vehicle on the road, or a traffic-related video provided by other personnel, etc.

[0097] Step S406, splitting the traffic video into video frames to obtain an image set including multiple video frames;

[0098] As an optional embodiment, when splitting traffic videos, multiple splitting methods can be used. For example, predetermined software can be used for splitting. By flexibly configuring the splitting time interval, flexible splitting of traffic videos can be achieved to obtain an image set including multiple video frames that meets the requirements.

[0099] Step S408: clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant;

[0100] As an optional embodiment, the multiple initial image clusters are each configured to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant. For example, one of the multiple initial image clusters may be a set of images representing the same traffic toll booth, and another of the multiple initial image clusters may be a set of images representing a car with a flamingo pattern painted on its body, etc.

[0101] Step S410, determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set;

[0102] Step S412: Based on the image clustering result, the query result of the target traffic participant is fed back.

[0103] As an optional embodiment, when providing feedback on a query result for a target traffic participant based on the image clustering results, an image cluster corresponding to the target traffic participant is found from the image clustering results, and the images included in the image cluster are provided as feedback on the query result. It should be noted that when providing feedback on the images included in the image cluster as the query result, the images included in the image cluster may also be processed and the processed images provided as feedback on the query result. For example, the location of the target traffic participant in the images included in the image cluster may be marked, and the marked images may be displayed.

[0104] Through the above embodiment, based on the query request for querying the target traffic participant, the images in the image set obtained by splitting the traffic video frames are clustered to obtain multiple initial image clusters; at least two intermediate image clusters are determined from the multiple initial image clusters; the at least two intermediate image clusters are fused to obtain the image clustering results of the image set, and the query results of the target traffic participant are fed back based on the image clustering results. By fusing the at least two intermediate image clusters, the purpose of quickly clustering the target traffic participant based on cluster fusion is achieved. Compared with clustering each image as a sample, the technical effect of improving the clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when clustering images based on conventional clustering methods in related technologies. Moreover, when applied to the target traffic participant scenario, since the clustering effect of the target traffic participant is more clear, accurate and efficient, the query results fed back to the user can also be more accurate, effectively improving the user's query experience.

[0105] Based on the above embodiment and optional embodiment, an optional implementation manner is provided.

[0106] Given that the aforementioned technologies often struggle with poor clustering results when clustering products in images based on conventional clustering methods, human-machine collaboration to complete key tasks can significantly improve results in real-world applications. Maximizing clustering results with minimal manual intervention, or minimizing manual annotation to achieve the desired effect, is particularly important.

[0107] In related technologies, two main approaches are used to achieve active clustering: Active Constraint Acquisition and Active Clustering with Constraints. The difference lies in whether manual labeling and cluster adjustment are performed simultaneously. Some algorithms consider first determining all pairs that need to be labeled before performing constrained clustering, while others continuously adjust the clustering results using existing labels and then select new pairs based on this. Furthermore, there are two approaches to sample selection in active clustering: pair-based and sample-based. Pair-based refers to the independent selection of sample pairs, while sample-based methods use human labeling to construct a clustering skeleton: the relationships between representative samples of each category are annotated, and the remaining samples are then divided based on this cluster skeleton. Because the complexity of pair-based methods is O(N^2), where N is the number of samples, they are often only suitable for data with a small number of categories and samples.

[0108] Probabilistic clustering and conventional clustering often suffer from the problem of class splitting: the number of actual clusters is greater than the true number of classes, i.e., there is a certain split ratio. Probabilistic clustering often generates high-purity clusters, and isolating one or two outliers from such clusters has little effect on clustering.

[0109] That is, in an optional embodiment of the present invention, the clustering effect is iteratively optimized by quantitatively evaluating the improvement in clustering effect brought about by manual labeling, so that a large improvement in effect can be achieved with as little manual labeling as possible. In an optional embodiment of the present invention, miscellaneous classes are automatically judged by introducing a probability metric, and miscellaneous classes are converted into pure classes through maximum likelihood probability clustering, and the expected normalized mutual information (NMI) index improvement of the combination of different pure classes is calculated, and representative samples of the best class pairs are selected for labeling. The optional embodiment of the present invention can be applied to the clustering of large-scale urban data, and a significant improvement in effect can be achieved with a small amount of manual intervention.

[0110] In an optional embodiment of the present invention, the goals of active probabilistic clustering are: 1) to find the clusters corresponding to the classes that were split during the clustering process and merge them; 2) to purify the found mixed clusters.

[0111] To achieve the above goals, the following approaches can be adopted.

[0112] 1. Cluster fusion based on cluster purity and size

[0113] Assume that the maximum likelihood solution obtained by probabilistic clustering of N samples is Ω={w1,w2,…,w k}, the real clustering is divided into C = {c1,c2,…,c J}, the NMI value of Ω is n1. For any two clusters, let them be w1 and w2, if they both have a leading category, their purity is t1 and t2 respectively, and the cluster sizes are p and q respectively. Then there exists t∈(min(t1, t2), max(t1, t2)). When w1 and w2 are merged into one cluster, if n1≥2×(1-t), then the new cluster partition Ω′={w 1,2 ,w3,…,w k}’s NMI value satisfies n2≥n1.

[0114] Based on the above-mentioned class fusion operation, clusters can be fused based on their purity and size. For example, clusters with a high purity (not less than 50%) should be selected for fusion. For mixed clusters, the samples of mixed categories need to be separated before fusion.

[0115] 2. Clustering based on latent labeling reasoning

[0116] Pair-based active clustering query strategies suffer from an inefficient utilization problem. After manually annotating several sample pairs, some unannotated relationships between them may already be known (referred to as latent known pairs). For example, if x1 and x2 are of the same type, and x2 and x3 are also of the same type, then x1 and x3 must also be of the same type. Traditional active learning algorithms fail to utilize this latent information. Therefore, in this alternative implementation, an efficient annotation information inference algorithm is proposed to address this issue.

[0117] Each time a pair of samples {i, j} is manually labeled, the state value is updated using the algorithm. If there is no potential known sample pair related to {i, j} (ml or cl between i or j), the state value is 0.

[0118] Assume that the sample sets with i,j as ml are G i ,G j (including i, j), and the sample sets where i, j are cl are G′ i ,G′ j .

[0119] 1. When i=j:

[0120] Update the relevant status of sample i: G after ml of i is updated i =G i +j, while leading to G j =G j +G i , and after i's cl is updated G′ j =G′ j +G′ i , note that at this time j and G i ,G′ i The status between samples is updated, but the status between the remaining samples is not updated.

[0121] Update the state of sample j: j's ml is updated after G i =G i +G j , at this time G i ,G j The state values ​​of all sample pairs become 1. After the cl of j is updated, G′ i =G′ i +G′ j , at this time {G i ,G j} and {G′ i ,G′ j} all state values ​​between them become -1, that is, all potential known sample pairs are updated.

[0122] 2. When i≠j, the update of ml will not cause the state value to change

[0123] Update the cl-related state of sample i: G′ j =G′ j +G i Update the cl-related state of sample j: G′ i =G′ i +G j , at this time G i ,G j The state values ​​between them all become -1, that is, all potential known sample pairs are updated.

[0124] 3. Recall of clusters of the same category

[0125] When labeling pairs of manually annotated samples, we prioritize clusters that are likely to be of the same type. We then determine whether they share a leading category. We then ask the annotators to determine whether their leading categories are the same. If they are, we force the two clusters to merge into one. The key to this process is estimating the probability that two clusters can be merged.

[0126] When selecting cluster pairs, prioritize those that maximize NMI improvement. The aforementioned theory shows that the change in NMI after fusion is determined by the purity and size of the two clusters. Clusters derived from probabilistic clustering tend to have higher purity. Therefore, assuming purity is 1, the greater the change Δh from the pre-fusion entropy value H(Ω) to the post-fusion entropy value H(Ω′), the greater the NMI improvement. There are two conclusions regarding Δh: 1. When p+q is fixed, the smaller |pq|, the larger Δh; 2. The larger p and q, the larger Δh.

[0127] Finally, the following expected returns are used to recall cluster-pairs.

[0128] The leading class and representative samples can be determined using a greedy selection algorithm: A cluster's purity is determined to meet subsequent fusion requirements through at most two manual annotations. Here, the purity is randomly sampled from a uniform distribution in the range [0.5, 0.7] to avoid the small probability of encountering an outlier when the purity lower limit is fixed at 0.5. Optionally, Gibbs sampling can be used to determine the purity of a class.

[0129] It should be noted that the active probabilistic clustering algorithm can only recall cluster pairs with a fusion probability greater than a preset threshold. (The vast majority of other cluster pairs have a very low fusion probability and are unlikely to be of the same type, so they are not worth labeling.) Furthermore, in the above algorithm, each time a pair of samples is manually labeled, the algorithm is immediately called to update the state.

[0130] Based on the above optional implementation method, starting from the granularity of the category, the complexity of the calculation and judgment required for active clustering is greatly reduced. And from the statistical perspective of probability, the effect gain obtained by all category judgments is quantified, and the clustering index is optimized in a targeted manner. By quantitatively analyzing the improvement in clustering effect that may be brought about by the discrimination of category pairs, the best sample pairs are selected for annotation, which greatly reduces the amount of manual annotation required to achieve certain indicators, and at the same time maximizes the clustering effect through a small amount of manual annotation; this solution can be generally applied to the clustering results obtained by various clustering algorithms, and with the addition of a matching probability function, the class purity can be judged, the miscellaneous categories can be purified, and the pure categories with the highest expected improvement can be continuously selected for manual judgment.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0132] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.

[0134] Example 2

[0135] According to an embodiment of the present invention, a device configured to implement the above-mentioned image processing method 1 is also provided. Figure 5 is a structural block diagram of the image processing device 1 provided according to an embodiment of the present invention. As shown in Figure 5, the device includes: a first acquisition module 52, a first clustering module 54, a first determination module 56 and a first fusion module 58. The device is described below.

[0136] A first acquisition module 52 is configured to acquire an image set; a first clustering module 54 is connected to the first acquisition module 52 and is configured to cluster the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively configured to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; a first determination module 56 is connected to the first clustering module 54 and is configured to determine at least two intermediate image clusters from the plurality of initial image clusters; a first fusion module 58 is connected to the first determination module 56 and is configured to fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0137] It should be noted that the first acquisition module 52, the first clustering module 54, the first determination module 56, and the first fusion module 58 correspond to steps S202 to S208 in Example 1. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be run as part of the device in the computer terminal 10 provided in Example 1.

[0138] According to an embodiment of the present invention, a device configured to implement the above-mentioned image processing method 2 is also provided. Figure 6 is a structural block diagram of the image processing device 2 provided according to an embodiment of the present invention. As shown in Figure 6, the device includes: a first receiving module 62, a first processing module 64 and a first feedback module 66. The device is described below.

[0139] The first receiving module 62 is configured to receive a search request, wherein the search request carries product information of a target product to be searched; the first processing module 64 is connected to the above-mentioned first receiving module 62, and is configured to obtain an image set in response to the search request; cluster the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively configured to represent a set of images of different objects, and the same initial image cluster represents a set of images of the same object; at least two intermediate image clusters are determined from the plurality of initial image clusters, and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set; the first feedback module 66 is connected to the above-mentioned first processing module 64, and is configured to feed back the search results of the target product based on the image clustering result.

[0140] It should be noted that the first receiving module 62, the first processing module 64, and the first feedback module 66 correspond to steps S302 to S306 in Example 1. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of the device and can be run in the computer terminal 10 provided in Example 1.

[0141] According to an embodiment of the present invention, a device configured to implement the above-mentioned image processing method three is also provided. Figure 7 is a structural block diagram of the image processing device three provided according to an embodiment of the present invention. As shown in Figure 7, the device includes: a second receiving module 72, a second acquisition module 74, a splitting module 76, a second clustering module 78, a second determination module 710 and a second feedback module 712. The device is described below.

[0142] The second receiving module 72 is configured to receive a query request, wherein the query request carries information of a target traffic participant to be queried; the second acquiring module 74 is connected to the second receiving module 72 and is configured to acquire a traffic video in response to the query request; the splitting module 76 is connected to the second acquiring module 74 and is configured to split the traffic video into video frames to obtain an image set including a plurality of video frames; the second clustering module 78 is connected to the splitting module 76 and clusters the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively configured to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; the second determining module 710 is connected to the second clustering module 78 and is configured to determine at least two intermediate image clusters from the plurality of initial image clusters, and fuse the at least two intermediate image clusters to obtain an image clustering result of the image set; the second feedback module 712 is connected to the second determining module 710 and is configured to feedback the query result of the target traffic participant based on the image clustering result.

[0143] It should be noted that the second receiving module 72, the second acquisition module 74, the splitting module 76, the second clustering module 78, the second determination module 710, and the second feedback module 712 correspond to steps S402 to S412 in Example 1. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be run as part of the device in the computer terminal 10 provided in Example 1.

[0144] Example 3

[0145] The embodiment of the present disclosure may provide a computer terminal (or computer device), which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0146] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0147] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method: obtaining an image set; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the multiple initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0148] Alternatively, Figure 8 is a block diagram of a computer terminal according to an embodiment of the present invention. As shown in Figure 8, the computer terminal may include: one or more (one is shown in the figure) processors 82, a memory 84, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0149] Among them, the memory can be configured to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and device in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an image set; cluster the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determine at least two intermediate image clusters from the multiple initial image clusters; and fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0151] Optionally, the processor may further execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: determining a leadership image cluster having a leadership category from a plurality of initial image clusters; and determining at least two intermediate image clusters from leadership image clusters having the same leadership category.

[0152] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: determining the purity of the plurality of initial image clusters respectively; and determining at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0153] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from an initial image cluster whose purity is greater than a predetermined purity threshold, including: separately determining image representatives of the initial image cluster whose purity is greater than a predetermined purity threshold; determining the similarity between image representatives of the initial image cluster whose purity is greater than a predetermined purity threshold; and determining that the initial image clusters corresponding to the target image representatives whose similarity is greater than the similarity threshold are at least two intermediate image clusters.

[0154] Optionally, the processor may also execute program code for the following steps: respectively determining image representatives of initial image clusters having a purity greater than a predetermined purity threshold, including: for a target initial image cluster having a purity greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster having a purity greater than a predetermined purity threshold; determining the average similarity of any image in the target initial image cluster relative to other images; determining the image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; and based on a method for determining image representatives of the target initial image cluster, respectively determining image representatives of initial image clusters having a purity greater than a predetermined purity threshold.

[0155] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: determining the sizes of the plurality of initial image clusters respectively; and determining at least two intermediate image clusters from initial images whose sizes are greater than a predetermined size threshold.

[0156] Optionally, the processor may also execute program code for the following steps: determining at least two intermediate image clusters from a plurality of initial image clusters, including: determining a leadership image cluster having a leadership category and a miscellaneous cluster without a leadership category from the plurality of initial image clusters; dividing the miscellaneous cluster to obtain a leadership image cluster having a leadership category; and determining at least two intermediate image clusters based on the leadership image cluster determined from the plurality of initial image clusters and the leadership image cluster obtained by dividing the miscellaneous cluster.

[0157] Optionally, the processor may also execute program code for the following steps: clustering images in the image set to obtain multiple initial image clusters, including: obtaining similar information on whether image pairs in the image set belong to the same object; and clustering images in the image set based on similar information to obtain multiple initial image clusters.

[0158] Optionally, the processor may also execute program code for the following steps: fusing at least two intermediate image clusters to obtain an image clustering result of the image set, including: fusing at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters other than the at least two intermediate image clusters from a plurality of initial image clusters; and determining the clustering result including the fused image cluster and other image clusters as the image clustering result of the image set.

[0159] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving a search request, wherein the search request carries product information of the target product to be searched; obtaining an image set in response to the search request; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent sets of images of different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search results of the target product based on the image clustering result.

[0160] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving a query request, wherein the query request carries information of the target traffic participant to be queried; obtaining a traffic video in response to the query request; performing video frame splitting on the traffic video to obtain an image set including multiple video frames; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic reference person; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the query result of the target traffic participant based on the image clustering result.

[0161] According to an embodiment of the present invention, images in an image set are clustered to obtain a plurality of initial image clusters; at least two intermediate image clusters are determined from the plurality of initial image clusters; and the at least two intermediate image clusters are fused to obtain an image clustering result of the image set. By fusing the at least two intermediate image clusters, the purpose of rapidly clustering images based on cluster fusion is achieved. Compared with clustering based on an image as a sample, the technical effect of improving clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when clustering images based on conventional clustering methods in related technologies.

[0162] Those skilled in the art will appreciate that the structure shown in FIG8 is for illustration only, and the computer terminal may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG8 does not limit the structure of the above-mentioned electronic devices. For example, the computer terminal may include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG8 , or may have a configuration different from that shown in FIG8 .

[0163] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0164] Example 4

[0165] The embodiment of the present disclosure further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium may be configured to store the program code executed by the image processing method provided in the first embodiment.

[0166] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0167] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code configured to execute the following steps: acquiring an image set; clustering images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the multiple initial image clusters; and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set.

[0168] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code configured to execute the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining a leadership image cluster with a leadership category from multiple initial image clusters; and determining at least two intermediate image clusters from leadership image clusters with the same leadership category.

[0169] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code configured to perform the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: respectively determining the purity of the multiple initial image clusters; and determining at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold.

[0170] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code configured to execute the following steps: determining at least two intermediate image clusters from the initial image cluster whose purity is greater than a predetermined purity threshold, including: respectively determining image representatives of the initial image cluster whose purity is greater than the predetermined purity threshold; determining the similarity between the image representatives of the initial image cluster whose purity is greater than the predetermined purity threshold; and determining that the initial image clusters corresponding to the target image representatives whose similarity is greater than the similarity threshold are at least two intermediate image clusters.

[0171] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code configured to execute the following steps: respectively determining image representatives of initial image clusters having a purity greater than a predetermined purity threshold, including: for a target initial image cluster having a purity greater than a predetermined purity threshold, obtaining images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster having a purity greater than the predetermined purity threshold; determining the average similarity of any image in the target initial image cluster relative to other images; determining the image with the highest average similarity in the target initial image cluster as the image representative of the target initial image cluster; based on the method for determining the image representative of the target initial image cluster, respectively determining image representatives of the initial image clusters having a purity greater than the predetermined purity threshold.

[0172] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code configured to perform the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining the sizes of the multiple initial image clusters respectively; and determining at least two intermediate image clusters from the initial images whose sizes are greater than a predetermined size threshold.

[0173] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code configured to execute the following steps: determining at least two intermediate image clusters from multiple initial image clusters, including: determining a leadership image cluster with a leadership category and a miscellaneous cluster without a leadership category from the multiple initial image clusters; dividing the miscellaneous cluster to obtain a leadership image cluster with a leadership category; and determining at least two intermediate image clusters based on the leadership image cluster determined from the multiple initial image clusters and the leadership image cluster obtained by dividing the miscellaneous cluster.

[0174] Optionally, in this embodiment, the computer-readable storage medium is also configured to store program code configured to execute the following steps: clustering the images in the image set to obtain multiple initial image clusters, including: obtaining similar information of whether the image pairs in the image set belong to the same object; based on the similar information, clustering the images in the image set to obtain multiple initial image clusters.

[0175] Optionally, in this embodiment, the computer-readable storage medium is further configured to store program code configured to execute the following steps: fusing at least two intermediate image clusters to obtain an image clustering result of the image set, including: fusing at least two intermediate image clusters to obtain a fused image cluster; obtaining other image clusters other than the at least two intermediate image clusters in multiple initial image clusters; and determining the clustering result including the fused image cluster and other image clusters as the image clustering result of the image set.

[0176] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code configured to execute the following steps: receiving a search request, wherein the search request carries product information of a target product to be searched; obtaining an image set in response to the search request; clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent a set of images of different products, and the same initial image cluster represents a set of images of the same product; determining at least two intermediate image clusters from the plurality of initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the search results of the target product based on the image clustering result.

[0177] Optionally, in this embodiment, a computer-readable storage medium is configured to store a program code configured to execute the following steps: receiving a query request, wherein the query request carries information of a target traffic participant to be queried; obtaining a traffic video in response to the query request; performing video frame splitting on the traffic video to obtain an image set including multiple video frames; clustering the images in the image set to obtain multiple initial image clusters, wherein the multiple initial image clusters are respectively set to represent a set of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; and feeding back the query result of the target traffic participant based on the image clustering result.

[0178] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0179] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0180] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are schematic. For example, the division of the units is a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0181] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0184] The above is a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as within the scope of protection of the present disclosure. Industrial Applicability

[0185] The disclosed embodiments propose an image processing method, apparatus, and computer device, which cluster images in the image set to obtain multiple initial image clusters; determine at least two intermediate image clusters from the multiple initial image clusters; and fuse the at least two intermediate image clusters to obtain an image clustering result of the image set. By fusing the at least two intermediate image clusters, the purpose of quickly clustering images based on cluster fusion is achieved. Compared with clustering based on an image as a sample, the technical effect of improving clustering effect and clustering efficiency is effectively achieved, thereby solving the technical problem of poor clustering effect when clustering images based on conventional clustering methods in related technologies.

Claims

1. An image processing method, comprising: Get the image set; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; determining at least two intermediate image clusters from the plurality of initial image clusters; The at least two intermediate image clusters are fused to obtain an image clustering result of the image set.

2. The method according to claim 1, wherein: The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: determining a leadership image cluster having a leadership category from the plurality of initial image clusters; The at least two intermediate image clusters are determined from leader image clusters having the same leader category.

3. The method according to claim 1, wherein: The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: respectively determining the purity of the plurality of initial image clusters; The at least two intermediate image clusters are determined from the initial image clusters whose purity is greater than a predetermined purity threshold.

4. The method according to claim 3, wherein: The step of determining the at least two intermediate image clusters from the initial image clusters whose purity is greater than a predetermined purity threshold comprises: respectively determining image representatives of the initial image clusters having purity greater than the predetermined purity threshold; determining similarities between image representatives of the initial image cluster having a purity greater than the predetermined purity threshold; It is determined that the target images with similarities greater than a similarity threshold represent corresponding initial image clusters, which are the at least two intermediate image clusters.

5. The method according to claim 4, wherein: The step of respectively determining image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold comprises: For a target initial image cluster whose purity is greater than the predetermined purity threshold, acquiring images in the target initial image cluster, wherein the target initial image cluster is any initial image cluster whose purity is greater than the predetermined purity threshold; Determine the average similarity of any image in the target initial image cluster relative to other images; Determine an image with the highest average similarity in the target initial image cluster as an image representative of the target initial image cluster; Based on the method for determining the image representatives of the target initial image cluster, image representatives of the initial image clusters whose purity is greater than the predetermined purity threshold are respectively determined.

6. The method according to claim 1, wherein: Determining at least two intermediate image clusters from the plurality of initial image clusters comprises: respectively determining the sizes of the plurality of initial image clusters; The at least two intermediate image clusters are determined from initial images having a size greater than a predetermined size threshold.

7. The method according to claim 1, wherein: The step of determining at least two intermediate image clusters from the plurality of initial image clusters comprises: Determine a leading image cluster having a leading category and a miscellaneous cluster without a leading category from the plurality of initial image clusters; dividing the miscellaneous clusters to obtain a leadership image cluster having a leadership category; The at least two intermediate image clusters are determined based on the leading image cluster determined from the plurality of initial image clusters and the leading image cluster obtained by dividing the miscellaneous clusters.

8. The method according to claim 1, wherein: The clustering of the images in the image set to obtain a plurality of initial image clusters comprises: Obtaining information of the same type as to whether the image pairs in the image set belong to the same object; Based on the similar information, the images in the image set are clustered to obtain the multiple initial image clusters.

9. The method according to any one of claims 1 to 8, wherein: The fusing the at least two intermediate image clusters to obtain the image clustering result of the image set includes: fusing the at least two intermediate image clusters to obtain a fused image cluster; Acquire other image clusters except the at least two intermediate image clusters from the plurality of initial image clusters; A clustering result including the fused image cluster and the other image clusters is determined as the image clustering result of the image set.

10. An image processing method, comprising: Receiving a search request, wherein the search request carries product information of a target product to be searched; In response to the search request, acquiring an image set; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent sets of images of different commodities, and the same initial image cluster represents a set of images of the same commodity; Determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; Based on the image clustering result, the search result of the target product is fed back.

11. An image processing method, comprising: Receiving a query request, wherein the query request carries information of a target traffic participant to be queried; In response to the query request, acquiring traffic video; Splitting the traffic video into video frames to obtain an image set including a plurality of video frames; Clustering the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively set to represent sets of images of different traffic participants, and the same initial image cluster represents a set of images of the same traffic participant; Determining at least two intermediate image clusters from the multiple initial image clusters, and fusing the at least two intermediate image clusters to obtain an image clustering result of the image set; Based on the image clustering result, the query result of the target traffic participant is fed back.

12. An image processing device, comprising: An acquisition module, configured to acquire an image set; a clustering module, configured to cluster the images in the image set to obtain a plurality of initial image clusters, wherein the plurality of initial image clusters are respectively configured to represent sets of images of different objects, and the same initial image cluster represents a set of images of the same object; a determination module, configured to determine at least two intermediate image clusters from the plurality of initial image clusters; The fusion module is configured to fuse the at least two intermediate image clusters to obtain an image clustering result of the image set.

13. A computer-readable storage medium, the computer-readable storage medium comprising a stored executable program, wherein: When the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 11.

14. A computer device comprising: A memory storing an executable program; A processor is configured to run the program, wherein the program executes the image processing method according to any one of claims 1 to 11 when running.

Images