Image file inspection method, device, equipment and computer storage medium

Abstract

The present application discloses an image archive inspection method, apparatus, device and computer storage medium, which relate to the field of image clustering technology and are used to improve the accuracy of image archives. The method comprises: determining, from a plurality of candidate image archives, image archives to be processed that meet set inspection conditions; splitting the corresponding image archives to be processed based on the similarities between the archived images included in each of the determined image archives to be processed, to obtain sub-archive sets corresponding to each of the image archives to be processed; wherein, when the sub-archive set includes a plurality of sub-archives, the sum of the similarities between the archived images included in any two sub-archives is less than a splitting similarity threshold; and merging the sub-archives included in each of the obtained sub-archive sets based on the similarities between the archived images included in every two sub-archives, to obtain a plurality of target image archives.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of object retrieval technology, and provides an image archive inspection method, device, equipment and computer storage medium. Background Art

[0002] With the development of technology, we can use snapshot cameras deployed in various areas to capture images, and use image recognition technology and image clustering technology to archive these captured images to form different image archives.

[0003] Taking portrait archives as an example, each portrait archive is formed by archiving captured portrait images through portrait recognition and clustering technologies. Each portrait archive consists of multiple portrait images. However, most portrait clustering technologies are based on image recognition technology, clustering portraits based on the similarity between images. However, due to the different capture conditions in different scenes, different portrait images may have high similarities, which often leads to incorrect archiving. For example, for portrait archive A, the portrait images of pedestrians 1 and 2 are simultaneously included in portrait archive A. The mixing of portrait archives of different people will affect the representation of each portrait archive A, and subsequent recognition errors may occur when using portrait archive A for portrait recognition.

[0004] Similarly, similar problems exist in other types of image archives. Summary of the Invention

[0005] The embodiments of the present application provide an image file inspection method, apparatus, device, and computer storage medium for improving the accuracy of image files.

[0006] In one aspect, a method for inspecting an image archive is provided, the method comprising:

[0007] Determine the image files to be processed that meet the set inspection conditions from multiple candidate image files;

[0008] Based on the determined similarities between the archived images included in each of the image archives to be processed, the corresponding image archives to be processed are split to obtain sub-archive sets corresponding to each of the image archives to be processed; wherein, when the sub-archive set includes multiple sub-archives, the sum of the similarities between the archived images included in any two sub-archives is less than a splitting similarity threshold;

[0009] Based on the similarity between the archived images included in every two sub-archives, the sub-archives included in each obtained sub-archive set are merged to obtain multiple target image archives.

[0010] In one aspect, an image file inspection device is provided, comprising:

[0011] A determination unit, configured to determine an image file to be processed that meets a set inspection condition from a plurality of candidate image files;

[0012] a file splitting unit configured to split the corresponding image files to be processed based on the determined similarities between the archived images included in each image file to be processed, to obtain sub-file sets corresponding to each of the image files to be processed; wherein, when the sub-file set includes multiple sub-files, the sum of the similarities between the archived images included in any two sub-files is less than a splitting similarity threshold;

[0013] The file merging unit is used to merge the sub-files included in each sub-file set based on the similarity between the archived images included in each two sub-files to obtain multiple target image files.

[0014] Optionally, the determining unit is specifically configured to:

[0015] For the multiple candidate image files, perform the following operations respectively:

[0016] For a candidate image file, determining whether the number of archived images included in the candidate image file is greater than an inspection number threshold;

[0017] If it is greater than the inspection quantity threshold, it is determined that the candidate image file meets the inspection condition.

[0018] Optionally, the file splitting unit is specifically used to:

[0019] For each of the image files to be processed, perform the following operations respectively:

[0020] For an image archive to be processed, constructing an edge graph corresponding to the image archive to be processed based on the similarities between the archived images included therein; wherein a node in the edge graph corresponds to an archived image;

[0021] Based on the edge weights between the nodes included in the edge graph, the edge graph is sliced ​​to obtain at least one subgraph; wherein the edge weight between two nodes is positively correlated with the similarity between the corresponding two archived images;

[0022] Based on the at least one subgraph, the one to-be-processed image archive is split to obtain a corresponding set of sub-archives; wherein the archived images in each sub-archive correspond one-to-one to a node in a subgraph.

[0023] Optionally, the file splitting unit is specifically used to:

[0024] For each two archived images, determining whether the similarity between the two archived images is greater than a first similarity threshold;

[0025] If it is greater than, then there is an edge between each two archived images in the edge graph; and,

[0026] Based on the similarity between each two archived images, a weight of an edge between each two archived images is determined.

[0027] Optionally, the file splitting unit is specifically used to:

[0028] Determine multiple candidate graph-slicing methods for the edge graph based on the nodes included in the edge graph, wherein the intersection of any two subgraphs after the graph slicing is an empty set, and the union of all subgraphs is the edge graph;

[0029] Determining, based on the sum of the edge weights between two subgraphs after the graph is sliced ​​according to various candidate graph slicing methods among the multiple candidate graph slicing methods, a target graph slicing method whose sum of the edge weights is not greater than a split weight threshold from the multiple candidate graph slicing methods;

[0030] The edge graph is sliced ​​according to the determined target slicing method to obtain the at least one subgraph.

[0031] Optionally, the file merging unit is specifically configured to:

[0032] For every two sub-files in each sub-file, perform the following operations respectively:

[0033] If it is determined that the sum of the edge weights between the two subgraphs corresponding to each two sub-files is greater than the merge weight threshold, each two sub-files are merged; wherein the merge weight threshold is greater than the split weight threshold.

[0034] Optionally, the device further includes a similarity calculation unit, configured to:

[0035] Performing feature extraction on each of the archived images to obtain feature matrices corresponding to each of the archived images;

[0036] Based on the feature matrices corresponding to the respective archived images, the similarity between each two archived images is determined.

[0037] In one aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0038] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0039] In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above methods.

[0040] In the embodiment of the present application, the image archive is split by using the similarity between each archived image in each image archive, so that the sum of the similarities between the archived images included in any two sub-archives after the split is less than the split similarity threshold, while the sum of the similarities between the archived images in each sub-archive is greater than the split similarity threshold, indicating that the image archive may be incorrectly filed. Then, the image archive that was originally incorrectly filed is split, and the sub-archives are re-filed using the similarity between the archived images of each sub-archive obtained by the split. After the merging, the archived images in each image archive are more closely connected, thereby correcting the original incorrectly filed image archive, improving the accuracy of each image archive, and correspondingly improving the accuracy of downstream applications. In addition, image archives that meet the inspection conditions are selected as the archives to be processed and deselected. For example, image archives with a large number of images or a long archiving time are inspected. Since they contain sufficient archived images, the image feature distribution in the archive is relatively obvious, and the subsequent splitting and merging effects are good, further improving the accuracy of the image archive. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 Schematic diagram of application scenarios provided by embodiments of the present application;

[0043] Figure 2 A flowchart of the image file inspection method provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the process of performing file splitting using spectral clustering provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of an edge graph corresponding to an image file B to be processed provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of several possible candidate switching modes provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a connectivity graph formed after combining files according to an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of an image file inspection device provided in an embodiment of the present application;

[0049] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0051] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0052] Image Archives: To facilitate the description of the same person or object, multiple images of the same person or object can be archived in the same image archive, and multiple images in the same image archive can be used to represent the person or object to assist in downstream applications. For example, if a portrait archive A contains only portrait images of user 1, user 1 can be represented as a vector using the multiple portrait images in portrait archive A to facilitate subsequent user tracking and retrieval.

[0053] Image clustering method: a method used to implement image clustering, such as the k-means clustering algorithm (k-means) method, the density-based spatial clustering method (DBSCAN) method, and the spectral clustering method. Taking spectral clustering to inspect image archives as an example, it is equivalent to treating all images in an image archive as points in space. These points can be connected by edges. The edge weight value between two points that are farther apart, that is, two images with lower similarity, is lower, while the edge weight value between two points that are closer is higher. Then, by cutting the connected graph composed of all portrait feature data points in the archive, the sum of the edge weights between different subgraphs after cutting is as low as possible, and the sum of the edge weights within the subgraph is as high as possible, thereby achieving the purpose of clustering.

[0054] The following is a brief introduction to the design concept of the embodiments of this application.

[0055] At present, image archives are widely used. Taking portrait archives as an example, relevant administrative departments can use the generated portrait archive information to monitor the target objects in order to maintain social order.

[0056] Due to various factors and conditions, portrait files often have errors in filing, and similar problems exist not only in portrait files, but also in other image files. Therefore, it is very necessary to conduct inspections on the portrait files to identify and correct the errors.

[0057] In view of this, an embodiment of the present application provides an object-based retrieval method, in which an image archive is split by utilizing the similarity between each archived image in each image archive, so that the sum of the similarities between the archived images included in any two sub-archives after the split is less than the split similarity threshold, while the sum of the similarities between the archived images in each sub-archive is greater than the split similarity threshold, indicating that the image archive may be incorrectly archived. Then, the image archive that was originally incorrectly archived is split, and then the sub-archives are re-archived using the similarity between the archived images of each sub-archive obtained by the split. After the merging, the archived images in each image archive are more closely connected, thereby correcting the image archive that was originally incorrectly archived, improving the accuracy of each image archive, and correspondingly improving the accuracy of downstream applications.

[0058] Secondly, image archives that meet the inspection conditions are selected as the archives to be processed and deselected. For example, inspections are conducted on image archives with a large number of images or a long archiving time. Since they contain sufficient archived images, the image feature distribution in the archives is relatively obvious, and the subsequent splitting and merging effects are better, further improving the accuracy of the image archives.

[0059] In addition, the embodiment of the present application adopts the same method when opening and closing files, so as to prevent the occurrence of filing errors caused by opening and closing files at the same time.

[0060] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0061] The solution provided by the embodiment of the present application can be applied to most image archive application scenarios, such as object monitoring scenarios and object retrieval scenarios. Figure 1 As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, a front-end monitoring device 101 and a server 102 may be included.

[0062] The front-end monitoring device 101 can be, for example, a camera installed in each monitoring area. The front-end monitoring device 101 can include components such as a camera, a pan / tilt head (PTZ), a protective cover, a monitor, an alarm detector, a multi-function decoder, and a signal transceiver. The front-end monitoring device 101 can capture images through the camera and transmit them to the back-end server 102 via the signal transceiver.

[0063] The server 102 may be a backend server corresponding to the front-end monitoring device 101, and may be used to implement management functions for each front-end monitoring device 101, as well as management functions for captured images, to implement the steps of the image archive inspection method provided in the embodiment of the present application. For example, it may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited thereto.

[0064] The server 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 for interacting with a terminal. Furthermore, the server 102 may be configured with a database 1024, which may be used to store the archived images included in each image archive, the representation vectors of each archived image, and the representation vectors corresponding to each image archive. The memory 1022 of the server 102 may also store program instructions for the image archive inspection method provided in an embodiment of the present application. When executed by the processor 1021, these program instructions may be used to implement the steps of the image archive inspection method provided in an embodiment of the present application, thereby realizing the image archive inspection process.

[0065] Specifically, after the images captured by the front-end monitoring device 101 are transmitted to the server 102, the server 102 can cluster the images and file them into separate image archives. For example, the received images can be clustered regularly to form image archives. In addition, after the image archives have been formed, the newly received images can be compared with the image archives and then filed into the corresponding image archives.

[0066] After a long period of archiving, some image archives may contain a large number of archived images, and their image feature distribution may be more obvious. In this case, the image archive inspection method provided in the embodiment of the present application may be used to inspect these image archives to clean out the erroneous archived images in each image archive.

[0067] The image files obtained by the above process can be applied to downstream application scenarios.

[0068] For example, in an object retrieval scenario, the image feature representation vector of the retrieval object is extracted and matched with the representation vectors of each stored image archive to find an image archive with high similarity. In an object monitoring scenario, the corresponding image archive can also be found for the monitored object, and the object's travel trajectory can be tracked based on the archived images in the image archive.

[0069] In the embodiment of the present application, the front-end monitoring device 101 and the server 102 can be directly or indirectly connected to each other through one or more networks 103. The network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless Fidelity (WIFI) network. Of course, other possible networks are also possible, and the embodiment of the present invention does not limit this.

[0070] It should be noted that in the embodiment of the present application, the number of front-end monitoring devices 101 can be one or more, and similarly, the number of servers 102 can be one or more, that is, there is no restriction on the number of front-end monitoring devices 101 or servers 102.

[0071] In one possible application scenario, the relevant data involved in the embodiments of this application (such as archived images and representation vectors) can be stored using cloud storage technology. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (or storage nodes) in a network through application software or application interfaces to work together and provide external data storage and service access functions.

[0072] In a possible application scenario, in order to reduce the communication delay of retrieval, servers 102 can be deployed in various regions, or for load balancing, different servers 102 can serve front-end monitoring devices 101 in different regions respectively. For example, the front-end monitoring device 101 is located at location a and establishes a communication connection with the server 102 serving location a. The front-end monitoring device 101 is located at location b and establishes a communication connection with the server 102 serving location b. Multiple servers 102 form a data sharing system, and data sharing is achieved through blockchain.

[0073] Each server 102 in the data sharing system has a node identifier corresponding to that server 102. Each server 102 in the data sharing system can store the node identifiers of other servers 102 in the data sharing system so that the generated blocks can be broadcast to other servers 102 in the data sharing system based on the node identifiers of other servers 102. Each server 102 can maintain a node identifier list, storing the server 102 name and node identifier in the node identifier list. The node identifier can be an Internet Protocol (IP) address for interconnection between networks, or any other information that can be used to identify the node.

[0074] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of this application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in subsequent method embodiments and will not be elaborated on here.

[0075] The method flow provided in each embodiment of the present application can be Figure 1 The process may be executed by the server 102 or the front-end monitoring device 101, or may be executed by the server 102 and the front-end monitoring device 101 together. Here, the process is mainly described by taking the server 102 as an example.

[0076] See also Figure 2 As shown, it is a flow chart of the image file inspection method provided in an embodiment of the present application. In the subsequent introduction, the image file is mainly introduced as a portrait file as an example. Of course, other types of image files may also be applicable.

[0077] Step 201: Determine an image file to be processed that meets a set inspection condition from a plurality of candidate image files.

[0078] In the embodiment of the present application, since there are a large number of candidate image files stored in the database, when performing an inspection, it is necessary to select image files that meet certain inspection conditions for subsequent inspection processes.

[0079] In one embodiment, considering that the number of archived images accumulated in the image archive after a long period of clustering is large, the image objects in the archive have richer features, the connections between images of the same object are closer and have higher similarity, while the connections between different objects are sparser and have lower similarity, it is possible to effectively perform inspections and split the archive with high accuracy. Therefore, image archives that meet the inspection conditions can be selected for subsequent inspection processes.

[0080] Specifically, the inspection condition may be that the number of archived images needs to be greater than the inspection number threshold. Then, for each candidate image file stored in the database, it is necessary to determine whether the number of archived images included in the candidate image file is greater than the inspection number threshold. If the number of archived images is greater than the inspection number threshold, it is determined that the candidate image file meets the inspection condition, and these candidate image files that meet the inspection condition will serve as the image files to be processed in the subsequent inspection process; otherwise, if the number of archived images is not greater than the inspection number threshold, the inspection condition is not met, and the subsequent inspection process will not be performed on the candidate image file.

[0081] In one embodiment, the inspection conditions can also be set according to the time period. That is, considering the large number of candidate image archives, the inspection can be carried out in different times. That is, each inspection can target the archived images within a time period. Then, when conducting an inspection, it is necessary to determine which candidate image archives are targeted by this inspection and screen them out as image archives to be processed.

[0082] In one embodiment, inspection conditions can be set based on the feature distribution of the candidate image files. Specifically, candidate image files with dispersed feature distributions are likely to be misfiled, so inspections can be performed on these candidate image files. The feature distribution can be measured, for example, using discrete distribution parameters.

[0083] Taking into account that in actual applications, the activity areas of people or things are usually relatively fixed, it is possible to obtain all image files in the specified area, and perform feature extraction based on the archived images in the image files to obtain the feature matrix of each archived image. Then, when performing subsequent similarity calculations, the feature matrices corresponding to each archived image can be used to determine the similarity between each two archived images.

[0084] Taking facial images as an example, considering that pedestrian activity areas typically don't change significantly in real-world applications, we can obtain all portrait images within the area where portrait clustering is required. Feature extraction is then performed on each portrait image within the portrait image archive to obtain a feature matrix for each portrait image. If a portrait image consists solely of a face, the extracted feature matrix is ​​used to represent the facial features. If a portrait image also includes a human body, the extracted feature matrix is ​​used to represent both facial and body features, such as clothing, height, and hairstyle.

[0085] Step 202: Based on the determined similarities between the archived images included in each of the image archives to be processed, the corresponding image archives to be processed are split to obtain a sub-archive set corresponding to each of the image archives to be processed; wherein, when the sub-archive set includes multiple sub-archives, the sum of the similarities between the archived images included in any two sub-archives is less than the splitting similarity threshold.

[0086] In the embodiment of the present application, a clustering method can be used to clean the data of each determined image file to be processed, so that the incorrectly filed image files to be processed are separated.

[0087] In one embodiment, a clustering method such as K-means clustering can be used to perform the splitting process. In the splitting process, for example, all archived images included in each image file to be processed can be merged together and re-clustered to obtain a splitting result.

[0088] In another embodiment, considering that re-clustering essentially overturns the original clustering results and is computationally intensive, in order to retain the original clustering results and further reduce the computational effort, the present embodiment can also employ spectral clustering to perform the de-clustering process. Below, the de-clustering process is specifically described using spectral clustering as an example. Since de-clustering is performed for each image file to be processed, and the de-clustering process for each image file to be processed is relatively similar, this description will be specifically made using one image file to be processed, namely, image file B to be processed, as an example.

[0089] See also Figure 3 As shown in FIG, it is a flowchart of de-storing using spectral clustering.

[0090] Step 2021: Based on the similarities between the archived images included in the image file B to be processed, construct an edge graph corresponding to the image file B to be processed; wherein one node in the edge graph corresponds to one archived image.

[0091] For example, the image file B to be processed includes N archived images, and the image features of these N archived images are obtained, which are B1, B2, ..., B n Assuming that the image features of each archived image are N-dimensional vectors, the image features of these N archived images can be understood as N nodes in the N-dimensional space. Each node represents the image features of an archived image, and these N nodes are connected in pairs as edges between nodes. The similarity between the archived images corresponding to these N nodes is calculated to obtain the weight of each edge.

[0092] Specifically, the similarity between each two archived images can be represented by the distance in the above N-dimensional space, and the weight of each edge can be expressed as:

[0093]

[0094] Among them, w ij Indicates B i and B j The edge weight between two nodes corresponding to two archived images, i≠j, and i and j are both positive integers not less than 1 and not greater than N, Indicates B i and B j The two-normal distance between two nodes corresponding to two archived images; σ is the expansion constant, which can be set according to actual experimental results.

[0095] Based on the above method, the edge weights between the nodes corresponding to each archived image in the image file B to be processed can be obtained, thereby forming a connected graph.

[0096] In practical applications, if the distance between two archived images is very far, the similarity between the two is very small, the corresponding edge weight is also very small, and the connection between the two is also very weak. Therefore, when constructing the edge graph, the edges between the corresponding nodes of the two archived images with edge weights less than a certain weight threshold can be removed.

[0097] Specifically, for each two archived images in each archived image, determine whether the similarity between each two archived images is greater than a first similarity threshold; if it is greater than the first similarity threshold, there is an edge between each two archived images in the edge graph, and based on the similarity between each two archived images, determine the edge weight between each two archived images.

[0098] Here, take N as 10 as an example, see Figure 4 , which is a schematic diagram of the edge graph corresponding to the image file B to be processed.

[0099] Specifically, the edge weights between the 10 nodes in the image file B to be processed are calculated to form a connected graph. At the same time, if the edge weight between two nodes is less than a certain threshold, the edge between the two nodes is removed. Finally, the edge graph of the image file B to be processed is obtained as follows: Figure 4 The following figure shows the edge graph between the corresponding feature data points of the 10 archived images in the image archive B to be processed in two-dimensional space. Assuming the threshold is 0.3, that is, when the edge weight between two nodes corresponding to two archived images is greater than 0.3, the corresponding edge is retained in the edge graph.

[0100] like Figure 4 As shown, the image archive B to be processed contains a total of 10 archived images B1 to B10, and each archived image corresponds to a node. For example, the edge weight between B3 and B6 is 0.54, and the edge weight between B3 and B7 is 0.56.

[0101] Step 2022: Based on the edge weights between the nodes included in the edge graph, the edge graph is sliced ​​to obtain at least one subgraph; wherein the edge weight between two nodes is positively correlated with the similarity between the corresponding two archived images.

[0102] In an embodiment of the present application, for the edge graph obtained above, the edge graph can be segmented by trying to segment the graph. When the two segmented subgraphs meet the segmentation conditions, the segmentation method is retained.

[0103] The image cutting conditions may include one or more of the following conditions:

[0104] (1) The intersection of any two subgraphs after cutting is an empty set, and the union of all subgraphs is an edge graph.

[0105] For example, after cutting the graph, the subgraphs are G1,…,G n , then G1,…,G n Any two subgraphs satisfy And all subgraphs satisfy G1∪G2∪…∪G n =B

[0106] (2) The sum of the edge weights between any two subgraphs after slicing is no greater than the split weight threshold.

[0107] In an embodiment of the present application, multiple candidate cutting methods of the edge graph can be determined based on the nodes included in the edge graph. When the intersection of any two subgraphs after cutting is an empty set and the union of all subgraphs is an edge graph, then the cutting method can be used as a candidate cutting method.

[0108] See also Figure 5 As shown, it is a schematic diagram of several possible candidate switching methods provided by the embodiment of the present application. Among them, method 1 divides the edge graph into two subgraphs (B3, B7, B1) and (B6, B2, B8, B10, B5, B4, B9), method 2 divides the edge graph into two subgraphs (B3, B7, B1, B6, B2, B8) and (B10, B5, B4, B9), method 3 divides the edge graph into two subgraphs (B3, B7, B1, B6, B2, B8, B10) and (B5, B4, B9), and method 4 divides the edge graph into two subgraphs (B3, B7, B1, B6, B2, B8, B10, B5, B4) and (B9). Of course, in addition to these, there may be other possible candidate cutting methods, which are not listed here one by one.

[0109] Furthermore, for each candidate cutting method, it is necessary to calculate the sum of the edge weights between the two subgraphs after cutting as the cutting weight, and then determine the target cutting method from multiple candidate cutting methods based on the cutting weight, whose sum of the edge weights is not greater than the splitting weight threshold, that is, to determine whether to cut the graph in this way.

[0110] Specifically, if the sum of the edge weights between the two subgraphs after slicing is not greater than the split weight threshold, then this candidate slicing method is determined as the target slicing method, and then the edge graph is sliced ​​according to the determined target slicing method to obtain at least one subgraph.

[0111] The cut weight can be calculated as follows:

[0112]

[0113] Among them, G x ,G y Represent the two subgraphs after cutting, W(G x,G y ) characterizes G x ,G y The weight of the cut graph between .

[0114] Here, if the weight threshold of the split file is set to 0.8, then in the above method 1, W(G x ,G y ) is the sum of the edge weights between the two subgraphs (B3, B7, B1) and (B6, B2, B8, B10, B5, B4, B9). There are 6 edges between these two subgraphs, and their edge weights are 0.54, 0.37, 0.44, 0.41, 0.36 and 0.49 respectively. Obviously, the sum of the edge weights is obviously greater than 0.8, so method 1 cannot be used as the target graph cutting method.

[0115] In the above method 2, W(G x ,G y ) is the sum of the edge weights between the two subgraphs (B3, B7, B1, B6, B2, B8) and (B10, B5, B4, B9). There are two edges between these two subgraphs, and their edge weights are 0.39 and 0.34 respectively. Obviously, the sum of the edge weights is no more than 0.8. Therefore, method 2 can be used as the target cutting method to split it into two subgraphs.

[0116] In the above method 3, W(G x ,G y ) is the sum of the edge weights between the two subgraphs (B3, B7, B1, B6, B2, B8, B10) and (B5, B4, B9). There are two edges between these two subgraphs, and their edge weights are 0.45 and 0.32 respectively. Obviously, the sum of the edge weights is no more than 0.8. Therefore, method 3 can also be used as the target cutting method to split it into two subgraphs.

[0117] In the above-mentioned method 4, W(G x ,G y ) is the sum of the edge weights between the two subgraphs (B3, B7, B1, B6, B2, B8, B10, B5, B4) and (B9). There are two edges between these two subgraphs, and their edge weights are 0.32 and 0.48 respectively. Obviously, the sum of the edge weights is equal to 0.8, so method 4 cannot be used as the target cutting method.

[0118] Then, after trying to determine, we can find that there are two ways to cut the graph so that W(G x ,G y) are less than the splitting weight threshold of 0.8, which are method 2 and method 3 respectively. Therefore, the edge graph of the image file B to be processed can be split into 3 sub-graphs, namely (B3, B7, B1, B6, B2, B8), (B10) and (B5, B4, B9).

[0119] Step 2023: Based on at least one subgraph, the image file B to be processed is split to obtain a corresponding sub-file set; wherein the archived images in each sub-file correspond one-to-one to a node in a subgraph.

[0120] Specifically, a node of each subgraph corresponds to an archived image in the image file B to be processed, so that the splitting result of the image file B to be processed can be obtained according to the above correspondence, that is, it is also split into 3 sub-files.

[0121] For any image file to be processed, the above-described splitting process can be used to obtain corresponding sub-file sets, each of which corresponds to the split results of a single image file to be processed. It should be noted that in actual situations, there may be a case where a single image file to be processed does not meet the splitting conditions. In this case, the image file to be processed will not be split, and its corresponding sub-file set will only include one image file, namely the original image file to be processed.

[0122] Step 203: Based on the similarity between the archived images included in every two sub-files, the sub-files included in each obtained sub-file set are merged to obtain multiple target image files.

[0123] In the embodiment of the present application, since the files that were previously clustered incorrectly may be files of other people or things, but were clustered incorrectly, it is necessary to re-archive the sub-files obtained after the split.

[0124] In one embodiment, the representation vectors of the sub-files may be extracted, and then the similarity between the sub-files may be calculated based on the representation vectors. If the similarity is greater than a certain similarity threshold, the sub-files may be merged.

[0125] In one embodiment, the same method as that used for splitting files may be used for merging files. For example, if spectral clustering is used for splitting files, spectral clustering may also be used for merging files. The spectral clustering method is specifically used as an example for introduction.

[0126] Specifically, after splitting each image file to be processed, a set of sub-files corresponding to each file can be obtained. These sub-files can then be combined for merging. When merging, for each pair of sub-files, if the sum of the edge weights between the two sub-graphs corresponding to each sub-file is determined to be greater than the merging weight threshold, the two sub-files are merged.

[0127] Here we take sub-files C and D as an example. Sub-files C and D can be any two sub-files. The sum of the edge weights between sub-files C and D is:

[0128]

[0129] Among them, G C ,G D Represents the two subgraphs corresponding to sub-file C and sub-file D, W(G C ,G D ) characterizes G C ,G D The weight of the combined graph between them is the sum of the above-mentioned edge weights.

[0130] Then if there is a connected graph formed by combining sub-files C and D, the edge weights and W(G C ,G D ) is greater than the merging weight threshold, the two files are considered to be merged successfully.

[0131] In actual applications, if the weight threshold for splitting is greater than the weight threshold for closing, the cycle of splitting and closing will occur. Therefore, the weight threshold for closing is usually greater than the weight threshold for splitting. For example, if the weight threshold for closing is set to 0.9, then see Figure 6 As shown in the figure, it is a schematic diagram of the connected graph formed after the sub-file C and sub-file D are combined, where sub-file C includes (C1, C3, C5, C4, C2, C8), sub-file D includes (D1, D2, D3), and the edge weights between each node are as follows: Figure 6 As shown, if sub-file C and sub-file D are merged, there are three edges between them, and their edge weights are 0.42, 0.33 and 0.45 respectively, so the total edge weight is 1.2, which is obviously greater than the merging weight threshold of 0.9, so sub-file C and sub-file D can be merged.

[0132] In an embodiment of the present application, the processes of steps 202 and 203 can be executed once to complete the inspection; alternatively, the processes of steps 202 and 203 can be executed in a loop multiple times until a certain termination condition is reached, such as when the gear cannot be disassembled any further. This embodiment of the present application does not impose any restrictions on this.

[0133] In summary, the embodiments of the present application provide a method for finding incorrectly clustered files using spectral clustering, thereby splitting image files containing multiple incorrect clusters. For example, a file containing multiple incorrectly clustered portrait images of the same person can be split. Simultaneously, the split files can be re-merged using spectral clustering, effectively improving the accuracy and recall of image clustering. Furthermore, the same method can be used for both merging and splitting, preventing the situation where files are split while being merged simultaneously. Furthermore, by splitting and merging files based on files that have been generated over a long period of time, the image feature distribution within the files is more distinct, resulting in better results.

[0134] See Figure 7 Based on the same inventive concept, the embodiment of the present application further provides an image file inspection device 70, which includes:

[0135] The determining unit 701 is used to determine the image files to be processed that meet the set inspection conditions from multiple candidate image files;

[0136] The file splitting unit 702 is configured to split the corresponding image files to be processed based on the determined similarities between the archived images included in each image file to be processed, thereby obtaining sub-file sets corresponding to each image file to be processed; wherein, when the sub-file set includes multiple sub-files, the sum of the similarities between the archived images included in any two sub-files is less than the splitting similarity threshold;

[0137] The file merging unit 703 is configured to merge the sub-files included in each sub-file set based on the similarity between the archived images included in every two sub-files, so as to obtain a plurality of target image files.

[0138] Optionally, the determining unit 701 is specifically configured to:

[0139] For multiple candidate image files, perform the following operations respectively:

[0140] For a candidate image file, determining whether the number of archived images included in the candidate image file is greater than an inspection number threshold;

[0141] If it is greater than the inspection quantity threshold, it is determined that the candidate image file meets the inspection conditions.

[0142] Optionally, the file splitting unit 702 is specifically configured to:

[0143] For each image file to be processed, perform the following operations:

[0144] For an image archive to be processed, based on the similarity between each archived image included in the archive, a connection graph corresponding to the image archive to be processed is constructed; wherein a node in the connection graph corresponds to an archived image;

[0145] Based on the edge weights between the nodes included in the edge graph, the edge graph is sliced ​​to obtain at least one subgraph; wherein the edge weight between two nodes is positively correlated with the similarity between the corresponding two archived images;

[0146] Based on at least one subgraph, an image archive to be processed is split to obtain a corresponding sub-archive set; wherein the archived images in each sub-archive correspond one-to-one to a node in a subgraph.

[0147] Optionally, the file splitting unit 702 is specifically configured to:

[0148] For each two archived images in each archived image, determining whether the similarity between each two archived images is greater than a first similarity threshold;

[0149] If it is greater than, then there is an edge between every two archived images in the edge graph; and,

[0150] Based on the similarity between each two archived images, the edge weight between each two archived images is determined.

[0151] Optionally, the file splitting unit 702 is specifically configured to:

[0152] Based on the nodes included in the edge graph, multiple candidate graph-slicing methods are determined; wherein the intersection of any two subgraphs after the graph slicing is an empty set, and the union of all subgraphs is the edge graph;

[0153] Based on the sum of the edge weights between the two subgraphs after the graph is sliced ​​according to various candidate graph slicing methods among the multiple candidate graph slicing methods, a target graph slicing method is determined from the multiple candidate graph slicing methods, wherein the sum of the edge weights is not greater than the split weight threshold;

[0154] The edge graph is sliced ​​according to the determined target slicing method to obtain at least one subgraph.

[0155] Optionally, the file merging unit 703 is specifically configured to:

[0156] For every two sub-files in each sub-file, perform the following operations:

[0157] If it is determined that the sum of the edge weights between the two subgraphs corresponding to each two sub-files is greater than the merge weight threshold, each two sub-files are merged; wherein the merge weight threshold is greater than the split weight threshold.

[0158] Optionally, the apparatus further includes a similarity calculation unit 704, configured to:

[0159] Perform feature extraction on each archived image to obtain the feature matrix corresponding to each archived image;

[0160] Based on the feature matrices corresponding to the respective archived images, the similarity between each two archived images is determined.

[0161] The above device can use spectral clustering to identify incorrectly clustered files, allowing for the splitting of image files containing multiple incorrect clusters. For example, it can split files containing multiple incorrectly clustered portrait images of the same person. Simultaneously, these split files can be re-merged using spectral clustering, effectively improving the accuracy and recall of image clustering. Furthermore, the same method can be used for both merging and splitting, preventing the situation where files are split while being merged simultaneously. Furthermore, by combining and splitting files over a long period of time, the image feature distribution within the files is more distinct, resulting in better results.

[0162] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, please refer to the description of the aforementioned embodiments and no further details will be given.

[0163] See Figure 8 Based on the same technical concept, the embodiment of the present application further provides a computer device 80, which can be Figure 1 As shown in the terminal device or server, the computer device 80 may include a memory 801 and a processor 802 .

[0164] The memory 801 is used to store computer programs executed by the processor 802. The memory 801 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the computer device, etc. The processor 802 may be a central processing unit (CPU), or a digital processing unit, etc. The specific connection medium between the above-mentioned memory 801 and the processor 802 is not limited in the embodiment of the present application. The embodiment of the present application is Figure 8 In the embodiment, the memory 801 and the processor 802 are connected via a bus 803. The bus 803 is connected to the processor 802 via a bus 803. Figure 8 The bus 803 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0165] Memory 801 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 801 may be a combination of the above memories.

[0166] The processor 802 is configured to execute the method executed by the device in each embodiment of the present application when calling the computer program stored in the memory 801.

[0167] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the method executed by the device in each embodiment of the present application.

[0168] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0169] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0170] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for inspecting image files, characterized in that: The method comprises: Determine the image files to be processed that meet the set inspection conditions from multiple candidate image files; For each determined image file to be processed, perform the following operations respectively to obtain the sub-file set corresponding to each image file to be processed: For an image archive to be processed, constructing an edge graph corresponding to the image archive to be processed based on the similarities between the archived images included therein; wherein a node in the edge graph corresponds to an archived image; Based on the edge weights between the nodes included in the edge graph, the edge graph is sliced ​​to obtain at least one subgraph; wherein the edge weight between two nodes is positively correlated with the similarity between the corresponding two archived images; Based on the at least one subgraph, the image archive to be processed is split to obtain a sub-archive set corresponding to the image archive to be processed, wherein the archived images in each sub-archive correspond one-to-one to a node in a sub-graph; wherein, when the sub-archive set includes multiple sub-archives, the sum of the similarities between the archived images included in any two sub-archives is less than a split-archive similarity threshold; Based on the similarity between the archived images included in every two sub-archives, the sub-archives included in each obtained sub-archive set are merged to obtain multiple target image archives.

2. The method according to claim 1, wherein From multiple candidate image files, determine the image files to be processed that meet the set inspection conditions, including: For the multiple candidate image files, perform the following operations respectively: For a candidate image file, determining whether the number of archived images included in the candidate image file is greater than an inspection number threshold; If it is greater than the inspection quantity threshold, it is determined that the candidate image file meets the inspection condition.

3. The method according to claim 1, wherein For an image file to be processed, based on the similarities between the archived images included therein, a connection graph corresponding to the image file to be processed is constructed, including: For each two archived images, determining whether the similarity between the two archived images is greater than a first similarity threshold; If it is greater than, then there is an edge between each two archived images in the edge graph; and, Based on the similarity between each two archived images, a weight of an edge between each two archived images is determined.

4. The method according to claim 1, wherein Based on the edge weights between the nodes included in the edge graph, the edge graph is sliced ​​to obtain at least one subgraph, including: Determine multiple candidate graph-slicing methods for the edge graph based on the nodes included in the edge graph, wherein the intersection of any two subgraphs after the graph slicing is an empty set, and the union of all subgraphs is the edge graph; Determining, based on the sum of the edge weights between two subgraphs after the graph is sliced ​​according to various candidate graph slicing methods among the multiple candidate graph slicing methods, a target graph slicing method whose sum of the edge weights is not greater than a split weight threshold from the multiple candidate graph slicing methods; The edge graph is sliced ​​according to the determined target slicing method to obtain the at least one subgraph.

5. The method according to any one of claims 1 to 4, characterized in that: Based on the similarity between the archived images included in each two sub-archives, the sub-archives included in each sub-archive set are merged to obtain multiple target image archives, including: For every two sub-files in each sub-file, perform the following operations respectively: If it is determined that the sum of the edge weights between the two subgraphs corresponding to each two sub-files is greater than the merge weight threshold, each two sub-files are merged; wherein the merge weight threshold is greater than the split weight threshold.

6. The method according to any one of claims 1 to 4, characterized in that: Before constructing the edge graph corresponding to the image archive to be processed based on the similarities between the archived images, the method further includes: Performing feature extraction on each of the archived images to obtain feature matrices corresponding to each of the archived images; Based on the feature matrices corresponding to the respective archived images, the similarity between each two archived images is determined.

7. An image file inspection device, characterized in that: The device comprises: A determination unit, configured to determine an image file to be processed that meets a set inspection condition from a plurality of candidate image files; The file splitting unit is configured to perform the following operations for each determined image file to be processed, respectively, to obtain a sub-file set corresponding to each image file to be processed: for one image file to be processed, based on the similarities between the archived images included therein, construct an edge graph corresponding to the image file to be processed; wherein, one node in the edge graph corresponds to one archived image; based on the edge weights between the nodes included in the edge graph, the edge graph is cut to obtain at least one sub-graph; wherein, the edge weight between two nodes is positively correlated with the similarity between the corresponding two archived images; based on the at least one sub-graph, the image file to be processed is split to obtain a sub-file set corresponding to the image file to be processed, wherein the archived images in each sub-file correspond one-to-one to the nodes in a sub-graph; wherein, when the sub-file set includes multiple sub-files, the sum of the similarities between the archived images included in any two sub-files is less than the splitting similarity threshold; The file merging unit is used to merge the sub-files included in each sub-file set based on the similarity between the archived images included in each two sub-files to obtain multiple target image files.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images