Archive generation method, device and storage medium

Abstract

The embodiment of the present application discloses a method, device and storage medium for generating archives, and belongs to the field of data processing technology. The method comprises: obtaining target identification features and target attribute features of each image in a plurality of images, wherein the target identification features of each image are used to identify the shooting target in the corresponding image, and the target attribute features of each image are used to describe the attributes of the shooting target in the corresponding image. Based on the target identification features and target attribute features of each image in the plurality of images, the association confidence between each image in the plurality of images is determined, and the association confidence between any two images indicates the probability that the shooting targets in any two images belong to the same shooting target. Based on the association confidence between each image in the plurality of images, the plurality of images are clustered to obtain one or more archives, and each of the one or more archives includes part or all of the images in the plurality of images. The embodiment of the present application solves the problem of low archive accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and more particularly to a file generation method, device and storage medium. Background Art

[0002] When faced with a large number of images, in order to find images more conveniently and quickly, a method is often needed to divide these images. For example, these images can be divided into multiple categories according to their characteristics, and each category of images is a file. For example, a personnel file is the result of clustering and archiving a large number of personnel images based on the unique identity of each person.

[0003] For personnel files, related technologies have proposed a method for generating personnel files. The method first obtains a face image database, extracts face features in the database, clusters the face features, and generates a file based on the clustering results.

[0004] However, the above personnel file generation method has a low prediction accuracy for personnel files, and the accuracy of the personnel files finally generated is not high. Summary of the invention

[0005] The embodiment of the present application provides an archive method, device and storage medium, which can solve the problem of low archive accuracy in related technologies. The technical solution is as follows:

[0006] In one aspect, a method for generating a file is provided, the method comprising:

[0007] Obtaining target identification features and target attribute features of each image in the plurality of images, wherein the target identification features of each image are used to identify a photographed target in the corresponding image, and the target attribute features of each image are used to describe an attribute of a photographed target in the corresponding image;

[0008] Determining, based on the target identification feature and the target attribute feature of each image in the plurality of images, an association confidence between each image in the plurality of images, wherein the association confidence between any two images indicates a probability that the photographed objects in the any two images belong to the same photographed object;

[0009] Based on the association confidence between each image in the plurality of images, the plurality of images are clustered to obtain one or more archives, each of the one or more archives including part or all of the images in the plurality of images.

[0010] Optionally, the determining, based on the target identification feature and the target attribute feature of each image in the multiple images, the association confidence between the multiple images includes:

[0011] Each of the multiple images is sequentially used as a central node, and the images other than the central node are used as edge nodes. The following operations are performed on each central node:

[0012] Determine a relationship weight between each edge node in the plurality of edge nodes and the central node based on a target identification feature and a target attribute feature of each edge node in the plurality of edge nodes and a target identification feature and a target attribute feature of the central node, wherein the relationship weight indicates a correlation between two corresponding images;

[0013] Based on the relationship weight between each edge node in the plurality of edge nodes and the central node, performing convolution processing on the plurality of edge nodes to obtain a relative embedding feature of each edge node, wherein the relative embedding feature of each edge node indicates a difference between the corresponding edge node and the central node;

[0014] Based on the relative embedding characteristics of each edge node in the plurality of edge nodes, an association confidence between each edge node and the central node is determined.

[0015] Optionally, performing convolution processing on the plurality of edge nodes based on the relationship weight between each edge node in the plurality of edge nodes and the central node to obtain a relative embedding feature of each edge node includes:

[0016] Inputting the target identification feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a graph neural network, wherein the graph neural network outputs a relative embedding feature of each edge node based on a relationship weight between each edge node in the plurality of edge nodes and the central node;

[0017] The graph neural network includes at least two graph aggregation layers, the input of the first layer of the at least two graph aggregation layers is the target identification feature of each edge node among the multiple edge nodes, and the input of each layer except the first layer of the at least two graph aggregation layers is the feature of each edge node output by the previous layer minus the target identification feature of the central node.

[0018] Optionally, determining the relationship weight between each edge node in the plurality of edge nodes and the central node based on the target identification feature and the target attribute feature of each edge node in the plurality of edge nodes and the target identification feature and the target attribute feature of the central node includes:

[0019] For any edge node among the plurality of edge nodes, determining a similarity between a target identification feature of the any edge node and the central node, and a similarity between a target attribute feature of the any edge node and the central node;

[0020] Based on the similarity between the target identification features of the any edge node and the central node and the similarity between the target attribute features of the any edge node and the central node, the relationship weight between the any edge node and the central node is determined.

[0021] Optionally, determining the relationship weight between each edge node in the plurality of edge nodes and the central node based on the target identification feature and the target attribute feature of each edge node in the plurality of edge nodes and the target identification feature and the target attribute feature of the central node includes:

[0022] For any edge node among the multiple edge nodes, based on the target identification feature and target attribute feature of the any edge node and the target identification feature and target attribute feature of the central node, the relationship weight between the any edge node and the central node is determined through an adaptive learning network.

[0023] Optionally, the determining, based on the relative embedding feature of each edge node among the plurality of edge nodes, the association confidence between each edge node and the central node comprises:

[0024] Inputting the relative embedding feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a binary classifier, the binary classifier outputting the association confidence between each edge node in the plurality of edge nodes and the central node;

[0025] Among them, the binary classifier is used to identify which edge node among the multiple edge nodes has a shooting target and the shooting target in the central node belong to the same shooting target, and when it is determined that the shooting target in an edge node and the shooting target in the central node belong to the same shooting target, output the association confidence between the edge node and the central node.

[0026] Optionally, the method further comprises:

[0027] For a first file in the one or more files, one or more images are selected from the first file as cover images, the image quality of the cover image is higher than the image quality of other images in the first file, and the first file is any file in the one or more files;

[0028] Based on the association confidence between each other image in the first archive and the cover image, some images in the other images in the first archive are deleted to purify the images in the first archive.

[0029] Optionally, the method further comprises:

[0030] Determine a representative node of each of the one or more archives, wherein the feature of the representative node of each archive is an average feature of the photographed targets in each image included in the corresponding archive, wherein the average feature includes an average target identification feature and / or an average target attribute feature;

[0031] For a second file in the one or more files, determining the association confidence between the image to be aggregated and the representative node of the second file based on the features of the representative node of the second file and the target identification features and / or target attribute features of the image to be aggregated;

[0032] Based on the association confidence between the image to be archived and the representative node of the second archive, determine whether to add the image to be archived to the second archive.

[0033] In another aspect, a device for generating a file is provided, the device comprising:

[0034] An acquisition module, used to acquire a target identification feature and a target attribute feature of each image in the plurality of images, wherein the target identification feature of each image is used to identify a photographed target in the corresponding image, and the target attribute feature of each image is used to describe an attribute of a photographed target in the corresponding image;

[0035] A first determination module is used to determine the association confidence between each of the multiple images based on the target identification feature and the target attribute feature of each of the multiple images, wherein the association confidence between any two images indicates the probability that the photographed objects in the any two images belong to the same photographed object;

[0036] A clustering module is used to cluster the multiple images based on the association confidence between each image in the multiple images to obtain one or more archives, each of the one or more archives including part or all of the images in the multiple images.

[0037] Optionally, the first determining module is specifically configured to:

[0038] Each of the multiple images is sequentially used as a central node, and the images other than the central node are used as edge nodes. The following operations are performed on each central node:

[0039] Determine a relationship weight between each edge node in the plurality of edge nodes and the central node based on a target identification feature and a target attribute feature of each edge node in the plurality of edge nodes and a target identification feature and a target attribute feature of the central node, wherein the relationship weight indicates a correlation between two corresponding images;

[0040] Based on the relationship weight between each edge node in the plurality of edge nodes and the central node, performing convolution processing on the plurality of edge nodes to obtain a relative embedding feature of each edge node, wherein the relative embedding feature of each edge node indicates a difference between the corresponding edge node and the central node;

[0041] Based on the relative embedding characteristics of each edge node in the plurality of edge nodes, an association confidence between each edge node and the central node is determined.

[0042] Optionally, the first determining module is specifically configured to:

[0043] Inputting the target identification feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a graph neural network, wherein the graph neural network outputs a relative embedding feature of each edge node based on a relationship weight between each edge node in the plurality of edge nodes and the central node;

[0044] The graph neural network includes at least two graph aggregation layers, the input of the first layer of the at least two graph aggregation layers is the target identification feature of each edge node among the multiple edge nodes, and the input of each layer except the first layer of the at least two graph aggregation layers is the feature of each edge node output by the previous layer minus the target identification feature of the central node.

[0045] Optionally, the first determining module is specifically configured to:

[0046] For any edge node among the plurality of edge nodes, determining a similarity between a target identification feature of the any edge node and the central node, and a similarity between a target attribute feature of the any edge node and the central node;

[0047] Based on the similarity between the target identification features of the any edge node and the central node and the similarity between the target attribute features of the any edge node and the central node, the relationship weight between the any edge node and the central node is determined.

[0048] Optionally, the first determining module is specifically configured to:

[0049] For any edge node among the multiple edge nodes, based on the target identification feature and target attribute feature of the any edge node and the target identification feature and target attribute feature of the central node, the relationship weight between the any edge node and the central node is determined through an adaptive learning network.

[0050] Optionally, the first determining module is specifically configured to:

[0051] Inputting the relative embedding feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a binary classifier, the binary classifier outputting the association confidence between each edge node in the plurality of edge nodes and the central node;

[0052] Among them, the binary classifier is used to identify which edge node among the multiple edge nodes has a shooting target and the shooting target in the central node belong to the same shooting target, and when it is determined that the shooting target in an edge node and the shooting target in the central node belong to the same shooting target, output the association confidence between the edge node and the central node.

[0053] Optionally, the device further comprises:

[0054] a selection module, configured to select one or more images from a first file in the one or more files as a cover image, wherein the image quality of the cover image is higher than the image quality of other images in the first file, and the first file is any file in the one or more files;

[0055] A purification module is used to delete some images in other images in the first file based on the association confidence between each other image in the first file and the cover image, so as to purify the images in the first file.

[0056] Optionally, the device further comprises:

[0057] A second determination module is used to determine a representative node of each of the one or more archives, wherein the feature of the representative node of each archive is an average feature of the photographed targets in each image included in the corresponding archive, wherein the average feature includes an average target identification feature and / or an average target attribute feature;

[0058] A third determination module is used to determine, for a second file in the one or more files, based on the features of the representative node of the second file and the target identification features and / or target attribute features of the image to be aggregated, the association confidence between the image to be aggregated and the representative node of the second file;

[0059] The fourth determination module is used to determine whether to add the image to be aggregated into the second file based on the association confidence between the image to be aggregated and the representative node of the second file.

[0060] On the other hand, a computer device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of the archive generation method described above.

[0061] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of the archive generation method described above are implemented.

[0062] The technical solution provided by the embodiments of the present application can at least bring the following beneficial effects:

[0063] In the embodiment of the present application, the target identification features and target attribute features of each image in the multiple images are obtained to characterize the shooting target in the corresponding image, and the image features obtained are rich. Based on these rich image features, the association confidence between the multiple images is determined, so that the determined association confidence can more accurately characterize whether the images are shot of the same target, so as to better cluster the multiple images, thereby solving the problem of low accuracy of the archives. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 is a flow chart of a file generation method provided in an embodiment of the present application;

[0066] Figure 2 is a schematic diagram of a relationship diagram provided in an embodiment of the present application;

[0067] Figure 3 is a flow chart of a file generation method provided in an embodiment of the present application;

[0068] Figure 4 It is a flow chart of a file purification method provided in an embodiment of the present application;

[0069] Figure 5 It is a flow chart of a file purification method provided in an embodiment of the present application;

[0070] Figure 6 This is a flow chart of a method for adding files to a file provided in an embodiment of the present application;

[0071] Figure 7 This is a flow chart of a method for adding files to a file provided in an embodiment of the present application;

[0072] Figure 8 It is a structural schematic diagram of a file generation device provided in an embodiment of the present application;

[0073] Fig. 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0075] Before explaining in detail the archive generation method provided in the embodiment of the present application, the application scenario provided in the embodiment of the present application is first introduced.

[0076] When a camera captures a large number of images, it is necessary to cluster these images reasonably and effectively. For example, images with similar features can be clustered based on the similarities between certain features of these images, and an identity is established for each image category according to the clustering results, thereby generating one or more archives. The above process can be called archive clustering.

[0077] When searching for a certain image among these images later, it is convenient to search for the file to which the image belongs in one or more previously generated files according to the features of the image, thereby searching for the image in the files. Moreover, when receiving one or more new images, the files that are most similar to the features of the one or more new images can be searched in one or more files according to the features of the one or more new images, thereby adding the one or more new images to the one or more files. The aforementioned process can be called file addition.

[0078] For example, in the field of access control, the image acquisition device collects a large number of personnel images within a period of time, and these personnel images need to be sorted and clustered in preparation for the later extraction of certain personnel images to determine whether to open the access control. At this time, these personnel images can be clustered and archived according to the similarity of their features to generate one or more personnel files. When one or more new personnel images are collected, the files that are most similar to the features of the one or more new personnel images can be found in one or more files based on the features of the one or more new personnel images, so that the one or more new personnel images can be added to one or more personnel files. These one or more personnel files can establish identity identification based on the image features of the collected personnel images without personnel registration information, so as to perform applications such as flow monitoring and crowd statistics.

[0079] Moreover, each file is often disturbed by images that do not belong to the file during operation. For example, file 1 is used to cluster various images of person A, and file 2 is used to cluster various images of person B. For file 1, these various images of person A are positive samples in file 1, and various images of person B are negative samples in file 1. For file 2, various images of person B are positive samples in file 2, and various images of person A are negative samples in file 2. However, there may be an image among the various images of person B, which is very similar to the image of person A in file 1 under certain shooting angles or because the target in the image is blocked or backlit, resulting in the situation that this image may be added to file 1 during the file generation process, so that in the final generated file, the image is not correctly added to the corresponding file.

[0080] Therefore, in the process of generating one or more archives by clustering and archiving a large number of images, these images can be processed based on the images according to the method provided in the embodiments of the present application, and they can be clustered according to the characteristics of these images to eliminate the interference of negative samples on the archives and enhance the correlation between positive samples and images in the archives, thereby more accurately achieving clustering of a large number of images.

[0081] The technical solution provided by this application includes three aspects, namely, a method for generating archives, a method for purifying archives, and a method for increasing archives. These three methods will be described in detail below.

[0082] Figure 1 This is a flowchart of a method for generating a file provided in an embodiment of the present application, which is applied in a computer device. Figure 1 , the method comprises the following steps.

[0083] Step 101: A computer device obtains a target identification feature and a target attribute feature of each image in a plurality of images, wherein the target identification feature of each image is used to identify a photographed target in the corresponding image, and the target attribute feature of each image is used to describe an attribute of a photographed target in the corresponding image.

[0084] Among them, the target identification feature can be the identity feature or appearance feature of the target in the image, etc., which can uniquely identify the captured target in the corresponding image; the target attribute feature can be the age, gender, posture, gait feature, image quality, geographical information, time, lighting, etc. of the target in the image, which can describe the attributes of the captured target in the corresponding image.

[0085] For example, when the image is a person image, the target identification feature can be the facial features of the person, or the appearance features of the human body, etc. The appearance features of the human body are the overall body shape features of the person. The target attribute features can be the age of the person, the gender of the person, the facial posture, the gait features, the image quality, the geographical information, the time, the lighting, etc. Among them, the facial posture features are the frontal features or the side features of the person in the image, and the gait features are the features of the person in the image when he is in a walking state or the features of the person when he is in an upright state, etc.

[0086] In some embodiments, the above-mentioned method of obtaining the target identification features and target attribute features of each of the multiple images can be implemented as follows: a camera captures multiple images, and the multiple images are used as inputs of the first graph neural network model to obtain the target identification features and target attribute features of each of the multiple images output by the first graph neural network model. For each image, the first graph neural network model can output the target identification features and target attribute features of its image, and the target identification features of each image are used to identify the shooting target in the corresponding image, and the target attribute features of each image are used to describe the attributes of the shooting target in the corresponding image.

[0087] The first graph neural network model is trained in advance through small-scale samples, so it can be used to detect multiple images taken by a camera, thereby obtaining target identification features and target attribute features of each of the multiple images.

[0088] The implementation process of the above-mentioned training of the first graph neural network model can be: obtaining multiple training samples and labels for each of the multiple training samples, wherein each training sample is an image, and the label of each training sample is the target identification feature and target attribute feature in the corresponding image, and the label of each training sample is manually calibrated. Then, the multiple training samples and the labels for each of the multiple training samples are input into the initialized neural network model, and the initialized neural network model can obtain the above-mentioned first graph neural network model after learning. After this training process, the first graph neural network model can be used to obtain the target identification features and target attribute features of each of the multiple images.

[0089] As an example, when obtaining the target identification features and target attribute features of each of the multiple images, each image may be annotated by a manual annotation method to annotate the target identification features and target attribute features of each image.

[0090] Step 102: Based on the target identification features and target attribute features of each image in the multiple images, the computer device determines the association confidence between each image in the multiple images, and the association confidence between any two images indicates the probability that the shooting targets in any two images belong to the same shooting target.

[0091] In some embodiments, the implementation process of step 102 can be implemented through the following steps.

[0092] Step 1: Take each of the multiple images as a central node in turn, and the images other than the central node as edge nodes, and perform the following operations on each central node: Based on the target identification features and target attribute features of each edge node in the multiple edge nodes, and the target identification features and target attribute features of the central node, determine the relationship weight between each edge node in the multiple edge nodes and the central node, and the relationship weight indicates the correlation between the corresponding two images.

[0093] Step 2: Based on the relationship weight between each edge node and the central node in the multiple edge nodes, convolution processing is performed on the multiple edge nodes to obtain the relative embedding feature of each edge node, and the relative embedding feature of each edge node indicates the difference between the corresponding edge node and the central node.

[0094] Step 3: Based on the relative embedding features of each edge node among the multiple edge nodes, determine the association confidence between each edge node and the central node.

[0095] That is to say, each of the multiple images is taken as a central node in turn, and the images other than the central node are taken as edge nodes. Based on the target identification feature and target attribute feature of each image in the multiple images, the relationship weight between each edge node and the central node in the multiple edge nodes is determined. Based on the relationship weight between each edge node and the central node in the multiple edge nodes, the relative embedding feature of each edge node is obtained, and then the association confidence between each edge node and the central node is determined.

[0096] Each of the multiple images is taken as a central node in turn, and the target identification features and target attribute features of each central node are the target identification features and target attribute features of the corresponding image. Except for the central node, other images are taken as edge nodes, and the target identification features and target attribute features of the edge nodes are the target identification features and target attribute features of the corresponding image.

[0097] Since each of the multiple images is taken as a central node in turn, for each central node, the relationship weight between each edge node and the central node can be determined according to the above method, and based on the relationship weight between each edge node and the central node, the relative embedding features of each edge node except the central node are obtained, and then the association confidence between each edge node and the central node is determined. The following embodiment is explained using a central node as an example. When each of the multiple images is taken as a central node in turn and processed according to the above method, the association confidence between any two nodes among all nodes can be obtained.

[0098] The relationship weight between each edge node and the central node in the above step 1 may be determined by the following two implementation methods.

[0099] In a possible implementation, the implementation process of determining the relationship weights may be as follows: Step 1: For any edge node among the multiple edge nodes, determine the similarity between the target identification features of any edge node and the central node, and the similarity between the target attribute features of any edge node and the central node. Step 2: Based on the similarity between the target identification features of any edge node and the central node, and the similarity between the target attribute features of any edge node and the central node, determine the relationship weight between any edge node and the central node.

[0100] Among them, the implementation process of step 1 can be: by calculating the distance between the target identification feature of any edge node among the multiple edge nodes and the target identification feature of the central node, the similarity between the target identification feature of any edge node among the multiple edge nodes and the target identification feature of the central node is determined; similarly, by calculating the distance between the target attribute feature of any edge node among the multiple edge nodes and the target attribute feature of the central node, the similarity between the target attribute feature of any edge node among the multiple edge nodes and the target attribute feature of the central node is determined.

[0101] It should be noted that when determining the relationship weight between each edge node and the central node among multiple edge nodes, the target identification feature at this time may be the identity feature of the target in the image, or the appearance feature of the target in the image, or the identity feature and the appearance feature of the target in the image. The target attribute feature at this time may be one feature of the target attribute feature, or two features of the target attribute feature, or all features of the target attribute feature.

[0102] As an example, when performing step 1, for the edge node T among multiple edge nodes and the central node O, the target identification feature of the edge node T and the target identification feature of the central node O select the identity feature of the target, calculate the distance between the identity feature of the target of the edge node T and the identity feature of the target of the central node O, and obtain the similarity between the identity feature of the target of the edge node T and the identity feature of the target of the central node O.

[0103] Correspondingly, the target attribute features of the edge node T and the target attribute features of the central node O select the age features and picture quality features of the targets, calculate the distance between the age features of the target of the edge node T and the age features of the target of the central node O, as well as the distance between the picture quality features of the edge node T and the picture quality features of the central node O, and obtain the similarity between the age features of the target of the edge node T and the age features of the target of the central node O, as well as the similarity between the picture quality features of the edge node T and the picture quality features of the central node O respectively.

[0104] The implementation process of step 2 can be: the similarity between the target identification feature of any edge node among multiple edge nodes and the target identification feature of the central node is used as the weight subject, that is, when calculating the relationship weight, the proportion of the similarity of the target identification feature is 100%. At the same time, the similarity between the target attribute feature of any edge node among multiple edge nodes and the target attribute feature of the central node is used as the weighted item to adjust the weight subject, that is, when calculating the relationship weight, the proportion of the similarity of the target attribute feature is set according to the various features contained in the target attribute feature. Finally, the relationship weight between any edge node among multiple edge nodes and the central node is obtained by normalization. The relationship weight can be a {0,1} discrete value or a continuous value. The larger the value, the stronger the correlation between any edge node and the central node.

[0105] When used as a weighted item to adjust the weight subject, the proportion of the target attribute feature can be set in advance, and it is only necessary to ensure that the total proportion of the similarity between the target attribute features of the two selected nodes is 100%. For example, the target attribute feature selects the gender feature and the illumination feature of the target. When calculating the relationship weight, if the proportion of the similarity between the target gender features of the two nodes is set to 80%, the proportion of the similarity between the illumination features of the two nodes is 20%.

[0106] For example, when performing step 2, the ratio of the similarity between the identity characteristics of the target of edge node T and the identity characteristics of the target of central node O is set to 100%, the ratio of the similarity between the age characteristics of the target of edge node T and the age characteristics of the target of central node O is set to 70%, and the ratio of the similarity between the image quality characteristics of edge node T and the image quality characteristics of central node O is set to 30%. That is, the relationship weight between the edge node T and the central node O = the similarity between the identity characteristics of the targets * 100% + the similarity between the age characteristics of the targets * 70% + the similarity between the image quality characteristics * 30%. Finally, the relationship weights between all nodes are normalized to obtain the relationship weight between the edge node T and the central node O, so that the relationship weight is between 0-1.

[0107] In another possible implementation, the implementation process of determining the relationship weight between each edge node among multiple edge nodes and the central node can be: for any edge node among the multiple edge nodes, based on the target identification feature and target attribute feature of any edge node, and the target identification feature and target attribute feature of the central node, the relationship weight between any edge node and the central node is determined through an adaptive learning network.

[0108] That is, the target identification features and target attribute features of any edge node among multiple edge nodes and the target identification features and target attribute features of the central node are used as inputs of the adaptive learning network to obtain the relationship weight between any edge node and the central node output by the adaptive learning network.

[0109] The adaptive learning network is trained in advance with small-scale samples. The implementation process of training the adaptive learning network can be: obtaining multiple training samples, labels for each training sample in the multiple training samples, and target identification features and target attribute features of each training sample. Each training sample is an image, and the label of each training sample is the relationship weight between the corresponding image and other images. The label of each training sample is manually labeled according to the target identification features and target attribute features of each training sample. Then, multiple training samples, labels for each training sample in the multiple training samples, and target identification features and target attribute features of each training sample are input into the initialized adaptive learning network, and the initialized adaptive learning network can obtain the above-mentioned adaptive learning network after learning. After this training process, the adaptive learning network can be used to determine the relationship weight between any edge node and the central node among multiple edge nodes.

[0110] Optionally, when determining the relationship weight between any edge node among the multiple edge nodes and the central node, it can also be determined directly by the staff. When obtaining the target identification feature of any edge node among the multiple edge nodes and the target identification feature of the central node, and the target attribute feature of any edge node among the multiple edge nodes and the target attribute feature of the central node, the staff directly sets the relationship weight between any edge node among the multiple edge nodes and the central node according to the similarity of the target identification feature and the target attribute feature of any two nodes.

[0111] In addition, after obtaining the relationship weights of each node, a relationship graph can be constructed based on the relationship weights between any edge node and the central node among the multiple edge nodes. The relationship graph is used to represent the relationship weights between any edge node and the central node among the multiple edge nodes obtained above. The larger the relationship weight value, the stronger the association between the two nodes, that is, the shorter the connection line between the two nodes, the closer the distance between the two nodes. Figure 2 As shown, the larger circular pattern in the figure is used to represent the central node, and the other small circular patterns represent other nodes. The length of the connecting line between the central node and other nodes indicates the strength of the association between the two nodes.

[0112] The relationship graph can also be constructed within a 1-hop or multi-hop range. For example, with A as the central node, node B and central node A establish a one-hop range based on the relationship weight. If B is the central node, node C and central node B establish a one-hop range based on the relationship weight. At this time, node C and node A are within a two-hop range. An increase in the number of hops means that the features between nodes are more interactive. However, too many hops may cause the F node in the multi-hop range to be eventually added to the archive generated with A as the central node, and the F node may have a stronger correlation with other central nodes, so that the F node is not correctly added to the corresponding archive.

[0113] In addition, in step 2, based on the relationship weight between each edge node and the central node, the multiple edge nodes are convolved to obtain the relative embedding feature of each edge node. The implementation process is as follows: the target identification feature of each edge node and the target identification feature of the central node are input into the graph neural network model, and the graph neural network model outputs the relative embedding feature of each edge node based on the relationship weight between each edge node and the central node. Wherein, the graph neural network model includes at least two graph aggregation layers, the input of the first layer of at least two graph aggregation layers is the target identification feature of each edge node of the multiple edge nodes, and the input of each layer other than the first layer of at least two graph aggregation layers is the feature of each edge node output by the previous layer minus the target identification feature of the central node.

[0114] At this time, the target identification feature of each edge node in the multiple edge nodes and the target identification feature of the central node can be the identity feature of the target in the image, or the appearance feature of the target in the image, or the identity feature and appearance feature of the target in the image. It is only necessary to ensure that the target identification feature of each edge node in the multiple edge nodes and the target identification feature of the central node are consistent.

[0115] It should be noted that the graph neural network model in step 2 has a different structure from the first graph neural network model mentioned above.

[0116] The implementation process of the above-mentioned training graph neural network model can be: obtaining multiple training samples, labels for each training sample in the multiple training samples, and the relationship weight between each training sample and any other sample in the multiple training samples. Each training sample is an image, and the label of each training sample is the relative embedding feature of the corresponding image. The label of each training sample is manually labeled according to the relationship weight between each training sample and any other sample. Then, multiple training samples, labels for each training sample in the multiple training samples, and the relationship weight between each training sample and any other sample in the multiple training samples are input into the initialization neural network model. After learning, the initialization neural network model can obtain the above-mentioned graph neural network model. After this training process, the graph neural network model can be used to output the relative embedding feature of each edge node.

[0117] For example, the graph neural network model includes at least two graph aggregation layers, and the relationship weight between each edge node and the central node among the multiple edge nodes obtained above is recorded as W, the target identification feature of one of the multiple edge nodes is recorded as B, and the target identification feature of the central node is recorded as A. σ(·) represents the convolution operation. For each edge node among the multiple edge nodes, in the first convolution layer of the graph aggregation layer of the graph neural network model, the feature F = σ[(BA)*W] of the edge node is obtained. The above operation is performed on each edge node among the multiple edges to obtain the feature of each edge node. Except for the first layer of the graph aggregation layer, the input of each other layer is the feature obtained after the convolution operation of the multiple edge nodes output by the previous layer. The feature is used as the input of each edge node among the multiple edge nodes in this layer, and the above convolution operation is performed again. After the convolution processing of each edge node among the multiple edge nodes in at least two graph aggregation layers, the relative embedding feature of each edge node can be obtained.

[0118] When performing the convolution processing on each edge node, in at least two graph aggregation layers in the graph neural network model, the target identification feature of each edge node is subtracted from the target identification feature of the central node, thereby amplifying the difference between the target identification features of each edge node and the central node. Then, the relative embedding feature of each edge node is obtained, so as to better perform the following operation of determining the association confidence between each edge node and the central node.

[0119] In the aforementioned step 3, the implementation process of determining the association confidence between each edge node and the central node based on the relative embedding feature of each edge node in the plurality of edge nodes can be as follows: the relative embedding feature of each edge node in the plurality of edge nodes and the target identification feature of the central node are input into a binary classifier, and the binary classifier outputs the association confidence between each edge node in the plurality of edge nodes and the central node. The binary classifier is used to identify which edge node in the plurality of edge nodes and the shooting target in the central node belong to the same shooting target, and outputs the association confidence between an edge node and the central node when it is determined that the shooting target in an edge node and the shooting target in the central node belong to the same shooting target.

[0120] It should be noted that the binary classifier is trained in advance with small-scale samples. The implementation process of training the binary classifier can be: obtaining multiple training samples, labels for each training sample in the multiple training samples, target identification features of each training sample, and relative embedding features of each training sample. Among them, each training sample is an image, and the label of each training sample is whether each training sample and any other training sample belong to the same shooting target. The label of each training sample is manually marked according to the target identification features of each training sample and the relative embedding features of each training sample. Then, multiple training samples, labels for each training sample in the multiple training samples, target identification features of each training sample, and relative embedding features of each training sample are input into the initialized binary classifier, and the initialized binary classifier can obtain the above binary classifier after learning. After this training process, the binary classifier can be used to determine the association confidence between each edge node and the central node.

[0121] The binary classifier identifies which edge node among multiple edge nodes has a shooting target and a shooting target in a central node that belong to the same shooting target. If there is a shooting target in an edge node and a shooting target in a central node that belong to the same shooting target, the image of the edge node is determined to be a positive sample, otherwise, the image of the edge node is a negative sample. And when it is determined that a shooting target in an edge node and a shooting target in the central node belong to the same shooting target, that is, when the image of the edge node is determined to be a positive sample, the association confidence between the edge node and the central node is output, and the association confidence between the edge node and the central node indicates the probability that the shooting targets in the images of the edge node and the central node belong to the same shooting target.

[0122] It should be noted that, assuming that there are currently three nodes, namely node A, node B and node C. Then, taking A as the central node, according to the above operations, the association confidences between node B, node C and central node A are obtained. If B is taken as the central node, also according to the above operations, the association confidences between node A, node C and central node B are obtained. At this time, the association confidences between node A and node B may be the same or different. If the association confidences between node A and node B obtained above are different, the two association confidences can be averaged, or the larger value of the two association confidences can be selected as the final association confidences of the two nodes.

[0123] Step 103: Based on the association confidence between each image in the multiple images, the computer device clusters the multiple images to obtain one or more archives, each of the one or more archives including part or all of the multiple images.

[0124] After executing step 101 and step 102, the association confidence between any two images in the multiple images is obtained. At this time, an identity can be assigned to each node according to the association confidence between any two images in the multiple images, and nodes with high association confidence are assigned the same identity, thereby dividing multiple nodes, i.e., multiple images, into one or more archives.

[0125] For example, in the relationship graph obtained above, a label propagation algorithm can be used to assign an identity to each node. The implementation process of the label propagation algorithm is: start from a small first threshold, compare the association confidence between any two nodes in the relationship graph with the first threshold, if there is an association confidence between two nodes greater than the first threshold, that is, the connection line between the two nodes in the relationship graph is regarded as connected, and these connected nodes are recorded as one type of node. Then, nodes whose number of connected nodes is greater than the second threshold are removed from these connected nodes, and these removed nodes become unclustered nodes. Then increase the first threshold, apply the above method to these unclustered nodes, and continue the above operation until all nodes have been successfully clustered, and then assign an identity to each type of node, that is, each type of image, according to the clustering result, so that the multiple images are divided into one or more archives. At this time, each of the one or more archives includes part or all of the multiple images. As Figure 2 As shown in the figure, the solid connecting line in the figure indicates that the central node is connected to the node, and the dotted connecting line in the figure and the "×" symbol on the dotted line indicate that the central node is not connected to the node. The central nodes are divided into four categories by the lines that are connected or disconnected between them, and these four categories of nodes are four archives.

[0126] It should be noted that the size of the first threshold can be set in advance, and the first threshold is a probability value. The second threshold is also set in advance, and the second threshold is an integer value greater than 1, and the second threshold can empirically estimate the maximum number of cluster nodes according to the application scenario, so that multiple nodes can be successfully clustered in this way.

[0127] Optionally, the classification of each image based on the association confidence between each image is not limited to the above-mentioned label propagation algorithm, and may also be implemented by using other clustering algorithms such as K-means clustering algorithm.

[0128] Based on the above steps 101 to 103, the file generation can be realized. Figure 3 For example Figure 1 The illustrated embodiment is further explained.

[0129] Figure 3 This is a flow chart of a method for generating a file provided in an embodiment of the present application. Figure 3As shown, first obtain the target identification features and target attribute features of each image. Then, take each image as the central node and build a relationship graph based on the target identification features and target attribute features of the image. Deduction is performed in the graph convolutional neural network model to obtain the relative embedding features of each image. Then, in the binary classifier, predict the association confidence of each image that the image and the shooting targets in other images belong to the same shooting target (the same shooting target is a positive sample). Finally, assign an identity to each image through the label propagation algorithm to generate one or more archives.

[0130] In an embodiment of the present application, in order to obtain more accurate association information between each node and reduce the interference of negative sample images on the archive, the target identification features and target attribute features of the image are fully utilized in the archive generation process. And the target identification features and target attribute features of the image participate in the adjustment of the relationship weight between the two images, and participate in the generation of the relative embedding features of each node. Secondly, through the relative embedding features of each node obtained, the differences in the target identification features of each edge node and the central node are magnified, so as to better judge whether each edge node is associated with the central node. According to the obtained association confidence between each node, each node is clustered, and then an identity is assigned to each node, thereby generating one or more archives, thereby improving the accuracy of archive generation.

[0131] In addition, based on the archive generation method provided in the above embodiment, the generated one or more archives may be purified. The archive purification method will be described in detail below.

[0132] Figure 4 This is a flowchart of a file purification method provided in an embodiment of the present application, which is applied in a computer device. Figure 4 , the method comprises the following steps.

[0133] Step 401: For a first file in one or more files, a computer device selects one or more images from the first file as a cover image, the image quality of the cover image is higher than the image quality of other images in the first file, and the first file is any file in the one or more files.

[0134] For each of the one or more archives generated above, the cover image of the archive can be determined according to the above method, so the embodiment of the present application takes the first archive as an example for description. After each of the one or more archives is processed according to the above method, the cover image of each archive can be obtained.

[0135] For example, an image with the highest image clarity and no obstruction of the target can be selected from the first file as the cover image. For example, for a person image, a frontal image of the person with the highest image clarity and no obstruction of the target can be selected as the cover image. It is also possible to sort each image in the first file according to the image quality in the first file, and select one or more images with higher image quality than other images as the cover image according to the sorting result.

[0136] When selecting a cover image from the first archive, the image quality can be determined based on the target attribute feature of the image, and all images in the first archive are sorted. According to the sorting result, one or more images are selected as the cover image, and the quality of the cover image must be higher than the quality of other images. The target attribute feature at this time can be one feature of the target attribute feature, or it can be other features in the target attribute feature. Among them, the method for obtaining the target attribute feature of each image in the one or more archives can refer to step 101 of the archive generation method in the previous document, and will not be repeated here.

[0137] Step 402: The computer device determines the association confidence between each other image in the first archive and the cover image.

[0138] Based on the above description, the implementation process of determining the association confidence between each other image in the first archive and the cover image can refer to the steps 101 and 102 in the previous archive generation method, which will not be repeated here.

[0139] Among them, for any image P in the first file except the cover image, if an image is selected as the cover image in the first file, the association confidence between the image P and the cover image is obtained through step 102. If multiple images are selected as cover images in the first file, after the association confidence between the image P in the first file and each of the multiple cover images is obtained, the association confidence between the multiple cover images and the image P can be averaged and used as the association confidence between the image P and the cover image.

[0140] Step 403: Based on the association confidence between each other image in the first archive and the cover image, the computer device deletes some images from the other images in the first archive to purify the images in the first archive.

[0141] As an example, after obtaining the association confidence between each other image in the first file and the cover image, the association confidence can be sorted, and one or more images ranked at the bottom can be deleted according to the sorting result, thereby completing the purification of the images in the first file.

[0142] As another example, a suitable third threshold may be set, and the value of the third threshold may be set in advance. Then, the association confidence between each other image in the first file and the cover image is compared with the third threshold. If there is an image whose association confidence with the cover image is lower than the third threshold, the image is deleted, and finally the purification of the images in the first file is completed.

[0143] Among them, the implementation method of obtaining the third threshold can be: input each image in the first file of the generated one or more files into the second graph convolutional neural network model and the binary classifier for multiple trainings, so as to obtain the precision-recall curve in the first file according to the accuracy of each training result, and set the third threshold according to the precision-recall curve.

[0144] Specifically, if there is a shooting target in an edge node and a shooting target in a central node that belong to the same shooting target, the image of the edge node is determined as a positive sample, and the precision measures the image that matches the actual state among all images determined as positive samples, that is, the probability that the images correctly determined as positive samples account for all images determined as positive samples. The recall rate measures how many images are correctly determined as positive samples among the images whose actual state is positive samples. Then, with the precision as the y-axis and the recall rate as the x-axis, the precision-recall curve is obtained. Determine the precision value required for clustering according to the application scenario, and then find the corresponding recall rate on the precision-recall curve according to the precision value and set it as the third threshold, such as the threshold under 95% precision.

[0145] Based on the above steps 401 to 403, file purification can be achieved. Figure 5 For example Figure 4 The illustrated embodiment is further explained.

[0146] Figure 5 This is a flow chart of a file purification method provided in the embodiment of the present application. Figure 5 As shown in the figure, first, the cover image is obtained according to the target attribute characteristics of the images in the archive. Then, with each image as the central node, a relationship graph is constructed based on the target identification characteristics and target attribute characteristics of the image. The same method as the archive generation is used to deduce the graph convolutional neural network model. Then, in the binary classifier, the association confidence of each image and the shooting targets in other images belonging to the same shooting target is predicted. The images are sorted according to the association confidence of each image with other images, and the images with association confidence less than the set threshold are deleted, and the archive is purified.

[0147] In the embodiment of the present application, in the one or more archives finally obtained by the above archive generation, the second graph convolutional neural network model and the dichotomizer are used to obtain the association confidence between each node in each archive and the cover node, and then the association confidence between each node and the cover node is sorted, and the nodes with confidence less than the set threshold are deleted to complete the purification of the archive. Moreover, when there are multiple cover nodes for an archive, the association confidence between the multiple cover nodes and each other node is averaged to improve the accuracy of archive purification.

[0148] In addition, based on the archive generation method and archive purification method provided in the above embodiments, when there are new images that need to be clustered, the archive enhancement method can be used to complete the archive enhancement. The archive enhancement method will be described in detail below.

[0149] Figure 6 This is a flowchart of a method for increasing the file size provided in an embodiment of the present application, which is applied in a computer device. Figure 6 , the method comprises the following steps.

[0150] Step 601: The computer device determines a representative node of each file in one or more files, and the characteristics of the representative node of each file are average characteristics of the photographed targets in each image included in the corresponding file, and the average characteristics include average target identification characteristics and / or average target attribute characteristics.

[0151] That is to say, in the one or more archives finally generated by the above archive purification, the representative node of each archive in the one or more archives is determined according to the target identification feature and target attribute feature of each archive in the one or more archives.

[0152] As an example, for any file A in one or more files generated after purification, the target identification features and target attribute features of each image in file A can be averaged to obtain the average features of the photographed targets in each image included in file A, and the average features can be used as the features of the representative node of file A.

[0153] Among them, the method for obtaining the target identification feature and the target attribute feature of each image in the one or more archives can refer to the step 101 of the above-mentioned archive generation method, and will not be repeated here. Moreover, the target identification feature of each image in the one or more archives can be the identity feature of the target in the image, or the appearance feature of the target in the image, or the identity feature and appearance feature of the target in the image. The target attribute feature of each image in the one or more archives can be one feature of the target attribute feature, or two features of the target attribute feature, or all features of the target attribute feature.

[0154] As another example, in the above-mentioned archive purification process, cover images of one or more archives are obtained. At this time, the one or more cover images obtained in the above-mentioned archive purification process can also be directly used as representative nodes of the corresponding archives in the one or more archives.

[0155] It should be noted that the representative node of each file in the one or more files may be one representative node or multiple representative nodes, which is not limited in the embodiment of the present application.

[0156] Step 602: For a second archive in one or more archives, based on features of a representative node of the second archive and target identification features and / or target attribute features of the image to be archived, the computer device determines the association confidence between the image to be archived and the representative node of the second archive.

[0157] Among them, the second file is any one of the one or more files after the above-mentioned aggregation or purification. That is, for each of the one or more files generated above, the association confidence between the image to be aggregated and the representative node of the file can be determined according to the above method, so the embodiment of the present application takes the second file as an example for explanation. After each file in the one or more files is processed according to the above method, the association confidence between the image to be aggregated and the representative node of each file can be obtained.

[0158] Based on the above description, the implementation process of determining the association confidence between the image to be archived and the representative node of the second archive can refer to steps 101 and 102 in the previous archive generation method, which will not be repeated here.

[0159] Step 603: Based on the association confidence between the image to be archived and the representative node of the second archive, the computer device determines whether to add the image to be archived to the second archive.

[0160] When the representative node of the second file is a representative node and there are only a few images to be aggregated, a fourth threshold can be set in advance. The size of the fourth threshold is not limited. The association confidence between the image to be aggregated and the representative node of the second file obtained above is compared with the fourth threshold. If the association confidence between the image to be aggregated and the representative node of the second file is higher than the fourth threshold, it is determined that the image to be aggregated is associated with the second file, and then the image to be aggregated is added to the second file. Otherwise, a new file is generated for the image to be aggregated.

[0161] When the representative node of the second archive is one representative node and there are many images to be clustered, at this time, according to step 103 in the above-mentioned archive generation process, the images to be clustered and the representative nodes of the second archive are clustered through step 103, and the nodes with high association confidence between the images to be clustered and the representative nodes of the second archive are associated together, and these images to be clustered associated with the representative nodes of the second archive are added to the second archive.

[0162] When there are multiple representative nodes of the second archive, whether the representative nodes of the archive image to be aggregated and the second image are associated is determined based on the association confidence between the archive image to be aggregated and the representative nodes of the second archive. If some of the multiple representative nodes are associated with the archive image to be aggregated, and the remaining nodes are not associated with the archive image to be aggregated, a fifth threshold value may be set in advance, and the size of the fifth threshold value is not limited. The number of associations between the archive image to be aggregated and the representative nodes of the second archive is counted. If the number is greater than the fifth threshold value, it is considered that the archive image to be aggregated and the representative nodes of the second archive are associated, and the archive image to be aggregated is added to the second archive. Otherwise, it is considered that the representative nodes of the archive image to be aggregated and the second archive are not associated, and a new archive is generated for the archive image to be aggregated.

[0163] Based on the above steps 601 to 603, file purification can be achieved. Figure 7 For example Figure 6 The illustrated embodiment is further explained.

[0164] Figure 7 This is a flow chart of a method for adding files to a file provided in an embodiment of the present application. Figure 7 As shown, first, a representative node is obtained from the archive generated above, and then the representative node and the new archive image to be aggregated are subjected to the above-mentioned archive generation method process, and finally, based on the correlation between the archive image to be aggregated and the representative node, it is determined whether the archive image to be aggregated should be added to the archive where the representative node is located or become a new archive.

[0165] In the embodiment of the present application, when there is a new image to be aggregated, based on one or more archives generated as described above, a second graph convolutional neural network model and a binarizer are used to obtain the association confidence between the representative node of each archive and the image to be aggregated, and then determine whether the image to be aggregated is associated with the representative node. If the image to be aggregated is associated with the representative node, the image to be aggregated is added to the archive to which the representative node belongs. Otherwise, the image to be aggregated forms a new archive.

[0166] Figure 8 This is a schematic diagram of the structure of a file generation device provided in an embodiment of the present application. The file generation device can be implemented as part or all of a computer device by software, hardware, or a combination of both. Figure 8The device includes: an acquisition module 801, a determination module 802 and a clustering module 803.

[0167] The acquisition module 801 is used to acquire the target identification feature and the target attribute feature of each image in the multiple images, wherein the target identification feature of each image is used to identify the photographed target in the corresponding image, and the target attribute feature of each image is used to describe the attribute of the photographed target in the corresponding image.

[0168] A first determination module 802 is used to determine the association confidence between each of the multiple images based on the target identification feature and the target attribute feature of each of the multiple images, and the association confidence between any two images indicates the probability that the shooting targets in any two images belong to the same shooting target;

[0169] The clustering module 803 is used to cluster the multiple images based on the association confidence between each image in the multiple images to obtain one or more archives, each of the one or more archives including part or all of the multiple images.

[0170] Optionally, the first determining module 802 is specifically configured to:

[0171] Take each of the multiple images as a central node, and other images except the central node as edge nodes, and perform the following operations on each central node:

[0172] Determine a relationship weight between each edge node in the plurality of edge nodes and the central node based on a target identification feature and a target attribute feature of each edge node in the plurality of edge nodes and a target identification feature and a target attribute feature of the central node, wherein the relationship weight indicates a correlation between the corresponding two images;

[0173] Based on the relationship weight between each edge node and the central node in the plurality of edge nodes, a convolution process is performed on the plurality of edge nodes to obtain a relative embedding feature of each edge node, where the relative embedding feature of each edge node indicates the difference between the corresponding edge node and the central node;

[0174] Based on the relative embedding features of each edge node among the plurality of edge nodes, an association confidence between each edge node and the central node is determined.

[0175] Optionally, the first determining module 802 is specifically configured to:

[0176] Inputting the target identification feature of each edge node among the multiple edge nodes and the target identification feature of the central node into the graph neural network, the graph neural network outputs the relative embedding feature of each edge node based on the relationship weight between each edge node among the multiple edge nodes and the central node;

[0177] Among them, the graph neural network includes at least two graph aggregation layers, the input of the first layer of at least two graph aggregation layers is the target identification feature of each edge node among multiple edge nodes, and the input of each layer except the first layer of at least two graph aggregation layers is the feature of each edge node output by the previous layer minus the target identification feature of the central node.

[0178] Optionally, the first determining module 802 is specifically configured to:

[0179] For any edge node among the plurality of edge nodes, determining the similarity between the target identification features of any edge node and the central node, and the similarity between the target attribute features of any edge node and the central node;

[0180] Based on the similarity between the target identification features of any edge node and the central node, and the similarity between the target attribute features of any edge node and the central node, the relationship weight between any edge node and the central node is determined.

[0181] Optionally, the first determining module 802 is specifically configured to:

[0182] For any edge node among the multiple edge nodes, based on the target identification feature and target attribute feature of any edge node and the target identification feature and target attribute feature of the central node, the relationship weight between any edge node and the central node is determined through an adaptive learning network.

[0183] Optionally, the first determining module 802 is specifically configured to:

[0184] Inputting the relative embedding feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a binary classifier, the binary classifier outputting the association confidence between each edge node in the plurality of edge nodes and the central node;

[0185] Among them, the binary classifier is used to identify which edge node among multiple edge nodes has a shooting target and a shooting target in the central node belonging to the same shooting target, and when it is determined that a shooting target in an edge node and a shooting target in the central node belong to the same shooting target, output an association confidence between an edge node and the central node.

[0186] Optionally, the device further comprises:

[0187] a selection module, for selecting one or more images from a first file in one or more files as a cover image, wherein the image quality of the cover image is higher than the image quality of other images in the first file, and the first file is any file in the one or more files;

[0188] The purification module is used to delete some images in other images in the first file based on the association confidence between each other image in the first file and the cover image, so as to purify the images in the first file.

[0189] Optionally, the device further comprises:

[0190] A second determination module is used to determine a representative node of each file in one or more files, wherein the feature of the representative node of each file is an average feature of the shooting target in each image included in the corresponding file, and the average feature includes an average target identification feature and / or an average target attribute feature;

[0191] A third determination module is used to determine, for a second file in the one or more files, the association confidence between the image to be aggregated and the representative node of the second file based on the features of the representative node of the second file and the target identification features and / or target attribute features of the image to be aggregated;

[0192] The fourth determination module is used to determine whether to add the image to be aggregated into the second archive based on the association confidence between the image to be aggregated and the representative node of the second archive.

[0193] In an embodiment of the present application, in order to obtain more accurate association information between each node and reduce the interference of negative sample images on the archive, the target identification features and target attribute features of the image are fully utilized in the archive generation process. And the target identification features and target attribute features of the image participate in the adjustment of the relationship weight between the two images, and participate in the generation of the relative embedding features of each node. Secondly, through the relative embedding features of each node obtained, the differences in the target identification features of each edge node and the central node are magnified, so as to better judge whether each edge node is associated with the central node. According to the obtained association confidence between each node, each node is clustered, and then an identity is assigned to each node, thereby generating one or more archives, thereby improving the accuracy of archive generation.

[0194] It should be noted that: the archive generation device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when generating archives. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the archive generation device provided in the above embodiment and the archive generation method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0195] Fig. 91 is a block diagram of a computer device 900 provided in an embodiment of the present application. The computer device 900 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The computer device 900 may also be referred to as a user device, a portable terminal, a laptop computer device, a desktop computer device, or other names.

[0196] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0197] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0198] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the archive generation method provided in the method embodiment of the present application.

[0199] In some embodiments, the computer device 900 may also optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a touch display screen 905, a camera 906, an audio circuit 907, a positioning component 908 and a power supply 909.

[0200] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0201] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in the embodiments of the present application.

[0202] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, and the front panel of the computer device 900 is set; in other embodiments, the display screen 905 can be at least two, which are respectively set on different surfaces of the computer device 900 or are folded; in some other embodiments, the display screen 905 can be a flexible display screen, which is set on the curved surface or folded surface of the computer device 900. Even, the display screen 905 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0203] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0204] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the computer device 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0205] The positioning component 908 is used to locate the current geographical location of the computer device 900 to implement navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0206] The power supply 909 is used to power various components in the computer device 900. The power supply 909 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0207] In some embodiments, the computer device 900 further includes one or more sensors 910 , including but not limited to: an acceleration sensor 911 , a gyroscope sensor 912 , a pressure sensor 913 , a fingerprint sensor 914 , an optical sensor 915 , and a proximity sensor 916 .

[0208] The acceleration sensor 911 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the computer device 900. For example, the acceleration sensor 911 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the touch display screen 905 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used for collecting game or user motion data.

[0209] The gyro sensor 912 can detect the body direction and rotation angle of the computer device 900, and the gyro sensor 912 can cooperate with the acceleration sensor 911 to collect the user's 3D actions on the computer device 900. The processor 901 can implement the following functions based on the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0210] The pressure sensor 913 can be set on the side frame of the computer device 900 and / or the lower layer of the touch display screen 905. When the pressure sensor 913 is set on the side frame of the computer device 900, it can detect the user's grip signal of the computer device 900, and the processor 901 performs left and right hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 913. When the pressure sensor 913 is set on the lower layer of the touch display screen 905, the processor 901 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0211] The fingerprint sensor 914 is used to collect the user's fingerprint, and the processor 901 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 914 can be set on the front, back, or side of the computer device 900. When a physical button or a manufacturer logo is set on the computer device 900, the fingerprint sensor 914 can be integrated with the physical button or the manufacturer logo.

[0212] The optical sensor 915 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the touch display screen 905 according to the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera component 906 according to the ambient light intensity collected by the optical sensor 915.

[0213] The proximity sensor 916, also called a distance sensor, is usually disposed on the front panel of the computer device 900. The proximity sensor 916 is used to collect the distance between the user and the front of the computer device 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the computer device 900 is gradually decreasing, the processor 901 controls the touch display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 916 detects that the distance between the user and the front of the computer device 900 is gradually increasing, the processor 901 controls the touch display screen 905 to switch from the screen-off state to the screen-on state.

[0214] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the computer device 900, and the computer device 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0215] In some embodiments, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the archive generation method in the above embodiment are implemented. For example, the computer-readable storage medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0216] It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0217] It should be understood that all or part of the steps to implement the above embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-mentioned computer readable storage medium.

[0218] That is, in some embodiments, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of the above-mentioned archive generation method.

[0219] It should be understood that the "at least one" mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0220] The above are embodiments provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a file, characterized in that: The method comprises: Obtaining target identification features and target attribute features of each image in the plurality of images, wherein the target identification features of each image are used to identify a photographed target in the corresponding image, and the target attribute features of each image are used to describe an attribute of a photographed target in the corresponding image; Each of the multiple images is sequentially used as a central node, and the images other than the central node are used as edge nodes, and the following operations are performed on each central node: based on the target identification feature and the target attribute feature of each edge node in the multiple edge nodes, and the target identification feature and the target attribute feature of the central node, a relationship weight between each edge node in the multiple edge nodes and the central node is determined, and the relationship weight indicates the correlation between the corresponding two images; based on the relationship weight between each edge node in the multiple edge nodes and the central node, a convolution process is performed on the multiple edge nodes to obtain a relative embedding feature of each edge node, and the relative embedding feature of each edge node indicates the difference between the corresponding edge node and the central node; based on the relative embedding feature of each edge node in the multiple edge nodes, an association confidence between each edge node and the central node is determined, and the association confidence between any two images indicates the probability that the shooting targets in the any two images belong to the same shooting target; Based on the association confidence between each image in the plurality of images, the plurality of images are clustered to obtain one or more archives, each of the one or more archives including part or all of the images in the plurality of images.

2. The method according to claim 1, characterized in that The step of performing convolution processing on the plurality of edge nodes based on the relationship weight between each edge node in the plurality of edge nodes and the central node to obtain a relative embedding feature of each edge node includes: Inputting the target identification feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a graph neural network, wherein the graph neural network outputs a relative embedding feature of each edge node based on a relationship weight between each edge node in the plurality of edge nodes and the central node; The graph neural network includes at least two graph aggregation layers, the input of the first layer of the at least two graph aggregation layers is the target identification feature of each edge node among the multiple edge nodes, and the input of each layer except the first layer of the at least two graph aggregation layers is the feature of each edge node output by the previous layer minus the target identification feature of the central node.

3. The method according to claim 1, characterized in that The determining, based on the target identification feature and the target attribute feature of each edge node in the plurality of edge nodes and the target identification feature and the target attribute feature of the central node, the relationship weight between each edge node in the plurality of edge nodes and the central node comprises: For any edge node among the plurality of edge nodes, determining a similarity between a target identification feature of the any edge node and the central node, and a similarity between a target attribute feature of the any edge node and the central node; Based on the similarity between the target identification features of the any edge node and the central node and the similarity between the target attribute features of the any edge node and the central node, the relationship weight between the any edge node and the central node is determined.

4. The method according to claim 1, characterized in that The determining, based on the target identification feature and the target attribute feature of each edge node in the plurality of edge nodes and the target identification feature and the target attribute feature of the central node, the relationship weight between each edge node in the plurality of edge nodes and the central node comprises: For any edge node among the multiple edge nodes, based on the target identification feature and target attribute feature of the any edge node and the target identification feature and target attribute feature of the central node, the relationship weight between the any edge node and the central node is determined through an adaptive learning network.

5. The method according to claim 1, characterized in that The determining, based on the relative embedding feature of each edge node in the plurality of edge nodes, the association confidence between each edge node and the central node comprises: Inputting the relative embedding feature of each edge node in the plurality of edge nodes and the target identification feature of the central node into a binary classifier, the binary classifier outputting the association confidence between each edge node in the plurality of edge nodes and the central node; Among them, the binary classifier is used to identify which edge node among the multiple edge nodes has a shooting target and the shooting target in the central node belong to the same shooting target, and when it is determined that the shooting target in an edge node and the shooting target in the central node belong to the same shooting target, output the association confidence between the edge node and the central node.

6. The method according to claim 1, characterized in that The method further comprises: For a first file in the one or more files, one or more images are selected from the first file as cover images, the image quality of the cover image is higher than the image quality of other images in the first file, and the first file is any file in the one or more files; Based on the association confidence between each other image in the first archive and the cover image, some images in the other images in the first archive are deleted to purify the images in the first archive.

7. The method according to claim 1, characterized in that The method further comprises: Determine a representative node of each of the one or more archives, wherein the feature of the representative node of each archive is an average feature of the photographed targets in each image included in the corresponding archive, wherein the average feature includes an average target identification feature and / or an average target attribute feature; For a second file in the one or more files, determining the association confidence between the image to be aggregated and the representative node of the second file based on the features of the representative node of the second file and the target identification features and / or target attribute features of the image to be aggregated; Based on the association confidence between the image to be archived and the representative node of the second archive, determine whether to add the image to be archived to the second archive.

8. A file generating device, characterized in that: The device comprises: An acquisition module, used to acquire a target identification feature and a target attribute feature of each image in the plurality of images, wherein the target identification feature of each image is used to identify a photographed target in the corresponding image, and the target attribute feature of each image is used to describe an attribute of a photographed target in the corresponding image; The first determination module is used to sequentially use each of the multiple images as a central node, and other images except the central node as edge nodes, and perform the following operations on each central node: based on the target identification feature and target attribute feature of each edge node in the multiple edge nodes, and the target identification feature and target attribute feature of the central node, determine the relationship weight between each edge node in the multiple edge nodes and the central node, and the relationship weight indicates the correlation between the corresponding two images; based on the relationship weight between each edge node in the multiple edge nodes and the central node, perform convolution processing on the multiple edge nodes to obtain the relative embedding feature of each edge node, and the relative embedding feature of each edge node indicates the difference between the corresponding edge node and the central node; based on the relative embedding feature of each edge node in the multiple edge nodes, determine the association confidence between each edge node and the central node, and the association confidence between any two images indicates the probability that the shooting targets in the any two images belong to the same shooting target; A clustering module is used to cluster the multiple images based on the association confidence between each image in the multiple images to obtain one or more archives, each of the one or more archives including part or all of the images in the multiple images.

Images