Electronic archive generation method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of data processing technology, and provides an electronic archive generation method, device, terminal equipment and storage medium, the method comprising: obtaining multiple images to be archived; identifying the target object of each image to be archived, and obtaining visual feature information of all target objects corresponding to the multiple images to be archived; obtaining auxiliary feature information corresponding to each target object from the multiple images to be archived, the auxiliary feature information representing the time or space information associated with the corresponding target object; clustering all target objects according to the visual feature information and auxiliary feature information corresponding to each target object, and obtaining multiple electronic archives. The visual feature information and auxiliary feature information of the target object are used to cluster all target objects in the image to be archived, and the auxiliary feature information introduces the time information or space information of the target object into the archiving process, thereby improving the clustering accuracy and meeting the business requirements in actual engineering applications.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to an electronic archive generation method, apparatus, terminal equipment and storage medium. Background Art

[0002] As the Internet and the Internet of Things play an increasingly profound role in our daily lives, data digitization is a very important link. Electronic archives, as electronic and archival data, are indispensable for mining potential facts and laws. Electronic and archival data can be used for big data analysis and mining, for machine learning model training, and even for actual business implementation by mining valuable intelligence. Especially in the past decade, machine learning models in the field of machine vision have reached a level that can replace manual labor. As deep learning has made important progress in the field of machine vision, electronic archives of visual data have gradually been integrated into all aspects of people's lives.

[0003] Visual data generally includes video data and image data. Automatic generation of electronic archives of visual data is the process of data management and analysis of user-defined target objects and clustering data according to certain rules. For example, given a certain number of terminal camera snapshots, the user-defined target object is a person, then the automatic generation of electronic archives of visual data is to group the snapshots of the same person together and separate the snapshots of different people; finally, a snapshot image archive of each person is formed, which is called one person one file in engineering business. According to the target object defined by the user, the target object of the archive automatically generated by the electronic archive of visual data can be a collection of people, objects, or even a certain abstract concept. In engineering business, there is one person one file, one car one file, one relationship one file, etc. In one relationship one file, the target object defined by the user is an abstract concept. Specific relationships can be friend relationships and family relationships, etc. One relationship one file of visual data can form the modeling, analysis and mining of people's social relationships.

[0004] The current automatic generation of electronic archives of visual data generally clusters visual data based on feature information of a single dimension, especially visual feature information of a single dimension. For example, in one person one archive, traditional models and algorithms are based on the visual features of the target object in the picture or video (similar to the key point features of the target body). The traditional automatic generation method of electronic archives of visual data has low clustering accuracy and often fails to meet business requirements in actual engineering applications. Summary of the invention

[0005] The embodiments of the present application provide an electronic archive generation method, apparatus, terminal device and storage medium, which can solve the technical problem that the traditional visual data electronic archive automatic generation method has low clustering accuracy and often fails to meet business requirements in actual engineering applications.

[0006] In a first aspect, an embodiment of the present application provides a method for generating an electronic archive, comprising:

[0007] Acquire multiple images to be archived;

[0008] Identify the target object for each of the images to be archived, and obtain visual feature information of all the target objects corresponding to the multiple images to be archived;

[0009] Acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object;

[0010] All the target objects are clustered according to the visual feature information and the auxiliary feature information corresponding to each target object to obtain a plurality of electronic files.

[0011] In a second aspect, an embodiment of the present application provides an electronic archive generation device, including:

[0012] An image acquisition unit, used for acquiring a plurality of images to be archived;

[0013] A first feature information acquisition unit, configured to identify a target object for each of the images to be archived, and obtain visual feature information of all target objects corresponding to the multiple images to be archived;

[0014] A second feature information acquisition unit, configured to acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object;

[0015] The electronic archive forming unit is used to cluster all the target objects according to the visual feature information and the auxiliary feature information corresponding to each of the target objects to obtain a plurality of electronic archives.

[0016] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the steps of the method described in the first aspect above.

[0019] As can be seen from the above, the embodiment of the present application first identifies the target objects of multiple images to be archived, and clusters all the target objects to obtain multiple electronic archives based on the visual feature information and auxiliary feature information of all the target objects corresponding to the multiple images to be archived, thereby avoiding the problem of low clustering accuracy caused by relying on the visual feature information of a single target object for clustering. At the same time, the auxiliary feature information represents the time or space information associated with the target object. Therefore, the time information or space information of the target object is introduced into the archiving process through the auxiliary features, which can effectively supplement the visual feature information of the target object. In particular, for visual data with insufficient visual feature information of the target object or visual data of low data quality, the accuracy of its archiving can be effectively improved, thereby meeting the business requirements in actual engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0022] Figure 2 It is a flowchart of a method for generating an electronic file provided in an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of the structure of the electronic archive generation device provided in an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The electronic archive in the embodiment of the present application specifically refers to clustering visual data with the same target object attribute into one archive, and visual data with different target object attributes are divided into different archives. The target object attribute of the archive is user-defined. For example, when the target object attribute is defined as a person, then the electronic archive generated in the embodiment of the present application is a collection of visual data in units of people. Of course, the target object attribute can be set as needed, for example, the target object attribute can be a person or a car, etc.

[0032] The electronic archive generation method provided in this application can be applied to Figure 1In the scene shown, the scene includes multiple acquisition devices 11 and electronic file generating devices 12. The multiple acquisition devices are set at different positions. The multiple acquisition devices 11 send the multiple images they have taken to the electronic file generating device 12. The electronic file generating device 12 generates an electronic file, establishes a file for each target object individual involved in all images, and classifies the corresponding images into the corresponding electronic files.

[0033] Multiple collection devices 11 can be set at multiple different locations in a fixed area, for example, they may be set at different locations in a shopping mall, such as the elevator entrance, the entrance, the escalator entrance, the exit, etc. The electronic file generating device 12 archives the people who appear in different collection devices and generates a file for each person for the shopping mall.

[0034] The electronic archive generation device in the embodiment of the present application may be a terminal device such as a server, a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.

[0035] In another embodiment, the acquisition device 11 has a certain data processing function. While taking images, it can also pre-process the images before sending them to the electronic archive generation device 12. The pre-processing can be to perform structural processing on the captured images and extract the visual feature information of the target object therein; the structurally processed images and the corresponding visual feature information of the target object are transmitted to the electronic archive generation device 12, thereby further completing the subsequent archiving operation. The data processing function of the acquisition device can reduce the data processing pressure of the electronic archive generation device 12.

[0036] The acquisition device in the embodiment of the present application can be a camera with data processing function. In addition, the acquisition device can also be a terminal device such as a mobile phone, a tablet computer, a laptop computer, etc. with a camera function. The embodiment of the present application does not impose any restrictions on the specific type of the acquisition device. Of course, in some embodiments, the acquisition device can also be an ordinary camera.

[0037] The electronic archive generation method provided by the present application is exemplarily described below in conjunction with specific embodiments.

[0038] Figure 2 A schematic flow chart of the electronic archive generation method provided by the present application is shown in FIG. Figure 2 , the method comprising:

[0039] Step S201, obtaining a plurality of images to be archived.

[0040] The images to be archived in the embodiments of the present application are from visual data, which may be picture data and / or video data, wherein the images to be archived may be images directly obtained from an acquisition device, or may be image frames obtained from video data obtained by an acquisition device. Multiple images to be archived may be obtained directly from an acquisition device, or may be obtained and stored in advance from an acquisition device.

[0041] In one embodiment, the image to be archived needs to be structured before feature extraction. The structured processing of the image refers to extracting the pixel value of each pixel point of the image to obtain the pixel value of each pixel point.

[0042] In one embodiment, a pooling operation may also be performed on the structured images to make all images of uniform size, thereby facilitating subsequent feature information extraction and feature fusion steps.

[0043] In the embodiment of the present application, each image to be archived may contain one or more target objects.

[0044] Step S202: identifying a target object for each of the images to be archived, and obtaining visual feature information of all target objects corresponding to the multiple images to be archived.

[0045] In the embodiment, the visual feature information is the visual feature of the target object itself, specifically, the feature that can distinguish different target objects from a visual perspective. Different target objects have different visual feature information. For example, if the target object for archiving is a person, that is, the image to be archived is to be formed into an electronic file according to one file for each person, then the visual feature information can be a facial feature or a body feature.

[0046] In the embodiment, all target objects corresponding to multiple images to be archived refer to all target objects included in the multiple images to be archived. For example, assuming that electronic archives need to be generated for four images to be archived, there is 1 target object in the first image to be archived, 3 target objects in the second image to be archived, 3 target objects in the third image to be archived, and 2 target objects in the fourth image to be archived; then the sum of the numbers of all target objects corresponding to the four images to be archived, that is, the number of all target objects corresponding to the four images to be archived is 9.

[0047] Exemplarily, the recognition of the target object is the process of extracting the visual feature information of the target object. The visual feature information can be obtained by extracting features from the archived image using a pre-trained deep learning network model. The deep learning network model in the embodiment of the present application can be a convolutional neural network CNN, a residual network ResNet, a graph convolutional neural network GCN, a Transformer, etc., which are not specifically limited here. The extraction of visual feature information using a deep learning network model is a conventional technical means in the field and will not be elaborated here.

[0048] Step S203: Acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object.

[0049] The auxiliary feature information in the embodiment is time-related information or space-related information associated with the corresponding target object. The auxiliary feature information can supplement the visual feature information of the target object from the perspective of time or space, thereby making full use of the relevant information in the image to be archived and laying the foundation for the subsequent accurate clustering of the image to be archived.

[0050] In one embodiment, the auxiliary feature information may include background feature information. The background feature information specifically refers to the background information after the visual feature information of the target object is removed from the image to be archived corresponding to the target object. It is understandable that the method for obtaining the background feature information may be: performing a difference process on the image to be archived corresponding to each target object and the visual feature information of the target object to obtain the background feature information of each target object. For example, when the visual feature information of the target object in the image to be archived is incomplete or unclear, the background feature information can be a good supplement to the visual feature information of the target object. For example, other images to be archived with background feature information of high similarity can be used to cluster the target objects in the image to be archived.

[0051] For example, suppose that a certain image to be archived includes three target objects, numbered a, b and c respectively, wherein the background feature information of target object a refers to the image information after the target object a is removed from the image to be archived, that is, in the image to be archived, target objects b and target objects c are both the background feature information of target object a. By analogy, the background feature information of target objects b and target objects c can be obtained.

[0052] In one embodiment, the auxiliary feature information may also include acquisition device attribute information. Acquisition device attribute information refers to information that a target object possesses based on certain attributes of the acquisition device. Acquisition device attribute information particularly refers to information related to the acquisition device that affects the visual feature information of the target object. Acquisition device attribute information can provide additional information support for the formation of a high-accuracy visual data electronic archive.

[0053] Exemplarily, the acquisition device attribute information in the embodiment may include target object density information, target object acquisition angle information and / or clarity information. The target object density information, target object acquisition angle information and clarity information all have a great influence on the quality of the extracted visual feature information of the target object. The specific requirements for the visual feature information in the archiving process can be adjusted according to the target object density information, target object acquisition angle information and clarity information. For example, when comparing the similarity of two target objects belonging to different images to be archived according to the visual feature information, different similarity thresholds can be set for target objects with different acquisition device attribute information (target object density information, target object acquisition angle information and / or clarity information).

[0054] In the embodiment of the present application, the target object density information refers to the number of target objects in the image to be archived corresponding to the target object. Generally speaking, the target object density information is related to the installation location of the acquisition device, so in the embodiment of the present application, the target object density information is classified as the acquisition device attribute information, and it can be seen that the target object density information is associated with the spatial information of the image to be archived.

[0055] The application scenario in which the target object is a person is specifically explained. When the target object is a person, the target object density refers to the density of people in the image to be archived corresponding to a certain target object, that is, the number of people; the density of people has a great influence on the archiving result, and the density of people will have a great influence on the quality of the extraction of the visual feature information of the target object: Generally speaking, the quality of the visual feature information of the target object extracted from the image to be archived with a large density of people is low, so when clustering it later, it is necessary to lower the requirements for the image to be archived in order to achieve accurate clustering. For example, assuming that clustering is performed by comparing the similarity of the visual feature information of the target object, for the image to be archived with a small density of people, it is judged to be the same person when the similarity is above 80%, and for the image to be archived with a large density of people, it can be judged to be the same person when the similarity is above 60%.

[0056] In the embodiment of the present application, the target object density information may be the target object density information in the current image to be archived corresponding to the target object, or may be the average target object density information in a preset historical time period. The average target object density information refers to the average value of the target object density information corresponding to the historical images acquired by the acquisition device for acquiring the image to be archived in the preset historical time period.

[0057] When the target object density information is the target object density information in the current image to be archived corresponding to the target object, the method for obtaining the target object density information includes: counting the target objects in the image to be archived corresponding to each target object to obtain the number of target objects, and the number of target objects is used as the target object density information corresponding to each of the target objects.

[0058] When the target object density information is the average target object density information within a preset historical time period, the method for obtaining the target object density information includes: obtaining multiple historical images, the multiple historical images have the same corresponding acquisition conditions as the current image to be archived (the same acquisition conditions refer to having the same acquisition parameters at the same installation position, for example, belonging to the same acquisition device); counting the target objects in each historical image, and averaging the counting results to obtain the average target object density information, and the average target object density information is used as the target object density information of the corresponding target object.

[0059] In the embodiment, the target object acquisition angle information refers to the deflection angle of the target object, and the optional deflection angles may include three angle information: roll angle, pitch angle, and yaw angle. The target object acquisition angle is related to the installation position and direction of the device.

[0060] Taking the scene where the target object is a person as an example, the target object acquisition angle information refers to the angle of the collected face. Generally speaking, it can be assumed that the acquisition device is fixed, so the target object acquisition angle is related to the activity pattern of the person and the installation of the acquisition device. Therefore, in the embodiment of the present application, the target object acquisition angle information is classified as device attribute information, and it can be known that the target object acquisition angle information is associated with the spatial information of the image to be archived corresponding to the target image.

[0061] In the embodiment of the present application, the target object acquisition angle information may be the target object acquisition angle information in the current image to be archived corresponding to the target object, or may be a historical average value within a preset historical time period.

[0062] When the target object acquisition angle information is the target object acquisition angle information in the current image to be archived corresponding to the target object, the method for obtaining the target object acquisition angle may include: inputting the current image to be archived into a pre-trained acquisition angle acquisition model for processing to obtain the target object acquisition angle information.

[0063] When the target object acquisition angle information is a historical average, the method for obtaining the target object acquisition angle may include: obtaining multiple historical images, the multiple historical images have the same acquisition conditions corresponding to the current image to be archived (the same acquisition conditions refer to having the same acquisition parameters at the same installation position, for example, belonging to the same acquisition device); inputting the multiple historical images into a pre-trained acquisition angle acquisition model for processing to obtain multiple historical acquisition angle information; taking an average of the multiple historical acquisition angle information to obtain the target object acquisition angle information.

[0064] In the embodiment of the present application, the clarity information may be the clarity information of the current image to be archived corresponding to the target object, or may be a historical average value within a preset historical time period.

[0065] When the definition information is the definition information of the current image to be archived corresponding to the target object, the method for obtaining the definition information may include: inputting the current image to be archived into a pre-trained definition model for processing to obtain the definition information.

[0066] When the clarity information is a historical average, the method for obtaining the clarity information may include: obtaining multiple historical images, the multiple historical images having the same acquisition conditions as the current image to be archived (the same acquisition conditions refer to having the same acquisition parameters at the same installation location, for example, belonging to the same acquisition device); inputting the multiple historical images into a pre-trained clarity model for processing to obtain multiple historical clarity information; averaging the multiple historical clarity information to obtain the clarity information.

[0067] The acquisition angle acquisition model and clarity model in the embodiments of the present application can both be deep learning network models. Specifically, the deep learning network model can be a convolutional neural network CNN, a residual network ResNet, a graph convolutional neural network GCN, a Transformer, etc. Deep learning network models for different purposes are trained using conventional training methods using different data sets. Extracting target object acquisition angle information and clarity information using a deep learning network model is a conventional technical means in the field and will not be elaborated here.

[0068] In the embodiment of the present application, since the target object acquisition angle information and clarity information corresponding to a single image are highly accidental, this accidental nature will generate a large amount of noise when used for clustering. The use of historical average values ​​can reduce the noise caused by accidental nature, making the acquired target object acquisition angle information and clarity information more representative, and reducing the impact of noise on clustering accuracy.

[0069] The target object acquisition angle information and clarity information have a great impact on the extraction of the target object's visual feature information. The following is a specific description of the application scenario where the target object is a person. The target object acquisition angle can be the person's front face or the person's side face. When the image to be archived is the person's front face, the extracted visual feature information (face feature) has better quality than the side face, so the subsequent clustering effect is better. In terms of clarity information, the higher the clarity of the face, the better the subsequent clustering effect.

[0070] In one embodiment, the auxiliary feature information may also include spatiotemporal sequence feature information, which characterizes the activity pattern of the corresponding target object in the target scene. The spatiotemporal sequence feature information is associated with the target object, and the spatiotemporal sequence feature information is obtained based on the summary of the activity pattern of the target object in all the images to be archived. The specific role of the spatiotemporal sequence feature information is to complete the visual data that is missing due to insufficient visual feature information of the target object, thereby completing the clustering of these visual data. For example, in the case of a person, the image to be archived cannot be correctly clustered due to the angle problem of the feature information of the face or body therein, then, the spatiotemporal sequence feature information associated with the target object can solve this type of visual data missing due to the angle. For example, from the spatiotemporal sequence feature information, it can be known that the target object A often appears in the acquisition device 1 at 8 am. If the visual features of the target object in a certain image to be archived taken by the acquisition device 1 at 8 am are unclear, it can be judged that the probability that the target object is A is relatively large, thereby achieving the role of completing the visual data.

[0071] Exemplarily, when the auxiliary feature information includes spatiotemporal sequence feature information, acquiring the auxiliary feature information corresponding to each target object from the plurality of images to be archived may include:

[0072] Pre-clustering all target objects according to the visual feature information corresponding to each target object to obtain a set of Y target objects;

[0073] If all target objects can be clustered into Y target object sets, then the spatiotemporal feature information of the image to be archived corresponding to the target object in each target object set is obtained, and then the spatiotemporal sequence feature information corresponding to each target object set is calculated, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived;

[0074] The spatiotemporal sequence feature information corresponding to each target object set is used as the spatiotemporal sequence feature information of each target object in the target object set.

[0075] In the embodiment of the present application, Y target object sets are obtained by pre-clustering, wherein all target objects included in each target object set are the same target object individual according to the pre-clustering standard. If a to-be-archived image includes multiple target objects, different target objects belong to different target object sets. For example, in an application scenario where the target object is a person, if Y target object sets are obtained by pre-clustering, it can be considered that after pre-clustering, it is preliminarily determined that all to-be-archived images include Y different people.

[0076] In the embodiment of the present application, the calculation to obtain the spatiotemporal sequence feature information corresponding to each target object set specifically includes:

[0077] Using the spatiotemporal feature information of the image to be archived corresponding to the target object in each target object set as the spatiotemporal feature information of the corresponding target object;

[0078] Sort all target objects in each target object set according to the acquisition time to obtain a sorted target object set corresponding to the target object set;

[0079] The acquisition time series and the acquisition device number series corresponding to each of the sorted target object sets are obtained, and the acquisition time series and the acquisition device number series are used as the spatiotemporal sequence feature information of the corresponding target object set.

[0080] In the embodiment of the present application, in the same target object set, the spatiotemporal sequence feature information corresponding to all target objects is the same. The spatiotemporal sequence feature information is used to characterize the target object individuals corresponding to the target object set appearing in different acquisition devices according to time.

[0081] In one embodiment, the step of acquiring auxiliary feature information corresponding to each target object from the plurality of images to be archived further includes:

[0082] If there is a first target object that does not belong to the set of Y target objects among all the target objects, then the visual feature information corresponding to the first target object is compared with the visual feature information corresponding to each target object belonging to the set of Y target objects for similarity, and the top K target objects with the highest similarity are obtained;

[0083] Adding the first target object to a candidate target object set, and then updating Y target object sets to obtain Y updated target object sets, wherein the candidate target object set is a target object set that includes most of the first K target objects in the Y image sets;

[0084] Acquire the spatiotemporal feature information of the image to be archived corresponding to the target object in each updated target object set, and then calculate the spatiotemporal sequence feature information corresponding to each updated target object set, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived;

[0085] The spatiotemporal sequence feature information of each updated target object set is used as the spatiotemporal sequence feature information of all target objects in the updated target object set.

[0086] In the embodiment of the present application, the method for calculating the spatiotemporal sequence feature information corresponding to each updated target object set is basically the same as the method for calculating the spatiotemporal sequence feature information corresponding to each target object set, and will not be described in detail here.

[0087] For ease of understanding, the following describes a method for obtaining spatiotemporal sequence feature information in an application scenario where the target object is a person.

[0088] Assume that there are 1,000 images to be archived, and the 1,000 archived images come from 10 acquisition devices, and the target object is the person in the image. In the process of forming an electronic file for one person from the 1,000 images to be archived, target object recognition (for example, face recognition) is performed on each image to be archived in step S202, and visual feature information (for example, face features) of all target objects in the 1,000 images to be archived is obtained; in step S203, auxiliary feature information corresponding to each target object is obtained, and the background feature information and device attribute information in the auxiliary feature information have been described in detail in the previous text, and will not be repeated here. The following is an example of the method of obtaining spatiotemporal sequence feature information in the auxiliary feature information in step S203:

[0089] The first case is: assuming that there are 1500 target objects in total from 1000 images to be archived, all target objects are pre-clustered according to the visual feature information of the 1500 target objects corresponding to the 1000 images to be archived, assuming that 20 target object sets are obtained. If all target objects can be clustered into 20 target object sets, 20 image sets correspond to 20 target object individuals. In this case, the 1000 images to be archived are roughly about 20 different target object individuals. Assuming that target object A corresponds to target object set B1, target object set B1 contains 60 target objects, each of which corresponds to an image to be archived; the 60 target objects in target object set B1 are sorted according to the time of the corresponding image to be archived, and then the acquisition time and acquisition device number of the image to be archived corresponding to the sorted 60 target objects are extracted according to the sorting, and the acquisition time series and acquisition device number series corresponding to the target object set B1 are obtained; the spatiotemporal sequence feature information of each target object in target object set B1 is the obtained acquisition time series and acquisition device number series. The above operation is performed once for each target object set to obtain the spatiotemporal sequence feature information corresponding to all target objects.

[0090] Exemplarily, the collection time may be divided into 24 hours a day, that is, the collection time may be one of 24 values. In this embodiment, there are 10 collection device numbers in total.

[0091] The second case is: assuming that 1500 target objects are obtained from 1000 images to be archived, all target objects are pre-clustered according to the visual feature information of the 1500 target objects corresponding to the 1000 images to be archived, and assuming that 20 target object sets are obtained, the 20 target object sets correspond to 20 target object individuals. If a target object X cannot be clustered into the 20 target object sets. In this case, the target object X needs to be processed separately, and the specific processing method includes: comparing the visual feature information corresponding to the target object X with the visual feature information corresponding to the 1499 target objects for similarity, selecting the top 100 target objects with the highest similarity, and if the target object set B2 includes the most of the 100 target objects in the 20 target object sets, then the target object set B2 is the candidate target object set, and the target object X is added to the candidate target object set B2; thereby, the 20 target object sets are updated to obtain 20 updated target object sets. For the 20 updated target object sets, the same operation method as that for the target object set B1 in the first case is adopted to obtain the spatiotemporal sequence feature information corresponding to all target objects.

[0092] In the embodiment of the present application, the specific method for obtaining the spatiotemporal sequence feature information can process the target object X that cannot be pre-clustered. If the conventional clustering method for forming an electronic archive is used, in the second case, the target object X may be randomly clustered into a target object set or the target object X may be abandoned. Therefore, based on the method in the embodiment of the present application, the target object X can be scientifically pre-clustered to avoid arbitrary classification or abandonment of the target object X. Therefore, compared with the conventional clustering method, the method in the present application can obtain a more accurate electronic archive in the subsequent process.

[0093] Step S204: clustering all the target objects according to the visual feature information and the auxiliary feature information corresponding to each of the target objects to obtain a plurality of electronic files.

[0094] In the embodiment, each electronic file corresponds to a specific target object individual, that is, the electronic files are archived according to the target object. Each electronic file includes at least one image to be archived. All images to be archived in each electronic file contain the same specific target object individual, which can be applied to one file for one person or one file for one vehicle in engineering business, etc.

[0095] In an embodiment of the present application, all target objects are clustered according to the visual feature information and auxiliary feature information of the target objects, and the time information or spatial information of the target objects is introduced into the archiving process by introducing the auxiliary feature information, thereby effectively supplementing the visual feature information of the target objects. In particular, for visual data with insufficient visual feature information of the target objects or visual data of low data quality, the accuracy of archiving can be effectively improved to meet the business requirements of actual engineering applications.

[0096] Exemplarily, the specific implementation method of step S204 may be: inputting the visual feature information and the auxiliary feature information corresponding to all the target objects into a pre-trained archiving model for processing to obtain similarity results between the target objects; clustering all the target objects according to the similarity results to obtain multiple electronic archives corresponding to the individual target objects.

[0097] In order to enhance the clustering effect of the archiving model, it is preferred that the source of the training samples of the archiving model is the same or similar to the source of the images to be archived. The same source means that the acquisition devices corresponding to the training samples and the acquisition devices corresponding to the images to be archived are from the same batch; similar sources mean that the acquisition devices corresponding to the training samples and the acquisition devices corresponding to the images to be archived have similar setting scenes and similar topological structures.

[0098] In one embodiment, the expression of the archiving model is shown in formula (1):

[0099] D=f1(A1)+f2(A2)+f3(A3)+f4(A4) (1)

[0100] Where D is the archive matrix with a dimension of M×M, M is the number of all target objects corresponding to the image to be archived; A1 represents the visual feature information set of all target objects, A2 represents the background feature information set of all target objects, A3 represents the spatiotemporal sequence feature information set of all target objects, and A4 represents the device attribute information set of all target objects. Functions f1, f2, f3, and f4 transform the corresponding sets into matrices respectively to obtain the component matrices corresponding to the sets that represent the similarity between all target objects in each pair, and the four component matrices are added together to obtain the final archive matrix.

[0101] If the transformation function is linear, the expression of the archiving model is as shown in formula (2):

[0102] D=W1A1+W2A2+W3A3+W4A4 (2)

[0103] In formula (2), the dimension of D is M×M, where M is the number of all target objects corresponding to the image to be archived; the dimension of W1 is M×P1, and the dimension of A1 is P1×M; the dimension of W2 is M×P2, and the dimension of A2 is P2×M; the dimension of W3 is M×P3, and the dimension of A3 is P3×M; the dimension of W4 is M×P4, and the dimension of A4 is P4×M. Among them, P1, P2, P3 and P4 are the dimensions of the data corresponding to A1, A2, A3 and A4 respectively.

[0104] Exemplarily, the archive matrix obtained by the model is a square matrix, and the dimension M of the archive matrix is ​​equal to the number of all target objects corresponding to the images to be archived. Generally speaking, the number of all target objects is greater than or equal to the number of images to be archived.

[0105] For ease of understanding, the following uses parameter W1 and the visual feature information set A1 of all target objects as an example to illustrate the data structure and archive matrix, where α1 is the weight coefficient, is the transposed matrix of matrix A1; each column of A1 corresponds to the visual feature information of a target object, Each row of corresponds to the visual feature information of a target object. For example, the first column of A1 represents the visual feature information corresponding to the first target object. The second row of represents the visual feature information corresponding to the second target object, and the first column of A1 is multiplied by The second row after the second row is the similarity result related to the visual feature information between the first target object and the second target object. Similarly, the similarity results related to the visual feature information between any two target objects in all target objects are obtained to obtain the matrix W1A1. The calculation methods of matrices W2A2, W3A3, and W4A4 are similar. Finally, the archive matrix D is obtained by matrix addition.

[0106] In the embodiment of the present application, W1-W4 are the archiving model parameters of the linear transformation. The parameters of the archiving model can be obtained by training the deep learning network. The deep learning network can be a convolutional neural network CNN, a residual network ResNet, a graph convolutional neural network GCN, a Transformer, etc.

[0107] In the embodiment, if the training sample data is labeled data, a supervised method is directly used to train the deep learning network. For example, the training samples are several groups of sample images, and each group of sample images is a training sample. The electronic file of each group of sample images about the target object is known, that is, the similarity results of each pair of sample images in all sample images in each group of sample images are known; first, the visual feature information and auxiliary feature information of each sample image in each group of sample images (for specific methods, see the method for extracting the visual feature information and auxiliary feature information of the target object in the method for generating electronic files in the embodiment of the present application); the several groups are divided into training sets and test sets, and the training sets and test sets respectively include multiple groups of sample images; the training method of the deep learning network includes:

[0108] Use the samples in the training set to iteratively train the initial deep learning network model to minimize the loss function and obtain the corresponding model parameters;

[0109] The accuracy of the model is evaluated on the training set and the test set respectively. If the evaluation results meet the preset conditions, the training of the deep learning network is completed.

[0110] In the embodiment, if the training samples are unlabeled data, high-quality partial electronic files can be first obtained by feature search similarity, and then the high-quality partial electronic files can be used as training samples to perform supervised training on the deep learning network. The specific method is the same as the previous article and will not be repeated here.

[0111] For example, in the archive matrix D, each calculated value is not necessarily 0 or 1, but a value close to 1 or 0. In the final processing, the values ​​in the archive matrix can be binary classified. For example, 0.5 can be taken as the threshold. When the value in the archive matrix D is greater than 0.5, it is 1, and when it is less than or equal to 0.5, it is 0. It can be understood that the value in the archive matrix is ​​0 or 1, and 1 represents that the picture with the row ID and the picture with the column ID belong to the same electronic archive.

[0112] Exemplarily, as shown in formula (3), the archive matrix D0 indicates that the number of all target objects corresponding to the images to be archived that participate in the automatic generation of electronic archives is four, and the generation result is that the first target object and the second target object belong to the same target object individual, and the third target object and the fourth target object belong to the same target object individual. The image to be archived corresponding to the first target object and the image to be archived corresponding to the second target object belong to one electronic archive, and the image to be archived corresponding to the third target object and the image to be archived corresponding to the fourth target object belong to another electronic archive.

[0113]

[0114] The embodiment of the present application clusters all target objects to obtain electronic archives based on the visual feature information and auxiliary feature information of all target objects corresponding to multiple images to be archived, thereby avoiding the problem of low clustering accuracy caused by relying on the visual feature information of a single target object for clustering. At the same time, the auxiliary feature information represents the time or space information associated with the corresponding target object. Therefore, the time information or space information of the target object is introduced into the archiving process through the auxiliary features, which can effectively supplement the visual feature information of the target object. In particular, for visual data with insufficient visual features of the target object or visual data of low data quality, the accuracy of its archiving can be effectively improved, thereby meeting the business requirements in actual engineering applications. It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0115] Corresponding to the electronic archive generation method described in the above embodiment, Figure 3 A structural block diagram of an electronic archive generating device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0116] Reference Figure 3 , the electronic archive generating device 3 comprises:

[0117] An image acquisition unit 31 is used to acquire a plurality of images to be archived;

[0118] A first feature information acquisition unit 32 is used to identify a target object for each of the images to be archived, and obtain visual feature information of all target objects corresponding to the multiple images to be archived;

[0119] A second feature information acquisition unit 33 is used to acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object;

[0120] The electronic profile forming unit 34 is used to cluster all target objects according to the visual feature information and the auxiliary feature information corresponding to each target object to obtain a plurality of electronic profiles.

[0121] In one embodiment, the auxiliary feature information includes background feature information. When the second feature information acquisition unit 33 is used to acquire the auxiliary feature information corresponding to each target object from a plurality of images to be archived, it is specifically used to: perform difference processing on the image to be archived corresponding to each target object and the visual feature information of the target object to obtain the background feature information corresponding to each target object.

[0122] Optionally, the auxiliary feature information also includes acquisition device attribute information. When the second feature information acquisition unit 33 is used to acquire the auxiliary feature information corresponding to each target object from the plurality of images to be archived, it is also specifically used to acquire the acquisition device attribute information corresponding to each target object.

[0123] Optionally, the acquisition device attribute information may include target object density information, target object acquisition angle information and / or clarity information.

[0124] Optionally, when the second feature information acquisition unit 33 is used to acquire the acquisition device attribute information corresponding to each target object, it is specifically used to: count the target objects in the image to be archived corresponding to each target object to obtain the number of target objects, and the number of target objects is used as the target object density information corresponding to each of the target objects.

[0125] Optionally, when the second feature information acquisition unit 33 is used to acquire the acquisition device attribute information corresponding to each target object, it is also specifically used to: acquire multiple historical images, and the multiple historical images have the same acquisition conditions as the current image to be archived; input the multiple historical images into a pre-trained acquisition angle acquisition model for processing to obtain multiple historical acquisition angle information; and average the multiple historical acquisition angle information to obtain the target object acquisition angle information of the corresponding target object.

[0126] Optionally, when the second feature information acquisition unit 33 is used to obtain the acquisition device attribute information corresponding to each target object, it is specifically used to: obtain multiple historical images, and the multiple historical images have the same acquisition conditions as the current image to be archived; input the multiple historical images into a pre-trained clarity model for processing to obtain multiple historical clarity information; and average the multiple historical clarity information to obtain the clarity information.

[0127] Optionally, the auxiliary feature information also includes spatiotemporal sequence feature information, and the spatiotemporal sequence feature information represents the activity pattern of the corresponding target object in the target scene. The second feature information acquisition unit 33 is used to acquire the auxiliary feature information corresponding to each target object from the plurality of images to be archived; it is also specifically used to:

[0128] Pre-clustering all target objects according to the visual feature information corresponding to each target object to obtain a set of Y target objects;

[0129] If all target objects can be clustered into Y target object sets, then the spatiotemporal feature information of the image to be archived corresponding to the target object in each target object set is obtained, and then the spatiotemporal sequence feature information corresponding to each target object set is calculated, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived;

[0130] The spatiotemporal sequence feature information corresponding to each target object set is used as the spatiotemporal sequence feature information of each target object in the target object set.

[0131] Optionally, when the second feature information acquiring unit 33 is used to acquire the auxiliary feature information corresponding to each target object from the plurality of images to be archived, it is further specifically used to:

[0132] If there is a first target object that does not belong to the set of Y target objects among all the target objects, then the visual feature information corresponding to the first target object is compared with the visual feature information corresponding to each target object belonging to the set of Y target objects for similarity, and the top K target objects with the highest similarity are obtained;

[0133] Adding the first target object to a candidate target object set, and then updating Y target object sets to obtain Y updated target object sets, wherein the candidate target object set is a target object set that includes most of the first K target objects in the Y image sets;

[0134] Acquire the spatiotemporal feature information of the image to be archived corresponding to the target object in each updated target object set, and then calculate the spatiotemporal sequence feature information corresponding to each updated target object set, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived;

[0135] The spatiotemporal sequence feature information of each updated target object set is used as the spatiotemporal sequence feature information of all target objects in the updated image set.

[0136] Optionally, when the electronic archive forming unit 34 is used to cluster all the target objects according to the visual feature information and the auxiliary feature information corresponding to each of the target objects to obtain a plurality of electronic archives, it is also specifically used to: input the target object visual feature information and the auxiliary feature information into a pre-trained archiving model for processing to obtain a similarity result between the target objects; cluster all the target objects according to the similarity result to obtain a plurality of electronic archives.

[0137] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0138] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0139] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 3 As shown, the terminal device 4 of this embodiment includes: at least one processor 40 ( Figure 4 Only one is shown in the figure) a processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, and when the processor 40 executes the computer program 42, the steps in any of the above-mentioned electronic archive generation method embodiments are implemented.

[0140] The terminal device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4It is only an example of the terminal device 4 and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0141] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0142] In some embodiments, the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0143] An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program.

[0144] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0145] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the electronic archive generation device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0149] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for generating an electronic archive, characterized in that: include: Acquire multiple images to be archived; Identify the target object for each of the images to be archived, and obtain visual feature information of all the target objects corresponding to the multiple images to be archived; Acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object; Clustering all the target objects according to the visual feature information and the auxiliary feature information corresponding to each of the target objects to obtain a plurality of electronic files; The auxiliary feature information also includes spatiotemporal sequence feature information, which represents the activity pattern of the corresponding target object in the target scene; The step of acquiring auxiliary feature information corresponding to each target object from the plurality of images to be archived includes: Pre-clustering all target objects according to the visual feature information corresponding to each target object to obtain a set of Y target objects; If there is a first target object that does not belong to the set of Y target objects among all the target objects, then the visual feature information corresponding to the first target object is compared with the visual feature information corresponding to each target object belonging to the set of Y target objects for similarity, and the top K target objects with the highest similarity are obtained; Adding the first target object to a candidate target object set, and then updating Y target object sets to obtain Y updated target object sets, wherein the candidate target object set is a target object set that includes most of the first K target objects in the Y image sets; Acquire the spatiotemporal feature information of the image to be archived corresponding to the target object in each updated target object set, and then calculate the spatiotemporal sequence feature information corresponding to each updated target object set, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived; The spatiotemporal sequence feature information of each updated target object set is used as the spatiotemporal sequence feature information of all target objects in the updated image set.

2. The method according to claim 1, characterized in that The auxiliary feature information includes background feature information; The step of acquiring auxiliary feature information corresponding to each target object from the plurality of images to be archived includes: The image to be archived corresponding to each of the target objects and the visual feature information of the target object are subjected to difference processing to obtain background feature information corresponding to each of the target objects.

3. The method according to claim 1, characterized in that The auxiliary feature information also includes acquisition device attribute information.

4. The method according to claim 1, characterized in that The method further comprises: If all target objects can be clustered into Y target object sets, then the spatiotemporal feature information of the image to be archived corresponding to the target object in each target object set is obtained, and then the spatiotemporal sequence feature information corresponding to each target object set is calculated, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived; The spatiotemporal sequence feature information corresponding to each target object set is used as the spatiotemporal sequence feature information of each target object in the target object set.

5. The method according to any one of claims 1 to 4, characterized in that: The step of clustering all the target objects according to the visual feature information and the auxiliary feature information corresponding to each target object to obtain a plurality of electronic files includes: Inputting the visual feature information and the auxiliary feature information corresponding to all the target objects into a pre-trained archiving model for processing to obtain similarity results between the target objects; All the target objects are clustered according to the similarity results to obtain a plurality of electronic files.

6. An electronic archive generation device, characterized in that: include: An image acquisition unit, used for acquiring a plurality of images to be archived; A first feature information acquisition unit, configured to identify a target object for each of the images to be archived, and obtain visual feature information of all target objects corresponding to the multiple images to be archived; A second feature information acquisition unit, configured to acquire auxiliary feature information corresponding to each target object from the plurality of images to be archived, wherein the auxiliary feature information represents time or space information associated with the corresponding target object; An electronic profile forming unit, configured to cluster all the target objects according to the visual feature information and the auxiliary feature information corresponding to each of the target objects, to obtain a plurality of electronic profiles; The auxiliary feature information also includes spatiotemporal sequence feature information, which represents the activity pattern of the corresponding target object in the target scene; The second feature information acquisition unit is used to acquire the auxiliary feature information corresponding to each target object from the plurality of images to be archived, and specifically includes: Pre-clustering all target objects according to the visual feature information corresponding to each target object to obtain a set of Y target objects; If there is a first target object that does not belong to the set of Y target objects among all the target objects, then the visual feature information corresponding to the first target object is compared with the visual feature information corresponding to each target object belonging to the set of Y target objects for similarity, and the top K target objects with the highest similarity are obtained; Adding the first target object to a candidate target object set, and then updating Y target object sets to obtain Y updated target object sets, wherein the candidate target object set is a target object set that includes most of the first K target objects in the Y image sets; Acquire the spatiotemporal feature information of the image to be archived corresponding to the target object in each updated target object set, and then calculate the spatiotemporal sequence feature information corresponding to each updated target object set, wherein the spatiotemporal feature information is the acquisition time and acquisition device number of the image to be archived; The spatiotemporal sequence feature information of each updated target object set is used as the spatiotemporal sequence feature information of all target objects in the updated image set.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images