Method, device and equipment for constructing personnel profile and storage medium

Abstract

The application discloses a personnel archive construction method and device, equipment and a storage medium. The personnel archive construction method comprises the following steps: obtaining an image to be archived, wherein the image to be archived comprises a face image and a body image; extracting feature information of the image to be archived to obtain first feature information of the face image and second feature information of the body image; fusing the first feature information and the second feature information to obtain first fused features; and performing feature clustering on the first fused features to obtain a target personnel archive. The first feature information of the face image and the second feature information of the body image of the image to be archived are fused and clustered to construct the target personnel archive, so that the face image and the body image do not need to be respectively archived and then merged, the efficiency of personnel archive construction is improved, the feature information of the body image and the face image is fused, the feature information is enriched, and the accuracy of personnel archive construction is improved.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a method, apparatus, device and storage medium for constructing personnel files. Background Technology

[0002] With the development of the security industry and artificial intelligence, personnel file systems have gradually become a key deployment target.

[0003] Existing personnel profile systems combining facial and human body data in related technologies have the following two approaches: 1) For facial data, sequential similarity comparisons or direct clustering algorithms are used to obtain facial profile results. Then, the human body images associated with the facial profiles are used as the base database cover. All human body snapshots are sequentially compared and sorted with the human body base database cover for similarity. Finally, the snapshots are assigned to the profile corresponding to the human body cover that ranks first and has a similarity greater than a specified threshold. 2) Facial data and human body data are clustered separately to obtain their respective clustering results. Then, based on the association relationship, the facial profiles and human body profiles are merged to obtain the final personnel profile.

[0004] The above method for constructing personnel files involves clustering and fusing facial and human images separately, followed by a final merging. This process is complex and involves less information fusion, resulting in low efficiency and accuracy in constructing personnel files. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, device, and storage medium for constructing personnel files, aiming to solve the technical problem of low efficiency in personnel file construction in the prior art.

[0006] To achieve the above objectives, this application provides a method for constructing personnel files, the method comprising:

[0007] Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0008] Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0009] The first feature information and the second feature information are fused to obtain the first fused feature;

[0010] The first fused feature is subjected to feature clustering to obtain the target personnel file.

[0011] Optionally, the step of fusing the first feature information and the second feature information to obtain the first fused feature includes:

[0012] The first feature information and the second feature information are classified to obtain feature sets of different types;

[0013] The feature sets from different types are fused to obtain the first fused feature.

[0014] Optionally, after the step of classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes:

[0015] The feature sets of each type are fused using single features of different dimensions to obtain the second fused feature of each type of feature set;

[0016] The step of fusing the feature sets of different types to obtain the first fused feature includes:

[0017] The second fusion feature is fused between the feature sets of different types to obtain the first fusion feature.

[0018] Optionally, the step of performing feature fusion on the feature sets of each type using different dimensions of a single feature to obtain the second fused feature of the feature sets of each type includes:

[0019] The feature sets of each type are divided according to the feature dimensions to obtain a preset first number of dimensional blocks for each type of feature set;

[0020] By fusing features between different dimensional blocks, a second fused feature for each type of feature set is obtained.

[0021] Optionally, after the step of classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes:

[0022] The same type of features are fused in the feature set to obtain a third fused feature;

[0023] The step of fusing the feature sets of different types to obtain the first fused feature includes:

[0024] The third fusion feature is fused with the feature sets of other different types to obtain the first fusion feature.

[0025] Optionally, the step of fusing similar features in the feature set to obtain a third fused feature includes:

[0026] The second feature information includes a preset second number of oriented human body features, wherein the second number is greater than or equal to 2;

[0027] In the feature set, human features with different orientations are fused to obtain a third fused feature.

[0028] Optionally, the step of fusing the first feature information and the second feature information to obtain the first fused feature includes:

[0029] Obtain the spatiotemporal features of the image to be archived;

[0030] The first feature information, the second feature information, and the spatiotemporal feature are fused to obtain the first fused feature.

[0031] This application also provides a personnel file construction apparatus, the personnel file construction apparatus comprising:

[0032] An acquisition module is used to acquire images to be archived, wherein the images to be archived include face images and human body images;

[0033] The extraction module is used to extract feature information from the image to be archived, and obtain the first feature information of the face image and the second feature information of the human body image;

[0034] The fusion module is used to fuse the first feature information and the second feature information to obtain a first fused feature;

[0035] The clustering module is used to perform feature clustering on the first fused features to obtain the target personnel file.

[0036] Optionally, the fusion module includes:

[0037] A classification module is used to classify the first feature information and the second feature information to obtain feature sets of different types; a first feature fusion module is used to fuse the feature sets of different types to obtain a first fused feature.

[0038] And / or, the fusion module further includes: a single feature fusion module, used to perform single feature fusion of different dimensions on the feature sets of each type to obtain a second fused feature of the feature sets of each type; and a second feature fusion module, used to fuse the second fused features between the feature sets of different types to obtain a first fused feature;

[0039] And / or, the single feature fusion module includes: a dimension partitioning module, used to partition the feature sets of each type according to the feature dimensions to obtain a preset first number of dimension blocks of the feature sets of each type; and a dimension block feature fusion module, used to fuse features between different dimension blocks to obtain a second fused feature of the feature sets of each type.

[0040] And / or, the fusion module further includes: a same-type feature fusion module, used to perform same-type feature fusion in the feature set to obtain a third fused feature; and a third feature fusion module, used to perform feature fusion with other feature sets of different types to obtain a first fused feature;

[0041] And / or, the same type feature fusion module includes: a human body feature fusion module with different orientations, used to fuse human body features with different orientations in the feature set to obtain a third fused feature;

[0042] And / or, the fusion module further includes: a spatiotemporal feature acquisition module, used to acquire the spatiotemporal features of the image to be archived; and an information fusion module, used to fuse the first feature information, the second feature information and the spatiotemporal features to obtain a first fused feature.

[0043] This application also provides a personnel file construction device, which includes: a memory, a processor, and a program stored in the memory for implementing the personnel file construction method.

[0044] The memory is used to store the program that implements the method for constructing personnel files;

[0045] The processor is used to execute a program that implements the method for constructing the personnel file, so as to implement the steps of the method for constructing the personnel file.

[0046] This application also provides a storage medium storing a program for implementing a method for constructing personnel files, wherein the program for implementing the method for constructing personnel files is executed by a processor to implement the steps of the method for constructing personnel files.

[0047] This application provides a method, apparatus, device, and storage medium for constructing personnel files. Compared with related technologies, which involve complex processes and limited information fusion, resulting in low efficiency and accuracy in personnel file construction, this application involves: acquiring images to be archived, including face images and body images; extracting feature information from the images to be archived to obtain first feature information of the face image and second feature information of the body image; fusing the first feature information and the second feature information to obtain a first fused feature; and performing feature clustering on the first fused feature to obtain the target personnel file. In other words, this application constructs the target personnel file by fusing and clustering the first feature information of the face image and the second feature information of the body image, eliminating the need to separately archive and merge face and body images, thus improving the efficiency of personnel file construction. Furthermore, this application integrates feature information from both body and face images, enriching the feature information and thereby improving the accuracy of personnel file construction. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0049] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application;

[0050] Figure 2 This is a flowchart illustrating the first embodiment of the method for constructing personnel files according to this application;

[0051] Figure 3 This is a schematic diagram of the modules of the device for constructing personnel files for this application;

[0052] Figure 4 This is a flowchart illustrating the fourth embodiment of the method for constructing personnel files in this application;

[0053] Figure 5 This is a schematic diagram of a multi-source information fusion network for the fourth embodiment of the method for constructing personnel files in this application.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0056] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of this application.

[0057] The terminal in this application embodiment can be a PC, or a smartphone, tablet computer, e-book reader, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, portable computer, or other portable terminal devices with display functions.

[0058] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0059] Optionally, the terminal may also include a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, and so on. Sensors may include light sensors, motion sensors, and other sensors. Specifically, light sensors may include ambient light sensors and proximity sensors. The ambient light sensor can adjust the display brightness according to the ambient light level, while the proximity sensor can turn off the display and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, a gravity accelerometer can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that identify the mobile terminal's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition functions (such as pedometers, taps), etc. Of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which will not be elaborated here.

[0060] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating device, a network communication module, a user interface module, and a program for constructing personnel files.

[0062] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the client (user terminal) and communicate with the client; and the processor 1001 can be used to call the personnel file construction program stored in the memory 1005.

[0063] Reference Figure 2 This application provides a method for constructing personnel files, the method comprising:

[0064] Step S100: Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0065] Step S200: Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0066] Step S300: The first feature information and the second feature information are fused to obtain the first fused feature;

[0067] Step S400: Perform feature clustering on the first fused features to obtain the target personnel file.

[0068] In this embodiment, the application scenario is:

[0069] As an example, the scenario for constructing personnel files can be that enterprises need to build personnel files to facilitate personnel management. Existing personnel file systems combining facial and human images have the following two solutions: 1) For facial data, sequential similarity comparisons or direct clustering algorithms are used to obtain facial file results. Then, the human images associated with the facial files are used as the base database cover. All human images are sequentially compared and sorted with the human base database cover for similarity, and finally, the images are assigned to the file corresponding to the human cover with the highest similarity score and a specified threshold. 2) Facial and human data are clustered separately to obtain their respective clustering results. Then, based on the association relationships, the facial and human files are merged to obtain the final personnel file. However, the above methods of constructing personnel files involve separate clustering and information fusion of facial and human images, followed by a merging process. This process is complex and involves limited information fusion, resulting in low efficiency and accuracy in personnel file construction. In response to this scenario, the personnel file construction method of this embodiment fuses the first feature information of the face image and the second feature information of the human body image of the image to be archived and performs feature clustering to construct the target personnel file. This eliminates the need to separately archive and merge the face image and the human body image, thus improving the efficiency of personnel file construction. Furthermore, this application integrates the feature information of the human body image and the face image, enriching the feature information and thereby improving the accuracy of personnel file construction.

[0070] As an example, the application scenarios for building personnel files are not limited to the aforementioned enterprise personnel management, but also include various other personnel file construction scenarios, without specific limitations here.

[0071] This embodiment aims to improve the efficiency and accuracy of personnel file creation.

[0072] In this embodiment, the method for constructing personnel files is applied to the personnel file construction apparatus.

[0073] The specific steps are as follows:

[0074] Step S100: Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0075] In this embodiment, the image to be archived is a human image captured by a shooting device (such as a surveillance camera), including facial images and human body images, which can be an image of one person or multiple people.

[0076] In this embodiment, the device can acquire the image to be archived by receiving the image to be archived sent by a preset surveillance camera or by receiving the image to be archived uploaded by the user; no specific limitation is made here.

[0077] Step S200: Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0078] In this embodiment, the feature information of the image to be archived includes face information, human body information, and spatiotemporal information, etc., wherein face information (first feature information) includes but is not limited to face features, face score, face attributes, and face key points; human body information (second feature information) includes but is not limited to human body features, human body score, human body attributes, and human body key points; and spatiotemporal information includes but is not limited to the latitude and longitude information of the person and time information.

[0079] In this embodiment, the method by which the device extracts feature information from the image to be archived includes: the device performs face and body structuring on the image to be archived, and generates a sequence of face and body location points and corresponding modeling features through target detection; the device obtains face and body modeling, scoring reasoning, attribute reasoning and key point reasoning based on face and body respectively.

[0080] Before step S300, which involves fusing the first feature information and the second feature information to obtain the first fused feature, the following step A100 is further included:

[0081] Step A100: Associating face images and body images in the image to be archived. Specifically, the device generates an association ID using a face-body association algorithm, binding the face and body of the same person together. That is, in application scenarios where the image to be archived contains face and body images of multiple people, face-body association is required to bind the face and body images of the same person under one association ID. For example, if the image to be archived contains face image X1, body image Y1, and body image Y2 of user A, and face image X2 and body image Y3 of user B, then the device binds the face and body of the same person together according to the face-body association algorithm, obtaining association ID Q1 for user A and association ID Q2 for user B. Q1 includes face image X1, body image Y1, and body image Y2 of user A; Q2 includes face image X2 and body image Y3 of user B.

[0082] Step S300: The first feature information and the second feature information are fused to obtain the first fused feature;

[0083] In this embodiment, the device fuses the first feature information and the second feature information to obtain a first fused feature. The information fusion method includes, but is not limited to, stacking, weighting, fully connected network, and Transformer. Specifically, it fuses different types of features in the first feature information and the second feature information. For example, the first feature information includes face features and face scoring information, and the second feature information includes human body features and human body scoring information. The device fuses the face features, face scoring information, human body features, and human body scoring information using a fully connected network to obtain the first fused feature.

[0084] Specifically, step S300 includes the following steps S310-S320:

[0085] Step S310: Classify the first feature information and the second feature information to obtain feature sets of different types;

[0086] In this embodiment, the device classifies the first feature information and the second feature information to obtain feature sets of different types. Each type of feature set includes multi-dimensional features of that type. For example, the first feature information and the second feature information include face feature type, human body feature type and other feature types (such as face score, face attribute, face key points, human body score, human body attribute, human body key points). The feature set of the face feature type contains 512-dimensional face features.

[0087] Step S320: The feature sets of different types are fused to obtain the first fused feature.

[0088] In this embodiment, the device fuses the feature sets of different types to obtain a first fused feature. For example, the device fuses the feature sets of face feature type and the feature sets of human body feature type using a fully connected network to obtain the first fused feature.

[0089] In this embodiment, step S300 further includes the following steps B100-B200:

[0090] Step B100: Obtain the spatiotemporal features of the image to be archived;

[0091] Step B100: The first feature information, the second feature information, and the spatiotemporal feature are fused to obtain the first fused feature.

[0092] In this embodiment, feature fusion is performed on all information of the face and body, as well as spatiotemporal information, which can effectively complement the face and body information and greatly improve the clustering accuracy.

[0093] Step S400: Perform feature clustering on the first fused features to obtain the target personnel file.

[0094] In this embodiment, the device performs feature clustering on the first fused features to obtain the target personnel file. Specifically, the device uses a clustering algorithm to cluster all fused features, and the obtained clustering result serves as the final target personnel file. The clustering algorithm includes traditional clustering algorithms and deep convolutional clustering algorithms. Traditional clustering algorithms include, but are not limited to, DBSCAN, K-means, spectral clustering, hierarchical clustering, and Infomap; deep clustering algorithms include, but are not limited to, deep convolutional clustering networks, such as DNC (Deep Nonparametric Clustering), DEC (Deep Embedded Clustering), and CCNN (Clustering Convolutional Neural Network); deep graph convolutional clustering networks, such as DAEGC (Deep Attentional Embedded Graph Clustering), LGCN (Linkage Graph Convolution Network), and DA-NET (Density Aware Feature Embedding Network); and deep Transformer clustering networks, such as Clusformer (A Transformer-based Clustering Approach) and FaceT (Learning to Cluster Faces via Transformer).

[0095] This application provides a method for constructing personnel files. Compared with related technologies, which involve complex processes and limited information fusion, resulting in low efficiency and accuracy in personnel file construction, this application involves: acquiring images to be archived, including face images and body images; extracting feature information from the images to be archived to obtain first feature information of the face image and second feature information of the body image; fusing the first feature information and the second feature information to obtain a first fused feature; and performing feature clustering on the first fused feature to obtain the target personnel file. In other words, this application constructs the target personnel file by fusing and clustering the first feature information of the face image and the second feature information of the body image, eliminating the need to separately archive and merge face and body images, thus improving the efficiency of personnel file construction. Furthermore, this application integrates the feature information of body and face images, enriching the feature information and thereby improving the accuracy of personnel file construction.

[0096] Based on the first embodiment described above, this application also provides another embodiment, wherein the method for constructing the personnel file includes the following steps C100-C600:

[0097] Step C100: Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0098] Step C200: Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0099] Step C300: Classify the first feature information and the second feature information to obtain feature sets of different types;

[0100] In this embodiment, steps C100-C300 refer to steps S100-S300 described above, and will not be repeated here.

[0101] Step C400: Perform feature fusion of different dimensions of single features on the feature sets of each type to obtain the second fused feature of the feature sets of each type;

[0102] In this embodiment, the device performs single-feature feature fusion of different dimensions on the feature sets of each type to obtain the second fused feature of each type of feature set. The single-feature feature fusion of different dimensions is to further fuse the feature sets of each type according to different dimensions, thereby improving the richness of the fused feature.

[0103] Specifically, step C400 includes the following steps C410-C420:

[0104] Step C410: Divide the feature sets of each type according to the feature dimensions to obtain a preset first number of dimensional blocks for each type of feature set;

[0105] In this embodiment, the device divides the feature sets of each type according to the feature dimensions to obtain a preset first number of dimensional blocks for each type of feature set. For example, in the feature set of the face feature type, 512 dimensions are extracted by the feature extractor and divided into 4 blocks. The feature dimensions and the number of blocks can be set.

[0106] Step C420: Feature fusion is performed between the different dimensional blocks to obtain the second fused feature of each type of feature set.

[0107] In this embodiment, the device performs feature fusion between different dimensional blocks to obtain a second fused feature of each type of feature set. For example, each dimensional block is fused with other dimensional blocks through a Transformer network; for example, each dimensional block is randomly fused with other dimensional blocks (such as the first block and the fourth block) through an Mlp network.

[0108] Step C500: The second fusion features between the feature sets of different types are fused to obtain the first fusion feature;

[0109] In this embodiment, the device fuses the second fused features between different types of feature sets to obtain a first fused feature.

[0110] Step C600: Perform feature clustering on the first fused features to obtain the target personnel file.

[0111] In this embodiment, based on the fusion of different types of features, feature sets of various types are added to perform feature fusion of single features in different dimensions, which further improves the richness of the fused features, thereby improving the accuracy of personnel file construction.

[0112] Based on the first and second embodiments described above, this application also provides another embodiment, wherein the method for constructing the personnel file includes the following steps D100-D600:

[0113] Step D100: Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0114] Step D200: Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0115] Step D300: Classify the first feature information and the second feature information to obtain feature sets of different types;

[0116] In this embodiment, steps D100-D300 refer to steps S100-S300 as described above, and will not be repeated here.

[0117] Step D400: Perform feature fusion of the same type in the feature set to obtain a third fused feature;

[0118] In this embodiment, the device performs same-type feature fusion in the feature set to obtain a third fused feature. The same-type feature fusion is a further feature fusion of the same type of feature set in the multi-dimensional feature set. For example, human body features include human body front features and human body back features. Human body front features and human body back features belong to the same type of features. Therefore, human body front features and human body back features are further fused.

[0119] Specifically, step D400 includes the following step D410:

[0120] The second feature information includes a preset second number of oriented human body features, wherein the second number is greater than or equal to 2;

[0121] Step D410: In the feature set, feature fusion is performed between human body features of different orientations to obtain a third fused feature;

[0122] In this embodiment, the device fuses human features with different orientations in the feature set to obtain a third fused feature. This application adds multi-angle options (such as front, back, left, and right) to the human features, meaning one face image corresponds to one frontal human image or multiple orientation human images, making the fused features richer. For example, orientation includes front, side, and back, and the 512-dimensional features of the frontal human feature are fused with the 512-dimensional features of the side and back orientations through a Transformer network; or, for example, orientation includes front and side, and the 512-dimensional features of the frontal human feature are randomly fused with the 512-dimensional features of the side orientation through an Mlp network.

[0123] Step D500: The third fusion feature is fused with the feature sets of other different types to obtain the first fusion feature;

[0124] In this embodiment, the device fuses the third fusion feature with other feature sets of different types to obtain a first fusion feature. For example, the feature sets of different types include a face feature set, a human body front feature set, and a human body back feature set. The human body front feature set and the human body back feature set are first fused with the same type of features to obtain a third fusion feature. The third fusion feature is then fused with the features of the face feature set to obtain the first fusion feature.

[0125] Step D600: Perform feature clustering on the first fused features to obtain the target personnel file.

[0126] In this embodiment, multi-angle options are added to human body features, and feature sets of various types are added to fuse features of the same type on the basis of different types of feature fusion, which further improves the richness of fused features and thus improves the accuracy of personnel profile construction.

[0127] Based on the first, second, and third embodiments described above, this application also provides another embodiment, referred to... Figure 4 The method for constructing the personnel files includes the following steps E100-E500:

[0128] Step E100: Face and body structuring, generating image sequences of face and body location points and corresponding modeling features through object detection;

[0129] In step E200, the face and body are modeled, scored, inferred, attributed, and key pointed respectively to obtain the modeling, scoring, attribute, and key point information of the face and body.

[0130] Step E300, face and body association: Through the face and body association algorithm, an association ID is generated to bind the face and body of the same person.

[0131] Step E400, multi-source information fusion clustering, obtains the modeling features, scores, attributes, key point information, and spatiotemporal information (latitude and longitude information, time information, etc.) of faces and bodies with associated IDs and fuses them together (fusion methods include but are not limited to stacking, weighting, fully connected networks, Transformer, etc.) to obtain a brand new feature (if there are missing items, such as single face or single body, the missing items are filled with 0), and then a clustering algorithm is used to cluster all the fused features.

[0132] In this embodiment, refer to Figure 5The entire multi-dimensional information fusion process consists of three steps: 1) Fusion of single features across different dimensions, such as single facial features. A feature extractor extracts 512 dimensions and divides them into four blocks (feature dimensions and the number of blocks are variable). In Specific Implementation 1, each block is fused with all other blocks via a Transformer network. In Specific Implementation 2, each block is randomly fused with features from another block (e.g., the first and fourth blocks) via an MLP network; 2) Fusion of similar features, such as the fusion of features from different orientations of the human body (front, back, and side). In Specific Implementation 1, the 512-dimensional features of each orientation are fused with the 512-dimensional features of other orientations via a Transformer network. In specific embodiment 2, the 512-dimensional features of each orientation are randomly fused with other 512-dimensional features of a certain orientation (such as front and side) through the Mlp network; 3) Fusion of different types of features, such as face fusion features, fusion features of different orientations of the human body and other fusion features of face and human body (scores, key points, spatiotemporal, etc.) are finally fused. In specific embodiment 1, the 512-dimensional features of each type are fused with other 512-dimensional features of other types through the Transformer network.

[0133] After fusing multiple information sources, we obtain a new and richer feature. Then, we use a clustering algorithm to cluster all the fused features, and the clustering result serves as the final personnel file.

[0134] Step D500, Personnel Files: Each cluster in the clustering results is used as a file, with faces and bodies sorted according to scores, and the top K face and body images are displayed as the cover.

[0135] This application proposes a novel multi-source information fusion clustering method that integrates all information about faces and bodies, as well as spatiotemporal information, for clustering. This method effectively complements face and body information, greatly improving clustering accuracy. It eliminates the need to separately archive and merge face and body images, thus improving the efficiency of personnel file creation and enhancing the accuracy of personnel files.

[0136] This application also provides a device for constructing personnel files, referring to... Figure 3 The personnel file construction device includes:

[0137] The acquisition module 10 is used to acquire images to be archived, wherein the images to be archived include face images and human body images;

[0138] Extraction module 20 is used to extract feature information of the image to be archived, and obtain the first feature information of the face image and the second feature information of the human body image;

[0139] The fusion module 30 is used to fuse the first feature information and the second feature information to obtain a first fused feature;

[0140] Clustering module 40 is used to perform feature clustering on the first fused features to obtain target personnel files.

[0141] Optionally, the fusion module 30 includes:

[0142] A classification module is used to classify the first feature information and the second feature information to obtain feature sets of different types; a first feature fusion module is used to fuse the feature sets of different types to obtain a first fused feature.

[0143] And / or, the fusion module 30 further includes: a single feature fusion module, used to perform single feature fusion of different dimensions on the feature sets of each type to obtain a second fused feature of the feature sets of each type; and a second feature fusion module, used to fuse the second fused features between the feature sets of different types to obtain a first fused feature;

[0144] And / or, the single feature fusion module includes: a dimension partitioning module, used to partition the feature sets of each type according to the feature dimensions to obtain a preset first number of dimension blocks of the feature sets of each type; and a dimension block feature fusion module, used to fuse features between different dimension blocks to obtain a second fused feature of the feature sets of each type.

[0145] And / or, the fusion module 30 further includes: a same-type feature fusion module, used to perform same-type feature fusion in the feature set to obtain a third fused feature; and a third feature fusion module, used to perform feature fusion of the third fused feature with other feature sets of different types to obtain a first fused feature;

[0146] And / or, the same type feature fusion module includes: a human body feature fusion module with different orientations, used to fuse human body features with different orientations in the feature set to obtain a third fused feature;

[0147] And / or, the fusion module 30 further includes: a spatiotemporal feature acquisition module, used to acquire the spatiotemporal features of the image to be archived; and an information fusion module, used to fuse the first feature information, the second feature information and the spatiotemporal features to obtain a first fused feature.

[0148] The specific implementation method of the personnel file construction device in this application is basically the same as the various embodiments of the personnel file construction method described above, and will not be repeated here.

[0149] Reference Figure 1 , Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of this application.

[0150] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0151] Optionally, the personnel profile building device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard. Optionally, the rectangular user interface may also include a standard wired interface or a wireless interface. The network interface may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0152] Those skilled in the art will understand that Figure 1 The personnel file construction equipment structure shown does not constitute a limitation on the personnel file construction equipment, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0153] like Figure 1 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a personnel file creation program. The operating system is a program that manages and controls the hardware and software resources of the personnel file creation device, supporting the operation of the personnel file creation program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the personnel file creation system.

[0154] exist Figure 1In the personnel file construction device shown, the processor 1001 is used to execute the personnel file construction program stored in the memory 1005 to implement the steps of the personnel file construction method described in any of the above claims.

[0155] The specific implementation method of the personnel file construction device in this application is basically the same as the above-mentioned personnel file construction method embodiments, and will not be repeated here.

[0156] This application also provides a storage medium storing a program implementing a method for constructing personnel files, the program being executed by a processor to implement the method for constructing personnel files as follows:

[0157] Obtain images to be archived, wherein the images to be archived include face images and human body images;

[0158] Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image;

[0159] The first feature information and the second feature information are fused to obtain the first fused feature;

[0160] The first fused feature is subjected to feature clustering to obtain the target personnel file.

[0161] Optionally, the step of fusing the first feature information and the second feature information to obtain the first fused feature includes:

[0162] The first feature information and the second feature information are classified to obtain feature sets of different types;

[0163] The feature sets from different types are fused to obtain the first fused feature.

[0164] Optionally, after the step of classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes:

[0165] The feature sets of each type are fused using single features of different dimensions to obtain the second fused feature of each type of feature set;

[0166] The step of fusing the feature sets of different types to obtain the first fused feature includes:

[0167] The second fusion feature is fused between the feature sets of different types to obtain the first fusion feature.

[0168] Optionally, the step of performing feature fusion on the feature sets of each type using different dimensions of a single feature to obtain the second fused feature of the feature sets of each type includes:

[0169] The feature sets of each type are divided according to the feature dimensions to obtain a preset first number of dimensional blocks for each type of feature set;

[0170] By fusing features between different dimensional blocks, a second fused feature for each type of feature set is obtained.

[0171] Optionally, after the step of classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes:

[0172] The same type of features are fused in the feature set to obtain a third fused feature;

[0173] The step of fusing the feature sets of different types to obtain the first fused feature includes:

[0174] The third fusion feature is fused with the feature sets of other different types to obtain the first fusion feature.

[0175] Optionally, the step of fusing similar features in the feature set to obtain a third fused feature includes:

[0176] The second feature information includes a preset second number of oriented human body features, wherein the second number is greater than or equal to 2;

[0177] In the feature set, human features with different orientations are fused to obtain a third fused feature.

[0178] Optionally, the step of fusing the first feature information and the second feature information to obtain the first fused feature includes:

[0179] Obtain the spatiotemporal features of the image to be archived;

[0180] The first feature information, the second feature information, and the spatiotemporal feature are fused to obtain the first fused feature.

[0181] The specific implementation of the storage medium in this application is basically the same as the embodiments of the above-mentioned personnel file construction method, and will not be repeated here.

[0182] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for constructing personnel files.

[0183] The specific implementation method of the computer program product of this application is basically the same as the various embodiments of the above-mentioned personnel file construction method, and will not be repeated here.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0185] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0186] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0187] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of constructing a personnel profile, characterized by, The method for constructing the personnel files includes: Obtain images to be archived, wherein the images to be archived include face images and human body images; Extract the feature information of the image to be archived to obtain the first feature information of the face image and the second feature information of the human body image; The first feature information and the second feature information are fused to obtain the first fused feature; The first fused features are clustered to obtain the target personnel file; The step of fusing the first feature information and the second feature information to obtain the first fused feature includes: The first feature information and the second feature information are classified to obtain feature sets of different types; The feature sets from different types are fused to obtain the first fused feature; Wherein, after the step of classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes: The feature sets of each type are fused using single features of different dimensions to obtain the second fused feature of each type of feature set; The step of fusing the feature sets of different types to obtain the first fused feature includes: The second fusion feature is fused between the feature sets of different types to obtain the first fusion feature.

2. The method of constructing a personnel file according to claim 1, wherein, The step of fusing features of different dimensions of single features into the feature sets of each type to obtain the second fused features of the feature sets of each type includes: The feature sets of each type are divided according to the feature dimensions to obtain a preset first number of dimensional blocks for each type of feature set; By fusing features between different dimensional blocks, a second fused feature for each type of feature set is obtained.

3. The method of claim 1, wherein the step of constructing a personnel profile further comprises the step of: After classifying the first feature information and the second feature information to obtain feature sets of different types, the method includes: ​ The same type of features are fused in the feature set to obtain a third fused feature; The step of fusing the feature sets of different types to obtain the first fused feature includes: The third fusion feature is fused with the feature sets of other different types to obtain the first fusion feature.

4. The method of claim 3, wherein the step of constructing a profile of the person is performed by a computer. The step of fusing similar features in the feature set to obtain a third fused feature includes: The second feature information includes a preset second number of oriented human body features, wherein the second number is greater than or equal to 2; In the feature set, human features with different orientations are fused to obtain a third fused feature.

5. The method for constructing personnel files as described in claim 1, characterized in that, The step of fusing the first feature information and the second feature information to obtain the first fused feature includes: Obtain the spatiotemporal features of the image to be archived; The first feature information, the second feature information, and the spatiotemporal feature are fused to obtain the first fused feature.

6. A device for constructing personnel files, characterized in that, The personnel file construction device includes: An acquisition module is used to acquire images to be archived, wherein the images to be archived include face images and human body images; The extraction module is used to extract feature information from the image to be archived, and obtain the first feature information of the face image and the second feature information of the human body image; The fusion module is used to fuse the first feature information and the second feature information to obtain a first fused feature; The clustering module is used to perform feature clustering on the first fused features to obtain the target personnel file; The fusion module includes: A classification module is used to classify the first feature information and the second feature information to obtain feature sets of different types; a first feature fusion module is used to fuse the feature sets of different types to obtain a first fused feature. The fusion module further includes: a single feature fusion module, used to perform single feature fusion of different dimensions on the feature sets of each type to obtain a second fused feature of the feature sets of each type; and a second feature fusion module, used to fuse the second fused features between the feature sets of different types to obtain a first fused feature.

7. The personnel file construction apparatus as described in claim 6, characterized in that, The single feature fusion module includes: a dimension partitioning module, used to partition the feature sets of each type according to the feature dimensions to obtain a preset first number of dimension blocks for each type of feature set; and a dimension block feature fusion module, used to fuse features between different dimension blocks to obtain a second fused feature for each type of feature set. And / or, the fusion module further includes: a same-type feature fusion module, used to perform same-type feature fusion in the feature set to obtain a third fused feature; and a third feature fusion module, used to perform feature fusion with other feature sets of different types to obtain a first fused feature; And / or, the same type feature fusion module includes: a human body feature fusion module with different orientations, used to fuse human body features with different orientations in the feature set to obtain a third fused feature; And / or, the fusion module further includes: a spatiotemporal feature acquisition module, used to acquire the spatiotemporal features of the image to be archived; and an information fusion module, used to fuse the first feature information, the second feature information and the spatiotemporal features to obtain a first fused feature.

8. A device for constructing personnel files, characterized in that, The personnel file construction device includes: a memory, a processor, and a program stored in the memory for implementing the personnel file construction method. The memory is used to store the program that implements the method for constructing personnel files; The processor is configured to execute a program that implements the method for constructing the personnel file, thereby implementing the steps of the method for constructing the personnel file as described in any one of claims 1 to 5.

9. A storage medium, characterized in that, The storage medium stores a program for implementing a method for constructing personnel files, which is executed by a processor to implement the steps of the method for constructing personnel files as described in any one of claims 1 to 5.

Images