Image archiving method, device, terminal equipment, and computer-readable storage medium

Abstract

The application is suitable for the technical field of image processing, and provides a picture archiving method and device, terminal equipment and computer readable storage medium, which comprises the following steps: acquiring a plurality of pictures to be archived, each of the pictures to be archived comprising a face and / or a human body; if the picture to be archived comprises a face, extracting face feature information in the picture to be archived; if the picture to be archived comprises a human body, extracting human body feature information in the picture to be archived; archiving the plurality of pictures to be archived according to the face feature information to obtain at least one face archive; archiving the plurality of pictures to be archived according to the human body feature information to obtain at least one human body archive; and matching at least one face archive and at least one human body archive according to track information to which the picture to be archived belongs, and merging the matched face archive and human body archive into one archive. Through the above method, the reliability of the picture archiving result can be effectively improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a method, apparatus, terminal device and computer-readable storage medium for archiving images. Background Art

[0002] Image archiving is the process of categorizing snapshot images captured by a camera into multiple archives. The goal of image archiving is to ensure that each archive contains the same subject. Image archiving is widely used in fields such as identity recognition and trajectory tracking. For example, when tracking multiple target individuals, multiple snapshot images can be archived, ensuring that each archive contains the same target individuals.

[0003] Existing image archiving methods typically extract facial features from captured images, calculate the similarity between two captured images based on this information, and then determine whether to group the two captured images into the same file based on the similarity. When the angle of the face is changed or the facial features are blurred, the calculated similarity becomes less reliable, resulting in poor image archiving results. Summary of the Invention

[0004] The embodiments of the present application provide a picture archiving method, apparatus, terminal device, and computer-readable storage medium, which can effectively improve the reliability of picture archiving results.

[0005] In a first aspect, an embodiment of the present application provides a method for archiving images, comprising:

[0006] Acquire multiple images to be archived, each of which includes a face and / or a body;

[0007] If the picture to be archived includes a face, extracting facial feature information from the picture to be archived;

[0008] If the picture to be archived includes a human body, extracting human body feature information from the picture to be archived;

[0009] Archiving the plurality of images to be archived according to the facial feature information to obtain at least one facial file;

[0010] Archiving the plurality of images to be archived according to the human body feature information to obtain at least one human body file;

[0011] Matching processing is performed on the at least one face file and the at least one body file according to the trajectory information to which the to-be-archived pictures belong, and the matched face file and body file are merged into one file, wherein the two to-be-archived pictures belonging to the same set of trajectory information contain the same photographed subject.

[0012] In the embodiment of the present application, facial feature information and body feature information are extracted from the image to be archived, and the image to be archived is archived according to the facial feature information and body feature information, respectively, to obtain a facial file and a body file; finally, the facial file and the body file are matched according to the trajectory information of the image to be archived. Through the above method, on the basis of facial feature information, the body feature information is taken into account, the dimension of the feature information is increased, the inaccuracy of the similarity caused by the inaccuracy of the feature information of a single dimension is avoided, and the reliability of the image archiving result is effectively improved; and the facial file and the body file are matched using the trajectory information of the image to be archived, and the matching facial file and body file are merged. Since the trajectory information is taken into account, the reliability of the archiving result is further increased.

[0013] In a possible implementation of the first aspect, archiving the multiple images to be archived according to the facial feature information to obtain at least one facial file includes:

[0014] Calculating an image quality value of each of a plurality of face pictures, wherein the face pictures are pictures to be archived that include faces;

[0015] Calculating the face fusion similarity between every two face images according to the image quality value and the face feature information;

[0016] The plurality of face images are archived according to the face fusion similarity to obtain the at least one face file.

[0017] In a possible implementation of the first aspect, calculating the image quality value of each of the multiple face images includes:

[0018] For each face image, obtaining respective parameter values ​​of a plurality of image quality parameters of the face image;

[0019] The parameter values ​​are weighted and summed to obtain the image quality value of the face image.

[0020] In a possible implementation of the first aspect, calculating the face fusion similarity between every two face images based on the image quality value and the facial feature information includes:

[0021] Calculating the facial feature similarity between every two facial images based on the facial feature information;

[0022] dividing the plurality of face images into a plurality of image groups according to the image quality values;

[0023] Obtaining a preset coefficient matrix, wherein the coefficient matrix includes weight coefficients between the picture groups to which each two face pictures belong;

[0024] The face fusion similarity between each two face pictures is calculated according to the face feature similarity between each two face pictures and the weight coefficient.

[0025] In a possible implementation of the first aspect, archiving the plurality of images to be archived according to the human body feature information to obtain at least one human body profile includes:

[0026] Calculating the human body feature similarity between every two human body pictures according to the human body feature information, wherein the human body pictures are pictures to be archived that include human bodies;

[0027] Calculating the spatiotemporal similarity between every two human body pictures;

[0028] Calculating the human body fusion similarity between every two human body pictures according to the human body feature similarity and the spatiotemporal similarity;

[0029] The plurality of human body images are archived according to the human body fusion similarity to obtain the at least one human body archive.

[0030] In a possible implementation of the first aspect, calculating the spatiotemporal similarity between every two human body pictures includes:

[0031] Calculating the actual distance between the photographing devices corresponding to each two of the human body pictures;

[0032] Calculating the time similarity between the shooting times of each two human body pictures;

[0033] The spatiotemporal similarity between every two human body pictures is calculated according to the actual distance and the temporal similarity.

[0034] In a possible implementation of the first aspect, matching the at least one face profile and the at least one body profile based on the trajectory information of the image to be archived, and merging the matched face profile and body profile into one profile, includes:

[0035] For any of the human body files, respectively calculating a matching value between the human body file and each of the face files, wherein the matching value represents the number of face images in the human body file that belong to the trajectory information corresponding to the face file;

[0036] The face file and the body file corresponding to the largest matching value are merged into one file.

[0037] In a second aspect, an embodiment of the present application provides a picture archiving device, comprising:

[0038] A picture acquisition unit, configured to acquire a plurality of pictures to be archived, each picture to be archived including a face and / or a body;

[0039] a first extraction unit, configured to extract facial feature information from the image to be archived if the image to be archived includes a face;

[0040] a second extraction unit, configured to extract human feature information from the image to be archived if the image to be archived includes a human body;

[0041] A face archiving unit, configured to archive the plurality of images to be archived according to the plurality of facial feature information to obtain at least one face file;

[0042] A human body archiving unit, configured to archive the plurality of images to be archived according to the human body feature information to obtain at least one human body file;

[0043] The image archiving unit is configured to match the at least one face file and the at least one body file according to the trajectory information to which the images to be archived belong, and merge the matched face files and body files into one file, wherein the two images to be archived belonging to the same set of trajectory information contain the same photographed subject.

[0044] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a picture archiving method as described in any one of the first aspects above when executing the computer program.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program, when executed by a processor, implements the image archiving method as described in any one of the above-mentioned first aspects.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the image archiving method described in any one of the above-mentioned first aspects.

[0047] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0049] Figure 1 is a flowchart of a picture archiving method provided by the embodiments of the present application;

[0050] Figure 2 is a schematic diagram of a face angle provided by the embodiments of the present application;

[0051] Figure 3 is a schematic diagram of the position of a shooting device provided by the embodiments of the present application;

[0052] Figure 4 is a schematic diagram of a picture archiving flow provided by the embodiments of the present application;

[0053] Figure 5 is a schematic diagram of the structure of a picture archiving device provided by the embodiments of the present application;

[0054] Figure 6 is a schematic diagram of the structure of a terminal device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0055] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known systems, devices, circuits, and methods have not been described in detail so as not to obscure the description of the present application.

[0056] It should be understood that, when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0057] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context.

[0058] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0060] See also Figure 1 , is a flow chart of a method for archiving pictures provided in an embodiment of the present application. As an example and not a limitation, the method may include the following steps:

[0061] S101: Acquire multiple pictures to be archived, each picture to be archived includes a face and / or a body.

[0062] In the embodiment of the present application, the multiple pictures to be archived can be taken by different shooting devices or by the same shooting device.

[0063] For example, in an application scenario, 10 cameras are installed in a shopping mall, and each camera takes 100 snapshots. These 10 cameras take a total of 1,000 snapshots. Some of these 1,000 snapshots include only facial parts, some include only human body parts, and some include the entire person (including both facial and human body parts). These 1,000 snapshots constitute a plurality of images to be archived. In subsequent embodiments of this application, images to be archived that include facial parts will be recorded as facial images, and images to be archived that include human body parts will be recorded as human body images.

[0064] In order to accurately extract feature information later, the picture including the entire person can also be segmented into a face picture including only the face part and a body picture including only the body part, and then feature extraction is performed on the segmented face picture and body picture respectively.

[0065] In practical applications, images to be archived can be manually sorted, such as by dividing them into face images and body images, or by manually dividing entire person images into face images and body images. Alternatively, trained recognition models can be used to sort images to be archived, without further limitation.

[0066] S102: If the picture to be archived includes a face, extract facial feature information from the picture to be archived.

[0067] S103: If the image to be archived includes a human body, extracting human body feature information from the image to be archived.

[0068] In the embodiments of the present application, existing image feature extraction methods can be used to extract facial and body feature information. For example, feature information can be extracted using a trained neural network model, using a scale-invariant feature variation method, or using a histogram of oriented gradients, etc., without specific limitations here.

[0069] It should be noted that when a picture to be archived includes both a face and a body, the picture to be archived is both a face picture and a body picture, and it is necessary to extract the face feature information and the body feature information of the picture to be archived respectively.

[0070] S104: Archiving multiple pictures to be archived according to facial feature information to obtain at least one facial file.

[0071] Step S104 is equivalent to archiving the face images in the archived images. In one embodiment, the step of archiving the face images includes:

[0072] Calculate the facial feature similarity between the facial feature information of each two facial images; if the facial feature similarity is greater than a first preset threshold, the two facial images are classified into the same facial file.

[0073] The distance between the facial feature information of two facial images can be calculated using distance calculation methods such as Euclidean distance and Mahalanobis distance, and then the distance can be subtracted from a preset value to obtain the facial feature similarity. The facial feature similarity between the facial feature information of two facial images can also be calculated using similarity calculation methods such as cosine similarity, Pierce correlation coefficient, and Jaccard similarity coefficient. This application does not specifically limit the method for calculating facial feature similarity.

[0074] In practical applications, it's common to capture a face in profile. Because profiles contain less facial feature information than full-face images, the facial feature similarity between two profile images may be greater than the facial feature similarity between two full-face images. In this case, the calculated facial feature similarity is inaccurate.

[0075] To solve the above problem, in one embodiment, the step of archiving facial images includes:

[0076] Calculate the image quality value of each of the multiple face images; calculate the face fusion similarity between every two face images based on the image quality value and face feature information; archive the multiple face images based on the face fusion similarity to obtain at least one face file.

[0077] Face images typically exist in a variety of states, including angle, size, presence of a mask, and image clarity. Each state of a face image is used as a quality parameter for evaluating image quality. The values ​​of each quality parameter are counted, and then the image quality value of the face image is calculated based on the parameter values.

[0078] For example, see angle Figure 2 , is a schematic diagram of the face angle provided by the embodiment of the present application. Figure 2 As shown in the figure, face angles include roll, pitch, and yaw. Roll represents the angle of rotation along the head's front-to-back axis, pitch represents the angle of rotation along the head's left-to-right axis, and yaw represents the angle of rotation along the head's bottom-to-top axis. These three angles can represent the rotation angle of the face relative to the camera.

[0079] Optionally, the image quality value may be calculated by:

[0080] For each face image, obtain parameter values ​​of multiple image quality parameters of the face image; and perform weighted summation of the parameter values ​​to obtain an image quality value of the face image.

[0081] Specifically, the formula Calculate the image quality value. Where Q represents the image quality value, n is the number of quality parameters, x i is the parameter value of the i-th quality parameter, ω i is the weight corresponding to the parameter value of the i-th quality parameter.

[0082] With the above Figure 2 Taking the angles described in the embodiment as an example, the three angle values ​​of roll, pitch and yaw can be used as a parameter value respectively. For the size of the picture, the length and width of the picture can be used as a parameter value respectively; the area of ​​the picture can also be used as a parameter value. For whether to wear a mask, parameter values ​​can be set separately for the cases of wearing a mask and not wearing a mask. For example, the parameter value for the case of wearing a mask is set to 1, and the parameter value for the case of not wearing a mask is set to 0. For the clarity of the picture, the resolution of the picture can be used as a parameter value. It should be noted that in addition to the several quality parameters listed in the embodiments of the present application, other quality parameters that affect the quality of the picture can also be selected, which are not specifically limited here.

[0083] The angle value of the face image can be obtained by identifying an existing face angle recognition model, which is not specifically limited here. The condition of the face wearing a mask can be obtained by identifying an existing face mask recognition model, which is not specifically limited here.

[0084] The greater the influence of a certain quality parameter on the image quality, the greater the corresponding weight. For example, the angle of a face in a picture has a greater influence on the image quality than the size of the picture, so the weight corresponding to the angle is increased and the weight corresponding to the size of the picture is reduced.

[0085] To improve the calculation accuracy of the image quality value, the weight can be continuously learned. For example, the weight is learned using the feature similarity between pictures and the parameter values of multiple quality parameters of each picture.

[0086] By the above method, various factors affecting the quality of the face picture are considered in the calculation of the face feature similarity, and the accuracy of the face similarity is effectively improved.

[0087] Based on the above description of the image quality value, the calculation method of the face fusion similarity can include: weighting and summing the face feature similarity between each two face pictures and the image quality value of each two face pictures to obtain the face fusion similarity between each two face pictures.

[0088] Optionally, another calculation method of the face fusion similarity includes:

[0089] According to the face feature information, the face feature similarity between each two face pictures is calculated; according to the image quality value, the multiple face pictures are divided into multiple picture groups; a preset coefficient matrix is obtained, the coefficient matrix including the weight coefficient between each two picture groups to which each two face pictures belong; according to the face feature similarity between each two face pictures and the weight coefficient, the face fusion similarity between each two face pictures is calculated.

[0090] The division method of the picture group can be: a division range of the image quality value is preset; the face pictures corresponding to the image quality values within the division range are divided into a picture group.

[0091] For example, it is assumed that the division range of the image quality value is 0-50, 50-80 and 80-100. The image quality values of face pictures A, B, C and D are 30, 60, 70 and 90 respectively. Face picture A belongs to the first picture group, face pictures B and C belong to the second picture group, and face picture D belongs to the third picture group. It should be noted that the above is only an example of picture group division, and the division range of the image quality value is not specifically limited.

[0092] In the embodiment of the application, the coefficient matrix can be preset by a person, can be calculated according to actual experience, or can be continuously adjusted during actual application.

[0093] In an embodiment of the present application, the face similarity and the weight coefficient can be multiplied to obtain the face fusion similarity. For example, assume that there are three face pictures A, B, and C, and the picture quality values ​​of A, B, and C are 40, 70, and 90, respectively. The three face pictures are divided into two picture groups according to the picture quality value. Specifically, the face pictures with a picture quality value greater than 50 are divided into one picture group (high-quality picture group), and the face pictures with a picture quality value less than 50 are divided into one picture group (low-quality picture group). That is, A belongs to the low-quality picture group, and B and C belong to the high-quality picture group. In the preset coefficient matrix, the weight coefficient between the high-quality picture group and the low-quality picture group is 0.9, the weight coefficient between the high-quality picture group and the high-quality picture group is 0.4, and the weight coefficient between the low-quality picture group and the low-quality picture group is 0.5.

[0094] Calculate the face fusion similarity between A and B: Calculate the facial feature similarity between A and B; the weight coefficient between the low-quality image group to which A belongs and the high-quality image group to which B belongs is 0.9; multiply the facial feature similarity between A and B by 0.9 to obtain the face fusion similarity between A and B.

[0095] Calculate the face fusion similarity between A and C: Calculate the facial feature similarity between A and B; the weight coefficient between the low-quality image group to which A belongs and the high-quality image group to which C belongs is 0.9; multiply the facial feature similarity between A and C by 0.9 to obtain the face fusion similarity between A and C.

[0096] Calculate the face fusion similarity between B and C: calculate the facial feature similarity between B and C; the weight coefficient between the high-quality image group to which B belongs and the high-quality image group to which C belongs is 0.4; multiply the facial feature similarity between B and C by 0.4 to obtain the face fusion similarity between B and C.

[0097] As can be seen from the above example, by modifying the facial feature similarity within each image quality value range, the facial feature similarity measurement for each quality range is unified. This allows facial images to be archived using a unified threshold, avoiding unreasonable archiving results caused by inconsistent feature similarity measurements.

[0098] S105 , archiving multiple pictures to be archived according to the human body feature information to obtain at least one human body file.

[0099] Step S105 is equivalent to archiving the human body picture in the archived picture. In one embodiment, the step of archiving the human body picture includes:

[0100] Calculate the human feature similarity between the human feature information of each two human body pictures; if the human feature similarity is greater than a second preset threshold, the two human body pictures are classified into the same human body file.

[0101] In actual applications, there may be two people wearing similar clothes, which will result in a high similarity of human features between the two human body images, resulting in inaccurate final archiving results.

[0102] To solve the above problem, in one embodiment, the step of archiving human body pictures includes:

[0103] Calculate the human feature similarity between every two human body pictures based on the human feature information; calculate the spatiotemporal similarity between every two human body pictures; calculate the human body fusion similarity between every two human body pictures based on the human feature similarity and the spatiotemporal similarity; archive multiple human body pictures based on the human body fusion similarity to obtain at least one human body file.

[0104] The spatiotemporal similarity includes similarity in temporal information and similarity in spatial information. In the embodiments of the present application, time may refer to the time it takes for a camera to capture a target object and obtain a human body image. Because multiple cameras may be present in a real scene, each installed in a different position, there may be an actual distance between them. This actual distance constitutes the spatial information.

[0105] For example, see Figure 3 , is a schematic diagram of the position of the shooting device provided in the embodiment of the present application. Figure 3 As shown, the position points of camera A and camera B are obtained, where the position point of camera A is the intersection point O1 of the center of the field of view of camera A and the center line of the illuminated road, and the position point of camera B is the intersection point O2 of the center of the field of view of camera B and the center of the illuminated road. The actual distance between cameras A and B is the actual distance from O1 to O2 (as shown in Figure 2). Figure 3 The line segments O1M, MN and NO2 are shown).

[0106] Optionally, the spatiotemporal similarity can be calculated by:

[0107] The actual distance between the shooting devices corresponding to each two human body pictures is calculated; the time similarity between the shooting times of each two human body pictures is calculated; and the spatiotemporal similarity between each two human body pictures is calculated based on the actual distance and the time similarity.

[0108] The method for calculating the actual distance may be: determining the position points of the two shooting devices in the application scene map, determining the path between the two position points in the application scene map, calculating the actual length of the path, and determining the actual length as the actual distance.

[0109] Temporal similarity can be calculated by multiplying the shooting times of two human images to obtain the temporal similarity between them. Alternatively, temporal similarity can be calculated by calculating the similarity between the shooting times of two human images. For example, the difference between the two shooting times can be calculated and used as the temporal similarity. Alternatively, the cosine similarity between the two shooting times can be calculated and used as the temporal similarity. Of course, other similarity calculation methods, such as Euclidean distance and Mahalanobis distance, can also be used, and are not specifically limited here.

[0110] Optionally, one implementation method for calculating the spatiotemporal similarity between two human body images based on the actual distance and temporal similarity can be: normalizing the calculated actual distance to reduce the distance to between 0 and 1, and then subtracting the normalized distance from 1 to obtain the spatial similarity; then multiplying the spatial similarity and the temporal similarity between each two human body images to obtain the spatiotemporal similarity between each two human body images.

[0111] One implementation method of calculating the human body fusion similarity between every two human body pictures based on the human body feature similarity and the spatiotemporal similarity is: multiplying the human body feature similarity and the spatiotemporal similarity between every two human body pictures to obtain the human body fusion similarity between every two human body pictures.

[0112] Furthermore, multiple human body pictures are archived according to human body fusion similarity to obtain at least one human body file, including: if the human body fusion similarity between two human body pictures is greater than a second preset threshold, the two human body pictures are summarized into the same human body file.

[0113] After S104 and S105, community graph cutting methods (such as infomap, louvain algorithm, etc.) can be used for post-processing to improve the accuracy of archiving.

[0114] S106 , performing matching processing on at least one face file and at least one body file according to the trajectory information of the image to be archived, and merging the matched face files and body files into one file.

[0115] In one embodiment, the matching of the face profile and the body profile is implemented by:

[0116] For any human body file, the matching value between the human body file and each face file is calculated respectively, and the face file and human body file corresponding to the largest matching value are merged into one file.

[0117] The matching value indicates the number of face images in the human body file that have trajectory information corresponding to the face file.

[0118] Optionally, the face and body files corresponding to the larger N matching values ​​can be merged into a single file. In practical applications, the value of N can be determined based on the accuracy of the file. A larger N value results in lower accuracy, while a smaller N value results in higher accuracy.

[0119] In embodiments of the present application, trajectory information can be used when matching human profiles with facial profiles. Multiple snapshots of the same subject captured by a single camera device belong to the same set of trajectory information. The trajectory information can be identified using existing trajectory tracking technology, which is not limited here.

[0120] In practical applications, a camera can use a tracking algorithm to track and photograph a subject. During the tracking process, multiple snapshots are obtained, which form a set of trajectory information. The multiple snapshots in a set of trajectory information may include both facial and body images. Facial and body images belonging to the same set of trajectory information are considered matching images.

[0121] For example, body file A is matched with each face file. Assume there are two face files, B and C, where body file A contains 10 body images, and face files B and C each contain 10 face images. Eight body images in body file A match eight face images in face file B, meaning eight body images in A belong to the trajectory information corresponding to B, and the matching value between A and B is 8. Three body images in body file A match three face images in face file C, meaning three body images in A belong to the trajectory information corresponding to C, and the matching value between A and C is 3. The face file corresponding to the largest matching value (i.e., 8) is merged with body file A into one file, meaning the face file B corresponding to the matching value of 8 is merged with body file A into one file.

[0122] See also Figure 4 , is a schematic diagram of the picture archiving process provided by the embodiment of this application. Figure 4 As shown in the figure, based on image quality and facial image features (i.e., facial feature information), face fusion similarity is calculated. Face images are archived based on the face fusion similarity to obtain face archives. Based on spatiotemporal similarity and body image features (i.e., body feature information), body fusion similarity is calculated. Body images are archived based on the body fusion similarity to obtain body archives. Finally, face archives and body archives are clustered based on trajectory information.

[0123] Through the above method, human body feature information is taken into consideration on the basis of facial feature information; image quality is taken into consideration when archiving faces, and spatiotemporal information is taken into consideration when archiving human bodies, thereby increasing the dimension of feature information and avoiding inaccurate similarity caused by inaccurate feature information of a single dimension, thereby effectively improving the reliability of image archiving results.

[0124] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0125] Corresponding to the picture archiving method described in the above embodiment, Figure 5 This is a structural block diagram of the picture archiving device provided in an embodiment of the present application. For the sake of convenience, only the parts related to the embodiment of the present application are shown.

[0126] Reference Figure 5 , the device comprises:

[0127] The picture acquisition unit 51 is used to acquire multiple pictures to be archived, each of which includes a face and / or a body.

[0128] The first extraction unit 52 is configured to extract facial feature information from the image to be archived if the image to be archived includes a face.

[0129] The second extraction unit 53 is configured to extract human feature information from the picture to be archived if the picture to be archived includes a human body.

[0130] The face archiving unit 54 is configured to archive the plurality of images to be archived according to the facial feature information to obtain at least one face file.

[0131] The human body archiving unit 55 is used to archive the multiple pictures to be archived according to the human body feature information to obtain at least one human body file.

[0132] The image archiving unit 56 is configured to match the at least one face file and the at least one body file according to the trajectory information of the images to be archived, and merge the matched face files and body files into one file, wherein the two images to be archived belonging to the same set of trajectory information contain the same subject.

[0133] Optionally, the face archiving unit 54 is further configured to:

[0134] Calculating image quality values ​​for each of a plurality of face pictures, wherein the face pictures are pictures to be archived that include faces; calculating face fusion similarities between every two face pictures based on the image quality values ​​and the face feature information; and archiving the plurality of face pictures based on the face fusion similarities to obtain the at least one face archive.

[0135] Optionally, the face archiving unit 54 is further configured to:

[0136] For each face picture, respective parameter values ​​of a plurality of image quality parameters of the face picture are obtained; and the parameter values ​​are weighted and summed to obtain the image quality value of the face picture.

[0137] Optionally, the face archiving unit 54 is further configured to:

[0138] Calculating the facial feature similarity between every two facial images based on the facial feature information;

[0139] dividing the plurality of face images into a plurality of image groups according to the image quality values;

[0140] Obtaining a preset coefficient matrix, wherein the coefficient matrix includes weight coefficients between the picture groups to which each two face pictures belong;

[0141] The face fusion similarity between each two face pictures is calculated according to the face feature similarity between each two face pictures and the weight coefficient.

[0142] Optionally, the human body archiving unit 55 is further used for:

[0143] The method further comprises calculating the human feature similarity between each two human body pictures based on the human feature information, wherein the human body pictures are pictures to be archived that include human bodies; calculating the spatiotemporal similarity between each two human body pictures; calculating the human body fusion similarity between each two human body pictures based on the human feature similarity and the spatiotemporal similarity; and archiving the multiple human body pictures based on the human body fusion similarity to obtain the at least one human body archive.

[0144] Optionally, the human body archiving unit 55 is further used for:

[0145] Calculating the actual distance between the shooting devices corresponding to each two of the human body pictures; calculating the time similarity between the shooting times of each two of the human body pictures; and calculating the spatiotemporal similarity between each two of the human body pictures based on the actual distance and the time similarity.

[0146] Optionally, the picture archiving unit 56 is further configured to:

[0147] For any of the human body profiles, the matching values ​​between the human body profile and the trajectory information corresponding to each of the facial profiles are calculated respectively, wherein the matching value represents the number of facial images in the human body profile that belong to the trajectory information corresponding to the facial profile; the facial profile and the human body profile corresponding to the largest matching value are merged into one profile.

[0148] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0149] in addition, Figure 5 The picture archiving device shown can be a software unit, a hardware unit, or a combination of software and hardware units built into an existing terminal device, or can be integrated into the terminal device as an independent accessory, or can exist as an independent terminal device.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0151] Figure 6 This is a schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 6 As shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned picture archiving method embodiments when executing the computer program 62.

[0152] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 6 It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0153] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0154] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Furthermore, the memory 61 may also include both an internal storage unit of the terminal device 6 and an external storage device. The memory 61 is used to store an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is about to be output.

[0155] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0156] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable medium cannot be an electric carrier signal or a telecommunication signal.

[0158] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0159] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0162] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for archiving pictures, characterized in that: include: Acquire multiple images to be archived, each of which includes a face and / or a body; If the picture to be archived includes a face, extracting facial feature information from the picture to be archived; If the picture to be archived includes a human body, extracting human body feature information from the picture to be archived; Calculating an image quality value of each of a plurality of face pictures, wherein the face pictures are pictures to be archived that include faces; Calculating the facial feature similarity between every two facial images based on the facial feature information; dividing the plurality of face images into a plurality of image groups according to the image quality values; Obtaining a preset coefficient matrix, wherein the coefficient matrix includes weight coefficients between the picture groups to which each two face pictures belong; Calculating the face fusion similarity between each two face images based on the face feature similarity between each two face images and the weight coefficient; Archiving the plurality of face images according to the face fusion similarity to obtain at least one face file; Archiving the plurality of images to be archived according to the human body feature information to obtain at least one human body file; Matching processing is performed on the at least one face file and the at least one body file according to the trajectory information to which the to-be-archived pictures belong, and the matched face file and body file are merged into one file, wherein the two to-be-archived pictures belonging to the same set of trajectory information contain the same photographed subject.

2. The image archiving method according to claim 1, wherein: Calculating the image quality value of each of the plurality of face images includes: For each face image, obtaining respective parameter values ​​of a plurality of image quality parameters of the face image; The parameter values ​​are weighted and summed to obtain the image quality value of the face image.

3. The image archiving method according to claim 1, wherein: Archiving the plurality of images to be archived according to the human body feature information to obtain at least one human body archive includes: Calculating the human body feature similarity between every two human body pictures according to the human body feature information, wherein the human body pictures are pictures to be archived that include human bodies; Calculating the spatiotemporal similarity between every two human body pictures; Calculating the human body fusion similarity between every two human body pictures according to the human body feature similarity and the spatiotemporal similarity; The plurality of human body images are archived according to the human body fusion similarity to obtain the at least one human body archive.

4. The image archiving method according to claim 3, wherein: The calculating of the spatiotemporal similarity between each two human body pictures comprises: Calculating the actual distance between the photographing devices corresponding to each two of the human body pictures; Calculating the time similarity between the shooting times of each two human body pictures; The spatiotemporal similarity between every two human body pictures is calculated according to the actual distance and the temporal similarity.

5. The image archiving method according to claim 1, wherein: The matching processing of the at least one face file and the at least one body file according to the trajectory information of the image to be archived, and merging the matched face file and body file into one file, includes: For any of the human body files, respectively calculating a matching value between the human body file and each of the face files, wherein the matching value represents the number of face images in the human body file that have trajectory information corresponding to the face file; The face file and the body file corresponding to the largest matching value are merged into one file.

6. A picture archiving device, characterized in that: include: A picture acquisition unit, configured to acquire a plurality of pictures to be archived, each picture to be archived including a face and / or a body; a first extraction unit, configured to extract facial feature information from the image to be archived if the image to be archived includes a face; a second extraction unit, configured to extract human feature information from the image to be archived if the image to be archived includes a human body; A face archiving unit is configured to calculate an image quality value for each of a plurality of face pictures, wherein the face pictures are pictures to be archived that include faces; calculate a facial feature similarity between each pair of the face pictures based on the facial feature information; divide the plurality of face pictures into a plurality of picture groups based on the image quality values; obtain a preset coefficient matrix, the coefficient matrix including weight coefficients between the picture groups to which each pair of the face pictures belongs; calculate a face fusion similarity between each pair of the face pictures based on the facial feature similarity and the weight coefficients; and archive the plurality of face pictures based on the face fusion similarity to obtain at least one face archive; A human body archiving unit, configured to archive the plurality of images to be archived according to the human body feature information to obtain at least one human body file; The image archiving unit is configured to match the at least one face file and the at least one body file according to the trajectory information to which the images to be archived belong, and merge the matched face files and body files into one file, wherein the two images to be archived belonging to the same set of trajectory information contain the same photographed subject.

7. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images