Portrait file gathering method and device, electronic equipment and storage medium

Abstract

The invention provides a portrait file gathering method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring at least two figure images to be subjected to file gathering; determining credibility values of the various human body features based on change types of the various human body features and a shooting time difference value and a shooting place difference value of the at least two figure images, the change types reflecting change degrees of the human body features changing at any time and space; based on the credibility values of the various human body features, determining a file gathering credibility threshold value; and carrying out portrait file gathering on the at least two figure images based on the similarity between the human body features in the at least two figure images and the file gathering credibility threshold. In the calculation process of the credibility value of the human body characteristics and the file gathering credibility threshold, the time difference value and the place difference value of shooting of the at least two character images to be subjected to file gathering and the change degree of various human body characteristics at any time and space change are considered, and the portrait file gathering accuracy is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a portrait aggregation method, device, electronic device and storage medium. Background Art

[0002] In the field of visual data, the video structured image data collected by the camera is clustered according to the image's vector feature information and spatiotemporal feature information, ultimately forming a personnel profile for analyzing personnel trajectories, behaviors, and information such as fellow travelers.

[0003] In the related art, facial features and body features are combined for similarity clustering. However, facial features and body features are affected by shooting angle, brightness, time and space, and there are certain degrees of deviation in different environments and time and space. In particular, body features may have large deviations, resulting in poor clustering accuracy. Summary of the invention

[0004] In response to the above-mentioned problems existing in the related technologies, the present application provides a portrait aggregation method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present application provides a portrait gathering method, comprising:

[0006] Obtain at least two person images to be assembled;

[0007] Determine the credibility value of various human features based on the change type to which the various human features belong, and the shooting time difference and shooting location difference of the at least two person images, wherein the change type reflects the degree of change of the human features with time and space;

[0008] Determining a confidence threshold of the aggregate file based on the confidence values ​​of the various human features;

[0009] Based on the similarity between the human body features in the at least two human images and the clustering credibility threshold, the at least two human images are clustered.

[0010] In some embodiments, the determining of the credibility values ​​of various human features based on the change types to which the various human features belong and the shooting time difference and shooting location difference of the at least two images of the person includes:

[0011] Determine the initial credibility of various human features based on the image quality scores of the various human features, the shooting location difference and the shooting time difference; wherein the image quality scores of the various human features are determined based on the clarity of the areas related to the various human features in the at least two human images;

[0012] Based on the magnitudes of the shooting time difference and the shooting location difference, selecting an influencing factor from the shooting time difference and the shooting location difference;

[0013] Based on the influencing factors and the change types to which the various human body features belong, the initial credibility of the various human body features is adjusted to obtain credibility values ​​of the various human body features.

[0014] In some embodiments, adjusting the initial credibility of the various human features based on the influencing factors and the types of changes to which the various human features belong includes:

[0015] Determining the credibility level of various human features based on the influencing factors and the change types to which the various human features belong;

[0016] Based on the credibility levels of the various human features, the initial credibility of the various human features is adjusted.

[0017] In some embodiments, adjusting the initial credibility of the various human features based on the credibility levels of the various human features includes:

[0018] Based on the credibility level, the linear adjustment parameter and the exponential adjustment parameter of the shooting location difference and the linear adjustment parameter and the exponential adjustment parameter of the shooting time difference included in the initial credibility are adjusted.

[0019] In some embodiments, the change types include: a first type, a second type and a third type; wherein the degree of change of the human body features corresponding to the first type, the second type and the third type decreases with time and space changes.

[0020] In some embodiments, the selecting an influencing factor from the shooting time difference and the shooting location difference based on the size of the shooting time difference and the shooting location difference includes:

[0021] In a case where the shooting time difference is greater than the first time difference, and the shooting location difference is greater than the first location difference, determining the shooting time difference as the influencing factor;

[0022] In a case where the shooting time difference is smaller than the second time difference, and the shooting location difference is smaller than the second location difference, the shooting location difference is determined as the influencing factor.

[0023] In some embodiments, the determining of the confidence threshold of the aggregate file based on the confidence values ​​of the various human features includes:

[0024] Determining the clustering credibility threshold based on the credibility values ​​of the various human features and the credibility values ​​of the facial features; wherein the credibility value of the facial features is determined based on the clarity of the area related to the facial features in the at least two person images;

[0025] The performing portrait clustering on the at least two person images based on the similarity between the human features in the at least two person images and the clustering credibility threshold comprises:

[0026] Based on the similarity between the human body features and the similarity between the facial features in the at least two human images and the clustering credibility threshold, the at least two human images are clustered.

[0027] In a second aspect, the embodiment of the present application further provides a portrait gathering device, comprising:

[0028] An acquisition unit, used for acquiring at least two person images to be assembled;

[0029] A first determining unit is used to determine the credibility values ​​of various human features based on the change types to which the various human features belong and the shooting time difference and shooting location difference of the at least two human images, wherein the change type reflects the degree of change of the human features with time and space;

[0030] A second determining unit, configured to determine a confidence threshold of the aggregate file based on the confidence values ​​of the various human features;

[0031] A clustering unit is used to cluster the portraits of the at least two human images based on the similarity between the human features in the at least two human images and the clustering credibility threshold.

[0032] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a portrait aggregation method as described above is implemented.

[0033] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the portrait aggregation methods described above.

[0034] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements any of the portrait aggregation methods described above.

[0035] The portrait clustering method, device, electronic device and storage medium provided in the embodiments of the present application take into account the time difference and shooting location difference of at least two human images to be clustered, as well as the degree of change of various human features with time and space, during the calculation process of the credibility value of human features and the clustering credibility threshold, thereby effectively improving the accuracy of portrait clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 It is a flowchart of a portrait aggregation method provided in an embodiment of the present application;

[0038] Figure 2 is a structural schematic diagram of a portrait gathering device provided in an embodiment of the present application;

[0039] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In the field of visual data, the video structured image data collected by the camera is clustered according to the image's vector feature information and spatiotemporal feature information, and ultimately a personnel profile is formed for analyzing personnel trajectories, behaviors, peers and other information.

[0041] Since there is no fixed pattern in the time and space of personnel activities, when the trajectory information of the archives is scattered in similar time and space, it is necessary to accurately aggregate the trajectory information of the same person to form a dynamic personnel archive.

[0042] In related technologies, facial features in personnel trajectory images are often used as the first features, and the facial image with the highest feature quality is taken as the file cover image. When clustering, the feature vector similarity between the facial features in the image library and the file cover image is calculated, and the files are clustered when the similarity is higher than a preset threshold. At the same time, the person's body-related feature attributes can also be combined as the second feature, the third feature or the Nth feature, and the preset similarity thresholds of different features are set. According to the similarity results of each body feature attribute, it is determined whether to cluster the pictures to be archived.

[0043] However, due to factors such as shooting angle, light brightness, time and space, both human face and human body features will have a certain degree of deviation in different environments and time and space, and some human body features will even have large errors. Therefore, in the related technology, even if the human face features are combined with human body features for similarity calculation and clustering, there will be many clustering errors, the clustering accuracy is low, and there are problems such as multiple files for one person and multiple files for one file.

[0044] In response to the above-mentioned problems existing in the related technologies, the embodiments of the present application provide a portrait clustering method, device and storage medium. In the process of calculating the credibility value of human features and the clustering credibility threshold, the time difference and the shooting location difference of at least two human images to be clustered, as well as the degree of change of various human features with time and space are considered, thereby effectively improving the accuracy of portrait clustering.

[0045] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Figure 1 is a flow chart of a portrait gathering method provided in an embodiment of the present application, such as Figure 1 As shown, the method comprises at least the following steps:

[0047] Step 101: Obtain at least two person images to be aggregated.

[0048] Specifically, at least two person images of the to-be-assembled file are obtained, that is, at least two person images of the to-be-assembled file are captured by a camera, a video camera, etc. Optionally, the person images of the to-be-assembled file may be initial images, or may be obtained after screening the initial images.

[0049] By extracting and preprocessing the face and body structured data in at least two person images, facial features, body features, etc. can be obtained. In the embodiment of the present application, the body features include but are not limited to the following attributes: age, height, gender, clothing color, body shape, hair, whether to wear a mask, whether to wear a hat, etc.

[0050] For example, the original captured images t1 and t2 are obtained from the camera, and the face and body structured data are preprocessed to obtain face features r1 and r2, and several body features rt1...rt n .

[0051] Step 102: Determine the credibility value of various human features based on the change type of various human features and the shooting time difference and shooting location difference of the at least two human images, wherein the change type reflects the degree of change of the human features with time and space.

[0052] Specifically, in the embodiment of the present application, the error in calculating the credibility of the aggregated file caused by the change of human features with time and space is taken into account. The credibility values ​​of various human features are determined based on the change types to which various human features belong, and the difference in shooting time and shooting location between at least two images of the person. The change types to which various human features belong reflect the degree of change of human features with time and space.

[0053] It can be understood that, for human features under any type of change, if the degree of change of the human features with time and space changes is large, assuming that the shooting time difference and / or shooting location difference is large, the human features may change greatly, and the credibility of the portrait grouping based on the human features is relatively low, that is, the credibility value of the human features is low. If the degree of change of the human features with time and space changes is small or remains unchanged, assuming that the shooting time difference and / or shooting location difference is small, the possibility that the human features may change greatly is not high, and the credibility of the portrait grouping based on the human features is relatively high, that is, the credibility value of the human features is low.

[0054] Step 103: Determine a confidence threshold based on the confidence values ​​of the various human features.

[0055] Specifically, according to the credibility values ​​of various human features obtained in step 102, the clustering credibility threshold can be determined. It can be understood that the clustering credibility threshold here is a threshold used to measure whether the similarity between human features meets the portrait clustering condition. The higher the credibility value of the human feature, the higher the clustering credibility threshold corresponding to the similarity under the human feature, and the lower the credibility value of the human body, the lower the clustering credibility threshold corresponding to the similarity under the human feature.

[0056] Step 104: Based on the similarity between the human features in the at least two human images and the clustering credibility threshold, perform portrait clustering on the at least two human images.

[0057] Specifically, based on the existing human feature similarity algorithm, the similarity between different human features in at least two human images can be obtained, combined with the clustering credibility threshold determined in step 104, to finally decide whether to perform portrait clustering processing on at least two human images.

[0058] The portrait clustering method provided in the embodiment of the present application considers the time difference and location difference between at least two images of the person to be clustered, as well as the degree of change of various human features with time and space, during the calculation process of the credibility value of human features and the clustering credibility threshold, thereby effectively improving the accuracy of portrait clustering.

[0059] In some embodiments, step 103: determining the credibility values ​​of various human features based on the change types to which the various human features belong and the shooting time difference and shooting location difference of the at least two human images specifically includes:

[0060] Determine the initial credibility of various human features based on the image quality scores of the various human features, the shooting location difference and the shooting time difference; wherein the image quality scores of the various human features are determined based on the clarity of the areas related to the various human features in the at least two human images;

[0061] Based on the magnitudes of the shooting time difference and the shooting location difference, selecting an influencing factor from the shooting time difference and the shooting location difference;

[0062] Based on the influencing factors and the change types to which the various human body features belong, the initial credibility of the various human body features is adjusted to obtain credibility values ​​of the various human body features.

[0063] Specifically, the credibility of various human features is determined by combining the change types of various human features and the shooting time difference and shooting location difference of at least two human images, which may specifically include:

[0064] First, determine the image quality scores rtq of various human features i , the image quality score is determined based on the clarity of the areas related to various human features in the person image. The higher the clarity, the higher the image quality score of the human features.

[0065] Secondly, based on the image quality scores of various human features, as well as the shooting location difference and shooting time difference, the initial credibility of various human features is determined.

[0066] Among them, the shooting location difference and the shooting time difference can be calculated as follows:

[0067] Get the capture time of the original image and the latitude and longitude information of the camera when capturing, and calculate the shooting time difference Δt and the shooting location difference Δd; the calculation method is as follows:

[0068] For two image capture times t1 and t2, the capture time difference Δt is:

[0069] Δt=t2-t1

[0070] For the longitude and latitude coordinates (φ1, λ1) and (φ2, λ2) of the two image capture locations, the Haversine formula is used to calculate the location difference Δd:

[0071]

[0072] Where R is the radius of the earth, usually 6371 kilometers. Before calculation, longitude and latitude need to be converted to radians, as follows:

[0073] φ1=φ1·π / 180

[0074] φ2=φ2·π / 180λ1=λ1·π / 180

[0075] λ2=λ2·π / 180

[0076] Here, π represents the ratio of a circle to a circle.

[0077] After obtaining the image quality scores rtq1,...,rtq of various human features n , and the shooting time difference Δt and the shooting location difference Δd, the initial credibility of each human feature is determined. Among them, the credibility value of the human feature should be the inverse linear result of the shooting time difference Δt and the shooting location difference Δd.

[0078] Specifically, the initial credibility rt of various human features can be determined according to the following formula: i cv:

[0079]

[0080] Among them, Δt represents the difference in shooting time, Δd represents the difference in shooting location, and rtq i is the image quality score of each human feature, m i 、n i 、p i ,q i They are all constants and can be adjusted according to actual conditions. i and n i is the linear adjustment parameter for the difference in shooting time and location, p i ,q i The parameters are adjusted at an exponential level, and these parameters are dynamically adjusted according to the characteristics of the human body of a specific classification.

[0081] Again, based on the size of the shooting time difference and the shooting location difference, one of the two is selected as an influencing factor. The size of the shooting time difference and the shooting location difference determines the degree of influence of the shooting time difference and the shooting location difference on the credibility of the human body features. For example, when the shooting time difference and the shooting time difference are both small, it means that the influence of time change on the credibility of human body features is small, while the influence of location change on the credibility of human body features is large.

[0082] Finally, the initial credibility of various human features is adjusted in combination with the selected influencing factors and the types of changes to which various human features belong, and the credibility values ​​of various human features are determined. Specifically, the types of changes to which human features belong are used to reflect the degree of change of human features as they change with time and space. In an embodiment of the present application, the types of changes described in human features are classified into three categories: a first type, a second type, and a third type. The degree of change of the human features corresponding to the three types increases with time and space. In actual implementation, different classifications can be performed according to classification requirements.

[0083] For example, human characteristics that will change significantly in similar time and space are variable characteristics (first type), such as clothing color, whether wearing a mask, whether wearing a hat, whether carrying a backpack, etc.; human characteristics that may change significantly during the entire validity period of the data file (ranging from 3 months, half a year to even a year) are short-term variable characteristics (second type), such as hairstyle, height, body shape, clothing type (short sleeves, long sleeves, summer clothes, winter clothes), etc.; human characteristics that basically do not change during the entire validity period of the data file are short-term immutable characteristics (third type), such as age, gender, etc.

[0084] The portrait clustering method provided in the embodiment of the present application selects one from the shooting time difference and the shooting location difference as an influencing factor of the credibility of the human body feature, and at the same time determines the credibility level of various human body features in combination with the change type to which various human body features belong, and determines the credibility value according to the credibility level of various human body features, thereby effectively reducing the weight of negative feedback human body features in the portrait clustering process and further improving the accuracy of portrait clustering.

[0085] In some embodiments, the selecting an influencing factor from the shooting time difference and the shooting location difference based on the size of the shooting time difference and the shooting location difference includes:

[0086] In a case where the shooting time difference is greater than the first time difference, and the shooting location difference is greater than the first location difference, determining the shooting time difference as the influencing factor;

[0087] In a case where the shooting time difference is smaller than the second time difference, and the shooting location difference is smaller than the second location difference, the shooting location difference is determined as the influencing factor.

[0088] Specifically, when the initial images have been screened, the ratio of the difference in shooting location to the difference in shooting time should fall within a reasonable range. For example, if it is determined from multiple initial images that the moving scene of a person is traveling by bicycle, the moving speed of the person (in km / h) should be within the range of (0, 30).

[0089] The ratio of the shooting location difference to the shooting time difference is in a reasonable range, that is, in accordance with the normal movement rate of personnel, the shooting location difference and the shooting time difference should be both small or both large.

[0090] Case 1: The shooting time difference is greater than the first time difference and the shooting location difference is greater than the first location difference. The first time difference and the first location difference can be preset. In different movement scenarios of people, the first time difference and the first location difference have different values. At this time, it indicates that both the shooting time difference and the shooting location difference are large, which means that the shooting time difference has a greater impact on the calculation of the credibility of human features, so the shooting time difference is used as an influencing factor.

[0091] Case 2: The shooting time difference is less than the second time difference and the shooting location difference is less than the second location difference. The second time difference and the second location difference can be preset. In different movement scenarios of people, the second time difference and the second location difference have different values. In this case, it indicates that both the shooting time difference and the shooting location difference are small, which means that the shooting location difference has a greater impact on the calculation of the credibility of human features, so the shooting location difference is used as an influencing factor.

[0092] Case 3: The shooting time difference is greater than the first time difference and the shooting location difference is less than the second location difference. In this case, it indicates that the shooting time difference is larger and the shooting location difference is smaller, which means that the shooting time difference has a greater impact on the calculation of the credibility of the human features, so the shooting time difference is used as an influencing factor.

[0093] Case 4: The shooting time difference is smaller than the second time difference and the shooting location difference is larger than the first location difference. In this case, it indicates that the shooting time difference is smaller and the shooting location difference is larger, which means that the shooting location difference has a greater impact on the calculation of the credibility of the human features, so the shooting location difference is used as an influencing factor.

[0094] For cases 3 and 4, when the person is in a specific moving scene, his or her moving speed should fall within a specific range, that is, the shooting location difference and the shooting time difference are either both small or both large. When the shooting location difference and the shooting time difference are one large and one small respectively, the credibility of the image is low. In extreme cases, the possibility that the two images are of the same person can be directly ruled out, and the image will not be involved in the subsequent clustering process.

[0095] The portrait clustering method provided in the embodiment of the present application selects the value that has a greater impact on the credibility of human features as an influencing factor based on the difference in shooting locations and shooting times of at least two human images to be clustered, thereby further improving the accuracy of portrait clustering.

[0096] In some embodiments, based on the influencing factors and the types of changes to which the various human features belong, the initial credibility of the various human features is adjusted, including:

[0097] Determining the credibility level of various human features based on the influencing factors and the change types to which the various human features belong;

[0098] Based on the credibility levels of the various human features, the initial credibility of the various human features is adjusted.

[0099] Specifically, according to the change type of the human body feature and the selected influencing factors, the credibility level of the human body feature can be determined, for example, high credibility level, relatively high credibility level, medium credibility level, low credibility level, untrustworthy, etc.

[0100] For example, the target moving average speed can be calculated by first using the time difference Δt and the location difference Δd as follows:

[0101]

[0102] Generally speaking, the moving scene of a person can be judged based on at least two person images. In different moving scenes, the moving speed of the target should fall into a reasonable speed range. The average speed range of the moving mode of a person can be referred to in Table 1:

[0103] Table 1: Average speed range of personnel movement

[0104]

[0105] (1) For variable features, such as clothing color, whether wearing a mask, or whether wearing a hat.

[0106] If the calculated s interval is reasonable, that is, it conforms to the normal movement rate of personnel, the differences in Δt and Δd should be small or large.

[0107] ① When the difference between Δt and Δd is small, Δd should be used as the influencing factor for the adjustment of the characteristic credibility parameter. (The unit of Δt is seconds).

[0108] Table 2: Confidence level of variable features determined by shooting location differences

[0109]

[0110] Among them, the interval value of the shooting location difference Δd should be adjusted according to the actual scenario of the aggregation business.

[0111] ② When the difference between Δt and Δd is large, Δt should be used as the influencing factor for the adjustment of the characteristic credibility parameter. (The unit value of Δt is hours). When the calculated s interval range does not conform to the normal movement rate of personnel, that is, when Δt is large and Δd is small, the interval value of Δt should also be taken as the main factor.

[0112] Table 3: Trust levels of mutable features determined by shooting time differences

[0113]

[0114] Among them, the interval value of Δt should be adjusted according to the actual scenario of the aggregation business.

[0115] (2) For short-term variable features, such as hairstyle, body shape, and clothing type.

[0116] If the calculated s interval is reasonable, that is, it conforms to the normal movement rate of personnel, the differences in Δt and Δd should be small or large.

[0117] ① When the differences between Δt and Δd are small, Δd should be used as an influencing factor in the adjustment of the feature credibility parameter.

[0118] Table 4: Short-term variable features determine the credibility level based on the difference in shooting locations

[0119]

[0120] The interval value of Δd should be adjusted according to the actual scenario of the archiving service, such as the number of cameras involved in the archiving and range factors.

[0121] ② When the difference between Δt and Δd is large, the adjustment of the characteristic credibility parameter should be based on the interval value of Δt. If the calculated s interval range does not conform to the normal movement rate of personnel, that is, when Δt is large and Δd is small, the interval value of Δt should also be the main one.

[0122] Table 5: Short-term variable features determine the credibility level based on the difference in shooting time

[0123]

[0124] Among them, the interval value of Δt should be adjusted according to the actual scenario of the aggregation business.

[0125] (3) For short-term immutable features, such as age and gender.

[0126] If the calculated s interval is reasonable, that is, it conforms to the normal movement rate of personnel, the differences in Δt and Δd should be small or large.

[0127] ① When the differences between Δt and Δd are small, Δd should be used as an influencing factor in the adjustment of the feature credibility parameter.

[0128] Table 6: Trust levels of short-term immutable features based on differences in shooting locations

[0129]

[0130] ② When the differences between Δt and Δd are large, the feature credibility does not change with Δt and Δd.

[0131] Table 7: Confidence levels of short-term invariant features

[0132]

[0133] If the calculated s interval range does not conform to the normal movement rate of personnel, that is, Δt is large and Δd is small, the results are the same as Table 7.

[0134] Among them, for “not related to Δt and Δd”, the credibility level can be determined as untrustworthy.

[0135] After determining the credibility level of the human features, the initial credibility is adjusted according to the credibility level of the human features to obtain the credibility value of the human features. The initial credibility of the human features obtained in the above steps is not static and immutable. The initial credibility should be dynamically adjusted in combination with the credibility level of the human features to obtain the credibility values ​​of various human features.

[0136] The portrait clustering method provided in the embodiment of the present application selects influencing factors from the shooting time difference and the shooting location difference, and divides the credibility level of each human feature based on the selected influencing factors and the change type of the human feature. This credibility level includes the influence of each human feature on the time and space changes. The credibility level is used to dynamically adjust the initial credibility of the human feature to obtain the credibility value of the human feature, so that the credibility threshold of the clustering can be dynamically adjusted to reduce the clustering rejection rate and improve the clustering accuracy.

[0137] In some embodiments, based on the credibility levels of the various human features, the initial credibility of the various human features is adjusted, including:

[0138] Based on the credibility level, the linear adjustment parameter and the exponential adjustment parameter of the shooting location difference and the linear adjustment parameter and the exponential adjustment parameter of the shooting time difference included in the initial credibility are adjusted.

[0139] Specifically, based on the credibility level of the human features, the initial credibility level of the human features is adjusted. The linear adjustment parameter and exponential adjustment parameter of the shooting location difference and the linear adjustment parameter and exponential adjustment parameter of the shooting time difference included in the initial credibility are adjusted. That is, m in the above formula is i 、n i 、p i ,q i The value of is adjusted.

[0140] For example, for human features with shooting location differences as influencing factors, m i 、n i 、p i ,q i The value adjustment can refer to Table 8:

[0141] Table 8: Parameter adjustment for mutable and short-term variable features with the difference in shooting locations as the influencing factor

[0142]

[0143] For example, for human features with shooting time difference as an influencing factor, m i 、n i 、p i ,q i The value adjustment can refer to Table 9:

[0144] Table 9: Parameter adjustment for mutable and short-term variable features with shooting time difference as the influencing factor

[0145]

[0146] The portrait clustering method provided in the embodiment of the present application dynamically adjusts the initial credibility of human features through the credibility level to obtain the credibility value of the human features, so as to dynamically adjust the clustering credibility threshold, reduce the clustering rejection rate, and improve the clustering accuracy rate.

[0147] In some embodiments, obtaining at least two person images to be aggregated includes:

[0148] Obtain at least two initial images of the file to be aggregated;

[0149] Based on a shooting time difference and a shooting location difference between the at least two initial images, performing a spatiotemporal credibility check on the at least two initial images;

[0150] When the spatiotemporal credibility check passes, the at least two initial images are used as at least two character images to be assembled.

[0151] Specifically, the at least two person images to be aggregated may be person images obtained by screening the initial images before calculating the credibility values ​​of the human features. The screening method is as follows:

[0152] Obtain at least two initial images of the file to be aggregated. The initial images are captured by a camera or a webcam.

[0153] The spatiotemporal credibility check is introduced. Based on the shooting time difference Δt and shooting location difference Δd between at least two initial images, the spatiotemporal credibility check is performed on at least two initial images. Considering the influence of the shooting location difference on the movement path of the person, if the shooting time difference is small and the movement path of the person is normal, that is, the shooting location difference is also small, the credibility of the feature is high; but if the shooting time difference is small, but the shooting location difference is large, the credibility of the feature is low.

[0154] In extreme cases, a very small difference in shooting time corresponds to a large difference in shooting location (for example, through multiple initial images, it is judged that the person should be in a moving scene of walking, but the speed of the person calculated by the difference in shooting location and shooting time is far beyond the maximum speed of walking). In this case, the possibility that the two initial images are of the same person can be directly excluded, that is, the spatiotemporal credibility check is not passed and the subsequent aggregation processing is not involved. Of course, the movement path of a person may be affected by other factors, such as road conditions, transportation, etc., which need to be considered in combination with multiple factors.

[0155] Only when the spatiotemporal credibility check is passed, the initial image is used as the character image to be aggregated for subsequent processing.

[0156] Specifically, the target moving average speed can be calculated by the time difference Δt and the location difference Δd as follows:

[0157]

[0158] In scenarios without the aid of transportation, the maximum walking and running speed of a person is between 10-15 km / h. If s is greater than this maximum value, the credibility check fails. In scenarios with the aid of transportation, for example, the average speed of a bicycle is between 20-30 km / h. If s is greater than this maximum value, the credibility check fails.

[0159] The portrait clustering method provided in the embodiment of the present application verifies the spatiotemporal credibility of the initial image before calculating the credibility of the portrait clustering, eliminates the initial image with obvious abnormalities, excludes the human features with negative feedback, strengthens the weight of the human features with positive feedback participating in the calculation, and further improves the accuracy of the portrait clustering.

[0160] In some embodiments, the determining of the confidence threshold of the aggregate file based on the confidence values ​​of the various human features includes:

[0161] Determining the aggregation credibility threshold based on the credibility values ​​of the various human features and the credibility values ​​of the facial features;

[0162] The credibility value of the facial feature is determined based on the clarity of the area related to the facial feature in the at least two character images.

[0163] Specifically, in the process of portrait gathering, in addition to various human body features, facial features are often more important, as facial features can more accurately locate a person.

[0164] In the process of determining the credibility threshold of the aggregate file based on the credibility values ​​of various human features, the credibility values ​​of facial features can also be combined.

[0165] In some embodiments, the performing portrait clustering on the at least two person images based on the similarity between the human features in the at least two person images and the clustering credibility threshold comprises:

[0166] Based on the similarity between the human body features and the similarity between the facial features in the at least two human images and the clustering credibility threshold, the at least two human images are clustered.

[0167] Specifically, during the clustering process, facial features can also be combined. Based on the existing facial similarity algorithm, the similarity between facial features in at least two person images can be obtained, combined with the loudness of facial features and the similarity of body features, as well as the clustering credibility threshold, to ultimately decide whether to perform portrait clustering on at least two person images.

[0168] The portrait clustering method provided in the embodiment of the present application not only uses human body features but also combines facial features in the process of determining the clustering credibility threshold and clustering processing, thereby further improving the accuracy of portrait clustering.

[0169] The following is an explanation of the clustering credibility threshold calculation process and the clustering process combining facial features and body features:

[0170] First, calculate the image quality scores of the face and body features. The image quality scores rlq1 and rlq2 of the face can be obtained according to the clarity of the relevant areas of the face features. The clearer the face, the higher the image quality score. At the same time, obtain the image quality scores rtq1...rtq of each body feature. n The more obvious the human features are, the higher the image quality score is. The value range of the image quality score is (0, 1).

[0171] Then, the credibility values ​​of face and body features are calculated.

[0172] By judging the interval in which the total of the credibility values ​​of various human features lies, a preset initial total weight value x of the face similarity and an initial value rtCT of the human similarity threshold are given.

[0173] The initial weight of face similarity is calculated based on the average image quality score of the face:

[0174]

[0175] Among them, the value range of x is the initial weight of the face similarity, that is, the proportion in the calculation of face and body similarity.

[0176] According to the credibility value of each human feature, the dynamic weight rt of each human feature involved in the calculation is calculated. i qz:

[0177]

[0178] The clustering credibility threshold CT is calculated based on the dynamic weights of n human features, the calculation weights of the facial feature vector, and the preset facial feature dynamic threshold and human feature dynamic threshold. The preset facial feature dynamic threshold and human feature dynamic threshold can be adjusted according to the actual situation.

[0179] Then, the aggregate file credibility threshold is calculated.

[0180] The aggregate file credibility threshold is calculated as follows:

[0181]

[0182] Among them, CT is the clustering credibility threshold, lCT is the dynamic threshold of facial features before adjustment, and rt i CT is the dynamic threshold of human characteristics before adjustment.

[0183] Finally, the similarity of faces and bodies is calculated, and the credibility threshold of the cluster is judged.

[0184] Based on the existing face similarity algorithm and human feature vector similarity algorithm, the similarity results of each item are calculated, and the credibility score is obtained by combining the weights of each face and body feature. It is compared with the clustering credibility threshold to decide whether to perform clustering processing in the end.

[0185] The portrait clustering method provided in the embodiment of the present application dynamically allocates the weights of the respective participants in the calculation according to the existing facial feature clarity quality and body feature credibility when calculating the credibility of personnel profile clustering; time and location factors are introduced in the calculation of body feature weights, and by calculating the credibility of each feature and verifying the spatiotemporal credibility, the body features with negative feedback are eliminated, and the weights of the positive feedback features participating in the calculation are strengthened; by calculating the credibility of each body feature, the credibility threshold of the clustering is dynamically adjusted to reduce the rejection rate of the clustering, thereby optimizing and improving the clustering effect.

[0186] The portrait gathering device provided in the present application is described below. The portrait gathering device described below and the portrait gathering method described above can be referenced to each other.

[0187] Figure 2 is a structural schematic diagram of a portrait gathering device provided in an embodiment of the present application, such as Figure 2 As shown, the device at least includes:

[0188] An acquisition unit 201 is used to acquire at least two person images to be aggregated;

[0189] A first determining unit 202 is used to determine the credibility values ​​of various human features based on the change types to which the various human features belong and the shooting time difference and shooting location difference of the at least two human images, wherein the change type reflects the degree of change of the human features with time and space;

[0190] A second determining unit 203 is used to determine a confidence threshold of the aggregation file based on the confidence values ​​of the various human features;

[0191] The clustering unit 204 is used to perform portrait clustering on the at least two human images based on the similarity between the human body features in the at least two human images and the clustering credibility threshold.

[0192] In some embodiments, the first determining unit 202 specifically includes:

[0193] A first determination module is used to determine the initial credibility of various human features based on the image quality scores of the various human features, the shooting location difference and the shooting time difference; wherein the image quality scores of the various human features are determined based on the clarity of the areas related to the various human features in the at least two human images;

[0194] A selection module, configured to select an influencing factor from the shooting time difference and the shooting location difference based on the magnitude of the shooting time difference and the shooting location difference;

[0195] The adjustment module is used to adjust the initial credibility of the various human features based on the influencing factors and the change types to which the various human features belong, so as to obtain the credibility values ​​of the various human features.

[0196] In some embodiments, the adjustment module includes:

[0197] A determination submodule, configured to determine the credibility level of various human features based on the influencing factors and the change types to which the various human features belong;

[0198] The adjustment submodule is used to adjust the initial credibility of the various human features based on the credibility levels of the various human features.

[0199] In some embodiments, the adjustment submodule is specifically used for:

[0200] Based on the credibility level, the linear adjustment parameter and the exponential adjustment parameter of the shooting location difference and the linear adjustment parameter and the exponential adjustment parameter of the shooting time difference included in the initial credibility are adjusted.

[0201] In some embodiments, the change types include: a first type, a second type and a third type; wherein the degree of change of the human body features corresponding to the first type, the second type and the third type decreases with time and space changes.

[0202] In some embodiments, the selection module includes:

[0203] A first selection submodule is used to determine that the shooting time difference is the influencing factor when the shooting time difference is greater than the first time difference and the shooting location difference is greater than the first location difference;

[0204] The second selection submodule is used to determine that the shooting location difference is the influencing factor when the shooting time difference is less than the second time difference and the shooting location difference is less than the second location difference.

[0205] In some embodiments, the second determining unit 203 includes:

[0206] A second determination module is used to determine the clustering credibility threshold based on the credibility values ​​of the various human features and the credibility value of the facial features; wherein the credibility value of the facial features is determined based on the clarity of the area related to the facial features in the at least two person images;

[0207] The clustering unit 204 includes: a clustering module, which is used to perform portrait clustering on the at least two person images based on the similarity between the human body features and the similarity between the facial features in the at least two person images, and the clustering credibility threshold.

[0208] Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303 and a communication bus 303, wherein the processor 301, the communication interface 302 and the memory 303 communicate with each other through the communication bus 303. The processor 301 may call the logic instructions in the memory 303 to execute the portrait gathering method, which includes:

[0209] Acquire at least two human images to be clustered; determine the credibility values ​​of various human features based on the change types to which they belong and the difference in shooting time and shooting location between the at least two human images, wherein the change types reflect the degree of change of the human features with time and space; determine a clustering credibility threshold based on the credibility values ​​of the various human features; perform portrait clustering on the at least two human images based on the similarity between the human features in the at least two human images and the clustering credibility threshold.

[0210] In addition, the logic instructions in the above-mentioned memory 303 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0211] On the other hand, the present application also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the portrait gathering method provided by the above methods, the method includes:

[0212] Acquire at least two human images to be clustered; determine the credibility values ​​of various human features based on the change types to which they belong and the difference in shooting time and shooting location between the at least two human images, wherein the change types reflect the degree of change of the human features with time and space; determine a clustering credibility threshold based on the credibility values ​​of the various human features; perform portrait clustering on the at least two human images based on the similarity between the human features in the at least two human images and the clustering credibility threshold.

[0213] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for performing the portrait gathering method provided by the above methods is implemented, and the method includes:

[0214] Acquire at least two human images to be clustered; determine the credibility values ​​of various human features based on the change types to which they belong and the difference in shooting time and shooting location between the at least two human images, wherein the change types reflect the degree of change of the human features with time and space; determine a clustering credibility threshold based on the credibility values ​​of the various human features; perform portrait clustering on the at least two human images based on the similarity between the human features in the at least two human images and the clustering credibility threshold.

[0215] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0216] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0217] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for gathering portrait images, characterized in that: include: Obtain at least two person images to be assembled; Determine the credibility value of various human features based on the change type to which the various human features belong, and the shooting time difference and shooting location difference of the at least two person images, wherein the change type reflects the degree of change of the human features with time and space; Determining a confidence threshold of the aggregate file based on the confidence values ​​of the various human features; Based on the similarity between the human body features in the at least two human images and the clustering credibility threshold, the at least two human images are clustered.

2. The portrait gathering method according to claim 1, characterized in that: The determining of the credibility values ​​of various human features based on the change types of various human features and the shooting time difference and shooting location difference of the at least two human images includes: Determine the initial credibility of various human features based on the image quality scores of the various human features, the shooting location difference and the shooting time difference; wherein the image quality scores of the various human features are determined based on the clarity of the areas related to the various human features in the at least two human images; Based on the magnitudes of the shooting time difference and the shooting location difference, selecting an influencing factor from the shooting time difference and the shooting location difference; Based on the influencing factors and the change types to which the various human body features belong, the initial credibility of the various human body features is adjusted to obtain credibility values ​​of the various human body features.

3. The portrait gathering method according to claim 2, characterized in that: The adjusting of the initial credibility of the various human features based on the influencing factors and the change types of the various human features includes: Determining the credibility level of various human features based on the influencing factors and the change types to which the various human features belong; Based on the credibility levels of the various human features, the initial credibility of the various human features is adjusted.

4. The portrait gathering method according to claim 3, characterized in that: The adjusting the initial credibility of the various human features based on the credibility levels of the various human features includes: Based on the credibility level, the linear adjustment parameter and the exponential adjustment parameter of the shooting location difference and the linear adjustment parameter and the exponential adjustment parameter of the shooting time difference included in the initial credibility are adjusted.

5. The portrait gathering method according to claim 3, characterized in that: The change types include: a first type, a second type and a third type; wherein the degree of change of the human body features corresponding to the first type, the second type and the third type decreases with time and space changes.

6. The portrait gathering method according to claim 2, characterized in that: The selecting an influencing factor from the shooting time difference and the shooting location difference based on the size of the shooting time difference and the shooting location difference includes: In a case where the shooting time difference is greater than the first time difference, and the shooting location difference is greater than the first location difference, determining the shooting time difference as the influencing factor; In a case where the shooting time difference is smaller than the second time difference, and the shooting location difference is smaller than the second location difference, the shooting location difference is determined as the influencing factor.

7. The portrait gathering method according to any one of claims 1 to 6, characterized in that: The step of determining the credibility threshold of the aggregated file based on the credibility values ​​of the various human features includes: Determining the clustering credibility threshold based on the credibility values ​​of the various human features and the credibility values ​​of the facial features; wherein the credibility value of the facial features is determined based on the clarity of the area related to the facial features in the at least two person images; The performing portrait clustering on the at least two person images based on the similarity between the human features in the at least two person images and the clustering credibility threshold comprises: Based on the similarity between the human body features and the similarity between the facial features in the at least two human images and the clustering credibility threshold, the at least two human images are clustered.

8. A portrait focusing device, characterized in that: include: An acquisition unit, used for acquiring at least two person images to be assembled; A first determining unit is used to determine the credibility values ​​of various human features based on the change types to which the various human features belong and the shooting time difference and shooting location difference of the at least two human images, wherein the change type reflects the degree of change of the human features with time and space; A second determining unit, configured to determine a confidence threshold of the aggregate file based on the confidence values ​​of the various human features; A clustering unit is used to cluster the portraits of the at least two human images based on the similarity between the human body features in the at least two human images and the clustering credibility threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the portrait gathering method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the portrait gathering method as described in any one of claims 1 to 7 is implemented.

Images