Deep learning-based barefoot footprint comparison method and device

Abstract

The invention relates to the field of footprint recognition, and discloses a barefoot footprint comparison method and device based on deep learning, and the method comprises the steps: obtaining a to-be-recognized barefoot footprint, and determining a toe region and a sole region in the to-be-recognized barefoot footprint through a footprint segmentation unit; determining a plurality of intermediate target barefoot footprints corresponding to the to-be-identified barefoot footprints in a barefoot footprint database according to the toe region through a toe region identification unit; determining a final target barefoot formation footprint corresponding to the to-be-identified barefoot footprint in the plurality of intermediate target barefoot formation footprints according to the sole region through a sole region identification unit, and taking the final target barefoot formation footprint as an identification retrieval result; wherein the toe area identification unit is obtained through training of a plurality of barefoot-footprint toe area data sets. According to the method and the device, the footprint recognition and retrieval speed in the barefoot footprint can be increased.

Description

Technical Field

[0001] The present application relates to the field of footprint recognition technology, and more specifically, to a barefoot footprint comparison method and device based on deep learning. Background Art

[0002] Footprints, in criminal science and technology, are traces left on the ground or other trace-forming objects by the weight (human body gravity) and muscle force of the suspect's feet when standing, walking, or other activities during the course of committing a crime. Footprints are an important piece of physical evidence frequently used in criminal investigations. Analysis and identification of these footprints can be used to determine a person's height, age, weight, walking posture, duration of stay, direction of walking, and more. Footprint inspection and identification technology is a crucial step in the inspection of trace evidence, possessing high evidentiary value and playing a crucial role in solving cases. However, traditional footprint comparisons rely primarily on manual labor, resulting in difficulties in extracting image features, slow recognition speeds, and low accuracy.

[0003] Patent CN118210941B (application number: CN202410325445.0) provides a cross-domain footprint pressure image retrieval system based on high-order spatiotemporal relationships. The footprint retrieval calculation process is as follows: first, the shod footprint pressure image to be queried is acquired through the acquisition window. After processing by the CPU processor, the trained optimal model is added to the system directory and the barefoot footprint pressure images in the database are loaded; then the system's metric function is used to calculate the similarity between the query image and the database image. Specifically, the query image is input into the trained network model. The backbone network in the model first extracts sequence-level features, and then processes them through the 2D block module and the spatial attention module respectively. Then, a hypergraph is constructed through the high-order spatiotemporal relationship module, and the adjacency matrix and related cosine distance are calculated to obtain the hypergraph features; finally, based on the calculated similarity, with the help of the cross-domain footprint pressure retrieval function, the barefoot footprint pressure image that is most similar to the shod footprint pressure image to be queried is found from the database, and the retrieval results are printed in the display window. When the method in patent CN118210941B recognizes and processes the pressure image of barefoot footprints, since the pressure image of barefoot footprints contains more than four footprint images, the computational complexity of identifying and retrieving barefoot footprints is large, resulting in a low recognition speed of barefoot footprints. Summary of the Invention

[0004] The purpose of this application is to provide a barefoot footprint comparison method and device based on deep learning, which solves the technical problems of large computational complexity and slow retrieval speed in footprint recognition and retrieval in barefoot footprints, and achieves the technical effect of improving the speed of footprint recognition and retrieval in barefoot footprints.

[0005] A method for barefoot footprint comparison based on deep learning provided by an embodiment of the present application, the method includes: obtaining a to-be-identified barefoot footprint, and determining a toe region and a sole region in the to-be-identified barefoot footprint through a footprint segmentation unit; through a toe region recognition unit, determining a plurality of intermediate target barefoot running footprints corresponding to the to-be-identified barefoot footprint in a barefoot running footprint database according to the toe region; through a sole region recognition unit, determining a final target barefoot running footprint corresponding to the to-be-identified barefoot footprint among the plurality of intermediate target barefoot running footprints according to the sole region, and taking the final target barefoot running footprint as an identification retrieval result; wherein, the toe region recognition unit is trained through a toe region data set of a plurality of barefoot running footprints, and the sole region recognition unit is trained through a sole region data set of a plurality of barefoot running footprints.

[0006] In a possible implementation manner, the method further includes: determining the toe region similarity corresponding to the to-be-identified barefoot footprint and each intermediate target barefoot footprint among the plurality of intermediate target barefoot running footprints respectively in the barefoot running footprint database according to the toe region, and determining the variance value of the plurality of toe region similarities corresponding to each intermediate target barefoot running footprint respectively; through the sole region recognition unit, determining the final target barefoot running footprint corresponding to the to-be-identified barefoot footprint among the plurality of intermediate target barefoot running footprints in turn according to the ascending order of the variance values of the plurality of toe region similarities corresponding to the plurality of intermediate target barefoot running footprints respectively according to the sole region.

[0007] In another possible implementation manner, determining the final target barefoot running footprint corresponding to the to-be-identified barefoot footprint among the plurality of intermediate target barefoot running footprints through the sole region recognition unit according to the sole region includes: determining the sole region similarity between the to-be-identified barefoot footprint and each intermediate target barefoot footprint among the plurality of intermediate target barefoot running footprints respectively through the sole region recognition unit according to the sole region, and determining the variance value of the plurality of sole region similarities corresponding to each intermediate target barefoot running footprint respectively; determining the minimum value among the variance values of the plurality of sole region similarities corresponding to the plurality of intermediate target barefoot running footprints respectively, and taking the intermediate target barefoot running footprint corresponding to the minimum value among the variance values of the plurality of sole region similarities as the final target barefoot running footprint.

[0008] In another possible implementation, the method further includes: obtaining the footprint environmental feature corresponding to each intermediate target barefoot footprint of each intermediate target barefoot in-trip footprint, and obtaining the footprint influence factor corresponding to the footprint environmental feature; multiplying the toe region similarity corresponding to each intermediate target barefoot footprint of multiple intermediate target barefoot in-trip footprints by the footprint influence factor to adjust the toe region similarity corresponding to each intermediate target barefoot footprint of multiple intermediate target barefoot in-trip footprints; multiplying the sole region similarity corresponding to each intermediate target barefoot footprint of multiple intermediate target barefoot in-trip footprints by the footprint influence factor to adjust the sole region similarity corresponding to each intermediate target barefoot footprint of multiple intermediate target barefoot in-trip footprints.

[0009] In another possible implementation, the method further includes: obtaining the toe region influence factor and the sole region influence factor corresponding to each intermediate target barefoot footprint of each intermediate target barefoot in-trip footprint, and determining the difference between the toe region influence factor and the sole region influence factor corresponding to each intermediate target barefoot footprint as the region influence difference; wherein, the toe region influence factor represents the footprint influence feature of the footprint environmental feature on the toe region, and the sole region influence factor represents the footprint influence feature of the footprint environmental feature on the sole region; when determining the variance value of the multiple toe region similarities corresponding to each intermediate target barefoot in-trip footprint, deleting the toe region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference; when determining the variance value of the multiple sole region similarities corresponding to each intermediate target barefoot in-trip footprint, deleting the sole region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference.

[0010] In another possible implementation, the method further includes: determining the environmental influence factor corresponding to each of the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trip footprint, and determining the mean value of the environmental influence factors corresponding to each of the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trip footprint as the in-trip footprint influence factor mean value; wherein, the environmental influence factor represents the overall influence degree of the ground material and the ground humidity on the footprint; determining the product of the variance value of the multiple toe region similarities corresponding to each intermediate target barefoot in-trip footprint and the in-trip footprint influence factor mean value to adjust the variance value of the multiple toe region similarities; determining the product of the variance value of the multiple sole region similarities corresponding to each intermediate target barefoot in-trip footprint and the in-trip footprint influence factor mean value to adjust the variance value of the multiple sole region similarities.

[0011] In another possible implementation, the method further includes: determining the area of each toe sub-region corresponding to each toe in the toe region of the barefoot footprint to be recognized; determining the ratio of the minimum area value to the maximum area value of the toe sub-region areas as the confidence of the toe region, and obtaining the number of consecutive footprints corresponding to the confidence of the toe region. Determining multiple intermediate target barefoot consecutive footprints with the number of consecutive footprints in ascending order of the variance values of the multiple toe region similarities corresponding to the multiple intermediate target barefoot consecutive footprints; through the sole region recognition unit, according to the sole region, determining the final target barefoot consecutive footprint corresponding to the barefoot footprint to be recognized among the multiple intermediate target barefoot consecutive footprints with the number of consecutive footprints, and using the final target barefoot consecutive footprint as the recognition and retrieval result.

[0012] In another possible implementation, the method further includes: when the minimum area value among the multiple toe sub-region areas is less than the preset minimum area value, through the barefoot footprint to be recognized evaluation unit, determining the number of consecutive footprints according to the multiple toe sub-region areas of the barefoot footprint to be recognized; determining multiple intermediate target barefoot consecutive footprints with the number of consecutive footprints in ascending order of the variance values of the multiple toe region similarities corresponding to the multiple intermediate target barefoot consecutive footprints; through the sole region recognition unit, according to the sole region, determining the final target barefoot consecutive footprint corresponding to the barefoot footprint to be recognized among the multiple intermediate target barefoot consecutive footprints with the number of consecutive footprints, and using the final target barefoot consecutive footprint as the recognition and retrieval result.

[0013] In another possible implementation, determining the number of consecutive footprints according to the multiple toe sub-region areas of the barefoot footprint to be recognized through the barefoot footprint to be recognized evaluation unit includes: through the toe position recognition unit, determining the toe position information corresponding to the areas of the multiple toe sub-regions according to the multiple toe sub-regions; through the barefoot footprint to be recognized evaluation unit, determining the number of consecutive footprints according to the multiple toe sub-region areas of the barefoot footprint to be recognized and the toe position information corresponding to the areas of the multiple toe sub-regions.

[0014] The embodiment of the present application also provides a barefoot footprint comparison device based on deep learning, including a unit for executing the method described in any one of the above.

[0015] The beneficial effects of the embodiment of the present application compared with the prior art are:

[0016] An embodiment of the present application provides a barefoot footprint comparison method based on deep learning. The method includes: obtaining a barefoot footprint to be recognized, and determining a toe region and a sole region in the barefoot footprint to be recognized through a footprint segmentation unit; through a toe region recognition unit, determining multiple intermediate target barefoot in-line footprints corresponding to the barefoot footprint to be recognized in a barefoot in-line footprint database according to the toe region; through a sole region recognition unit, determining a final target barefoot in-line footprint corresponding to the barefoot footprint to be recognized from the multiple intermediate target barefoot in-line footprints according to the sole region, and taking the final target barefoot in-line footprint as the recognition retrieval result; wherein, the toe region recognition unit is trained through a toe region data set of multiple barefoot in-line footprints, and the sole region recognition unit is trained through a sole region data set of multiple barefoot in-line footprints. In the embodiment of the present application, the retrieval speed of four or more footprint images in a barefoot in-line footprint image can be improved, and the retrieval effect of the barefoot in-line footprint image is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of the first barefoot footprint comparison method based on deep learning provided by the embodiment of the present application;

[0019] Figure 2 It is a schematic working flowchart of the first barefoot footprint comparison method based on deep learning provided by the embodiment of the present application;

[0020] Figure 3 It is a schematic diagram for determining the toe region and the sole region in the barefoot footprint to be recognized in the embodiment of the present application;

[0021] Figure 4 It is a schematic flowchart of the second barefoot footprint comparison method based on deep learning provided by the embodiment of the present application;

[0022] Figure 5 It is a schematic flowchart of the third barefoot footprint comparison method based on deep learning provided by the embodiment of the present application;

[0023] Figure 6 It is a schematic logical structure diagram of a barefoot footprint comparison device based on deep learning provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations.

[0025] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] As used in the specification of the present application and the appended claims, the term "if" may be interpreted, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0027] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are used only for differentiating descriptions and cannot be understood as indicating or implying relative importance.

[0028] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] In the existing cross-domain in-step footprint retrieval methods, when identifying and processing the barefoot in-step footprint pressure image, since there are more than four footprint images in the barefoot in-step footprint pressure image, the computational amount for identifying and retrieving the barefoot in-step footprint is relatively large, resulting in a relatively low recognition speed of the barefoot in-step footprint.

[0030] For the above reasons, the embodiment of the present application provides a barefoot footprint comparison method based on deep learning. The method includes: obtaining a to-be-identified barefoot footprint, and determining the toe region and the sole region in the to-be-identified barefoot footprint through a footprint segmentation unit; through a toe region recognition unit, determining a plurality of intermediate target barefoot in-step footprints corresponding to the to-be-identified barefoot footprint in a barefoot in-step footprint database according to the toe region; through a sole region recognition unit, determining the final target barefoot in-step footprint corresponding to the to-be-identified barefoot footprint from the plurality of intermediate target barefoot in-step footprints according to the sole region, and taking the final target barefoot in-step footprint as the recognition and retrieval result; wherein, the toe region recognition unit is trained through a toe region data set of a plurality of barefoot in-step footprints, and the sole region recognition unit is trained through a sole region data set of a plurality of barefoot in-step footprints. In the embodiment of the present application, the retrieval speed of four or more footprint images in a barefoot in-step footprint image can be improved, and the retrieval effect of the barefoot in-step footprint image is improved.

[0031] In some scenarios, a barefoot footprint comparison method based on deep learning in the embodiment of the present application can be applied to the retrieval of barefoot in-step footprint images in criminal science and technology, and can improve the auxiliary effect on criminal cases involving barefoot in-step footprint images.

[0032] The following specifically describes a barefoot footprint comparison method based on deep learning provided by the embodiment of the present application with specific examples.

[0033] Figure 1 It is a schematic flowchart of the first barefoot footprint comparison method based on deep learning provided by the embodiment of the present application. As Figure 1 shown, the above method includes S110 to S120, and the following specifically describes S110 to S120.

[0034] S110. Obtain a to-be-identified barefoot footprint, and determine the toe region and the sole region in the to-be-identified barefoot footprint through a footprint segmentation unit.

[0035] Figure 2 It is a schematic working flowchart of the first barefoot footprint comparison method based on deep learning provided by the embodiment of the present application. As Figure 2 shown, in the embodiment of the present application, after obtaining the to-be-identified barefoot footprint, in order to efficiently retrieve the to-be-identified barefoot footprint to be retrieved, the toe region and the sole region in the to-be-identified barefoot footprint can be first determined through a footprint segmentation unit, and the recognition of the toe region and the sole region can be further realized.

[0036] Figure 3 It is a schematic diagram for determining the toe region and the sole region in the to-be-identified barefoot footprint in the embodiment of the present application. As Figure 3As shown in the figure, the toe region 11 and the sole region 12 can be determined in the barefoot footprint 1 to be recognized through the footprint segmentation unit. The area and pixel content of the toe region 11 are less, and the information of the toe region 11 can comprehensively reflect the information of the toes in the footprint. Furthermore, the barefoot footprint to be recognized can be preliminarily retrieved according to the toe region 11 with richer information and smaller image size. At the same time, the barefoot footprint to be recognized can be further retrieved and recognized according to the sole region 12, reducing the amount of calculation and realizing the high-speed retrieval and recognition of the barefoot footprint to be recognized.

[0037] Exemplarily, the original pressure image of the barefoot footprint to be recognized can be first collected by a high-precision optical sensor and input into the footprint segmentation unit. The footprint segmentation unit uses a pre-trained U-Net network model to extract the local and global features of the footprint through multi-scale convolutional layers and outputs the binary masks of the toe region and the sole region. Among them, the toe region is defined as the contour range from the front end of the footprint to the first metatarsophalangeal joint, and the sole region covers the continuous pressure distribution area from the arch to the heel.

[0038] S120. Through the toe region recognition unit, according to the toe region, a plurality of intermediate target barefoot gait footprints corresponding to the barefoot footprint to be recognized are determined in the barefoot gait footprint database. Through the sole region recognition unit, according to the sole region, the final target barefoot gait footprint corresponding to the barefoot footprint to be recognized is determined among the plurality of intermediate target barefoot gait footprints, and the final target barefoot gait footprint is used as the recognition and retrieval result. Among them, the toe region recognition unit is trained through the toe region data sets of a plurality of barefoot gait footprints, and the sole region recognition unit is trained through the sole region data sets of a plurality of barefoot gait footprints.

[0039] After obtaining the toe region and the sole region, the toe region recognition unit can be used to determine a plurality of intermediate target barefoot gait footprints corresponding to the barefoot footprint to be recognized in the barefoot gait footprint database according to the toe region, thereby realizing the purpose of preliminarily retrieving the footprint according to the toe region to obtain a plurality of intermediate target barefoot gait footprints. The plurality of intermediate target barefoot gait footprints are a plurality of alternative intermediate target barefoot gait footprints obtained by recognizing through the toe region.

[0040] Exemplarily, the segmented toe region image can be input into the toe region recognition unit. The toe region recognition unit is constructed based on the ResNet-50 network, and its training data is the toe region image set marked in the barefoot gait footprint database, including the dynamic change characteristics of toe pressure under different gait cycles. The toe region recognition unit calculates the feature similarity (using cosine similarity measurement) between the toe region to be recognized and the toe regions of each gait footprint in the database, and screens the top N gait footprints with the highest similarity as the intermediate target barefoot gait footprints.

[0041] After obtaining multiple intermediate target barefoot walking footprints, the sole area recognition unit can determine the final target barefoot walking footprint corresponding to the barefoot footprint to be recognized among the multiple intermediate target barefoot walking footprints according to the sole area, and use the final target barefoot walking footprint as the recognition and retrieval result to achieve the retrieval and positioning of the final target barefoot walking footprint.

[0042] Exemplarily, the sole area data of the intermediate target barefoot walking footprint can be input into the sole area recognition unit. The sole area recognition unit adopts a VGG-16 network structure, and its training data is a set of sole area images aligned with the toe area dataset, focusing on learning stability features such as arch shape and pressure peak distribution. The sole area recognition unit compares the sole area to be recognized with the sole area features of the intermediate target walking footprint, combines the dynamic time warping (DTW) algorithm to align the gait sequence, and finally outputs the walking footprint with the highest similarity as the final target barefoot walking footprint.

[0043] Exemplarily, when determining the final target barefoot walking footprint corresponding to the barefoot footprint to be recognized among the multiple intermediate target barefoot walking footprints, score values corresponding to the multiple intermediate target barefoot walking footprints can be obtained, and the intermediate target barefoot walking footprint with the highest score value is used as the final target barefoot walking footprint.

[0044] It should be noted that the toe area recognition unit is trained through the toe area datasets of multiple barefoot walking footprints, and the sole area recognition unit is trained through the sole area datasets of multiple barefoot walking footprints, so as to further retrieve and recognize the distribution of the final target barefoot walking footprint.

[0045] It should be noted that when comparing the barefoot footprint to be recognized with the barefoot walking footprint, when the barefoot footprint to be recognized is a left foot footprint, the left foot footprint in the barefoot walking footprint can be retrieved separately; when the barefoot footprint to be recognized is a right foot footprint, the right foot footprint in the barefoot walking footprint can be retrieved separately.

[0046] The beneficial effect of the above implementation method is that the barefoot footprint to be recognized is initially retrieved based on the toe area with richer information and smaller image size, and at the same time, the barefoot footprint to be recognized can be further retrieved and recognized according to the sole area, reducing the calculation amount. First, the intermediate targets are quickly screened through the toe area, and then the fine matching is performed through the sole area, taking into account both the retrieval efficiency and accuracy, and achieving the high-speed retrieval and recognition of the barefoot footprint to be recognized.

[0047] The beneficial effect of the above implementation method is also that by independently training the toe and sole area recognition units, the feature optimization is respectively carried out for the toe area sensitive to dynamic changes and the sole area with stronger stability, avoiding the problem of information mixing of a single feature model.

[0048] Figure 4 This is a flowchart of the second barefoot footprint comparison method based on deep learning provided by an embodiment of the present application. As shown in Figure 4 shown, the above method further includes S210 to S220, and the following is a specific description of S210 to S220.

[0049] S210. According to the toe region, determine the similarity of the toe region corresponding to the to-be-identified barefoot footprint and each intermediate target barefoot footprint in the barefoot footprint database for multiple intermediate target barefoot footprints, and determine the variance value of the multiple toe region similarities corresponding to each intermediate target barefoot footprint.

[0050] After determining multiple intermediate target barefoot footprints through the toe region recognition unit, for each intermediate target footprint, extract the toe region features of all single-step barefoot footprints it contains, and based on the pre-trained Siamese network model, calculate the similarity of the toe region of the to-be-identified barefoot footprint and the toe region of each single-step footprint in the footprint one by one. Use the normalized Euclidean distance as the similarity index, which is used as the toe region similarity, to calculate the similarity of the toe region corresponding to the to-be-identified barefoot footprint and each intermediate target barefoot footprint in multiple intermediate target barefoot footprints respectively.

[0051] After obtaining the multiple toe region similarities corresponding to each intermediate target barefoot footprint, for each intermediate target barefoot footprint, calculate the variance value of the toe region similarity values of all single-step footprints contained in each intermediate target barefoot footprint.

[0052] S220. Through the sole region recognition unit, according to the sole region, in the order from small to large of the variance values of the multiple toe region similarities corresponding to the multiple intermediate target barefoot footprints, sequentially determine the final target barefoot footprint corresponding to the to-be-identified barefoot footprint in the multiple intermediate target barefoot footprints.

[0053] After obtaining the variance values of the toe region similarity values of all single-step footprints contained in each intermediate target barefoot footprint, the multiple intermediate target footprints can be arranged in ascending order (from small to large) according to the toe region similarity variance value to generate a priority queue. Further, the sole region recognition unit can be processed in this order, and the footprint with the smallest variance value can be preferentially matched.

[0054] The beneficial effect of the above implementation method is that high-consistency footprints are screened through the variance of toe region similarities, and the candidate set with low variance is preferentially processed, reducing the amount of ineffective calculation, and significantly reducing the retrieval time.

[0055] The beneficial effects of the above implementation are also that the variance index can effectively filter out abnormal similarity fluctuations caused by local deformation of single-step footprints (such as toe curling), and improve the matching stability of the stride cycle in footprint retrieval.

[0056] In some implementations, in the above S120, through the sole area recognition unit, according to the sole area, the final target barefoot whole-track footprint corresponding to the to-be-recognized barefoot footprint is determined from multiple intermediate target barefoot whole-track footprints, including S121 to S122. The following is a specific description of S121 to S122.

[0057] S121: Through the sole area recognition unit, according to the sole area, determine the sole area similarity between the to-be-recognized barefoot footprint and each intermediate target barefoot footprint of each intermediate target barefoot whole-track footprint, and determine the variance value of the multiple sole area similarities corresponding to each intermediate target barefoot whole-track footprint respectively.

[0058] When performing sole area recognition, for each intermediate target barefoot whole-track footprint, the sole area features of all single-step footprints included therein can also be extracted. The sole area recognition unit can adopt a pre-trained Inception-v3 network model to perform multi-scale feature fusion on the arch shape, peak pressure distribution, and heel contact area of each single-step footprint to generate a 128-dimensional feature vector, and the 128-dimensional feature vector is the sole area similarity.

[0059] Exemplarily, by comparing the feature vector of the to-be-recognized sole area with the feature vector of the single-step sole area in the database, the normalized cross-correlation (NCC) similarity can be calculated to implement the calculation of the sole area similarity between the to-be-recognized barefoot footprint and each intermediate target barefoot footprint of each intermediate target barefoot whole-track footprint.

[0060] When performing sole area recognition, for each intermediate target whole-track footprint, the variance value of all single-step sole area similarity values can also be statistically calculated to implement the evaluation of each intermediate target barefoot whole-track footprint according to the variance value of the sole area similarity value.

[0061] Exemplarily, the sliding window method can be used to dynamically calculate the variance. The window size is 50% of the whole-track footprint length, the step size is 1 frame, and the final variance value is the median of all window variances to resist local fluctuation interference.

[0062] S122: Determine the minimum value among the variance values of the multiple sole area similarities corresponding to each intermediate target barefoot whole-track footprint respectively, and use the intermediate target barefoot whole-track footprint corresponding to the minimum value among the variance values of the multiple sole area similarities as the final target barefoot whole-track footprint.

[0063] After obtaining the variance values of the similarity of the sole regions of all intermediate target footstep sequences, the variance values of the similarity of the sole regions of all intermediate target footstep sequences can be traversed. A high variance indicates that there are significant dynamic matching features (such as arch elastic deformation and gait phase synchronization) in the sole region of the footstep sequence, which is more in line with the biomechanical pattern of real barefoot walking. Furthermore, the footstep sequence corresponding to the maximum value among the variance values of the similarity of multiple sole regions can be selected as the final target.

[0064] The beneficial effect of the above implementation method is that by screening footstep sequences with high consistency through the variance of the sole region similarity, preferentially processing the candidate set with low variance, reducing the amount of invalid calculations, and significantly reducing the retrieval time consumption.

[0065] The beneficial effect of the above implementation method is also that the combination of sliding window variance calculation and median statistics effectively suppresses the influence of single-frame noise (such as heel slippage) on the overall variance evaluation, and can greatly reduce the false detection rate in interference scenarios.

[0066] Figure 5 This is a schematic flowchart of the third barefoot footprint comparison method based on deep learning provided by the embodiments of the present application. As Figure 5 shown, the above method further includes S310 to S320, and the following is a specific description of S310 to S320.

[0067] S310. Obtain the footprint environment features corresponding to each intermediate target barefoot footprint of each intermediate target barefoot footstep sequence, and obtain the footprint influence factors corresponding to the footprint environment features.

[0068] In footprint recognition, the environment may affect the footprint. Therefore, for each single-step footprint in each intermediate target barefoot footstep sequence, the footprint environment features corresponding to the intermediate target barefoot footstep sequence can be obtained through the barefoot footstep sequence database, and the footprint can be retrieved according to the footprint environment features.

[0069] Exemplarily, the footprint environment features may include the type of ground material, environmental temperature and humidity, and light intensity, and the footprint environment features can be synchronously recorded when the footprint is recorded.

[0070] After obtaining the footprint environment features, the footprint influence factors corresponding to the footprint environment features can be obtained. The footprint influence factors can be obtained by inputting the three types of features into a pre-trained environment influence factor calculation model, and the output range of the footprint influence factors is [0.8, 1.2]. The calculation formula can be: influence factor = 0.8 + 0.4·σ(α·material coefficient + β·humidity normalization value + γ·light attenuation rate), where σ is the Sigmoid function, and α = 0.6, β = 0.3, γ = 0.1 are weight parameters.

[0071] S320. Multiply the toe region similarity corresponding to each intermediate target barefoot in-step footprint by the footprint influence factor to adjust the toe region similarity corresponding to each intermediate target barefoot in-step footprint among the multiple intermediate target barefoot in-step footprints. Multiply the sole region similarity corresponding to each intermediate target barefoot in-step footprint by the footprint influence factor to adjust the sole region similarity corresponding to each intermediate target barefoot in-step footprint among the multiple intermediate target barefoot in-step footprints.

[0072] After obtaining the footprint influence factor corresponding to the footprint environmental feature, dynamic adjustment of the similarity of each single-step footprint of each intermediate target barefoot in-step footprint can be performed. During the adjustment, the toe region similarity and the sole region similarity can be multiplied by the influence factor of the corresponding footprint respectively to adjust the toe region similarity corresponding to each intermediate target barefoot in-step footprint among the multiple intermediate target barefoot in-step footprints, and to adjust the sole region similarity corresponding to each intermediate target barefoot in-step footprint among the multiple intermediate target barefoot in-step footprints. After the adjustment, normalization processing can also be performed on the adjusted toe region similarity and sole region similarity.

[0073] After obtaining the adjusted toe region similarity and sole region similarity, the variance value of each intermediate target in-step footprint can be recalculated based on the adjusted toe region similarity and sole region similarity. The variance value of the toe region similarity is used to update the above-mentioned priority queue of the toe region, and the minimum variance value of the sole region similarity is used to re-screen the final target in-step footprint to form a closed-loop optimization.

[0074] The beneficial effect of the above implementation method is that by dynamically correcting the similarity of the footprint through the footprint influence factor, it is possible to eliminate the feature distortion caused by environmental factors such as slippery ground and too high hardness, and improve the retrieval accuracy in complex environments.

[0075] The beneficial effect of the above implementation method is also that after fusing environmental features such as material, light, and humidity, the cross-domain retrieval error rate of the system in different material scenarios such as cement floor, carpet, and sand ground decreases.

[0076] The beneficial effect of the above implementation method is also that by adopting a separate environmental factor calculation model and only performing scalar multiplication adjustment at the single-step footprint level, the impact on the overall calculation cost is small.

[0077] In some implementation methods, the above method further includes S330 to S340, which are specifically described below.

[0078] S330. Obtain the toe region influence factor and the sole region influence factor corresponding to each intermediate target barefoot footprint of each intermediate target barefoot in-step footprint, and determine the difference between the toe region influence factor and the sole region influence factor corresponding to each intermediate target barefoot footprint as the region influence difference. Among them, the toe region influence factor represents the footprint influence characteristics of the footprint environment characteristics on the toe region, and the sole region influence factor represents the footprint influence characteristics of the footprint environment characteristics on the sole region.

[0079] For each single-step footprint of each intermediate target barefoot in-step footprint, the absolute difference between its toe region influence factor and sole region influence factor can also be calculated as the region influence difference, and the region influence difference represents the difference in the environmental influence degree between the toe region and the sole region.

[0080] It should be noted that the toe region influence factor represents the footprint influence characteristics of the footprint environment characteristics on the toe region, and the toe region influence factor can be determined manually through the environmental characteristics of the toe region. The sole region influence factor represents the footprint influence characteristics of the footprint environment characteristics on the sole region, and the sole region influence factor can be determined manually through the environmental characteristics of the sole region.

[0081] S340. When determining the variance value of the multiple toe region similarities corresponding to each intermediate target barefoot in-step footprint, delete the toe region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference. When determining the variance value of the multiple sole region similarities corresponding to each intermediate target barefoot in-step footprint, delete the sole region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference.

[0082] After obtaining the region influence difference, when determining the variance value of the multiple toe region similarities corresponding to each intermediate target barefoot in-step footprint, the toe region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference can be deleted to exclude the toe region similarities of the intermediate target barefoot footprints with too high environmental influence degree.

[0083] After obtaining the region influence difference, when determining the variance value of the multiple sole region similarities corresponding to each intermediate target barefoot in-step footprint, the sole region similarities of the intermediate target barefoot footprints whose region influence difference is greater than or equal to the preset region influence difference can be deleted to exclude the sole region similarities of the intermediate target barefoot footprints with too high environmental influence degree.

[0084] After excluding the similarity of the toe area of the intermediate target barefoot footprint with too high environmental influence and the similarity of the sole area of the intermediate target barefoot footprint with too high environmental influence, based on the filtered effective similarity data, a priority queue of the intermediate target in-trail footprint can be regenerated, and the above-mentioned maximum sole variance matching process can be executed.

[0085] Exemplarily, the preset regional influence difference can be dynamically calibrated through a cross-scenario validation set, and its value range is [0.3, 0.6]. When the regional influence difference is greater than or equal to the preset regional influence difference, it is determined that there is regional-specific environmental interference in the footprint (for example, only the toes sink into the sand while the soles touch the hard ground).

[0086] The beneficial effect of the above implementation is that through the filtering of the regional influence difference, the abnormal interference of the local environment on specific foot areas (such as water stains affecting the toe area and the sole area being dry) can be effectively eliminated, the variance calculation error in complex scenarios can be reduced, and the accuracy of footprint retrieval can be improved.

[0087] In some implementation manners, the above method further includes S410 to S420, and the following is a specific description of S410 to S420.

[0088] S410. Determine the environmental influence factors corresponding to the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint, and determine the mean value of the environmental influence factors corresponding to the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint as the in-trail footprint influence factor mean value. Among them, the environmental influence factor characterizes the overall influence degree of the ground material and the ground humidity on the footprint.

[0089] When performing footprint retrieval, the environmental influence factors corresponding to the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint can also be determined, and the environmental influence factor characterizes the overall influence degree of the environment on the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint.

[0090] Exemplarily, the environmental influence factor corresponding to the environment where the intermediate target barefoot in-trail footprint is located can be determined through empirical values.

[0091] After obtaining the environmental influence factors corresponding to the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint, the mean value of the environmental influence factors corresponding to the multiple intermediate target barefoot footprints of each intermediate target barefoot in-trail footprint can be further determined as the in-trail footprint influence factor mean value, and the in-trail footprint influence factor mean value characterizes the comprehensive influence intensity of the environment where the entire in-trail footprint is located.

[0092] S420. Determine the product of the variance value of the similarities of multiple toe regions corresponding to each intermediate target barefoot in - step footprint and the mean value of the in - step footprint influence factor to adjust the variance value of the similarities of multiple toe regions. Determine the product of the variance value of the similarities of multiple sole regions corresponding to each intermediate target barefoot in - step footprint and the mean value of the in - step footprint influence factor to adjust the variance value of the similarities of multiple sole regions.

[0093] After obtaining the mean value of the in - step footprint influence factor, the variance of the toe - region similarity and the variance of the sole - region similarity of the in - step footprint can be multiplied by the mean value of the influence factor respectively to achieve environment - adaptive variance calibration. Subsequently, based on the adjusted variance values, the above - mentioned processes of prioritizing the intermediate target barefoot footprints and screening the final targets of the intermediate target barefoot footprints can be re - carried out to form a closed - loop optimization system for environment perception.

[0094] The beneficial effect of the above implementation method is that, through the mean - valued global environment influence factor, the variance index of the in - step footprint is uniformly calibrated, which can improve the retrieval accuracy in special scenarios such as sandy land and ice surface.

[0095] The beneficial effect of the above implementation method is also that the scaling mechanism of the variance value enables the system to automatically adapt to the interference intensity of the environment, reduces the false - matching rate in high - humidity environments, and reduces the missed - detection rate on hard ground.

[0096] In some implementation methods, the above - mentioned method further includes S510 to S520, which are specifically described below.

[0097] S510. Determine the area of each toe - sub - region corresponding to each toe in the toe region of the barefoot footprint to be recognized.

[0098] When performing footprint retrieval, the toe region of the barefoot footprint to be recognized can be finely segmented. Specifically, an improved Mask R - CNN model can be used. Based on the binary mask, the independent sub - region contours of the five toes are identified through a skeletonization algorithm. The area of each toe - sub - region can be calculated by pixel counting and recorded as A1, A2,..., A5.

[0099] S520. Determine the ratio of the minimum area value to the maximum area value of the toe - sub - region areas as the toe - region confidence level, and obtain the number of in - step footprints corresponding to the toe - region confidence level. According to the ascending order of the variance values of the similarities of multiple toe regions corresponding to multiple intermediate target barefoot in - step footprints, determine multiple intermediate target barefoot in - step footprints with the number of in - step footprints. Through the sole - region recognition unit, according to the sole region, determine the final target barefoot in - step footprint corresponding to the barefoot footprint to be recognized among the intermediate target barefoot in - step footprints with the number of in - step footprints, and take the final target barefoot in - step footprint as the recognition and retrieval result.

[0100] After obtaining the area of each toe sub-region, the maximum value Amax and the minimum value of the toe sub-region area can be extracted. And determine the ratio of the minimum area value to the maximum area value of the toe sub-region area as the toe region confidence level, which characterizes the accuracy of initially retrieving footprints through the toe region.

[0101] After obtaining the toe region confidence level, the number of consecutive footprints corresponding to the toe region confidence level can be obtained. Different toe region confidence levels correspond to different numbers of consecutive footprints. Subsequently, according to the principle of ascending toe region variance, the intermediate target barefoot consecutive footprints matching the number of consecutive footprints can be intercepted from the priority queue of the intermediate target barefoot consecutive footprints.

[0102] Exemplarily, when the number of consecutive footprints corresponding to the toe region confidence level is 20%, the first 20 intermediate target barefoot consecutive footprints can be selected from 100 intermediate target barefoot consecutive footprints.

[0103] The beneficial effect of the above implementation is that by adaptively adjusting the candidate quantity through the toe confidence level, a large amount of inefficient calculation can be greatly reduced while ensuring the accuracy.

[0104] The beneficial effect of the above implementation is also that the toe area ratio index can effectively identify abnormal situations such as missing toes and adhesions, and can reduce the false matching rate in the scenario of foot defects.

[0105] In some implementations, the above method further includes S530 to S540, which will be specifically described below.

[0106] S530. When the minimum area value among the areas of multiple toe sub-regions is less than the preset minimum area value, the number of consecutive footprints is determined by the to-be-identified barefoot footprint evaluation unit according to the areas of multiple toe sub-regions of the to-be-identified barefoot footprint.

[0107] After obtaining the minimum area value among the areas of multiple toe sub-regions, the preset minimum area value Amin can be obtained. When the minimum area A of the toe sub-region of the to-be-identified footprint is less than Amin, it indicates that the recognition may be inaccurate due to the too small toe sub-region of the to-be-identified footprint. Therefore, the to-be-identified barefoot footprint evaluation unit can be triggered to adjust the number of consecutive footprints to improve the accuracy of footprint retrieval.

[0108] Exemplarily, the preset minimum area value Amin can be 120 pixels (corresponding to an actual size of 3 cm 2 ).

[0109] When determining the number of footstep sequences, the number of footstep sequences can be determined by the barefoot footprint evaluation unit to be recognized according to the areas of multiple toe sub-regions of the barefoot footprint to be recognized.

[0110] S540. Determine multiple intermediate target barefoot footstep sequences of the number of footstep sequences in ascending order of the variance values of the similarities of multiple toe regions corresponding to multiple intermediate target barefoot footstep sequences. Through the sole region recognition unit, according to the sole region, determine the final target barefoot footstep sequence corresponding to the barefoot footprint to be recognized among the intermediate target barefoot footstep sequences of the number of footstep sequences, and use the final target barefoot footstep sequence as the recognition retrieval result.

[0111] Similarly, after obtaining the number of footstep sequences, the above method can be used to determine multiple intermediate target barefoot footstep sequences of the number of footstep sequences, and the final target barefoot footstep sequence corresponding to the barefoot footprint to be recognized can be determined among the intermediate target barefoot footstep sequences of the number of footstep sequences.

[0112] The beneficial effect of the above implementation is that by triggering dynamic quantity adjustment through the minimum area threshold, the candidate set scale can be adjusted in case of abnormalities such as broken toes and curling, improving the accuracy of footprint retrieval.

[0113] In some implementation manners, in S530 above, by the barefoot footprint evaluation unit to be recognized, according to the areas of multiple toe sub-regions of the barefoot footprint to be recognized, determining the number of footstep sequences includes S531 to S532, and the following will specifically describe S531 to S532.

[0114] S531. Through the toe position recognition unit, according to multiple toe sub-regions, determine the toe position information corresponding to the areas of multiple toe sub-regions respectively.

[0115] When retrieving footprints, the toe position information corresponding to the areas of multiple toe sub-regions can also be determined according to multiple toe sub-regions, and then the toe sub-regions after segmentation can be spatially located through the toe position recognition unit. Specifically, a key point detection model based on HRNet can be used to mark the three-dimensional coordinates (xi, yi, pi) at the center of each toe sub-region. The toe position information can be represented by the three-dimensional coordinates (xi, yi, pi), where xi, yi are the image plane coordinates and pi is the pressure peak intensity value.

[0116] S532. Through the barefoot footprint evaluation unit to be recognized, according to the areas of multiple toe sub-regions of the barefoot footprint to be recognized and the toe position information corresponding to the areas of multiple toe sub-regions respectively, determine the number of footstep sequences.

[0117] After obtaining the toe position information, the barefoot footprint evaluation unit to be recognized can determine the number of footprints in a series according to the areas of multiple toe sub-regions of the barefoot footprint to be recognized and the toe position information corresponding to the areas of the multiple toe sub-regions respectively, so as to realize the judgment of the reliability of the toe region based on the areas of the multiple toe sub-regions and the toe position information, and improve the retrieval accuracy of the footprint.

[0118] Exemplarily, a toe space distribution matrix can be constructed. The toe space distribution matrix converts the position coordinates of the five toes into a normalized vector relative to the heel origin as the toe space distribution matrix. The barefoot footprint evaluation unit to be recognized uses a multi-head attention mechanism to process the areas of multiple toe sub-regions and the toe position information corresponding to the areas of the multiple toe sub-regions respectively to output the number of footprints in a series.

[0119] The beneficial effect of the above implementation manner is that by effectively capturing the biomechanical correlation between toes (such as the big toe dominant effect), the number prediction error rate can be reduced.

[0120] The beneficial effect of the above implementation manner is also that through the weighted fusion mechanism of area and position features, the candidate number can adapt to the changes of footprint atrophy, offset and rotation, and the footprint retrieval accuracy under complex deformations can be improved.

[0121] The embodiment of the present application also provides a barefoot footprint comparison device based on deep learning, including a unit for executing the method described in any one of the above.

[0122] Figure 6 FIG. is a schematic logical structure diagram of a barefoot footprint comparison device based on deep learning provided by an embodiment of the present application. As Figure 6 shown, the device 2 of this embodiment includes a processing unit 21, a storage unit 22, and a transceiver unit 23. The processing unit 21 is used to process data, the storage unit 22 is used to store data, and the transceiver unit 23 is used to send and receive data. The processing unit 21, the storage unit 22, and the transceiver unit 23 cooperate with each other to implement the above method. The beneficial effects of the embodiment of the present application have been described in the above method and will not be repeated here.

[0123] It should be noted that the information interaction, execution process, etc. between the above devices / units, due to being based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part specifically, and will not be repeated here.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0125] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0126] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0128] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0129] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A barefoot footprint comparison method based on deep learning, characterized in that, The method includes: Obtain the barefoot footprint to be recognized, and determine the toe area and sole area in the barefoot footprint to be recognized through a footprint segmentation unit; Through a toe area recognition unit, based on the toe area, determine multiple intermediate target barefoot in-step footprints corresponding to the barefoot footprint to be recognized in a barefoot in-step footprint database; through a sole area recognition unit, based on the sole area, determine the final target barefoot in-step footprint corresponding to the barefoot footprint to be recognized from the multiple intermediate target barefoot in-step footprints, and use the final target barefoot in-step footprint as the recognition retrieval result; wherein, the toe area recognition unit is trained through a toe area data set of multiple barefoot in-step footprints, and the sole area recognition unit is trained through a sole area data set of multiple barefoot in-step footprints.

2. The method according to claim 1, wherein, The method further includes: Based on the toe area, determine the toe area similarity between the barefoot footprint to be recognized and each intermediate target barefoot in-step footprint in the barefoot in-step footprint database respectively, and determine the variance value of the multiple toe area similarities corresponding to each intermediate target barefoot in-step footprint respectively; Through the sole area recognition unit, based on the sole area, in the order from small to large of the variance values of the multiple toe area similarities corresponding to the multiple intermediate target barefoot in-step footprints respectively, sequentially determine the final target barefoot in-step footprint corresponding to the barefoot footprint to be recognized from the multiple intermediate target barefoot in-step footprints.

3. The method according to claim 2, characterized in that, Through the sole area recognition unit, based on the sole area, determining the final target barefoot in-step footprint corresponding to the barefoot footprint to be recognized from the multiple intermediate target barefoot in-step footprints includes: Through the sole area recognition unit, based on the sole area, determine the sole area similarity between the barefoot footprint to be recognized and each intermediate target barefoot in-step footprint of each intermediate target barefoot in-step footprint respectively, and determine the variance value of the multiple sole area similarities corresponding to each intermediate target barefoot in-step footprint respectively; Determine the minimum value among the variance values of the multiple sole area similarities corresponding to the multiple intermediate target barefoot in-step footprints respectively, and use the intermediate target barefoot in-step footprint corresponding to the minimum value of the variance values of the multiple sole area similarities as the final target barefoot in-step footprint.

4. The method according to claim 3, characterized in that The method further includes: Obtain the footprint environment feature corresponding to each intermediate target barefoot in-step footprint of each intermediate target barefoot in-step footprint, and obtain the footprint influence factor corresponding to the footprint environment feature; Multiply the toe area similarity corresponding to each intermediate target barefoot in-step footprint of the multiple intermediate target barefoot in-step footprints by the footprint influence factor to adjust the toe area similarity corresponding to each intermediate target barefoot in-step footprint of the multiple intermediate target barefoot in-step footprints; multiply the sole area similarity corresponding to each intermediate target barefoot in-step footprint of the multiple intermediate target barefoot in-step footprints by the footprint influence factor to adjust the sole area similarity corresponding to each intermediate target barefoot in-step footprint of the multiple intermediate target barefoot in-step footprints.

5. The method according to claim 4, wherein The method further includes: Obtain the toe area influence factor and the sole area influence factor corresponding to each intermediate target barefoot footprint of each intermediate target barefoot walking footprint, and determine the difference between the toe area influence factor and the sole area influence factor corresponding to each intermediate target barefoot footprint as the area influence difference; wherein, the toe area influence factor represents the influence characteristic of the footprint environment characteristic on the footprint in the toe area, and the sole area influence factor represents the influence characteristic of the footprint environment characteristic on the footprint in the sole area; When determining the variance value of the multiple toe area similarities corresponding to each intermediate target barefoot walking footprint, delete the toe area similarities of the intermediate target barefoot footprints whose area influence difference is greater than or equal to the preset area influence difference; when determining the variance value of the multiple sole area similarities corresponding to each intermediate target barefoot walking footprint, delete the sole area similarities of the intermediate target barefoot footprints whose area influence difference is greater than or equal to the preset area influence difference.

6. The method according to claim 5, wherein The method further includes: Determine the environmental influence factor corresponding to each of the multiple intermediate target barefoot footprints of each intermediate target barefoot walking footprint, and determine the mean value of the environmental influence factors corresponding to each of the multiple intermediate target barefoot footprints of each intermediate target barefoot walking footprint as the influence factor mean value of the walking footprint; wherein, the environmental influence factor represents the overall influence degree of the ground material and the ground humidity on the footprint; Determine the product of the variance value of the multiple toe area similarities corresponding to each intermediate target barefoot walking footprint and the influence factor mean value of the walking footprint to adjust the variance value of the multiple toe area similarities; determine the product of the variance value of the multiple sole area similarities corresponding to each intermediate target barefoot walking footprint and the influence factor mean value of the walking footprint to adjust the variance value of the multiple sole area similarities.

7. The method according to claim 6, wherein The method further includes: Determine the area of each toe sub-region corresponding to each toe in the toe area of the barefoot footprint to be identified; Determine the ratio of the minimum area value to the maximum area value of the toe sub-region area as the toe area confidence level, and obtain the number of walking footprints corresponding to the toe area confidence level. According to the ascending order of the variance values of the multiple toe area similarities corresponding to the multiple intermediate target barefoot walking footprints, determine the multiple intermediate target barefoot walking footprints with the number of walking footprints; through the sole area recognition unit, according to the sole area, determine the final target barefoot walking footprint corresponding to the barefoot footprint to be identified among the intermediate target barefoot walking footprints with the number of walking footprints, and use the final target barefoot walking footprint as the recognition retrieval result.

8. The method according to claim 7, wherein The method further includes: When the minimum area value among the multiple toe sub-region areas is less than the preset minimum area value, through the barefoot footprint to be identified evaluation unit, determine the number of walking footprints according to the multiple toe sub-region areas of the barefoot footprint to be identified; Determine the multiple intermediate target barefoot in-step footprints with the number of in-step footprints in ascending order of the variance values of the similarities of the multiple toe regions corresponding to the multiple intermediate target barefoot in-step footprints respectively; through the sole region recognition unit, determine the final target barefoot in-step footprint corresponding to the to-be-recognized barefoot footprint among the multiple intermediate target barefoot in-step footprints with the number of in-step footprints, and use the final target barefoot in-step footprint as the recognition and retrieval result.

9. The method according to claim 8, characterized in that, Through the to-be-recognized barefoot footprint evaluation unit, determine the number of in-step footprints according to the areas of multiple toe sub-regions of the to-be-recognized barefoot footprint, including: Through the toe position recognition unit, determine the toe position information corresponding to the areas of multiple toe sub-regions respectively according to the multiple toe sub-regions; Through the to-be-recognized barefoot footprint evaluation unit, determine the number of in-step footprints according to the areas of multiple toe sub-regions of the to-be-recognized barefoot footprint and the toe position information corresponding to the areas of multiple toe sub-regions respectively.

10. A barefoot footprint comparison device based on deep learning, characterized in that, It includes a unit for executing the method according to any one of claims 1 to 9.

Images