Deep learning based barefoot footprint comparison method and apparatus

Abstract

The application relates to the field of footprint identification, and discloses a barefoot footprint comparison method and device based on deep learning. The method comprises the following steps: obtaining a to-be-identified barefoot footprint, and determining a toe region and a sole region in the to-be-identified barefoot footprint through a footprint segmentation unit; determining a plurality of intermediate target barefoot footprint traces corresponding to the to-be-identified barefoot footprint in a barefoot footprint trace database according to the toe region through a toe region identification unit; determining a final target barefoot footprint trace corresponding to the to-be-identified barefoot footprint in the plurality of intermediate target barefoot footprint traces according to the sole region through a sole region identification unit, and taking the final target barefoot footprint trace as an identification search result; wherein the toe region identification unit is obtained through training of toe region data sets of a plurality of barefoot footprint traces. The application can improve the speed of footprint identification search in the barefoot footprint trace.

Description

Technical Field

[0001] This application relates to the field of footprint recognition technology, and more specifically, to a method and apparatus for comparing barefoot footprints based on deep learning. Background Technology

[0002] Footprints, as used in forensic science, are the traces left by a criminal suspect's weight (human gravity) and muscle force applied to the ground or other trace-forming objects during activities such as standing and walking. Footprints are a crucial piece of evidence frequently used in criminal investigations. Analysis and identification of these footprints can determine a person's height, age, weight, gait, as well as the duration of stillness and direction of movement. Footprint examination and identification techniques are a vital part of trace evidence examination, possessing high evidentiary value and playing a crucial role in solving cases. However, traditional footprint comparison relies primarily on manual methods, which suffers from difficulties in image feature extraction, slow recognition speed, and low accuracy.

[0003] Patent CN118210941B (application number: CN202410325445.0) provides a cross-domain footprint pressure image retrieval system based on high-order spatiotemporal relations. The footprint retrieval calculation process is as follows: First, the system acquires the footprint pressure image of the user wearing shoes through the acquisition window. After processing by the CPU processor, the system adds the trained optimal model to the system directory and loads the footprint pressure image of the user barefoot through the database. Next, the system's metric function is used to calculate the similarity between the image to be retrieved and the image in the database. Specifically, the image to be retrieved is input into the trained network model. The backbone network in this model first extracts sequence-level features, and then processes them through the 2D block module and the spatial attention module. After that, a hypergraph is constructed through the high-order spatiotemporal relation module, and the adjacency matrix and the relevant cosine distance are calculated to obtain the hypergraph features. Finally, based on the calculated similarity, the system uses the cross-domain footprint pressure retrieval function to find the barefoot footprint pressure image of the user wearing shoes that is most similar to the image to be retrieved from the database, and prints the retrieval results in the display window. The method in patent CN118210941B, when processing barefoot footprint pressure images, results in a low recognition speed because there are more than four footprint images in the barefoot footprint pressure images. The computational load for recognizing and retrieving barefoot footprints is large. Summary of the Invention

[0004] The purpose of this application is to provide a method and apparatus for barefoot footprint comparison based on deep learning, which solves the technical problems of large computational load and slow retrieval speed in footprint recognition and retrieval in barefoot footprints, and achieves the technical effect of improving the speed of footprint recognition and retrieval in barefoot footprints.

[0005] This application provides a deep learning-based barefoot footprint comparison method, which includes: acquiring a barefoot footprint to be identified, and determining the toe region and sole region in the barefoot footprint through a footprint segmentation unit; using a toe region identification unit, determining multiple intermediate target barefoot footprints corresponding to the barefoot footprint to be identified in a barefoot footprint database based on the toe region; using a sole region identification unit, determining the final target barefoot footprint corresponding to the barefoot footprint to be identified from the multiple intermediate target barefoot footprints based on the sole region, and using the final target barefoot footprint as the identification retrieval result; wherein, the toe region identification unit is trained using a toe region dataset of multiple barefoot footprints, and the sole region identification unit is trained using a sole region dataset of multiple barefoot footprints.

[0006] In one possible implementation, the method further includes: determining the similarity of the toe region between the barefoot footprint to be identified and each intermediate target barefoot footprint in the barefoot footprint database based on the toe region, and determining the variance value of the similarity of the multiple toe regions corresponding to each intermediate target barefoot footprint; and, through the foot region recognition unit, determining the final target barefoot footprint corresponding to the barefoot footprint to be identified in the multiple intermediate target barefoot footprints in ascending order of the variance value of the similarity of the multiple toe regions corresponding to the multiple intermediate target barefoot footprints, based on the foot region.

[0007] In another possible implementation, a foot region recognition unit determines the final target barefoot footprint corresponding to the barefoot footprint to be identified among multiple intermediate target barefoot footprints based on the foot region. This includes: using the foot region recognition unit, determining the similarity of the foot region between the barefoot footprint to be identified and each intermediate target barefoot footprint of each intermediate target barefoot footprint based on the foot region, and determining the variance of the multiple foot region similarities corresponding to each intermediate target barefoot footprint; determining the minimum value among the multiple foot region similarities corresponding to the multiple intermediate target barefoot footprints, and taking the intermediate target barefoot footprint corresponding to the minimum value among the multiple foot region similarities as the final target barefoot footprint.

[0008] In another possible implementation, the method further includes: obtaining the footprint environment features corresponding to each intermediate target barefoot footprint in each intermediate target barefoot journey, and obtaining the footprint influence factor corresponding to the footprint environment features; multiplying the toe region similarity corresponding to each intermediate target barefoot footprint in multiple intermediate target barefoot journeys by the footprint influence factor to adjust the toe region similarity corresponding to each intermediate target barefoot footprint in multiple intermediate target barefoot journeys; multiplying the sole region similarity corresponding to each intermediate target barefoot footprint in multiple intermediate target barefoot journeys by the footprint influence factor to adjust the sole region similarity corresponding to each intermediate target barefoot footprint in multiple intermediate target barefoot journeys.

[0009] In another possible implementation, the method further includes: obtaining the toe region influence factor and foot region influence factor corresponding to each intermediate target barefoot footprint for each intermediate target barefoot footprint, and determining the difference between the toe region influence factor and the foot region influence factor corresponding to each intermediate target barefoot footprint as the regional influence difference; wherein, the toe region influence factor represents the footprint influence feature of the footprint environment on the toe region, and the foot region influence factor represents the footprint influence feature of the footprint environment on the foot region; when determining the variance value of the similarity of multiple toe regions corresponding to each intermediate target barefoot footprint, the toe region similarity of intermediate target barefoot footprints with a regional influence difference greater than or equal to a preset regional influence difference is deleted; when determining the variance value of the similarity of multiple foot regions corresponding to each intermediate target barefoot footprint, the foot region similarity of intermediate target barefoot footprints with a regional influence difference greater than or equal to a preset regional influence difference is deleted.

[0010] In another possible implementation, the method further includes: determining the environmental impact factors corresponding to multiple intermediate target barefoot footprints for each intermediate target barefoot footprint, and determining the mean of the environmental impact factors corresponding to multiple intermediate target barefoot footprints for each intermediate target barefoot footprint as the mean of the footprint impact factors; wherein, the environmental impact factors characterize the overall influence of ground material and ground humidity on the footprints; determining the product of the variance of the similarity of multiple toe regions corresponding to each intermediate target barefoot footprint and the mean of the footprint impact factors, in order to adjust the variance of the similarity of multiple toe regions; determining the product of the variance of the similarity of multiple sole regions corresponding to each intermediate target barefoot footprint and the mean of the footprint impact factors, in order to adjust the variance of the similarity of multiple sole regions.

[0011] In another possible implementation, the method further includes: determining the area of ​​each toe sub-region corresponding to each toe in the toe region of the barefoot footprint to be identified; determining the ratio of the minimum area value to the maximum area value of the toe sub-region as the toe region confidence score, and obtaining the number of footprints corresponding to the toe region confidence score; determining multiple intermediate target barefoot footprints in the number of ...

[0012] In another possible implementation, the method further includes: when the minimum area value among the multiple toe sub-regions is less than a preset minimum area value, the barefoot footprint evaluation unit determines the number of footprints in a row based on the area of ​​the multiple toe sub-regions of the barefoot footprint to be identified; the intermediate target barefoot footprints in a row are determined in ascending order of the variance values ​​of the multiple toe regions corresponding to the multiple intermediate target barefoot footprints; the sole region identification unit determines the final target barefoot footprint corresponding to the barefoot footprint to be identified from the intermediate target barefoot footprints in the number of footprints in a row based on the sole region, and the final target barefoot footprint is used as the identification retrieval result.

[0013] In another possible implementation, the number of footprints in a row is determined by the barefoot footprint evaluation unit based on the area of ​​multiple toe sub-regions of the barefoot footprint to be identified. This includes: determining the toe position information corresponding to the area of ​​each toe sub-region based on the multiple toe sub-regions by the toe position identification unit; and determining the number of footprints in a row based on the area of ​​each toe sub-region and the toe position information corresponding to the area of ​​each toe sub-region by the barefoot footprint evaluation unit.

[0014] This application also provides a deep learning-based barefoot footprint comparison device, including a unit for performing the method described in any of the preceding claims.

[0015] The beneficial effects of the embodiments in this application compared with the prior art are:

[0016] This application provides a deep learning-based method for barefoot footprint comparison. The method includes: acquiring a barefoot footprint to be identified, and determining the toe region and sole region of the barefoot footprint using a footprint segmentation unit; using a toe region identification unit, identifying multiple intermediate target barefoot footprints corresponding to the barefoot footprint in a barefoot footprint database based on the toe region; using a sole region identification unit, identifying the final target barefoot footprint corresponding to the barefoot footprint from the multiple intermediate target barefoot footprints based on the sole region, and using the final target barefoot footprint as the identification retrieval result; wherein, the toe region identification unit is trained using a toe region dataset of multiple barefoot footprints, and the sole region identification unit is trained using a sole region dataset of multiple barefoot footprints. This application can improve the retrieval speed of more than four footprint images in a barefoot footprint image and improve the retrieval effect of barefoot footprint images. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the first deep learning-based barefoot footprint comparison method provided in this application embodiment;

[0019] Figure 2 A schematic diagram illustrating the workflow of the first deep learning-based barefoot footprint comparison method provided in this application embodiment;

[0020] Figure 3 This is a schematic diagram illustrating the determination of the toe region and the sole region in the barefoot footprint to be identified in this embodiment of the application;

[0021] Figure 4 A flowchart illustrating the second deep learning-based barefoot footprint comparison method provided in this application embodiment;

[0022] Figure 5 A flowchart illustrating the third deep learning-based barefoot footprint comparison method provided in this application embodiment;

[0023] Figure 6 This is a schematic diagram of the logical structure of a deep learning-based barefoot footprint comparison device provided in an embodiment of this application. Detailed Implementation

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] In existing cross-domain footprint retrieval methods, the recognition speed of barefoot footprints is low because there are more than four footprint images in the barefoot footprint pressure image.

[0030] Based on the above reasons, this application provides a deep learning-based barefoot footprint comparison method. The method includes: acquiring a barefoot footprint to be identified, and determining the toe region and sole region within the footprint using a footprint segmentation unit; using a toe region identification unit, identifying multiple intermediate target barefoot footprints corresponding to the barefoot footprint in a barefoot footprint database based on the toe region; using a sole region identification unit, identifying the final target barefoot footprint corresponding to the barefoot footprint from among the multiple intermediate target barefoot footprints based on the sole region, and using the final target barefoot footprint as the identification retrieval result; wherein the toe region identification unit is trained using a toe region dataset of multiple barefoot footprints, and the sole region identification unit is trained using a sole region dataset of multiple barefoot footprints. This application embodiment can improve the retrieval speed for more than four footprint images in a barefoot footprint image, and improves the retrieval effect of barefoot footprint images.

[0031] In some scenarios, the deep learning-based barefoot footprint comparison method of this application embodiment can be applied to the retrieval of barefoot footprint images in criminal science and technology, which can improve the auxiliary effect on criminal cases involving barefoot footprint images.

[0032] The following describes in detail, with specific examples, a deep learning-based barefoot footprint comparison method provided in the embodiments of this application.

[0033] Figure 1 A flowchart illustrating the first deep learning-based barefoot footprint comparison method provided in this application embodiment is shown below. Figure 1 As shown, the above method includes S110 to S120, and S110 to S120 will be described in detail below.

[0034] S110. Obtain the barefoot footprint to be identified, and determine the toe region and foot region in the barefoot footprint to be identified through the footprint segmentation unit.

[0035] Figure 2 A schematic diagram illustrating the workflow of the first deep learning-based barefoot footprint comparison method provided in this application embodiment is shown below. Figure 2 As shown in the embodiments of this application, after obtaining the barefoot footprint to be identified, in order to efficiently retrieve the barefoot footprint to be identified, the toe region and the sole region in the barefoot footprint to be identified can be determined first by the footprint segmentation unit, and then the identification of the toe region and the sole region can be further realized.

[0036] Figure 3 This is a schematic diagram illustrating the determination of the toe region and the sole region in the barefoot footprint to be identified in this embodiment of the application, as shown below. Figure 3As shown, the toe region 11 and the foot region 12 can be determined in the barefoot footprint 1 to be identified by the footprint segmentation unit. The toe region 11 has a smaller area and fewer pixels. At the same time, the information of the toe region 11 can comprehensively reflect the information of the toes in the footprint. Then, the barefoot footprint to be identified can be initially searched based on the toe region 11, which has richer information and smaller image size. At the same time, the barefoot footprint to be identified can be further searched and identified based on the foot region 12, which reduces the amount of computation and realizes high-speed retrieval and identification of the barefoot footprint to be identified.

[0037] For example, the raw pressure image of the barefoot footprint to be identified can first be acquired using a high-precision optical sensor and then input into the footprint segmentation unit. The footprint segmentation unit uses a pre-trained U-Net network model to extract local and global features of the footprint through multi-scale convolutional layers and outputs binary masks for the toe region and the sole region. The toe region is defined as the contour range from the front end of the footprint to the first metatarsal joint, and the sole region covers the continuous pressure distribution area from the arch of the foot to the heel.

[0038] S120. Using the toe region recognition unit, multiple intermediate target barefoot footprints corresponding to the barefoot footprint to be identified are determined from the barefoot footprint database based on the toe region. Using the sole region recognition unit, the final target barefoot footprint corresponding to the barefoot footprint to be identified is determined from the multiple intermediate target barefoot footprints based on the sole region, and the final target barefoot footprint is used as the recognition retrieval result. The toe region recognition unit is trained using a toe region dataset of multiple barefoot footprints, and the sole region recognition unit is trained using a sole region dataset of multiple barefoot footprints.

[0039] After obtaining the toe region and the foot region, the toe region recognition unit can determine multiple intermediate target barefoot footprints corresponding to the barefoot footprints to be identified in the barefoot footprint database based on the toe region. This achieves the purpose of preliminary retrieval of footprints based on the toe region to obtain multiple intermediate target barefoot footprints. The multiple intermediate target barefoot footprints are candidate intermediate target barefoot footprints obtained through toe region recognition.

[0040] For example, the segmented toe region image can be input into the toe region recognition unit, which is built based on the ResNet-50 network. Its training data is a set of toe region images labeled in the barefoot footprint database, which contains dynamic changes in toe pressure under different time periods. The toe region recognition unit calculates the feature similarity between the toe region to be identified and the toe regions of each footprint in the database (using cosine similarity as a metric), and selects the top N footprints with the highest similarity as intermediate target barefoot footprints.

[0041] After obtaining multiple intermediate target barefoot footprints, the foot region recognition unit can determine the final target barefoot footprint corresponding to the barefoot footprint to be identified from among the multiple intermediate target barefoot footprints based on the foot region. The final target barefoot footprint is then used as the recognition and retrieval result to achieve the retrieval and location of the final target barefoot footprint.

[0042] For example, the foot region data of the barefoot footprints of the intermediate target can be input into the foot region recognition unit. The foot region recognition unit adopts the VGG-16 network structure. Its training data is a set of foot region images aligned with the toe region dataset. It focuses on learning stability features such as arch shape and pressure peak distribution. The foot region recognition unit compares the foot region to be identified with the foot region features of the intermediate target footprints and aligns the gait sequence by combining the dynamic time warping (DTW) algorithm. Finally, it outputs the footprint with the highest similarity as the final target barefoot footprint.

[0043] For example, when determining the final target barefoot footprint corresponding to the barefoot footprint to be identified among multiple intermediate target barefoot footprints, the score values ​​corresponding to the multiple intermediate target barefoot footprints can be obtained, and the intermediate target barefoot footprint with the highest score value can be taken as the final target barefoot footprint.

[0044] It should be noted that the toe region recognition unit is trained using a dataset of toe regions from multiple barefoot footprints, and the sole region recognition unit is trained using a dataset of sole regions from multiple barefoot footprints. This allows for the retrieval and recognition of the distribution of the final target barefoot footprints.

[0045] It should be noted that when comparing barefoot footprints to be identified with a set of barefoot footprints, if the barefoot footprint to be identified is a left foot footprint, the left foot footprint in the set of barefoot footprints can be searched separately; if the barefoot footprint to be identified is a right foot footprint, the right foot footprint in the set of barefoot footprints can be searched separately.

[0046] The beneficial effects of the above implementation method are that it initially searches for the barefoot footprints to be identified based on the toe region, which has richer information and smaller image size, and at the same time, it can further search and identify the barefoot footprints to be identified based on the sole region, which reduces the amount of computation. It first quickly filters intermediate targets through the toe region and then performs fine matching through the sole region, taking into account both search efficiency and accuracy, and achieves high-speed search and identification of the barefoot footprints to be identified.

[0047] The beneficial effect of the above implementation method is that by independently training the toe and foot region recognition units, feature optimization is performed on the dynamically sensitive toe region and the more stable foot region respectively, avoiding the information mixing problem of a single feature model.

[0048] Figure 4 A flowchart illustrating the second deep learning-based barefoot footprint comparison method provided in this application embodiment is shown below. Figure 4 As shown, the above method also includes S210 to S220, which will be described in detail below.

[0049] S210. Based on the toe region, determine the similarity of the toe region between the barefoot footprint to be identified and each intermediate target barefoot footprint in the barefoot footprint database, and determine the variance of the similarity of the multiple toe regions corresponding to each intermediate target barefoot footprint.

[0050] After identifying multiple intermediate target barefoot footprints through the toe region recognition unit, the toe region features of all single-step barefoot footprints contained in each intermediate target footprint can be extracted. Based on the pre-trained Siamese network model, the similarity between the toe region of the barefoot footprint to be identified and the toe region of each single-step footprint in the footprint can be calculated one by one. Normalized Euclidean distance is used as the similarity index as the toe region similarity, so as to calculate the toe region similarity between the barefoot footprint to be identified and each intermediate target barefoot footprint in the multiple intermediate target footprints.

[0051] After obtaining the similarity of multiple toe regions corresponding to each intermediate target barefoot footprint, the variance of the toe region similarity values ​​of all single-step footprints contained in each intermediate target barefoot footprint can be calculated.

[0052] S220. Using the foot region recognition unit, based on the foot region, and in accordance with the order of the variance values ​​of the similarity of the multiple toe regions corresponding to the multiple intermediate target barefoot footprints from smallest to largest, the final target barefoot footprint corresponding to the barefoot footprint to be identified is determined sequentially among the multiple intermediate target barefoot footprints.

[0053] After obtaining the variance of the toe region similarity values ​​of all single-step footprints contained in each intermediate target barefoot footprint, multiple intermediate target footprints can be arranged in ascending order (from smallest to largest) according to the variance of the toe region similarity to generate a priority queue. Furthermore, the foot region recognition unit can be processed in this order, prioritizing the matching of the footprint with the smallest variance value.

[0054] The beneficial effect of the above implementation method is that by screening highly consistent footprints through the variance of toe region similarity, and prioritizing the processing of low variance candidate sets, the amount of invalid computation can be reduced, which can significantly reduce the retrieval time.

[0055] The beneficial effect of the above implementation method is that the variance index can effectively filter out abnormal similarity fluctuations caused by local deformation of single-step footprints (such as toe curling), and improve the matching stability of cross-gait cycle in footprint retrieval.

[0056] In some implementations, in S120 above, the foot region recognition unit determines the final target barefoot footprint corresponding to the barefoot footprint to be identified from multiple intermediate target barefoot footprints based on the foot region, including S121 to S122. S121 to S122 will be explained in detail below.

[0057] S121. Using the foot region recognition unit, based on the foot region, determine the similarity of the foot region between the barefoot footprint to be identified and each intermediate target barefoot footprint of each intermediate target barefoot footprint, and determine the variance value of the similarity of multiple foot regions corresponding to each intermediate target barefoot footprint.

[0058] When performing foot region recognition, the foot region features of all single-step footprints contained in each intermediate target barefoot trek can be extracted. The foot region recognition unit can use a pre-trained Inception-v3 network model to perform multi-scale feature fusion on the arch shape, pressure distribution peak and heel contact area of ​​each single-step footprint to generate a 128-dimensional feature vector. The 128-dimensional feature vector is the foot region similarity.

[0059] For example, the normalized cross-correlation (NCC) similarity can be calculated by comparing the feature vectors of the foot region to be identified with the single-step foot region in the database, thereby realizing the calculation of the similarity between the foot region of the barefoot footprint to be identified and the foot region of each intermediate target barefoot footprint in each intermediate target barefoot journey.

[0060] When performing foot region recognition, the variance of the similarity values ​​of all single-step foot regions can be calculated for each intermediate target's footprint, thus enabling the evaluation of each intermediate target's barefoot footprint based on the variance of the foot region similarity values.

[0061] For example, the variance can be dynamically calculated using the sliding window method, with the window size being 50% of the length of a single footprint and the step size being 1 frame. The final variance value is the median of the variances of all windows, in order to resist local fluctuation interference.

[0062] S122. Determine the minimum value among the variances of the similarity values ​​of multiple foot regions corresponding to the barefoot footprints of multiple intermediate targets, and take the barefoot footprints of the intermediate targets corresponding to the minimum value among the variances of the similarity values ​​of multiple foot regions as the final barefoot footprints.

[0063] After obtaining the variance values ​​of the foot region similarity of all intermediate target footprints, we can iterate through the variance values ​​of the foot region similarity of all intermediate target footprints. A high variance indicates that the footprint has significant dynamic matching features in the foot region (such as arch elastic deformation and gait phase synchronization), which is more in line with the biomechanical pattern of real barefoot walking. Therefore, we can select the footprint corresponding to the maximum value among multiple foot region similarity variance values ​​as the final target.

[0064] The beneficial effect of the above implementation method is that by screening highly consistent footprints through the similarity variance of the foot region and prioritizing the processing of low variance candidate sets, the amount of invalid computation can be reduced, which can significantly reduce the retrieval time.

[0065] The beneficial effect of the above implementation method is that the sliding window variance calculation combined with median statistics can effectively suppress the impact of single-frame noise (such as heel slippage) on the overall variance evaluation, and can greatly reduce the false detection rate in interference scenarios.

[0066] Figure 5 A flowchart illustrating the third deep learning-based barefoot footprint comparison method provided in this application embodiment is shown below. Figure 5 As shown, the above method also includes S310 to S320, which will be described in detail below.

[0067] S310. Obtain the footprint environment features corresponding to each intermediate target barefoot footprint in each intermediate target barefoot journey, and obtain the footprint influence factor corresponding to the footprint environment features.

[0068] In footprint recognition, the environment may affect the footprints. Therefore, for each intermediate target barefoot footprint, the environmental features corresponding to the barefoot footprint database can be obtained for each single step footprint in the barefoot footprint database, and the footprints can be retrieved based on the environmental features.

[0069] For example, footprint environmental features may include ground material type, ambient temperature and humidity, and light intensity, and footprint environmental features can be recorded simultaneously when the footprint is recorded.

[0070] After obtaining the footprint environmental features, the footprint impact factor corresponding to the footprint environmental features can be obtained. The footprint impact factor can be calculated by inputting the three types of features into a pre-trained environmental impact factor calculation model, and the output footprint impact factor in the range of [0.8, 1.2] can be calculated as follows: Impact factor = 0.8 + 0.4·σ(α·material coefficient + β·humidity normalized value + γ·light attenuation rate), where σ is the Sigmoid function, and α = 0.6, β = 0.3, and γ = 0.1 are weight parameters.

[0071] S320. Multiply the toe region similarity of each intermediate target barefoot footprint in the multiple intermediate target barefoot footprint sets by a footprint influence factor to adjust the toe region similarity of each intermediate target barefoot footprint in the multiple intermediate target barefoot footprint sets. Multiply the sole region similarity of each intermediate target barefoot footprint in the multiple intermediate target barefoot footprint sets by a footprint influence factor to adjust the sole region similarity of each intermediate target barefoot footprint in the multiple intermediate target barefoot footprint sets.

[0072] After obtaining the footprint influence factors corresponding to the footprint environmental features, the similarity of each single-step footprint of the barefoot footprints of each intermediate target can be dynamically adjusted. During adjustment, the similarity of the toe region and the similarity of the sole region can be multiplied by the corresponding footprint influence factors to adjust the similarity of the toe region of each intermediate target barefoot footprint of multiple intermediate targets, and the similarity of the sole region of each intermediate target barefoot footprint of multiple intermediate targets. After adjustment, the adjusted similarity of the toe region and the sole region can be normalized.

[0073] After obtaining the adjusted toe region similarity and foot region similarity, the variance of each intermediate target footprint can be recalculated based on these adjusted similarities. The variance of the toe region similarity is used to update the priority queue of the toe regions, and the minimum variance of the foot region similarity is used to re-select the final target footprints, forming a closed-loop optimization.

[0074] The beneficial effect of the above implementation method is that by dynamically correcting the similarity of footprints through the footprint influence factor, feature distortion caused by environmental factors such as slippery ground and excessive hardness can be eliminated, thereby improving the retrieval accuracy in complex environments.

[0075] The beneficial effect of the above implementation method is that, after integrating environmental characteristics such as material, lighting, and humidity, the system reduces the cross-domain retrieval error rate in different material scenarios such as cement floors, carpets, and sand floors.

[0076] The beneficial effect of the above implementation method is that by adopting a separate environmental factor calculation model, scalar multiplication adjustment is only performed at the single-step footprint level, which has a smaller impact on the overall computational overhead.

[0077] In some implementations, the above method also includes S330 to S340, which will be described in detail below.

[0078] S330. Obtain the toe region influence factor and foot region influence factor for each intermediate target barefoot footprint in a complete run, and determine the difference between the toe region influence factor and the foot region influence factor for each intermediate target barefoot footprint, as the region influence difference. The toe region influence factor represents the influence of footprint environmental features on the toe region, and the foot region influence factor represents the influence of footprint environmental features on the foot region.

[0079] For each intermediate target's barefoot footprint, the absolute difference between the toe region influence factor and the sole region influence factor can be calculated as the regional influence difference for each single footprint. The regional influence difference characterizes the difference in the degree of environmental influence between the toe region and the sole region.

[0080] It should be noted that the toe region influence factor represents the influence of footprint environmental characteristics on the toe region, and the toe region influence factor can be manually determined by the environmental characteristics of the toe region. The ball of the foot influence factor represents the influence of footprint environmental characteristics on the ball of the foot region, and the ball of the foot influence factor can be manually determined by the environmental characteristics of the ball of the foot region.

[0081] S340. When determining the variance of the similarity values ​​of multiple toe regions corresponding to each intermediate target barefoot footprint, delete the toe region similarity values ​​of intermediate target barefoot footprints whose regional influence difference is greater than or equal to the preset regional influence difference. When determining the variance of the similarity values ​​of multiple foot regions corresponding to each intermediate target barefoot footprint, delete the foot region similarity values ​​of intermediate target barefoot footprints whose regional influence difference is greater than or equal to the preset regional influence difference.

[0082] After obtaining the regional impact difference, when determining the variance of the similarity of multiple toe regions corresponding to each intermediate target barefoot footprint, the toe region similarity of intermediate target barefoot footprints with a regional impact difference greater than or equal to the preset regional impact difference can be deleted to exclude the toe region similarity of intermediate target barefoot footprints with an excessively high degree of environmental influence.

[0083] After obtaining the regional impact difference, when determining the variance of the similarity of multiple foot regions corresponding to each intermediate target barefoot footprint, the similarity of the foot regions of intermediate target barefoot footprints with regional impact differences greater than or equal to the preset regional impact difference can be deleted to exclude the similarity of the foot regions of intermediate target barefoot footprints with excessive environmental influence.

[0084] After excluding the similarity of the toe region of the barefoot footprints of intermediate targets with excessive environmental influence, and excluding the similarity of the sole region of the barefoot footprints of intermediate targets with excessive environmental influence, a priority queue of intermediate target footprints can be regenerated based on the filtered effective similarity data, and the above-mentioned maximum variance matching process for soles can be executed.

[0085] For example, the preset regional influence difference can be dynamically calibrated through a cross-scenario validation set, with a value range of [0.3, 0.6]. When the regional influence difference is greater than or equal to the preset regional influence difference, it is determined that the footprint has regional specific environmental interference (e.g., only the toes sink into the sand while the sole of the foot contacts the hard ground).

[0086] The beneficial effect of the above implementation method is that by filtering the regional influence difference, it can effectively eliminate abnormal interference of the local environment on specific foot areas (such as water stains affecting the toe area and the sole area being dry), reduce variance calculation errors in complex scenarios, and improve the accuracy of footprint retrieval.

[0087] In some implementations, the above method also includes S410 to S420, which are described in detail below.

[0088] S410. Determine the environmental impact factors corresponding to multiple intermediate target barefoot footprints for each intermediate target barefoot footprint, and determine the average value of the environmental impact factors corresponding to multiple intermediate target barefoot footprints for each intermediate target barefoot footprint, as the average value of the footprint impact factors. The environmental impact factors characterize the overall influence of ground material and ground moisture on the footprints.

[0089] When conducting footprint retrieval, it is also possible to determine the environmental impact factors corresponding to multiple intermediate target barefoot footprints for each intermediate target barefoot footprint. The environmental impact factors characterize the overall degree of environmental impact on the multiple intermediate target barefoot footprints for each intermediate target barefoot footprint.

[0090] For example, the environmental impact factors corresponding to the environment in which the intermediate target's barefoot footprints are located can be determined by empirical values.

[0091] After obtaining the environmental impact factors corresponding to the multiple intermediate target barefoot footprints for each intermediate target barefoot footprint, the mean value of the environmental impact factors corresponding to the multiple intermediate target barefoot footprints for each intermediate target barefoot footprint can be further determined as the mean value of the footprint impact factor. The mean value of the footprint impact factor represents the comprehensive impact intensity of the environment in which the entire footprint is located.

[0092] S420. Determine the product of the variance of the similarity of multiple toe regions corresponding to each intermediate target's barefoot footprints and the mean of the footprint influence factor, in order to adjust the variance of the multiple toe region similarity. Determine the product of the variance of the similarity of multiple foot regions corresponding to each intermediate target's barefoot footprints and the mean of the footprint influence factor, in order to adjust the variance of the multiple foot region similarity.

[0093] After obtaining the mean value of the influence factor of the entire footprint, the variance of the similarity of the toe region and the variance of the similarity of the foot region of the entire footprint can be multiplied by the mean value of the influence factor to achieve environmental adaptive variance calibration. Then, based on the adjusted variance value, the above-mentioned process of prioritizing the intermediate target barefoot footprints and the final target screening process of the intermediate target barefoot footprints can be repeated to form a closed-loop optimization system for environmental perception.

[0094] The beneficial effect of the above implementation method is that by uniformly calibrating the variance index of the footprints through the mean-based global environmental influence factor, the retrieval accuracy can be improved in special scenarios such as sandy areas and ice surfaces.

[0095] The beneficial effects of the above implementation method are that the variance scaling mechanism enables the system to automatically adapt to the intensity of environmental interference, thereby reducing the false matching rate in high humidity environments and reducing the false detection rate on hard surfaces.

[0096] In some implementations, the above method also includes S510 to S520, which are described in detail below.

[0097] S510. Determine the area of ​​the toe sub-region corresponding to each toe in the toe region of the barefoot footprint to be identified.

[0098] When performing footprint retrieval, the toe region of the barefoot footprint to be identified can be finely segmented. Specifically, an improved Mask R-CNN model can be used. Based on the binary mask, the skeletonization algorithm is used to identify the contours of the five independent sub-regions of the toes. The area of ​​each toe sub-region can be calculated by pixel counting and recorded as A1, A2, ..., A5.

[0099] S520. Determine the ratio of the minimum to the maximum area of ​​the toe sub-region as the toe region confidence score, and obtain the number of footprints corresponding to the toe region confidence score. Based on the ascending order of the variance values ​​of the similarity between the toe regions corresponding to the multiple intermediate target barefoot footprints, determine the number of intermediate target barefoot footprints for each footprint. Using the foot region recognition unit, determine the final target barefoot footprint corresponding to the barefoot footprint to be identified from the intermediate target barefoot footprints in the number of footprints for each footprint, and use the final target barefoot footprint as the recognition and retrieval result.

[0100] After obtaining the area of each toe sub-region, the maximum value Amax and the minimum value of the toe sub-region area can be extracted. And determine the ratio of the minimum area value to the maximum area value of the toe sub-region area as the toe region confidence level, which characterizes the accuracy of initially retrieving footprints through the toe region.

[0101] After obtaining the toe region confidence level, the number of consecutive footprints corresponding to the toe region confidence level can be obtained. Different toe region confidence levels correspond to different numbers of consecutive footprints. Subsequently, in accordance with the principle of ascending toe region variance, the intermediate target barefoot consecutive footprints matching the number of consecutive footprints can be intercepted from the priority queue of the intermediate target barefoot consecutive footprints.

[0102] Exemplarily, when the number of consecutive footprints corresponding to the toe region confidence level is 20%, the first 20 intermediate target barefoot consecutive footprints can be selected from 100 intermediate target barefoot consecutive footprints.

[0103] The beneficial effect of the above implementation method is that by adaptively adjusting the candidate quantity according to the toe confidence level, inefficient calculations can be greatly reduced while ensuring accuracy.

[0104] The beneficial effect of the above implementation method is also that the toe area ratio index can effectively identify abnormal conditions such as missing toes and adhesions, and can reduce the false matching rate in the scenario of incomplete feet.

[0105] In some implementation methods, the above method further includes S530 to S540, which will be specifically described below.

[0106] S530: When the minimum area value among the areas of multiple toe sub-regions is less than the preset minimum area value, the unit for evaluating the barefoot footprint to be recognized determines the number of consecutive footprints according to the areas of multiple toe sub-regions of the barefoot footprint to be recognized.

[0107] After obtaining the minimum area value among the areas of multiple toe sub-regions, the preset minimum area value Amin can be obtained. When the minimum area A of the toe sub-region of the footprint to be recognized is less than Amin, it indicates that the recognition may be inaccurate due to the excessively small toe sub-region of the footprint to be recognized. Therefore, the unit for evaluating the barefoot footprint to be recognized can be triggered to adjust the number of consecutive footprints to improve the accuracy of footprint retrieval.

[0108] Exemplarily, the preset minimum area value Amin can be 120 pixels (corresponding to an actual size of 3 cm 2 ).

[0109] When determining the number of footprints in a row, the number of footprints in a row can be determined by using the barefoot footprint assessment unit to identify, based on the area of ​​multiple toe sub-regions of the barefoot footprint to be identified.

[0110] S540. Determine the number of intermediate target barefoot footprints in ascending order of the variance values ​​of the similarity of the toe regions corresponding to the multiple intermediate target barefoot footprints. Using the foot region recognition unit, determine the final target barefoot footprint corresponding to the barefoot footprint to be identified from the intermediate target barefoot footprints in the number of footprints in the number of footprints, and use the final target barefoot footprint as the recognition and retrieval result.

[0111] Similarly, after obtaining the number of complete footprints, multiple intermediate target barefoot complete footprints can be determined using the above method, and the final target barefoot complete footprint corresponding to the barefoot footprint to be identified can be determined from the intermediate target barefoot complete footprints of the number of complete footprints.

[0112] The beneficial effect of the above implementation method is that by triggering dynamic quantity adjustment through the minimum area threshold, the size of the candidate set can be adjusted under abnormal conditions such as broken toes or curling, thereby improving the accuracy of footprint retrieval.

[0113] In some implementations, in S530 above, the number of footprints in a row is determined by the barefoot footprint evaluation unit based on the area of ​​multiple toe sub-regions of the barefoot footprint to be identified, including S531 to S532. S531 to S532 will be explained in detail below.

[0114] S531. Using the toe position recognition unit, determine the toe position information corresponding to the area of ​​each of the multiple toe sub-regions.

[0115] When retrieving footprints, multiple toe sub-regions can be used to determine the toe position information corresponding to the area of ​​each toe sub-region. Then, the segmented toe sub-regions can be spatially located using a toe position recognition unit. Specifically, a key point detection model based on HRNet can be used to mark the three-dimensional coordinates (xi,yi,pi) at the center of each toe sub-region. The toe position information can be represented by the three-dimensional coordinates (xi,yi,pi), where xi,yi are the image plane coordinates and pi is the peak pressure intensity value.

[0116] S532. Using the barefoot footprint assessment unit to be identified, determine the number of footprints in a row based on the area of ​​multiple toe sub-regions of the barefoot footprint to be identified and the toe position information corresponding to the areas of multiple toe sub-regions.

[0117] After obtaining the toe position information, the barefoot footprint evaluation unit can determine the number of footprints in a row based on the area of ​​multiple toe sub-regions of the barefoot footprint to be identified and the toe position information corresponding to the areas of multiple toe sub-regions. This enables the judgment of the reliability of the toe region based on the area of ​​multiple toe sub-regions and the toe position information, thereby improving the accuracy of footprint retrieval.

[0118] For example, a toe spatial distribution matrix can be constructed, which converts the position coordinates of the five toes into normalized vectors relative to the origin of the heel. As the toe spatial distribution matrix, the barefoot footprint evaluation unit to be identified uses a multi-head attention mechanism to process the area of ​​multiple toe sub-regions and the toe position information corresponding to the areas of multiple toe sub-regions, so as to output the number of footprints in a row.

[0119] The beneficial effect of the above implementation method is that by effectively capturing the biomechanical relationship between toes (such as the big toe dominance effect), the error rate of quantity prediction can be reduced.

[0120] The beneficial effect of the above implementation method is that, through the weighted fusion mechanism of area and location features, the number of candidates can adapt to the changes in footprint shrinkage, offset and rotation, which can improve the accuracy of footprint retrieval under complex deformation.

[0121] This application also provides a deep learning-based barefoot footprint comparison device, including a unit for performing the method described in any of the preceding claims.

[0122] Figure 6 A schematic diagram of the logic structure of a deep learning-based barefoot footprint comparison device provided in this application embodiment is shown below. Figure 6 As shown, the apparatus 2 in this embodiment includes a processing unit 21, a storage unit 22, and a transceiver unit 23. The processing unit 21 is used to process data, the storage unit 22 is used to store data, and the transceiver unit 23 is used to send and receive data. The processing unit 21, the storage unit 22, and the transceiver unit 23 cooperate with each other to implement the above-described method. The beneficial effects of the embodiments of this application have been described in the above-described method and will not be repeated here.

[0123] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0126] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0128] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A deep learning-based barefoot footprint comparison method, characterized in that, The method comprises: acquiring a to-be-identified barefoot footprint, and determining a toe region and a sole region in the to-be-identified barefoot footprint through a footprint segmentation unit; determining, through a toe region identification unit, a plurality of intermediate target barefoot footprint strides corresponding to the to-be-identified barefoot footprint in a barefoot footprint stride database according to the toe region; determining, through a sole region identification unit, a final target barefoot footprint stride corresponding to the to-be-identified barefoot footprint in the plurality of intermediate target barefoot footprint strides according to the sole region; the final target barefoot footprint stride is taken as a recognition search result; wherein the toe region identification unit is trained through a plurality of toe region data sets of barefoot footprint strides, and the sole region identification unit is trained through a plurality of sole region data sets of barefoot footprint strides; The method further comprises: determining, according to the toe region, a toe region similarity respectively corresponding to each intermediate target barefoot footprint stride of the plurality of intermediate target barefoot footprint strides in the barefoot footprint stride database, and determining a variance value of the plurality of toe region similarities respectively corresponding to each intermediate target barefoot footprint stride; determining, through the sole region identification unit, the final target barefoot footprint stride corresponding to the to-be-identified barefoot footprint in the plurality of intermediate target barefoot footprint strides according to the sole region in the order from small to large of the variance values of the plurality of toe region similarities respectively corresponding to the plurality of intermediate target barefoot footprint strides; determining, through the sole region identification unit, the final target barefoot footprint stride corresponding to the to-be-identified barefoot footprint in the plurality of intermediate target barefoot footprint strides according to the sole region comprises: determining, through the sole region identification unit, a sole region similarity respectively corresponding to each intermediate target barefoot footprint of each intermediate target barefoot footprint stride according to the sole region, and determining a variance value of the plurality of sole region similarities respectively corresponding to each intermediate target barefoot footprint stride; determining a minimum value in the variance values of the plurality of sole region similarities respectively corresponding to the plurality of intermediate target barefoot footprint strides, and taking the intermediate target barefoot footprint stride corresponding to the minimum value in the variance values of the plurality of sole region similarities as the final target barefoot footprint stride; The method further comprises: acquiring a footprint environment feature corresponding to each intermediate target barefoot footprint of each intermediate target barefoot footprint stride, and acquiring a footprint influence factor corresponding to the footprint environment feature; multiplying the toe region similarity respectively corresponding to each intermediate target barefoot footprint of the plurality of intermediate target barefoot footprint strides by the footprint influence factor to adjust the toe region similarity corresponding to each intermediate target barefoot footprint of the plurality of intermediate target barefoot footprint strides; and multiplying the sole region similarity respectively corresponding to each intermediate target barefoot footprint of the plurality of intermediate target barefoot footprint strides by the footprint influence factor to adjust the sole region similarity corresponding to each intermediate target barefoot footprint of the plurality of intermediate target barefoot footprint strides.

2. The method of claim 1, wherein, The method further comprises: The method further comprises: The method further comprises:

3. The method of claim 2, wherein, The method further comprises: The method further comprises: The method further comprises:

4. The method of claim 3, wherein, The method further comprises: The method further comprises: The method further comprises:

5. The method of claim 4, wherein, The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further comprises: The method further According to the order from small to large of the variance values of the plurality of toe area similarities of the plurality of intermediate target barefoot footprints corresponding to the plurality of footprints, determine the plurality of intermediate target barefoot footprints; through the sole area recognition unit, according to the sole area, determine the final target barefoot footprint corresponding to the to-be-recognized barefoot footprint from the plurality of intermediate target barefoot footprints, and take the final target barefoot footprint as the recognition search result.

6. The method of claim 5, wherein, Through the to-be-recognized barefoot footprint evaluation unit, determine the number of footprints according to the plurality of toe sub-area areas of the to-be-recognized barefoot footprint, including: Through the toe position recognition unit, determine the toe position information corresponding to the plurality of toe sub-area areas according to the plurality of toe sub-area areas; Through the to-be-recognized barefoot footprint evaluation unit, determine the number of footprints according to the plurality of toe sub-area areas of the to-be-recognized barefoot footprint and the toe position information corresponding to the plurality of toe sub-area areas.

7. A deep learning-based barefoot footprint matching device, characterized by, The unit for executing the method of any one of claims 1 to 6.

Images