An interpretable footprint classification system based on region-contour prototype network

Abstract

The application discloses an interpretable footprint classification system based on a region-contour prototype network and belongs to the technical field of image processing. The application adopts a double-branch collaborative prototype network architecture: the left and right foot standardized space alignment is completed after the footprint image is preprocessed; the prototype activation is accurately aligned with key semantic regions such as toes, metatarsophalangeal joints, arches and heels through the region prototype branch combined with the position alignment constraint; the contour prototype branch is constructed based on the binary contour map, and morphological information such as foot shape boundaries and toe arrangement is extracted; the identity category is output by fusing the double-branch classification scores through global standardization, and then the TopK prototype is generated according to the prototype quality for global screening to generate visualized interpretation. While maintaining high classification accuracy, the application enables the interpretation evidence output by the model to stably correspond to the key semantic regions in the footprint image, and can adaptively complement between the region evidence and the contour evidence.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an interpretable footprint classification system based on a region-contour prototype network, which is particularly suitable for the identification classification and interpretable analysis of barefoot footprint images. Background Technology

[0002] Footprint images contain a variety of biometric information, including foot morphology, distribution of sole contact areas, and foot contours. Barefoot footprint images, in particular, can further reflect toe arrangement, arch shape, heel shape, and local sole contact features, making them valuable in applications such as identity recognition, forensic identification, security access control, and personnel retrieval. Compared to biometrics such as faces and fingerprints, footprint morphology exhibits relative stability over a certain timeframe and can be passively collected in non-contact or natural walking scenarios; therefore, footprint classification technology has significant application potential.

[0003] Traditional footprint classification methods, especially barefoot footprint classification methods, largely rely on manually designed features such as foot length, width, forefoot and hindfoot area, arch area ratio, and contour geometry. While these methods offer strong interpretability, their feature representation capabilities are limited, and they are easily affected by factors such as capture posture, ground material, light reflection, foot contact deformation, and background noise, making it difficult to adapt to the complex variations in real-world scenarios. With the development of deep learning, models such as convolutional neural networks, metric learning, and Transformers have been applied to footprint image classification and retrieval, achieving significant improvements in classification accuracy.

[0004] However, existing deep learning models typically model footprint images as overall texture or contour, with their discrimination criteria implicit in a high-dimensional feature space. This makes it difficult to intuitively explain why the model assigns a particular identity. In high-risk applications such as criminal investigation and security, simply providing classification results without reliable explanatory evidence reduces the credibility and verifiability of the model's conclusions, and may even lead to erroneous judicial judgments or security decisions.

[0005] Interpretable methods based on partial-prototype networks learn several typical local patterns and perform similarity matching between local regions of the input image and a prototype, thus providing an interpretation path that "the local region of the image is similar to a certain training prototype." However, in barefoot footprint classification tasks, different regions may have similar skin textures and contact marks. For example, the local texture differences between the heel, arch, and metatarsophalangeal regions are small. Traditional prototype models are prone to prototype response misalignment, that is, the prototype activates different semantic regions in different samples, resulting in unstable and unreliable interpretations.

[0006] Furthermore, footprint identity features exist not only in the internal region of the sole but also in external geometric structures such as contour boundaries, toe arrangement, and foot width. If the model relies solely on regional texture features, it may overlook the complementary identity information provided by the contour morphology. Simultaneously, most existing interpretable footprint classification methods rely on high-performance servers for cloud-based inference, making local deployment and real-time analysis in field environments difficult, thus limiting their application in criminal scene investigation and mobile security scenarios. Therefore, it is necessary to provide an interpretable footprint classification system that simultaneously utilizes regional and contour information, ensures prototype semantic alignment through explicit position supervision, and supports embedded local deployment. Summary of the Invention

[0007] (a) Technical problems to be solved To address the problems of existing footprint classification models, especially barefoot footprint classification models, such as opaque discrimination criteria, semantic misalignment in prototype responses, tendency of models to focus on background noise or non-critical regions, difficulty in fully utilizing contour geometric information for single region features, lack of stability evaluation and screening mechanisms for interpretation results, and difficulty in localizing the algorithm for display on embedded devices, this invention provides an interpretable footprint classification system based on region-contour prototype networks.

[0008] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: an interpretable footprint classification system based on a region-contour prototype network, comprising a visible light image acquisition module, an image preprocessing module, a location-supervised region-contour collaborative prototype network, a decision fusion module, an interpretation fusion module, and an interactive display module.

[0009] The system's workflow includes the following steps: S1: Acquire the footprint image of the person to be identified and perform preprocessing; during the training phase, perform data augmentation on the training set and acquire the identity category label corresponding to the footprint image and the key point coordinates of multiple footprint semantic key regions; S2: Input the preprocessed footprint image into the normalized spatial alignment module, perform lateral discrimination on the footprint image, and mirror the right foot sample horizontally to a unified normalized geometric space according to the discrimination result. At the same time, perform synchronous transformation on the coordinates of the corresponding key points to obtain the normalized footprint image and normalized key point coordinates. S3: Input the normalized footprint image into the region prototype branch, obtain the region feature map through the first feature extraction network, construct the corresponding region prototype set through the first prototype generation layer, and calculate the cosine similarity between the region feature map and the preset region prototype set to obtain the region prototype activation map. S4: Construct position alignment constraints based on the normalized key point coordinates to align the peak response position of the region prototype activation map with the key point coordinates of the corresponding semantic key region; S5: Generate a binary contour map based on the footprint image, input the binary contour map into the contour prototype branch, obtain the contour feature map through the second feature extraction network, construct a contour prototype set through the second prototype generation layer, and calculate the contour prototype activation map to extract the footprint edge morphology information. S6: Globally standardize the first classification score output by the region prototype branch and the second classification score output by the outline prototype branch. After the decision layer merges them, the final classification score is obtained, and the footprint identity category is output according to the final classification score. S7: During the training phase, the consistency score and stability score of each region prototype and contour prototype are calculated based on the training set, and a fixed prototype quality score is calculated; during the inference phase, high-quality prototypes are selected from the joint candidate prototype set composed of region prototypes and contour prototypes based on the fixed prototype quality score, and visual prototype evidence is generated to explain the footprint identity category.

[0010] Furthermore, the footprint image is a barefoot footprint image. The preprocessing in step S1 includes image cropping, resizing, and normalization, uniformly adjusting the size of the original footprint image to 224×224 pixels. During the training phase, the processed image dataset is divided into a training set and a test set in an 8:2 ratio. Data augmentation strategies including random scaling, random rotation, and resizing are applied only to the training set, while the test set retains the original image features for model performance evaluation.

[0011] Furthermore, the semantic key regions of the footprints mentioned in step S1 include at least the center of the toe pads, the first metatarsophalangeal joint, the fifth metatarsophalangeal joint, the center of the arch, and the center of the heel in the barefoot footprint image; the preprocessing also includes one or more of random scaling, rotation, and noise perturbation to enhance the intra-class variation of the training samples.

[0012] Furthermore, the lateral discrimination in step S2 is implemented by a pre-trained binary classification network; when the lateral discrimination result is right foot, a horizontal mirror transformation is performed on the footprint image, and a corresponding mirror coordinate transformation is performed on the horizontal coordinate of the key point according to the image width; when the lateral discrimination result is left foot, the footprint image and key point coordinates remain unchanged.

[0013] Furthermore, in step S3, the first feature extraction network is a convolutional neural network with the fully connected layers removed, which can be selected from any one of VGG, ResNet, and DenseNet; each region prototype in the preset region prototype set is assigned to a preset semantic key region.

[0014] Furthermore, the loss function for the position alignment constraint mentioned in step S4 is: ,in Indicates the first Regional prototype activation map This represents the spatial positioning function obtained from the maximum response coordinates. Indicates the relationship with the first The normalized key point coordinates corresponding to each region prototype.

[0015] Further, the decision-level fusion in step S6 includes: calculating the mean and standard deviation of the classification scores of the region prototype branch and the contour prototype branch on the training set respectively; in the inference stage, using the mean and standard deviation to perform Z-score standardization on the first classification score and the second classification score to obtain the standardized region score and the standardized contour score; performing additive fusion, weighted fusion or global standardization fusion on the standardized region score and the standardized contour score to obtain the final classification score.

[0016] Further, the consistency score in step S7 is used to characterize the proportion of prototype activation regions that hit true semantic key regions, and the stability score is used to characterize the proportion of prototypes that maintain the same key region activation patterns between the original image and the perturbed image; the fixed prototype quality score satisfy ,in For consistency score, For stability score, and The weights are non-negative.

[0017] Furthermore, the binary contour map mentioned in step S5 is generated by manual segmentation, threshold segmentation, deep segmentation network, or segmentation network based on the fusion of multi-granularity footprint features and regional relationships; in step S7, during the inference stage, the regional prototypes and contour prototypes are combined into a joint candidate set, and globally sorted according to the fixed prototype quality score, and the top-K prototypes with the highest scores are selected as the final visual explanation evidence.

[0018] Furthermore, the total loss function during the model training phase includes at least two of the following: classification loss, prototype clustering loss, prototype separation loss, orthogonality constraint loss, and position alignment loss, and the total loss function is expressed as: ,in , , and This is the loss balance coefficient.

[0019] Furthermore, the system also includes a domestically produced embedded development board, the visible light image acquisition module is mounted on the embedded development board or connected to it via a USB interface, and the embedded interactive display module is connected to the embedded development board via a display interface.

[0020] (III) Beneficial Effects This invention provides an interpretable footprint classification system based on a region-contour prototype network, which has the following advantages: 1. Solved the problem of prototype response semantic misalignment: By normalizing spatial alignment and positional alignment constraints, in the application of barefoot footprint images, the high-response regions of the regional prototype are stably located in key semantic parts such as toes, metatarsophalangeal joints, arches and heels, reducing the probability of prototype response misalignment and background activation, and significantly improving the credibility of the interpretation results.

[0021] 2. Complementary fusion of region and contour features is achieved: The collaborative modeling of region branch and contour branch is adopted, which utilizes both the texture details of the footprint's internal region and the contour structure information such as toe arrangement, foot shape boundary and edge curvature, making the identity classification evidence more complete.

[0022] 3. Improved stability of interpretation results: By fixing the prototype quality score in the training set and using a global Top-K prototype screening mechanism, low-quality prototypes with inconsistent semantics or unstable responses are effectively filtered out, making the final interpretation evidence more concentrated, stable and credible.

[0023] 4. Supports embedded local deployment: It can be deployed on domestic embedded development boards such as Orange Pi, and can be used with visible light cameras and touch screens to realize local data acquisition, local inference and interpretable result display. The entire processing flow can be completed without a network connection, which improves the practicality and privacy security of the system in security access control, criminal technical auxiliary analysis and on-site biometric identification. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the overall process of the footprint classification method of the present invention. Figure 2 This is a framework diagram for model training and detection in this invention; Figure 3 This is a schematic diagram of the dual-branch collaborative prototype network structure of the present invention; Figure 4 This is a schematic diagram of the position alignment constraint module of the present invention; Figure 5 This is a schematic diagram of the decision fusion and interpretation fusion mechanism of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Please see Figure 1-5 The present invention provides a technical solution: In this embodiment, the footprint image is preferably a barefoot footprint image, such as... Figure 2 As shown, model training and footprint classification include the following steps: The first step involved acquiring, labeling, and preprocessing the training images: Visible light image acquisition equipment was used to collect footprint images of individuals with different identities. During the acquisition process, subjects walked naturally through the acquisition area, and the equipment captured images of barefoot footprints formed when the soles of their feet were in contact with the ground. The acquisition scenarios included ground environments with different textures and reflective properties, such as wooden floors, composite mats, and ordinary tiles. After acquisition, each footprint image was labeled with its corresponding identity category, and nine key points were marked, including the center of the five toe pads, the first metatarsophalangeal joint, the fifth metatarsophalangeal joint, the center of the arch, and the center of the heel. The original footprint images were cropped, scaled, and normalized to a uniform size of 224×224 pixels. During the training phase, only the training set underwent data augmentation such as random cropping, rotation, noise addition, brightness changes, and contrast changes. The test set retained its original acquisition features for performance evaluation, and the training and test sets were divided in an 8:2 ratio.

[0027] The second step is optimal network training: the preprocessed footprint images are input into a location-supervised region-contour collaborative prototype network for training. The network includes a region prototype branch module, a contour prototype branch module, a location alignment constraint module, a two-branch decision fusion module, and an interpretation fusion module. The network parameters are iteratively optimized using the training set, and the model's classification accuracy, interpretation consistency, and interpretation stability are evaluated using the test set.

[0028] The specific steps for training the network structure modules in the second step are as follows: (1) Constructing Normalized Spatial Alignment and Region Prototype Branches: Since the left and right feet are mirror images in spatial structure, directly inputting them into the network for training may result in the same semantic part being in different spatial positions in different samples. Therefore, this invention first uses a lateral discrimination network to determine whether the input image is a left or right foot; if it is a right foot, a horizontal mirror transformation is performed and the key point coordinates are transformed simultaneously; if it is a left foot, it remains unchanged. The lateral discrimination network uses a pre-trained binary classification convolutional network to output the left / right foot probability.

[0029] The normalized footprint image is input into a region feature extraction network, preferably using ResNet34 as the feature extraction backbone, to obtain the region feature map. Let the set of region prototypes be... By calculating the cosine similarity between local features in the region feature map and each region prototype, the region prototype activation map is obtained. The region prototype activation map is used to represent the local regions of interest to the model in the current footprint image. Each region prototype is explicitly assigned to a predefined semantic key region, such as toe, metatarsophalangeal joint, arch, heel, etc.

[0030] (2) Constructing the position alignment constraint module: Calculate the response center of the j-th region prototype based on the region prototype activation map. And make it consistent with the corresponding normalization key points. Alignment. The position alignment loss is expressed as: By minimizing this loss, region prototypes are guided to key regions with clear biological semantics, such as toes, metatarsophalangeal joints, arches, and heels, thereby reducing the interference of background noise and non-key regions on classification and interpretation results.

[0031] (3) Constructing a contour prototype branch: To supplement the insufficient representation of the internal details of barefoot footprints by the region branch, this invention further constructs a contour prototype branch. First, a binary contour map corresponding to the footprint image is obtained through footprint segmentation or contour extraction methods. Then, the binary contour map is input into the contour feature extraction network. Obtain the contour feature map Let the set of contour prototypes be... By calculating the similarity between contour features and contour prototypes, an activation map of the contour prototype is obtained to capture structural information such as toe arrangement, arch shape, heel width, and foot boundary curvature. Since contour features are usually continuously distributed along the foot boundary, and keypoints mostly represent local center positions, the contour prototype branch does not impose keypoint position constraints, enabling it to adaptively learn discriminative boundary morphological features.

[0032] (4) Constructing a joint optimization objective: The region branch and the contour branch participate in the training together. The overall loss function includes classification loss, prototype clustering loss, prototype separation loss, prototype orthogonality loss, and position alignment loss, expressed as: .in, Used to optimize the boundary for footprint identity recognition. Used to enhance the close relationship between similar samples and their corresponding prototypes. Used to amplify the differences between different categories of prototypes Used to reduce redundancy between similar prototypes Used to improve spatial consistency between the regional prototype and key parts of the barefoot footprint.

[0033] (5) Construct a dual-branch decision fusion module: the region branch and the contour branch output classification scores respectively. and Because the two branches have different input formats and feature distributions, their output scores may exhibit scale differences. Therefore, the mean and standard deviation of the classification scores for the region branch and contour branch are calculated separately on the training set, and then calculated separately during the inference phase. and The final classification score is then obtained through summation, weighted fusion, or global standardized fusion. and with As a predicted identity category.

[0034] (6) Constructing an interpretation fusion module: After training, calculate the consistency score between each region prototype and contour prototype based on the training set. and stability score And calculate the fixed prototype mass fraction The consistency score characterizes the proportion of prototype activation regions that hit true semantic key regions, while the stability score characterizes the proportion of prototypes that maintain the same key region activation patterns between the original and perturbed images. During the inference phase, the prototype quality score is not recalculated; instead, the prototype activation response is calculated in real-time based on the current image, and the region prototypes and contour prototypes are combined into a joint candidate set. The prototypes are globally sorted according to their fixed quality scores, and the top-K prototypes with the highest scores are selected as the final explanatory evidence.

[0035] The third step involves the embedded footprint classification and interpretation function: The optimal model parameters, region prototypes, contour prototypes, fixed prototype quality scores, and dual-branch score statistics are imported into the embedded development board. The embedded development board connects to a visible light camera via a USB interface and to a touchscreen via a display interface to acquire real-time images of the footprints of the person to be identified.

[0036] The system first preprocesses the footprint images to be identified and performs spatial alignment of the left and right feet. Then, it calculates the prototype matching response, branch classification score, and prototype activation map by inputting region prototype branches and contour prototype branches respectively. Next, it outputs the identity classification result through dual-branch decision fusion and outputs the interpretability result through Top-K prototype filtering. Finally, the footprint classification result, prototype evidence, and visualized activation region are displayed on the embedded system's touch screen interface.

[0037] Based on the above steps, such as Figure 1 As shown, the specific operation steps of an interpretable footprint classification system based on a region-contour prototype network are as follows: S1: An embedded development board is used as the core hardware unit to collect on-site footprint images of the target personnel through an external visible light camera; S2: The collected footprint images are cropped, scaled, normalized, and spatially aligned with the left and right feet. Identity labels and key point coordinate annotations are obtained during the training phase. S3: Input the preprocessed footprint image into the region prototype branch, and obtain the region branch score and region prototype activation map through region feature extraction, region prototype matching and position alignment constraints; S4: Generate a binary contour map based on the footprint image, input the binary contour map into the contour prototype branch, and obtain the contour branch score and contour prototype activation map through contour feature extraction and contour prototype matching. S5: The region branch score and the contour branch score are statistically standardized and fused using the dual-branch decision fusion module to obtain the footprint identity classification result; S6: The interpretation fusion module performs global Top-K prototype screening based on the prototype quality scores fixed and saved during the training phase, generates interpretability results, and displays the classification results and interpretability results on the touch screen interactive interface.

[0038] Example 1: Standard Dataset and Training Settings This embodiment uses the self-built real-world footprint image dataset RSBD-200 for testing. This dataset serves as the preferred experimental data for the footprint classification system of this invention, used to verify the system's classification and interpretation performance on barefoot footprint images. The dataset contains 8433 on-site optical footprint images from 200 volunteers, covering various ground environments such as wooden floors and composite mats, and includes both left and right foot samples. Each image was cropped and uniformly adjusted to 224×224 pixels, and divided into training and testing sets in an 8:2 ratio. During training, data augmentation strategies such as random cropping, rotation, and noise addition were employed. The network is implemented using PyTorch, employing a two-stage training strategy: the backbone network is frozen for the first 5 epochs, training only the prototype layer and classification head; subsequently, the entire network is unfrozen for end-to-end joint fine-tuning. The preferred backbone network is ResNet34, with 9 prototypes per identity category.

[0039] Example 2: Performance Testing of Different Backbone Networks To verify the adaptability of the method of this invention to different backbone networks, tests were conducted on ResNet, VGG, and DenseNet series backbones. Evaluation metrics included classification accuracy, consistency score, and stability score. The results are as follows:

[0040] As shown in the table, this invention maintains high classification and interpretation performance across multiple convolutional backbone networks. Among them, ResNet34 achieves a good overall balance between accuracy, interpretation consistency, and stability, with a classification accuracy of 97.89%, interpretation consistency of 98.00%, and interpretation stability of 93.75%. Therefore, in this embodiment, ResNet34 is preferred as the default training backbone before deployment in the embedded system.

[0041] Example 3: Comparison with some prototype network methods To further verify the effectiveness of the method of the present invention, the present invention was compared with some representative prototype networks such as ProtoPNet, Proto-Rset, and Eval-ProtoPNet. The results are as follows:

[0042] The results show that the method of the present invention improves the accuracy of Eval-ProtoPNet from 93.38% to 97.89% on the ResNet34 backbone, the interpretation consistency from 81.11% to 98.00%, and the stability remains at 93.75%. This indicates that position alignment constraints, contour branch supplementation, and high-quality prototype screening in the interpretation stage can simultaneously improve classification ability and interpretation reliability.

[0043] Example 4: Ablation Experiment of Key Modules To verify the effectiveness of the normalized spatial alignment module, the positional alignment constraint module, and the dual-branch fusion strategy, ablation tests were conducted with different module combinations. The results are as follows:

[0044] Experimental results show that: position alignment constraints can improve the spatial stability of the model for key regions; bi-branch fusion can significantly improve interpretability consistency; when all three are combined, the model achieves the best overall result in terms of accuracy, interpretability consistency and stability.

[0045] Example 5: Fusion Strategy and Interpretation Screening Experiment This embodiment is used to verify the impact of different bi-branch decision fusion strategies and different interpretation fusion strategies on model performance. The experiment compares the performance of the classification decision layer and the interpretation generation layer, and the results are shown in the table below.

[0046] (1) Integration strategy of different decision-making levels

[0047] As shown in the table, different decision-level fusion strategies have a certain impact on the accuracy of footprint classification. On the barefoot footprint dataset in this embodiment, the global Z-score normalization fusion strategy achieved a classification accuracy of 97.89%. The fusion method of directly taking the maximum value had an accuracy of 93.96%, which was significantly lower than other fusion strategies. This indicates that without sufficient scale calibration, simply selecting the larger response in the branch is easily affected by abnormally high scores or differences in branch scale. The classification accuracy of the additive fusion strategy and the intra-class Z-score normalization fusion strategy reached 97.81% and 97.85%, respectively, which are close to the optimal results. The above results show that reasonable fusion of the classification scores of the region branch and the contour branch can make full use of the complementary information of the two in the details of the footprint region and the contour structure, thereby improving the stability and accuracy of the classification results.

[0048] (2) Different interpretation fusion methods

[0049] As shown in the table, different interpretation fusion methods significantly impact the consistency and stability of the final interpretation results. The consistency score of the region single branch is 81.72%, higher than that of the contour single branch (58.94%), indicating that the region prototype is more likely to form a stable correspondence with the key semantic regions of the barefoot footprint; however, using only a single branch is still insufficient to fully utilize the contour structure information. The consistency scores of methods such as randomly selecting K prototypes, global averaging, and weighted averaging are all relatively low, indicating that if the quality of candidate prototypes is not effectively screened, low-quality or semantically unstable prototypes are more likely to enter the final interpretation set, thereby weakening the credibility of the interpretation results.

[0050] In contrast, the Top-K prototype selection strategy significantly improved the explanatory results. Specifically, the intra-class Top-K selection achieved consistency and stability scores of 96.17% and 92.59%, respectively; the global Top-K selection further improved these scores to 98.00% and 93.75%, achieving the best overall performance. These results demonstrate that the global Top-K prototype selection strategy based on a fixed quality score in the training set can filter out higher-quality explanatory evidence from a unified candidate set composed of region and contour prototypes, effectively filtering out low-quality prototypes with semantic inconsistencies or unstable responses, resulting in a more focused, stable, and reliable final visualization explanation.

[0051] Example 6 Embedded Board Deployment and Physical System This embodiment imports the trained optimal model parameters, region prototype library, contour prototype library, fixed prototype quality score, and bi-branch score statistics into an embedded development board. The embedded development board can be an Orange Pi development board or an equivalent domestic embedded development board, connected to an external USB visible light camera and a touchscreen display module, forming a portable physical system. During system operation, the camera captures footprint images from the site, preferably barefoot footprint images. The development board performs image preprocessing, region prototype matching, contour prototype matching, bi-branch score fusion, and Top-K prototype interpretation filtering locally, displaying the predicted identity category, Top-K prototype evidence, and their corresponding activation regions on the touchscreen interface. This deployment method does not rely on cloud uploads, offering good privacy protection and ease of on-site application.

[0052] This invention enables the model to gradually learn the spatial distribution patterns of local regional differences, contour morphology differences, and key semantic regions in footprint images throughout the entire network optimization process. Once the model obtains optimal parameters through iterative training, it can be used as a tool for footprint identification and interpretability analysis. This method not only enables footprint identification but also outputs prototype evidence and visualized activation regions to support the classification results, thereby improving the model's credibility in scenarios such as security identification, criminal forensic identification, and biometric identification.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An interpretable footprint classification system based on a region-contour prototype network, characterized in that, Includes the following steps: S1: Acquire the footprint image of the person to be identified and perform preprocessing; during the training phase, perform data augmentation on the training set and acquire the identity category label corresponding to the footprint image and the key point coordinates of multiple footprint semantic key regions; S2: Input the preprocessed footprint image into the normalized spatial alignment module, perform lateral discrimination on the footprint image, and mirror the right foot sample horizontally to a unified normalized geometric space according to the discrimination result. At the same time, perform synchronous transformation on the coordinates of the corresponding key points to obtain the normalized footprint image and normalized key point coordinates. S3: Input the normalized footprint image into the region prototype branch, obtain the region feature map through the first feature extraction network, construct the corresponding region prototype set through the first prototype generation layer, and calculate the cosine similarity between the region feature map and the preset region prototype set to obtain the region prototype activation map. S4: Construct position alignment constraints based on the normalized key point coordinates to align the peak response position of the region prototype activation map with the key point coordinates of the corresponding semantic key region; S5: Generate a binary contour map based on the footprint image, input the binary contour map into the contour prototype branch, obtain the contour feature map through the second feature extraction network, construct a contour prototype set through the second prototype generation layer, and calculate the contour prototype activation map to extract the footprint edge morphology information. S6: Globally standardize the first classification score output by the region prototype branch and the second classification score output by the outline prototype branch. After the decision layer merges them, the final classification score is obtained, and the footprint identity category is output according to the final classification score. S7: During the training phase, calculate the consistency score and stability score of each region prototype and contour prototype based on the training set, and calculate the quality score of the fixed prototype. During the reasoning phase, high-quality prototypes are selected from the joint candidate prototype set composed of regional prototypes and contour prototypes based on the fixed prototype quality score, and visual prototype evidence is generated to explain the footprint identity category.

2. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The footprint images are barefoot footprint images. The preprocessing in step S1 includes image cropping, resizing, and normalization, which uniformly adjusts the size of the original footprint images to 224×224 pixels. During the training phase, the processed image dataset is divided into a training set and a test set in an 8:2 ratio. Data augmentation strategies including random scaling, random rotation, and resizing are applied only to the training set. The test set retains the original image features for model performance evaluation.

3. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The semantic key regions of footprints mentioned in step S1 include at least the center of the toe pads, the first metatarsophalangeal joint, the fifth metatarsophalangeal joint, the center of the arch, and the center of the heel in the barefoot footprint image; the preprocessing also includes one or more of random scaling, rotation, and noise perturbation to enhance the intra-class variation of the training samples.

4. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The lateral discrimination described in step S2 is implemented by a pre-trained binary classification network; when the lateral discrimination result is right foot, a horizontal mirror transformation is performed on the footprint image, and the corresponding mirror coordinate transformation is performed on the horizontal coordinate of the key point according to the image width; when the lateral discrimination result is left foot, the footprint image and key point coordinates remain unchanged.

5. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: In step S3, the first feature extraction network is a convolutional neural network with the fully connected layers removed, which can be selected from any one of VGG, ResNet, and DenseNet; each region prototype in the preset region prototype set is assigned to a preset semantic key region.

6. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The loss function for the position alignment constraint mentioned in step S4 is: ,in Indicates the first A prototype activation map of a region represents the spatial localization function obtained from the coordinates of the maximum response. Indicates the relationship with the first The normalized key point coordinates corresponding to each region prototype.

7. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The decision-level fusion in step S6 includes: calculating the mean and standard deviation of the classification scores of the region prototype branch and the contour prototype branch on the training set respectively; in the inference stage, using the mean and standard deviation to perform Z-score standardization on the first classification score and the second classification score to obtain the standardized region score and the standardized contour score; performing additive fusion, weighted fusion or global standardization fusion on the standardized region score and the standardized contour score to obtain the final classification score.

8. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The consistency score in step S7 is used to characterize the proportion of prototype activation regions that hit true semantic key regions, and the stability score is used to characterize the proportion of prototypes that maintain the same key region activation patterns between the original image and the perturbed image; the fixed prototype quality score satisfy ,in For consistency score, For stability score, and The weights are non-negative.

9. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The binary contour map mentioned in step S5 is generated by manual segmentation, threshold segmentation, deep segmentation network, or segmentation network based on the fusion of multi-granularity footprint features and regional relationships; in step S7, during the inference stage, the regional prototype and the contour prototype are combined into a joint candidate set, and the prototypes are globally sorted according to the fixed prototype quality score. The top-K prototypes with the highest scores are selected as the final visual explanation evidence.

10. The interpretable footprint classification system based on a region-contour prototype network according to claim 1, characterized in that: The total loss function during model training includes at least two of the following: classification loss, prototype clustering loss, prototype separation loss, orthogonality constraint loss, and position alignment loss, and the total loss function is expressed as: ,in , , and This is the loss balance coefficient.

Images