Shoeprint point cloud identification method based on image processing

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a shoeprint point cloud identification method based on image processing, and the method comprises the steps: obtaining the original point cloud data of a real shoeprint through a shoeprint scanning device built by a structured light sensor; performing preprocessing operations such as cutting, normal vector consistency noise reduction and secondary noise reduction on the original point cloud data in sequence, and constructing a three-dimensional shoeprint data set; then, on the basis of PointNet + +, constructing a classification model containing a multi-scale feature extraction layer and a channel-space attention layer, and dividing a data set to train and test the model; and finally, to-be-identified shoeprint data is collected in real time, preprocessed and input into the model, and a shoeprint classification result is output through multi-scale feature extraction and channel and space attention calibration. According to the method, the three-dimensional point cloud data is accurately collected by using the structured light sensor, accurate recognition of shoeprints is realized through efficient preprocessing and an improved network model, and the method has important application value in the fields of criminal investigation and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a shoe print point cloud recognition method based on image processing. BACKGROUND

[0002] In the process of criminal investigation, shoe prints, as common important trace evidence in crime scenes, contain features such as sole patterns, sizes and gaits, which play an irreplaceable role in inferring the identity of suspects and narrowing the scope of investigation. According to statistics, about 30% of crime scenes have extractable shoe print traces. The geometric structure of the sole pattern can be used to infer the brand, model and production batch of the shoe. Combined with the size and contour features of the shoe print, the physiological parameters of the suspect, such as height and weight, can be further inferred. Even the gait features can reveal the movement habits and behavior patterns of the suspect. The classification of shoe brands can directly associate with the characteristics of the suspect's clothing, economic status and social background, providing a key index for clue screening. Therefore, efficient and accurate recognition of shoe prints is crucial for improving the efficiency of criminal investigation.

[0003] However, traditional shoe print recognition techniques have significant limitations in data collection and feature analysis, making it difficult to meet the needs of recognition accuracy and efficiency in criminal investigation scenarios. In terms of data collection, traditional methods mainly rely on manual casting methods (such as silicone or plaster molding) and two-dimensional photography recording: manual casting requires direct contact with the trace, which can easily cause shoe print deformation or object residue, damaging existing evidence, and is time-consuming and costly. Although two-dimensional photography improves the collection speed, it loses key depth information such as groove depth and edge curvature when compressing three-dimensional shoe print traces into two-dimensional images, and these information is important for fine differentiation of sole patterns.

[0004] In terms of feature analysis and recognition, early techniques are mostly based on two-dimensional image processing methods, such as designing rotationally and translationally invariant features through frequency domain analysis (DFT modulus, power spectral density) or geometric transformation (Radon transform pre-alignment, key point space matching) to achieve coarse-grained comparison and classification of shoe prints. In recent years, deep learning methods have gradually become mainstream. Although feature learning frameworks based on convolutional neural networks (CNN) (such as multi-part weighted convolutional neural networks and improved YOLO models) have improved accuracy in two-dimensional shoe print retrieval tasks, their core logic is still limited to two-dimensional image feature representation, and they cannot effectively integrate three-dimensional shoe print depth distribution and microscopic deformation information, resulting in insufficient classification robustness and difficulty in meeting the actual needs of shoe print patterns and complex shapes in criminal investigation scenarios.

[0005] To break through the limitations of two-dimensional technology, three-dimensional point cloud technology provides a new technical path for shoeprint identification. Non-contact three-dimensional measurement technology (such as structured light and laser scanning) can achieve high-precision topographic reconstruction of shoeprints. The structured light technology can complete three-dimensional reconstruction with micron-level precision within 0.1 seconds of a single frame by projecting coded gratings and analyzing phase information, and has the advantages of fast acquisition speed and high resolution. The non-contact feature can avoid damage to the evidence, and the high-resolution feature can completely preserve the microscopic geometric characteristics of the sole pattern, providing multi-dimensional data support for fine shoeprint identification. However, three-dimensional point cloud data processing still faces algorithm challenges: traditional point cloud analysis methods (such as point cloud registration and curvature mapping) rely on manual feature design and are difficult to adapt to the complex topological structure of shoeprints in criminal investigation scenes; although existing deep learning models (such as PointNet++) support adaptive feature learning, their general architecture lacks targeted optimization for local details of shoeprints (such as groove direction and pattern density), and the feature aggregation process easily weakens subtle differences, is sensitive to noise, and is difficult to distinguish between non-rigid deformation and real individual differences, resulting in limited shoeprint classification accuracy and inability to fully utilize the advantages of three-dimensional point cloud data.

[0006] In addition, existing shoeprint identification research relies heavily on two-dimensional image datasets, and there is a lack of high-quality three-dimensional shoeprint datasets covering various shoe types and gender attributes. The scarcity of data resources further limits the development and optimization of three-dimensional shoeprint identification algorithms. Meanwhile, during model training, traditional loss functions can easily lead to excessive confidence in labels, and the adaptability of optimizers and learning rate scheduling strategies is insufficient, which also affects the convergence efficiency and generalization ability of the model, making it difficult to meet the stringent requirements of real-time and accuracy in criminal investigation scenarios.

[0007] Therefore, there is still room for improvement in current shoeprint identification technology in terms of data acquisition integrity, three-dimensional feature analysis, and model recognition accuracy. The present invention aims to provide a shoeprint point cloud identification method based on image processing, which combines efficient data preprocessing, targeted feature extraction, and optimized training strategies to break through the technical bottlenecks of existing technology and provide more reliable technical support for intelligent shoeprint identification in the field of criminal investigation. SUMMARY

[0008] The present invention aims to provide a shoeprint point cloud identification method based on image processing to solve the technical problems of current shoeprint identification technology in terms of data acquisition integrity, three-dimensional feature analysis, and model recognition accuracy.

[0009] To solve the above technical problems, the technical solution adopted by the present invention is as follows: A shoeprint point cloud identification method based on image processing, comprising the following steps: S1: A shoe print scanning device is built based on a structured light sensor, and a real shoe print is scanned by the shoe print scanning device to obtain original point cloud data containing three-dimensional coordinates; S2: The original point cloud data is preprocessed by sequentially performing clipping, normal vector consistency denoising, and secondary denoising to construct a three-dimensional shoe print dataset; S3: Based on PointNet++, a classification model containing a multi-scale feature extraction layer and a channel-spatial attention layer is constructed, and a training set and a test set are divided from the three-dimensional shoe print dataset to input the classification model for training and testing; S4: Real-time shoe print data to be identified is collected, preprocessed, and input into the classification model, the multi-scale feature extraction layer performs twice farthest point downsampling on the preprocessed shoe print point cloud data input, and after extracting features of different scales, the features are spliced and fused to obtain a fused feature map; S5: The channel-spatial attention layer is composed of a channel attention submodule and a spatial attention submodule in cascade, the channel attention submodule performs spatial dimension compression on the fused feature map through adaptive average pooling and adaptive maximum pooling to obtain two channel description vectors, which are summed after being transformed by a multilayer perceptron, and then a channel attention weight is generated through a Sigmoid function; S6: The spatial attention submodule performs average pooling and maximum pooling on the feature map along the channel dimension to generate two spatial feature maps, which are spliced along the channel dimension and then modeled by a convolution layer to model the spatial relationship, and then a spatial attention weight is generated through a Sigmoid function, the feature is calibrated through the channel-spatial attention layer, and finally a shoe print classification result is output.

[0010] Preferably, the shoe print scanning device in step S1 comprises a main frame and a console, the console is arranged on the inner top wall of the main frame, and a structured light sensor is arranged on the console.

[0011] Preferably, the specific process of sequentially performing clipping, normal vector consistency denoising, and secondary denoising in step S2 is as follows: S21: The region containing the complete shoe print is framed, and irrelevant data such as ground points and device support points outside the shoe print in the region is removed; S22: The normal vector of each point in the clipped point cloud data is calculated, that is, by analyzing the spatial positional relationship of each point with the points in its neighborhood, the normal direction of the surface where the point is located is determined; The consistency of the normal vector of each point with the normal vectors of other points in the neighborhood is counted, if the normal vector of a point deviates greatly from the normal vectors of most points in the neighborhood, the point is determined to be a noise point and is removed; S23: The point cloud data is checked region by region by a point cloud visualization tool, and scattered outliers and irrelevant region points that do not conform to the geometric shape of the shoe print are deleted to eliminate noise points and irrelevant region points.

[0012] Preferably, the training process of step S3 sets a cross-entropy loss function, and the specific expression is as follows: ; Wherein, C is the total number of categories, and ϵ∈(0, 1) is a smoothing parameter, p t represents the predicted probability of the real category t, is the label value after label smoothing, is the cross-entropy loss function after label smoothing, p i is the predicted probability of the model to the first i category.

[0013] Preferably, the model training process in step S3 is as follows: S31: Initialize the parameters of the classification model, set the key training parameters, including fixing the number of input point clouds to 4096 points, setting the training batch size to 4, and setting the total training rounds to 300; At the same time, configure the basic parameters of the AdamW optimizer and the learning rate range of OneCycleLR; S32: In each training batch of each round, based on the model prediction result and the real label, the loss value of the current batch is calculated through the cross-entropy loss function of label smoothing, and the gradient calculation and update of the model parameters are completed through the AdamW optimizer: S33: During the entire 300-round training period, the OneCycleLR learning rate scheduling strategy dynamically adjusts the learning rate according to the linear rise-linear fall rule, and provides adaptive learning rate for different training stages; S34: After each round of training, the performance of the model is evaluated using the test set to monitor the training trend of the model; When the training round reaches the specified round preset value, or the performance of the model on the test set has no obvious improvement for several rounds in a row, the training is terminated, and the final trained model is obtained.

[0014] Preferably, the specific processing process of the multi-scale feature extraction layer in step S4 for the preprocessed shoeprint point cloud data input is as follows: S41: The multi-scale feature extraction layer performs two times of farthest point downsampling on the preprocessed shoeprint point cloud data input in turn, respectively obtaining 4096-point and 1024-point point clouds, and performing three different radius ball queries to form groups on each point at each layer. S42: Use a multi-scale feature extraction method to capture features of different scales in the point cloud, and splice the features of each scale; S43: After splicing, send to the channel-space attention module for weighting, use a learnable 1x1 convolution kernel to dynamically adjust the weight, implicitly learn the linear combination relationship between each scale feature, and realize feature fusion.

[0015] Preferably, the specific processing process of the channel attention submodule in step S5 is as follows: S51: The channel attention submodule receives the fused feature map, performs feature compression along the spatial dimension through adaptive average pooling and adaptive maximum pooling respectively, and obtains two channel description vectors F avg And F max ; S52: Transform through a multilayer perceptron, and the two channel description vectors F avg And F max are subjected to a nonlinear activation function Sigmoid to obtain channel attention weights; S53: The output channel attention weights are weighted with the original feature map channel by channel to capture the nonlinear dependence between channels.

[0016] Preferably, the specific processing process of the spatial attention submodule in step S6 is as follows: S61: Average pooling and maximum pooling operations are respectively performed along the channel dimension to generate two spatial feature maps F1 and F2; S62: After splicing along the channel dimension, a 1D convolution layer with symmetric padding is used to model the spatial relationship; S63: A spatial attention weight matrix is generated through a Sigmoid function, and finally the spatial dimension of the feature map is calibrated through position-by-position multiplication.

[0017] The beneficial effects of the present application include: 1. Accurate and efficient data acquisition: a structured light sensor is used to build a shoe print scanning device, non-contact scanning avoids damage to shoe print evidence, micron-level scanning accuracy and 0.1 second single-frame scanning time can complete and quickly obtain original point cloud data containing three-dimensional coordinates of shoe prints, and retain the micro geometric features of the shoe tread, providing a high-quality data foundation for subsequent accurate identification. Through pre-processing denoising: sequentially perform cropping, normal vector consistency denoising, secondary denoising and operation, the combination of cropping and normal vector analysis can effectively remove irrelevant data and noise, and secondary denoising further eliminates residual outlier noise, thereby retaining key geometric features of shoe prints to the greatest extent, and providing pure and efficient point cloud data for model training and identification.

[0018] 2. Strong model feature extraction capability: the classification model based on PointNet++ has a multi-scale feature extraction layer that can fully capture the features of shoeprint point clouds at different scales through multiple farthest point downsampling, spherical query grouping, and feature splicing and fusion. The channel-space attention level cascades the channel and spatial attention submodules to calibrate the feature map from the channel and spatial dimensions, respectively, to strengthen key features and suppress irrelevant features, significantly improving the model's extraction and differentiation ability of shoeprint features.

[0019] 3. Scientific and efficient model training: the label smoothing cross-entropy loss function is used to avoid the model's excessive confidence in the label and improve the generalization ability; the AdamW optimizer improves the weight decay method to effectively prevent overfitting; the OneCycleLR learning rate scheduling strategy dynamically adjusts the learning rate to make the model update parameters at an appropriate learning rate in different training stages, accelerating convergence and improving accuracy, and the trained model has better performance.

[0020] 4. Accurate and reliable recognition results: real-time acquisition of shoeprint data to be identified, preprocessed, and input into the trained classification model, which can accurately output shoeprint classification results through multi-scale feature extraction and channel-space attention calibration, and can be widely used in criminal investigation and other fields to provide strong technical support for shoeprint analysis and has important practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The flowchart of the shoeprint point cloud recognition method based on image processing of the present application.

[0022] Figure 2 The overall architecture of the classification model of the present application.

[0023] Figure 3 The architecture of the multi-scale feature extraction layer of the present application.

[0024] Figure 4 The structure diagram of the shoeprint scanning device of the present application. DETAILED DESCRIPTION

[0025] The following will be described in detail with reference to the accompanying drawings. Figures 1-4 Further detailed description of the present application: Example 1 Referring to the accompanying drawings, Figure 1 A shoeprint point cloud recognition method based on image processing includes the following steps: S1: A shoeprint scanning device is built based on a structured light sensor, and real shoeprints are scanned by the shoeprint scanning device to obtain original point cloud data containing three-dimensional coordinates (X, Y, Z); S2: preprocessing the original point cloud data, sequentially performing clipping, normal vector consistency denoising, secondary denoising and farthest point downsampling operations, and constructing a three-dimensional shoeprint dataset based on the preprocessed point cloud data; Clipping retains the shoeprint data and removes irrelevant information, normal vector consistency denoising extracts the shoeprint and pattern contour by calculating the normal vector to filter out points with excessive deviation, secondary denoising excludes residual outliers, and the preprocessed shoeprint point cloud data is obtained; S3: based on PointNet++, a classification model containing multi-scale feature extraction layers and channel-spatial attention layers is constructed, and the overall structure of the model is shown in Figure 2 The shoeprint point cloud classification network structure based on multi-scale feature and channel-spatial attention fusion is shown.

[0026] The network mainly includes the following modules: Data preprocessing module: used for preliminary processing of original shoeprint point cloud data to filter out irrelevant information as much as possible, and unify the original data format for subsequent multi-level feature extraction, including: manual clipping: removing background and irrelevant areas; normal vector filtering: calculating and filtering out points with rich contour information in the data; farthest point sampling: uniformly selecting a number of representative points according to spatial distance.

[0027] Multi-scale feature extraction module: extracting geometric features of point cloud under different sampling scales. The point cloud data at each scale is sequentially input into multiple multilayer perceptrons, and the point-by-point features are spliced and then mapped and reduced in channel through 1x1 convolution to form "fused features". The role of this module is to fully extract the geometric shape and texture information of shoeprints under different scales, and realize multi-level representation of local details and overall structure.

[0028] Channel-spatial attention module: this module uses a double attention mechanism to weight the features, including: Channel attention mechanism: adaptively assigning weights according to the importance of each feature channel to strengthen key geometric features; Spatial attention mechanism: generating an attention weight map in the spatial dimension to highlight the shoeprint area and significant morphological change area.

[0029] The two kinds of attention weights are multiplied and then fed back to the feature vector to enhance the discriminability of feature expression.

[0030] Classification module: after feature fusion, the network uses a combination of max-pooling and average-pooling operations to extract global features, and then outputs the classification results of shoeprints through a classification fully connected layer.

[0031] Multi-scale feature extraction layer, see Figure 3The module input is the original point cloud data (N x 3), N is the number of input original point cloud data points, after the farthest point sampling, the feature extraction and fusion operation are performed. The specific process is as follows: Farthest point sampling: N points are uniformly selected from the original point cloud according to the spatial distance, which is used to capture the geometric semantics at different scales.

[0032] Multi-layer perception feature extraction: in each layer, each point in the input point cloud data is constructed into three different perception domain subsets according to the geometric adjacent relationship, and the point cloud subsets in different perception domains are input into the multi-layer perception network.

[0033] Multi-layer perception is used to extract and abstract local features point by point, and the feature vectors output by multiple perceptrons are spliced to form a high-dimensional feature matrix containing different scale features.

[0034] 1x1 convolution and activation function processing: the spliced feature matrix is mapped and compressed through the 1x1 convolution layer, and the non-linear features are introduced through the ReLU activation function. The output "fused features" represent the comprehensive point cloud features at this scale.

[0035] Multi-scale feature fusion: the fused feature matrix is down-sampled again to extract comprehensive features at different scales layer by layer until a unique global feature vector is obtained. The global feature vector is used as the input of the subsequent channel-spatial attention module.

[0036] The training set divided from the three-dimensional shoeprint dataset is input into the classification model for training. During the training process, the label smoothing cross-entropy loss function, AdamW optimizer and OneCycleLR learning rate scheduling strategy are used for training. After the training is completed, the test set is used to test the classification model.

[0037] S4: Real-time acquisition of shoeprint data to be identified, preprocessed and input into the validated classification model. The multi-scale feature extraction layer performs two farthest point down-sampling on the input preprocessed shoeprint point cloud data, respectively obtaining 4096 points and 1024 points of point cloud. In each layer, three different radius ball queries are performed on each point to form groups. The features of each group are extracted by the multi-layer perception with shared weights, and the features at different scales are spliced and fused by 1x1 convolution to obtain a fused feature map. S5: The channel-spatial attention layer is composed of a channel attention submodule and a spatial attention submodule. The channel attention submodule performs spatial dimension compression on the fused feature map through adaptive average pooling and adaptive maximum pooling to obtain two channel description vectors. After being transformed by the multi-layer perception, the two channel description vectors are summed and the channel attention weight is generated by the Sigmoid function. S6: The spatial attention submodule performs average pooling and maximum pooling on the feature map along the channel dimension to generate two spatial feature maps, concatenates them along the channel dimension, models the spatial relationship through a 1D convolution layer with a kernel size of 7, and then generates spatial attention weights through a Sigmoid function. The feature is calibrated through the channel-spatial attention layer, and the shoe print classification result is output.

[0038] In this embodiment, referring to Figure 4 , the shoe print scanning device in step S1 includes a main body frame and a console, the console is arranged on the inner top wall of the main body frame, and a structured light sensor is arranged on the console.

[0039] Embodiment 2 Based on embodiment 1, the specific process of sequentially performing the operations of cropping, normal vector consistency denoising, secondary denoising, and farthest point downsampling in step S2 is as follows: S21: The region containing the complete shoe print is framed, irrelevant data such as ground points and device support points outside the shoe print in the region is removed, and it is ensured that only the shoe print main body is retained in the data set after cropping; S22: The normal vector of each point in the point cloud data after cropping is calculated, that is, the normal direction of the surface on which the point is located is determined by analyzing the spatial positional relationship of the point with the points in its neighborhood; The consistency of the normal vector of each point with the normal vectors of other points in its neighborhood is counted, and if the normal vector of a point deviates greatly from the normal vectors of most points in the neighborhood, the point is determined to be a noise point (such as an isolated environmental interference point or a false point on the edge of the shoe print) and is removed; Through this operation, noise points can be removed, and the pattern contour features of the shoe print surface can be strengthened, thereby reserving key geometric information for subsequent classification; S23: The point cloud data is checked region by region through a point cloud visualization tool, and outlying points that are sporadically distributed and do not conform to the geometric shape of the shoe print are deleted, so that the processed data contains almost no or only a small amount of irrelevant information, and noise points and irrelevant area points are eliminated.

[0040] The specific expression of the cross-entropy loss function in step S3 is as follows: ; wherein, C is the total number of categories, and ϵ∈(0, 1) is a smoothing parameter for controlling the degree of label smoothing to avoid excessive confidence of the model on the label, p t represents the predicted probability of the real category t, is the label value after label smoothing processing, is the cross-entropy loss function after label smoothing, and is used to measure the difference between the predicted probability distribution of the model and the smoothed label distribution, pi For the model to the first i The predicted probability of the class.

[0041] The model training process in step S3 is as follows: S31: Initialize the parameters of the classification model, including the convolution kernel weights of the multi-scale feature extraction layer, the perceptron parameters of the channel-space attention layer, and the fully connected weights of the classification layer, and set the key training parameters, including fixing the input point cloud number to 4096 points, the training batch size to 4, and the total training epoch to 300; at the same time, configure the basic parameters of the AdamW optimizer (such as the weight decay coefficient for regularization) and the learning rate range of OneCycleLR (the initial learning rate is 1e-3, and the maximum value is adapted according to the model training characteristics); S32: In each training batch of each epoch, based on the model prediction result and the true label, the loss value of the current batch is calculated by the cross-entropy loss function with label smoothing, and the gradient calculation and update of the model parameters are completed by the AdamW optimizer: Gradient backpropagation: taking the loss value LLS as the target, all learnable parameters of the improved PointNet++ network are traversed through the backpropagation algorithm to calculate the gradient (i.e. the partial derivative of the parameter with respect to the loss value) of each parameter, reflecting the influence of parameter change on the loss.

[0042] First-order moment and second-order moment estimation by AdamW optimizer: Calculate the first-order moment (momentum term) and second-order moment (adaptive learning rate term) of the gradient, the first-order moment is used to accelerate the gradient descent and reduce the training shock, and the second-order moment is used to adaptively adjust the learning rate for different parameters, so that the sparse gradient parameters obtain a larger update step; Weight decay application: AdamW directly applies weight decay (regularization) to the model parameters, which can be simplified as: θ t+1 = θ t − η · (Adam gradient update term + λ· θ t ), where θ is the model parameter, η is the current learning rate (provided dynamically by OneCycleLR), and λ is the weight decay coefficient, which effectively prevents model overfitting and improves generalization ability; Parameter update: based on the above calculation results, all learnable parameters of the network are iteratively updated to complete the training step of the current batch.

[0043] S33: During the entire 300-round training period, the OneCycleLR learning rate scheduling strategy dynamically adjusts the learning rate according to the linear rise-linear fall rule to provide adaptive learning rates for different training stages. The specific process is as follows: Learning rate rising stage: In the first half of the training period (about the first 150 rounds), the learning rate is linearly raised from the initial value 1e-3 to the preset maximum value. In this stage, the model is in the parameter rapid exploration period, and a higher learning rate helps the model to quickly converge to a better parameter region, avoiding falling into a local optimum.

[0044] Learning rate falling stage: In the second half of the training period (about the last 150 rounds), the learning rate is linearly decreased from the maximum value to a much smaller minimum value (such as 1e-5 order of magnitude) than the initial value. In this stage, the model enters the parameter fine-tuning period, and a lower learning rate allows the parameters to be updated slowly in the optimal region, further optimizing the model accuracy and reducing the risk of overfitting.

[0045] Adaptation with AdamW: Before each training round, OneCycleLR calculates and outputs the learning rate of that round according to the period position of the current training round, which is used as the update step of AdamW optimizer, realizing the collaborative work of optimizer-learning rate scheduling, and ensuring the efficiency and stability of the training process.

[0046] S34: After each training round, the model is evaluated for performance using the test set (using Top1 accuracy, Top3 accuracy, precision, recall, F1 value, etc.) to monitor the model training trend; when the training round reaches the preset value of 300 rounds, or the model performance on the test set does not significantly improve for several consecutive rounds (judged as convergence), the training is terminated, and the final trained model is obtained for subsequent shoeprint classification tasks.

[0047] Embodiment 3 Based on Embodiment 1 or Embodiment 2, the specific processing process of the multi-scale feature extraction layer for the preprocessed shoeprint point cloud data input in step S4 is as follows: S41: The multi-scale feature extraction layer performs two times of farthest point downsampling on the input preprocessed shoeprint point cloud data in turn, respectively obtaining 4096-point and 1024-point point clouds, and performing ball query with three different radii to form groups for each point at each layer; S42: A multi-scale feature extraction method is used to capture features of different scales in the point cloud, and the features of each scale are spliced; S43: After splicing, it is sent to the channel-spatial attention module for weighting, using a learnable 1x1 convolution kernel to dynamically adjust the weight, implicitly learning the linear combination relationship between the features of each scale, realizing feature fusion, thereby enhancing the perception ability of local features between different scales. The specific expression of feature fusion is as follows: ; wherein, is a weight coefficient, F i represent features in different scales, is a fused feature.

[0048] The specific processing process of the channel attention sub-module in step S5 is as follows: S51: The channel attention sub-module receives the fused feature map, performs feature compression along the spatial dimension through adaptive average pooling and adaptive maximum pooling respectively, to obtain two channel description vectors F avg and F max ; S52: Transform through a multi-layer perception, then sum the output of the two vectors, and obtain the required channel attention weight through a nonlinear activation function Sigmoid, the specific expression is as follows: ; wherein, σ represents a Sigmoid function, W channel represents a channel attention weight, MLP is a multi-layer perception, used for nonlinear transformation of two channel description vectors F avg and F max ; S53: The output channel attention weight W channel is weighted with the original feature map channel by channel, so as to capture the nonlinear dependence between channels.

[0049] The specific processing process of the spatial attention sub-module in step S6 is as follows: S61: Perform average pooling and maximum pooling operations along the channel dimension respectively to generate two spatial feature maps F1 and F2; S62: After splicing along the channel dimension, model the spatial relationship through a 1D convolution layer with symmetric padding, the specific expression is as follows: ; wherein, Conv1D k represents a 1D convolution with a convolution kernel size k, k is 7; S63: Generate a spatial attention weight matrix through a Sigmoid function, and finally realize spatial dimension calibration of the feature map through position-by-position multiplication; During feature propagation, the importance of each feature channel is first rescaled by channel attention, and then the response strength of key spatial positions is enhanced by spatial attention. This two-stage attention mechanism is formulated as: F out =W spatial · W channel ·F fused

[0050] The performance of the classification model is evaluated by setting up corresponding experiments: The point cloud number of input data is set to 4096, the batch size is set to 4, the AdamW optimizer is used, the model is trained for 300 epochs, the initial learning rate is set to 1e-3, and the OneCycleLR learning rate scheduling strategy is used to dynamically adjust the learning rate.

[0051] The overall accuracy Top1, overall accuracy Top3, precision, recall, and F1 value are used as performance evaluation indicators of each classification model. The calculation formulas are as follows: The overall accuracy Top1 and overall accuracy Top3 reflect the overall classification ability of the model; the precision and recall evaluate the accuracy and coverage of the prediction results respectively; the F1 value comprehensively evaluates the precision and recall, providing a more comprehensive performance evaluation.

[0052] ​​​​​​​​​​​To verify the performance of the model, the present application conducts comparative experiments on a self-built shoeprint dataset. PointNet++ is used as the experimental baseline, and the comparative algorithms cover traditional feature extraction and machine learning methods as well as deep learning models. The main methods include the following: Voxel-RF: Voxel-RF is a classification method based on traditional feature engineering. This method first converts the three-dimensional point cloud into a uniform voxel grid, and calculates the point density, mean normal vector, and geometric features in each voxel. Then, these local statistical features are spliced into a global feature vector, which is input into the Random Forest classifier for decision-making.

[0053] Projection-RF: The Projection-RF method uses the multi-view projection strategy in MVCNN to project the three-dimensional point cloud onto 6 orthogonal two-dimensional planes (front, back, left, right, up, down) to generate multiple depth images. For each view, HOG (Histogram of Oriented Gradients) and SIFT (Scale-Invariant Feature Transform) descriptors are extracted to capture the edge direction distribution and key point texture information in the image. After splicing the features of all views, the shoeprint category is determined by the Random Forest classifier.

[0054] FPFH-RF: The FPFH-RF method encodes local geometric information based on the Fast Point Feature Histogram (FPFH). For each point, the curvature, normal vector angle, distance distribution, and other statistics of the points in its neighborhood are calculated to generate a 33-dimensional FPFH descriptor. The global FPFH feature histogram is classified by the Random Forest.

[0055] ICP-RF: The ICP-RF method combines the Iterative Closest Point (ICP) algorithm and the Random Forest method. First, the average registration error of the test sample and each category in the template library is calculated by ICP, and the category with the smallest error is selected as the candidate. Then, the registration error is input into the Random Forest for fine-grained classification.

[0056] VoxNet: VoxNet is a representative method of early 3D convolutional neural networks in the field of point cloud processing. It first normalizes the original point cloud data to a unit cube space, and then discretizes the three-dimensional space into a 30x30x30 regular grid unit through the voxelization process. Each unit is assigned a binary feature or density statistic according to the existence of internal points. The voxel grid input is composed of a feature extraction network with two layers of three-dimensional convolution: the first layer uses 32 5x5x5 convolution kernels for spatial feature extraction, and after ReLU nonlinear activation, it is reduced through 3x3x3 max pooling; the second layer uses 64 convolution kernels of the same size to further capture high-order geometric patterns. Finally, the flattened features are mapped to the classification space through a fully connected layer. This method verifies the effectiveness of three-dimensional convolution operations in geometric representation, but the fixed resolution voxelization process has the inherent defect of losing detailed information.

[0057] MVCNN: MVCNN proposes a multi-view projection and feature fusion framework to solve the complexity of direct processing of three-dimensional data through spatial projection transformation. In specific implementation, 12 orthogonal virtual viewpoints are set on the periphery of the point cloud, and the three-dimensional structure is projected into a set of two-dimensional depth images. Each view image input shares a ResNet-18 network to extract 512-dimensional high-level semantic features, and through view-level max pooling operation, multi-angle discriminative features are fused to form a compact global description vector. The innovation of this method lies in the migration of mature two-dimensional convolutional neural networks to the three-dimensional field, but the dimension reduction of three-dimensional topological relationship in the projection process leads to the loss of part of the spatial geometric information.

[0058] PointNet: PointNet is an end-to-end deep learning framework that realizes unordered point sets. Its architecture design includes three core components. Through a multi-layer perceptron with shared weights, the three-dimensional coordinates of each point are transformed point by point to generate 1024-dimensional point-level features; a max-pooling symmetric function is used to aggregate global features, effectively solving the unordered point set problem; a learnable spatial transformation network (T-Net) is introduced to rigidly align the input point cloud, improving the robustness of the model to spatial transformation. This method lays a theoretical foundation for subsequent research, but the single-scale global feature aggregation mechanism limits the ability to capture local geometric patterns.

[0059] PointNet++: PointNet++ is a deep learning network framework that builds on the breakthrough of PointNet in hierarchical feature learning. It recursively constructs a point set hierarchy using the farthest point sampling algorithm and uses a ball query strategy to obtain local neighborhood point sets within a certain radius at each level. The Set Abstraction module is used to abstract the features of local regions: after extracting neighborhood features using shared MLP, local descriptors are generated through max-pooling. To further enhance multi-scale perception, a multi-scale grouping strategy is proposed to process local regions of different radius scales in parallel and concatenate the features. In the feature decoding stage, high-level semantic features are propagated to the bottom layer through reverse interpolation, and the jump connection is used to preserve geometric detail information.

[0060] PointMLP: PointMLP explores the application potential of pure MLP architecture in point cloud processing. This method realizes feature deepening by stacking residual MLP modules, each module containing two fully connected layers and batch normalization operations. The key innovation lies in the design of the geometric affine module: after centering the local neighborhood point set, a learnable affine transformation matrix is applied to adjust the feature distribution, enhancing the model's adaptability to spatial transformations. Through recursive local feature aggregation, the receptive field is gradually expanded, from micro-geometric patterns to macro-shape features, achieving progressive feature learning.

[0061] DGCNN: DGCNN proposes a dynamic graph convolutional neural network architecture, focusing on solving the problem of local geometric relationship modeling of point clouds.

[0062] This method dynamically constructs a k-nearest neighbor graph structure in the feature space and encodes the geometric relationship between points through edge convolution operations: the feature difference vector between the center point and the neighborhood points is calculated, and the neighborhood information is aggregated after shared MLP transformation. As the network depth increases, the graph structure is dynamically updated according to the high-level semantic features, achieving adaptive context perception. The multi-layer feature concatenation mechanism preserves the semantic information of different abstraction levels, and finally generates a discriminative feature vector through global pooling.

[0063] Table 1 Comparison of experimental results In summary, the image processing-based shoeprint point cloud recognition method provided by the application can be seen from the experimental results that in the shoeprint point cloud classification task, different methods have their own characteristics and limitations. The voxel-based three-dimensional convolution method VoxNet extracts three-dimensional features through a regular voxel grid, but the quantization operation in the voxelization process may cause the loss of some microscopic geometric details. In contrast, MVCNN fuses three-dimensional information through multi-view two-dimensional projection, and in the overall shape classification task, it takes into account efficiency and robustness, but the projection process may weaken the continuity features of the three-dimensional space, and the geometric constraints across views need to be explored to enhance the implicit modeling ability of three-dimensional topological relationships.

[0064] The method of directly processing the original point cloud, such as PointNet, performs outstandingly in global shape classification by avoiding the loss of quantized information, but its symmetric function aggregation mechanism may ignore the relevance between local structures. On this basis, the hierarchical point cloud sampling method of PointNet++ is introduced, which significantly improves the fine-grained classification accuracy by fusing multi-scale local features, and the accuracy is improved to 94%. The dynamic neighborhood construction mechanism of the dynamic graph convolution method DGCNN gives the model the ability to adapt to non-rigid deformation, but the frequent updating of neighborhood relationships leads to a non-linear increase in computational complexity with the density of point clouds, and the real-time challenge in high-density scenes needs to be solved. Compared with the experimental baseline PointNet++, the network improves the Top1 Accuracy by 2.38% and the Top3 Accuracy by 2.38% through optimized multi-scale feature extraction and application of the CBAM mechanism, reaching 100%. It shows that the optimization part strengthens the discrimination ability of fine-grained feature extraction, and good performance is achieved on the shoeprint point cloud dataset.

[0065] In summary, the application constructs a three-dimensional shoeprint point cloud dataset applicable to criminal investigation analysis, and proposes an improved feature fusion PointNet++ and a three-dimensional shoeprint classification network with attention mechanism weighting, providing a solution to the problem of shoeprint feature digital analysis. The dataset is collected by high-precision laser scanning equipment and covers multiple shoe samples, providing data support for the development of shoeprint intelligent analysis related fields. Experimental verification shows that the multi-scale feature fusion network proposed in this paper has significant advantages in shoeprint recognition: by designing a three-level feature pyramid structure and fusing it by convolution, combined with channel-spatial dual attention mechanism, the Top1 recognition accuracy of 95.63% is achieved on the test set, which is significantly improved compared with traditional methods.

Claims

1. A method for recognizing shoe print point clouds based on image processing, characterized in that, Includes the following steps: S1: A shoe print scanning device is built based on a structured light sensor, and the real shoe print is scanned by the shoe print scanning device to obtain raw point cloud data containing three-dimensional coordinates; S2: Preprocess the raw point cloud data: perform cropping, normal vector consistency denoising, and secondary denoising in sequence to construct a 3D shoe print dataset; S3: Based on PointNet++, a classification model is constructed that includes a multi-scale feature extraction layer and a channel-spatial attention layer. The training set and test set are divided from the 3D shoe print dataset and input into the classification model for training and testing. S4: Real-time acquisition of shoe print data to be identified, preprocessing and input into the classification model. The multi-scale feature extraction layer performs two farthest point downsampling operations on the preprocessed shoe print point cloud data, extracts features at different scales, and then splices and fuses them to obtain a fused feature map. S5: The channel-spatial attention layer is composed of a channel attention submodule and a spatial attention submodule. The channel attention submodule compresses the spatial dimension of the fused feature map through adaptive average pooling and adaptive max pooling to obtain two channel description vectors. After being transformed by a multilayer perceptron, they are summed and then the channel attention weights are generated by the Sigmoid function. S6: The spatial attention submodule performs average pooling and max pooling on the feature map along the channel dimension to generate two spatial feature maps. After concatenation along the channel dimension, the spatial relationship is modeled through a convolutional layer, and then spatial attention weights are generated through the Sigmoid function. Feature calibration is performed through the channel-spatial attention layer, and finally the shoe print classification result is output.

2. The shoe print point cloud recognition method based on image processing according to claim 1, characterized in that, The shoe print scanning device in step S1 includes a main frame and a control console. The control console is located on the inner top wall of the main frame, and a structured light sensor is installed on the control console.

3. The shoe print point cloud recognition method based on image processing according to claim 1, characterized in that, The specific process of performing clipping, normal vector consistency denoising, and secondary denoising in step S2 is as follows: S21: Select the area containing the complete shoe print and remove irrelevant data such as ground points and equipment bracket points outside the shoe print area; S22: Calculate the normal vector of each point in the clipped point cloud data, that is, determine the normal direction of the surface where the point is located by analyzing the spatial positional relationship between each point and its neighboring points; The consistency between the normal vector of each point and the normal vectors of other points in the neighborhood is statistically analyzed. If the normal vector of a point deviates too much from the normal vectors of most of the surrounding points, the point is judged as a noise point and is removed. S23: Use point cloud visualization tools to inspect point cloud data region by region, delete outliers that are scattered or do not conform to the geometric shape of shoe prints, and eliminate noise and irrelevant points.

4. The shoe print point cloud recognition method based on image processing according to claim 1, characterized in that, Step S3, during the training process, sets the cross-entropy loss function, with the specific expression as follows: ； in, C Let ϵ∈(0,1) be the total number of categories, and let ϵ∈(0,1) be the smoothing parameter. p t ∈[0,1] represents the predicted probability of the true class t. These are the label values ​​after label smoothing. The cross-entropy loss function is the result of label smoothing. p i For the model to the first i The predicted probability of a class.

5. The shoe print point cloud recognition method based on image processing according to claim 4, characterized in that, The model training process in step S3 is as follows: S31: Initialize the parameters of the classification model, set key training parameters, including a fixed input point cloud of 4096 points, a training batch size of 4, and a total training epochs of 300; at the same time, configure the basic parameters of the AdamW optimizer and the learning rate range of OneCycleLR. S32: In each training batch of each round, based on the model prediction results and the true labels, the loss value of the current batch is calculated using the label-smoothed cross-entropy loss function, and the gradient calculation and update of the model parameters are completed by the AdamW optimizer. S33: Throughout the entire 300 training cycles, the OneCycleLR learning rate scheduling strategy dynamically adjusts the learning rate according to a linear increase-linear decrease pattern, providing an appropriate learning rate for different training stages. S34: After each training round, use the test set to evaluate the model's performance and monitor the model's training trend; when the number of training rounds reaches the preset value of a specified round, or when the model's performance on the test set does not improve significantly for several consecutive rounds, terminate the training and obtain the final trained model.

6. The shoe print point cloud recognition method based on image processing according to claim 1, characterized in that, The specific processing procedure of the preprocessed shoe print point cloud data by the multi-scale feature extraction layer in step S4 is as follows: S41: The multi-scale feature extraction layer performs two rounds of farthest point downsampling on the preprocessed shoe print point cloud data to obtain point clouds of 4096 points and 1024 points respectively. In each layer, three sphere queries with different radii are performed on each point to form groups. S42: Employ a multi-scale feature extraction method to capture features at different scales in the point cloud and stitch together the features at each scale. S43: After concatenation, the data is fed into the channel-space attention module for weighting. The weights are dynamically adjusted using a learnable 1×1 convolutional kernel, implicitly learning the linear combination relationship between features at each scale to achieve feature fusion.

7. The shoe print point cloud recognition method based on image processing according to claim 6, characterized in that, The specific processing procedure of the channel attention submodule in step S5 is as follows: S51: The channel attention submodule receives the fused feature map and performs feature compression along the spatial dimension using adaptive average pooling and adaptive max pooling, respectively, to obtain two channel description vectors. F avg and F max ; S52: Transform the two channel description vectors using a multilayer perceptron. F avg and F max The channel attention weights are obtained by applying the non-linear activation function Sigmoid. S53: The output channel attention weights are weighted with the original feature map channel by channel to capture the non-linear dependencies between channels.

8. The shoe print point cloud recognition method based on image processing according to claim 6, characterized in that, The specific processing procedure of the spatial attention submodule in step S6 is as follows: S61: Perform average pooling and max pooling operations along the channel dimension to generate two spatial feature maps F1 and F2; S62: After splicing the two along the channel dimension, spatial relationship modeling is performed through a 1D convolutional layer with symmetrical filling; S63: A spatial attention weight matrix is ​​generated using the Sigmoid function, and the spatial dimension calibration of the feature map is finally achieved through position-by-position multiplication.

Images