A shell type recognition method and system based on deep learning

Abstract

The application relates to the field of image processing and recognition technology, and discloses a shell type recognition method and system based on deep learning, which comprises the following steps: collecting a two-dimensional image sequence, three-dimensional point cloud data and size parameters of a shell surface through an industrial camera array, and generating a complete three-dimensional model through ICP algorithm registration of the three-dimensional point cloud data; carrying out denoising, contrast enhancement and distortion correction on the two-dimensional image sequence to extract image features, carrying out voxelization and down-sampling on the point cloud to extract roughness features; fusing the image and point cloud features into a unified feature tensor through an attention mechanism, inputting a visual analysis large model to complete feature extraction and classification, and outputting initial recognition results and confidence of ammunition types, calibers and the like; when the confidence is insufficient, triggering an auxiliary recognition mechanism to correct the probability, and finally generating a recognition report. The application shortens the judgment time and reduces the probability of safety accidents caused by human misjudgment.

Description

Technical Field

[0001] This invention relates to the field of image processing and recognition technology, and more specifically, to a method and system for identifying projectile types based on deep learning. Background Technology

[0002] War-related ammunition, due to its long burial time and significant environmental impact, generally suffers from severe corrosion, structural damage, and extremely unstable performance, posing a constant risk of explosion or leakage. Some of these chemical agent munitions are even more dangerous.

[0003] To eliminate the aforementioned safety hazards, it is urgent to accurately identify and classify discovered war remnants so that scientific and reliable methods can be adopted for their handling, preservation, and destruction. However, existing identification methods mainly rely on human experience or single sensor information, such as manual visual comparison, two-dimensional image recognition, or dimensional measurement. These methods have the following shortcomings: low identification accuracy, poor environmental adaptability, insufficient identification efficiency, and lack of multi-dimensional information fusion.

[0004] Therefore, it is necessary to design a deep learning-based method and system for identifying projectile types to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a deep learning-based method and system for identifying projectile types, aiming to solve the problems of low accuracy and insufficient efficiency in identifying abandoned ammunition.

[0006] In one aspect, this invention proposes a deep learning-based method for identifying projectile types, comprising:

[0007] Multimodal data of the projectile to be identified is acquired using an industrial camera array. The multimodal data includes a two-dimensional image sequence of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is further registered using an ICP algorithm to form the final three-dimensional point cloud data.

[0008] The two-dimensional image sequence is subjected to nonlocal mean denoising, the contrast of the marked region is enhanced by CLAHE technology, and the image distortion is corrected by perspective transformation to generate two-dimensional image features; the final three-dimensional point cloud data is voxelized downsampled and curvature features are extracted to generate roughness features characterizing the surface properties of the projectile; the two-dimensional image features and the roughness features are fused through an attention mechanism to construct a unified feature tensor;

[0009] The unified feature tensor is input into the large visual analysis model for feature extraction, and the classification probability distribution is output.

[0010] An initial identification result and confidence level are determined based on the classification probability distribution, where the confidence level is the maximum probability value in the classification probability distribution. The initial identification result includes ammunition type, applicable weapon type, caliber, and ammunition type. When the confidence level is less than a confidence threshold, an auxiliary identification mechanism is triggered to correct the probability and generate a final identification report.

[0011] Furthermore, when acquiring multimodal data of the projectile to be identified based on an industrial camera array, this includes:

[0012] The industrial camera array is a ring array that surrounds and acquires circumferential images of the projectile to be identified, generating the two-dimensional image sequence and three-dimensional point cloud data.

[0013] Furthermore, when the 3D point cloud data is registered using the ICP algorithm to form the final 3D point cloud data, it includes:

[0014] Multiple local point cloud data of the three-dimensional point cloud data are input into the ICP algorithm to determine the corresponding point relationship of each adjacent local point cloud data. Based on the corresponding point relationship, the rigid body transformation matrix between local point clouds is calculated, and the spatial position of the local point cloud data is adjusted according to the rigid body transformation matrix.

[0015] Determine whether the local point cloud registration error meets the convergence condition. If the convergence condition is not met, update the corresponding point relationship and recalculate the rigid body transformation matrix. Continue until the local point cloud registration error meets the convergence condition. Perform overlapping region data fusion on the converged local point cloud data to generate the final three-dimensional point cloud data.

[0016] Furthermore, when generating two-dimensional image features by performing nonlocal mean denoising on the two-dimensional image sequence, enhancing the contrast of the marked region using CLAHE technology, and correcting image distortion through perspective transformation, the process includes:

[0017] The nonlocal mean denoising process includes: selecting multiple candidate pixel regions within a search window based on each pixel of the image data in the two-dimensional image sequence; calculating the gray-level similarity between the candidate pixel regions and the central pixel region; determining weight coefficients based on the gray-level similarity; and performing a weighted average of the candidate pixel regions to obtain the pixel value.

[0018] The CLAHE technique for enhancing the contrast of marked regions includes: dividing the image data within the two-dimensional image sequence into multiple regularly distributed local region grids; calculating a histogram of pixel grayscale distribution for each local region grid; redistributing the frequency of grayscale levels exceeding the contrast clipping threshold based on the histogram; performing an adaptive equalization transformation on the histogram to generate a locally contrast-enhanced image; and identifying the location of marked regions using an image segmentation algorithm and applying optimized contrast enhancement parameters to the marked regions.

[0019] The perspective transformation correction of image distortion includes: extracting feature points from the image data in the two-dimensional image sequence based on a corner detection algorithm; matching feature point descriptors to obtain feature point pairs between the image and a standard plane; selecting interior feature point pairs that conform to the homography model based on the RANSAC algorithm; calculating the homography matrix based on the interior feature point pairs; and using the homography matrix to perform bilinear interpolation resampling on the image data of the two-dimensional image sequence to correct geometric distortion caused by the shooting angle.

[0020] The image data in the two-dimensional image sequence after nonlocal mean denoising, CLAHE contrast enhancement, and perspective transformation correction are integrated into the two-dimensional image features.

[0021] Furthermore, when performing voxelization downsampling and extracting curvature features on the final three-dimensional point cloud data to generate roughness features characterizing the surface properties of the projectile, the process includes:

[0022] The final 3D point cloud data is divided into multiple cubic voxel units; the coordinates of representative points of the 3D point cloud data within each cubic voxel unit are calculated, and the 3D point cloud data is downsampled; the point cloud set within the neighborhood of each downsampled 3D point cloud data is determined; the covariance matrix is ​​calculated based on the point cloud set within the neighborhood and decomposed into eigenvalues; the Gaussian curvature and mean curvature are calculated based on the eigenvalues; the variation characteristics of the Gaussian curvature and mean curvature are statistically analyzed to generate the roughness feature characterizing the surface properties of the projectile.

[0023] Furthermore, when fusing the two-dimensional image features and the roughness features through an attention mechanism to construct a unified feature tensor, the process includes:

[0024] The feature responses of the two-dimensional image features and roughness features are calculated in the channel dimension to generate channel attention weights; the two-dimensional image features and roughness features are weighted according to the channel attention weights, and the feature responses in the spatial dimension are calculated to generate spatial attention weights.

[0025] The two-dimensional image features are adjusted to the same feature dimension as the roughness features through convolution transformation; the two-dimensional image features and roughness features are weighted and fused according to channel attention weight and spatial attention weight to generate fused features; the high-order representation of the fused features is extracted through three-dimensional convolution operation to construct the unified feature tensor.

[0026] Furthermore, when inputting the unified feature tensor into the large-scale visual analysis model for feature extraction and outputting the classification probability distribution, the process includes:

[0027] The large-scale visual analysis model is a deep convolutional neural network. The unified feature tensor is input into the initial processing layer of the large-scale visual analysis model. Local features are extracted in the spatial-channel dimension through deep convolutional kernels to capture the multi-scale features of the shell shape and surface markings.

[0028] After multiple feature extraction stages, each stage includes a 3D convolutional layer, a batch normalization layer, and a non-linear activation layer to extract high-level abstract feature representations. An attention mechanism module is embedded in the feature extraction stage to dynamically calculate the importance weights of each spatial location and feature channel in the unified feature tensor. Global average pooling is performed on the extracted abstract feature representations to compress the feature space dimension and retain discriminative information.

[0029] Abstract feature representations are input into fully connected layers, and the classification probability distribution is output through the fully connected network.

[0030] Furthermore, when determining the initial identification result and confidence level based on the classification probability distribution, the process includes:

[0031] The ammunition types in the classification probability distribution are sorted according to their probability values, and the ammunition type with the highest probability value is selected as the ammunition type of the initial identification result;

[0032] Based on the initial identification results of the ammunition type, the probability distribution of each dimension corresponding to the applicable weapon type, caliber and ammunition type in the classification probability distribution is classified and determined to identify the applicable weapon type, caliber and ammunition type.

[0033] The maximum probability value corresponding to the ammunition type in the initial identification result is determined as the confidence level.

[0034] Furthermore, when the confidence level is less than the confidence level threshold, the auxiliary recognition mechanism is triggered to perform probability correction, and when generating the final recognition report, it includes:

[0035] When the confidence level is less than the confidence threshold, the laser-induced breakdown spectral analysis module is triggered to perform micro-area elemental composition detection on the surface of the projectile to be identified; the spectral signal of the projectile material is acquired and converted into elemental feature information; the elemental feature information and the projectile size parameters are input into a Bayesian network to calculate the posterior probability distribution of each type of ammunition; the initial identification result is corrected according to the posterior probability distribution, and the final identification report is generated.

[0036] Compared with existing technologies, the advantages of this invention are as follows: By performing high-order fusion of 2D image texture features and 3D point cloud geometric features (including curvature / roughness) under an attention mechanism, the model can simultaneously utilize texture and shape discrimination information, improving the accuracy of ammunition type, caliber, and ammunition type discrimination. Preprocessing steps such as nonlocal mean denoising, CLAHE local contrast enhancement, and perspective correction can restore surface marker information degraded by corrosion, dirt, or shooting angle, reducing the misclassification rate caused by surface degradation. Based on stepwise ICP registration of local point clouds and judgment of convergence and overlapping area fusion, the accuracy and consistency of 3D reconstruction are ensured, providing a reliable foundation for subsequent geometric feature extraction (such as Gaussian curvature, average curvature, etc.), and enhancing the ability to distinguish ammunition with similar shapes but different geometric details. Channel and spatial level attention weights adaptively adjust the importance of each modal feature, achieving complementary use of information, enhancing the model's attention to key local markers and subtle geometric features, and improving discrimination performance in complex scenes. By setting a confidence threshold for classification probabilities, material composition detection, such as laser-induced breakdown spectroscopy (LIBS), is automatically triggered when the confidence level is insufficient. Posterior probability correction is then performed based on a Bayesian network, reducing the risk of misjudgment and missed judgment, and improving the credibility and security of the final decision. Automated identification and report generation significantly reduce the reliance on manual experience in judgment, providing rapid decision-making suggestions to on-site personnel, shortening judgment time, and reducing the probability of safety accidents caused by human error.

[0037] On the other hand, this application also provides a deep learning-based projectile type identification system for applying the aforementioned deep learning-based projectile type identification method, comprising:

[0038] The acquisition unit is configured to acquire multimodal data of the projectile to be identified based on an industrial camera array. The multimodal data includes a two-dimensional image sequence of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is further registered using an ICP algorithm to form the final three-dimensional point cloud data.

[0039] The processing unit is configured to perform nonlocal mean denoising on the two-dimensional image sequence, enhance the contrast of the marked region using CLAHE technology, and correct image distortion through perspective transformation to generate two-dimensional image features; perform voxelization downsampling on the final three-dimensional point cloud data and extract curvature features to generate roughness features characterizing the surface properties of the projectile; and fuse the two-dimensional image features and the roughness features through an attention mechanism to construct a unified feature tensor.

[0040] The output unit is configured to input the unified feature tensor into a large visual analysis model for feature extraction and output a classification probability distribution.

[0041] The generation unit is configured to determine an initial identification result and confidence level based on the classification probability distribution, wherein the confidence level is the maximum probability value in the classification probability distribution, and the initial identification result includes ammunition type, applicable weapon style, caliber, and ammunition type; when the confidence level is less than the confidence threshold, an auxiliary identification mechanism is triggered to perform probability correction and generate a final identification report.

[0042] It is understandable that the aforementioned deep learning-based shell type identification method and system have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0044] Figure 1 A flowchart illustrating a deep learning-based projectile type identification method provided in an embodiment of the present invention;

[0045] Figure 2 A functional block diagram of a deep learning-based shell type identification system provided in an embodiment of the present invention. Detailed Implementation

[0046] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0047] Current technologies for shell identification primarily rely on human experience or single sensor data, such as classification through two-dimensional image comparison or manual visual inspection. This approach has significant drawbacks: shells are diverse, and surface markings are often difficult to discern due to corrosion, coating peeling, etc.; single image information is highly susceptible to identification errors; manual judgment depends on the experience of professionals, resulting in low efficiency, strong subjectivity, and lack of reproducibility; the lack of support from multimodal data fusion and deep learning models leads to insufficient accuracy and robustness in complex environments, making it difficult to meet the actual needs of safe handling.

[0048] For this, please refer to Figure 1As shown, a deep learning-based method for identifying projectile types includes:

[0049] S100: Based on an industrial camera array, it acquires multimodal data of the projectile to be identified. The multimodal data includes a two-dimensional image sequence of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is also registered by the ICP algorithm to form the final three-dimensional point cloud data.

[0050] S200: Performs nonlocal mean denoising on the two-dimensional image sequence, uses CLAHE technology to enhance the contrast of the marked area, and corrects image distortion through perspective transformation to generate two-dimensional image features; performs voxelization downsampling on the final three-dimensional point cloud data and extracts curvature features to generate roughness features characterizing the surface properties of the projectile; and fuses the two-dimensional image features and roughness features through an attention mechanism to construct a unified feature tensor.

[0051] S300: Inputs a unified feature tensor into a large visual analysis model for feature extraction and outputs a classification probability distribution;

[0052] S400: Determine the initial identification result and confidence level based on the classification probability distribution. The confidence level is the maximum probability value in the classification probability distribution. The initial identification result includes ammunition type, applicable weapon style, caliber and ammunition type. When the confidence level is less than the confidence threshold, the auxiliary identification mechanism is triggered to correct the probability and generate the final identification report.

[0053] Specifically, a circular industrial camera array acquires multimodal data of the projectile to be identified, including a circumferentially covered two-dimensional image sequence, the original three-dimensional point cloud, and dimensional measurement parameters. The acquired local point cloud is first filtered and denoised, and then input into the ICP registration module in blocks according to a preset overlap strategy. Based on corresponding point matching, the rigid body transformation matrix is ​​iteratively solved, and the registration error is judged to converge. After convergence, the overlapping areas are fused to form a final consistent three-dimensional point cloud model. For the two-dimensional image sequence, a non-local means (NL-Means) method is first implemented to suppress noise by weighted averaging based on gray-level similarity within the local search window. Then, CLAHE adaptive histogram equalization is applied to the segmented marked / engraved regions to improve local contrast. Homography matrix is ​​calculated by corner detection and descriptor matching, and outliers are removed using RANSAC. Bilinear interpolation is used to perform perspective transformation correction on the image, thereby generating geometrically corrected and contrast-enhanced two-dimensional image features. The final 3D point cloud is downsampled using voxelization to reduce redundant points, and the covariance matrix of each voxel's neighborhood is calculated. Gaussian curvature and mean curvature are obtained through eigenvalue decomposition. Curvature distribution and variation characteristics are statistically analyzed to construct 3D features characterizing the surface roughness and micro-geometric details of the projectile. Attention responses are calculated for 2D image features and 3D roughness features in the channel and spatial dimensions, respectively, generating channel attention weights and spatial attention weights. The 2D features are first mapped to the same dimension as the 3D features through convolutional transformation. Then, the two modal features are weighted according to the aforementioned attention weights and spatially fused. A high-order representation of the fused features is extracted through 3D convolution to construct a unified feature tensor. The unified feature tensor is input into a hierarchical visual analysis model (the visual analysis model is a deep convolutional neural network containing several three-dimensional convolutional layers, batch normalization, nonlinear activation, and embedded attention modules). After global average pooling and fully connected layers, it outputs the classification probability distribution of ammunition type, applicable weapon style, caliber, and ammunition type. The maximum class probability is used as the confidence level of the initial recognition result. When the confidence level is lower than a preset threshold, an auxiliary recognition unit is automatically triggered (e.g., laser-induced breakdown spectrum sampling of the projectile surface and extraction of elemental features). The spectral elemental features and size parameters are input into a Bayesian network to calculate the posterior probability to correct the initial probability distribution. Finally, an recognition report containing the recognition conclusion, confidence level, and auxiliary evidence is generated for use in decision-making.

[0054] The confidence threshold is typically set based on a combination of statistical analysis and experimental verification to ensure a balance between classification accuracy and reliability. This can be achieved by running a large-scale visual analysis model on a large number of known shell samples, statistically analyzing the probability distribution of each category, and examining the concentration and fluctuation range of the maximum probability values. Based on actual recognition needs and error tolerance requirements, the confidence threshold is set higher than the lower limit of the maximum probability value for most correctly identified samples, while remaining lower than the peak probability that is prone to misclassification.

[0055] The working principle and process of this application are as follows: a circular industrial camera array acquires multimodal data of the projectile to be identified (a two-dimensional image sequence covering the circumference, the original three-dimensional point cloud, and dimensional parameters). The local point cloud is then segmented according to an overlap strategy and input into an ICP registration module for iterative solution of rigid body transformation until convergence to obtain a consistent final three-dimensional point cloud. Non-local mean denoising is first performed on the two-dimensional image; CLAHE local contrast enhancement is applied to the segmented marked regions, and perspective correction is performed using corner detection + RANSAC estimation of the homography matrix to restore clear image features. Voxelization downsampling is performed on the final point cloud, and the covariance matrix is ​​calculated in the neighborhood of each voxel to obtain curvature measures such as Gaussian curvature and mean curvature to characterize surface roughness. The two-dimensional image features are then combined with the three-dimensional... The roughness features are calculated with attention responses in both the channel and spatial dimensions. First, the image features are mapped to the same dimension as the point cloud features, and then fused with channel / spatial attention weights. A unified feature tensor is constructed by extracting high-order representations through 3D convolution. This tensor is input into a hierarchical deep convolutional neural network to extract discriminative representations and outputs the classification probability distribution of ammunition type, applicable weapon style, caliber, and ammunition type through a fully connected layer. The highest probability is used as the confidence level of the initial recognition result. When the confidence level is lower than a preset threshold, auxiliary recognition is automatically triggered (such as laser-induced breakdown spectrum sampling to obtain elemental features), and spectral and size evidence is input into a Bayesian network for posterior probability correction. Finally, a detailed recognition report containing the recognition conclusion, confidence level, and auxiliary evidence is generated for use in decision-making.

[0056] As a preferred embodiment, the solution of this application is implemented as follows: For example, after on-site personnel find a shell with severe corrosion and blurred markings at a construction site, they place it on the mandrel of the testing platform and start collecting data from 12 circular industrial cameras (approximately 5MP each) to obtain a two-dimensional image sequence covering the circumference, the original point cloud, and dimensional measurement data. The point cloud is divided into several local blocks according to the overlap strategy and input into the ICP registration module. If the RMSE of adjacent blocks after registration is less than 0.5mm or the number of iterations reaches 50, convergence is determined. The overlapping area is fused to form a consistent final three-dimensional point cloud. The image sequence is first denoised using NL-Means (the search window and similarity threshold are adaptively set according to the noise level at the scene). The segmented inscribed / marked regions are then enhanced with CLAHE (the local mesh size and contrast clipping limit are adjusted according to the region details) to enhance local contrast. Then, corner detection → descriptor matching → RANSAC to remove outliers → calculation of homography matrix and perspective correction using bilinear interpolation are performed to restore the true geometry. The final point cloud is downsampled by 2mm voxelization to reduce redundancy, and the covariance matrix is ​​calculated in the neighborhood of each voxel. Eigenvalue decomposition yields Gaussian curvature and mean curvature. The mean, variance, and extreme value distribution of curvature are statistically analyzed to construct a three-dimensional feature vector characterizing surface roughness. Two-dimensional features are mapped to the same dimension as three-dimensional features through convolution. Attention weights are calculated in both channel and spatial dimensions, and the two modal features are weighted and fused. Then, a high-order representation of the fused features is extracted using a 3×3×3 three-dimensional convolutional kernel to construct a unified feature tensor, which is then input into a hierarchical deep convolutional neural network for inference. The classification probabilities output by the network are, for example: 0.48 for high explosive shells, 0.38 for chemical shells, 0.07 for training shells, and 0.07 for practice shells. The maximum probability of 0.48 is used as the initial confidence level. If the preset confidence threshold is 0.85, assisted recognition is triggered. Laser-induced breakdown spectroscopy (LIBS) measurements were performed on a micro-area selected on the surface of the projectile. The extracted spectrum showed abnormal intensity of elements such as Cl and S, which highly matched the material composition distribution of a certain type of chemical agent projectile known in the database. After combining the elemental characteristics with the external dimensions measured on-site using a Bayesian network and prior probability, the posterior probability of the chemical projectile was calculated and increased to 0.92. Finally, an identification report was generated: it was identified as a "historical artillery shell containing chemical agent" (posterior probability 0.92), and it was recommended to isolate it according to the special chemical defense procedures and have a professional team take neutralization and destruction measures. The report also included two-dimensional / three-dimensional visualizations, LIBS spectral curves, confidence levels, and judgment criteria for use in on-site disposal decisions.

[0057] Through the above technical solutions, this invention improves the recognition accuracy and robustness of rusted, soiled, or partially damaged projectiles by employing multimodal acquisition with an industrial camera array, robust ICP point cloud registration, targeted image preprocessing (non-local mean denoising, CLAHE contrast enhancement, perspective correction), and 2D-3D feature fusion based on channel / spatial attention and deep convolutional neural network classification. The 3D roughness features extracted by voxelization downsampling and curvature statistics compensate for the lack of 2D texture information, enabling effective differentiation of projectiles with similar shapes but different geometric details. The confidence-driven auxiliary recognition mechanism (such as LIBS spectral sampling + Bayesian posterior correction) provides additional evidence when the model is uncertain, reducing the risk of misjudgment and missed judgment, and enhancing security.

[0058] This application further proposes a method for acquiring multimodal data of projectiles to be identified based on an industrial camera array, including:

[0059] The industrial camera array is a ring array that surrounds and acquires circumferential images of the projectile to be identified, generating a two-dimensional image sequence and three-dimensional point cloud data.

[0060] Specifically, during the multimodal data acquisition process for the projectiles to be identified, the industrial camera array is arranged in a ring, installed 360° around the circumference of the projectile to ensure complete coverage of the projectile surface from different angles and directions, achieving omnidirectional image acquisition. Each industrial camera can independently acquire high-precision two-dimensional image sequences according to preset resolution and exposure parameters, capturing the texture, markings, colors, and local wear information of the projectile surface. Simultaneously, point cloud data is slowly acquired along the longitudinal direction of the projectile, ensuring that the point cloud achieves high precision in both density and continuity, thereby obtaining the complete three-dimensional geometric structure of the projectile surface.

[0061] As a preferred embodiment, the solution of this application is implemented as follows: For example, when identifying a shell with a diameter of approximately 120 mm and a length of approximately 500 mm, the shell is fixed on a rotatable support to ensure its stability and prevent shaking during the scanning process. A circular industrial camera array consists of 12 high-resolution cameras, uniformly surrounding the shell. Each camera is set with different focal lengths and exposure parameters to capture markings, coatings, and wear marks on the shell's surface. During the scanning process, the camera moves spirally along the shell's longitudinal axis, collecting point cloud data layer by layer to obtain complete three-dimensional geometric information of the shell's surface.

[0062] Through the above technical solution, this application can simultaneously acquire high-precision two-dimensional images and complete three-dimensional point cloud information of the projectile, achieving accurate alignment and fusion of multimodal data. The circular industrial camera array can capture the surface texture, markings, and coating features of the projectile from all angles, avoiding information loss caused by single-angle shooting. Simultaneously, by adjusting exposure and focal length, image clarity is ensured under different surface materials and lighting conditions. It accurately records the projectile's geometric shape, size parameters, and minute surface irregularities, providing reliable data for subsequent curvature analysis and surface roughness feature extraction. Synchronous control based on hardware trigger signals ensures the temporal and spatial consistency between the two-dimensional image sequence and the three-dimensional point cloud data, avoiding data misalignment caused by motion or acquisition delays, and improving the accuracy of subsequent deep learning models in feature extraction and classification.

[0063] This application further proposes that when 3D point cloud data is registered using the ICP algorithm to form the final 3D point cloud data, it includes:

[0064] Multiple local point cloud data of 3D point cloud data are input into the ICP algorithm to determine the corresponding point relationship of each adjacent local point cloud data. Based on the corresponding point relationship, the rigid body transformation matrix between local point clouds is calculated, and the spatial position of the local point cloud data is adjusted according to the rigid body transformation matrix.

[0065] Determine whether the local point cloud registration error meets the convergence condition. If it does not meet the convergence condition, update the corresponding point relationship and recalculate the rigid body transformation matrix. Continue until the local point cloud registration error meets the convergence condition. Perform overlapping region data fusion on the converged local point cloud data to generate the final 3D point cloud data.

[0066] Specifically, in the process of registering 3D point cloud data using the ICP (Iterative Closest Point) algorithm to form the final 3D point cloud data, multiple collected local point cloud data are input into the ICP algorithm to precisely align each pair of adjacent local point clouds. The ICP algorithm establishes a set of corresponding point pairs between each local point cloud by calculating the nearest point correspondence between adjacent point clouds. Then, based on these corresponding point pairs, it calculates the rigid body transformation matrix between local point clouds, including rotation matrix and translation vector, thereby adjusting each local point cloud to a unified spatial coordinate system. The registration error, such as the mean Euclidean distance or root mean square error, is calculated for the registered local point clouds, and it is determined whether it meets the preset convergence condition. If the registration error does not meet the convergence condition, the rigid body transformation matrix is ​​recalculated based on the new corresponding point pairs, and the position and orientation of each local point cloud are iteratively adjusted. This process is repeated until the error converges. After completing the iterative registration, the overlapping areas between local point clouds are fused. Repeated points and redundant data are integrated through weighted averaging or optimal point selection strategies to generate high-precision, complete, and continuous final 3D point cloud data.

[0067] The convergence condition is set according to the following rules: when the average Euclidean distance or root mean square error between corresponding points of adjacent local point cloud data is lower than a preset threshold during the iterative registration process, and the error change in several consecutive iterations is less than the fine-tuning threshold, the registration is considered to have converged. At the same time, a maximum number of iterations can be set to avoid infinite loops and ensure that the ICP algorithm terminates when the error meets the stability requirement or reaches the upper limit of iteration, thus generating the final 3D point cloud data.

[0068] As a preferred embodiment, the solution of this application is implemented as follows: When performing a 3D scan on a historical artillery shell with a diameter of 120 mm and a length of approximately 500 mm, an industrial camera array acquires multiple local point cloud data of the shell's surface, each point cloud containing millions of 3D coordinate points. These local point clouds are input into the ICP algorithm. The algorithm identifies corresponding point pairs for each region of the shell's surface in different scan data by calculating the nearest point correspondence between adjacent point clouds. Then, based on these corresponding point pairs, it calculates the rigid body transformation matrix, including rotation angles and translation vectors, to accurately align each local point cloud to a unified coordinate system. The registration error, such as the mean Euclidean distance between points, is calculated, and it is determined whether the error meets the convergence condition (e.g., error less than 0.2 mm). If the error does not meet the convergence requirement, the corresponding point relationships are updated and the transformation matrix is ​​recalculated, iterating repeatedly until the error converges. The data in the overlapping areas are fused, redundant points are eliminated by weighted averaging, and a complete and continuous three-dimensional point cloud model is generated, which accurately reflects the geometric features of the projectile surface, such as the projectile curvature, the position of the joint, and external markings, providing a high-precision data foundation for subsequent voxelization downsampling, curvature analysis, and deep learning recognition.

[0069] Through the above technical solution, this application improves the accuracy and completeness of the three-dimensional model of the projectile surface by accurately registering the three-dimensional point cloud data using the ICP algorithm to form the final three-dimensional point cloud data. During the registration process, the rigid body transformation matrix is ​​calculated between corresponding points in each local point cloud, achieving spatial alignment at different scanning angles, eliminating displacement and rotation errors during the scanning process, and ensuring that the registration error converges to a set threshold through iterative updates, guaranteeing accurate matching of each point cloud data. Data fusion in overlapping areas further eliminates redundant points and noise, making the final generated three-dimensional point cloud data continuous and smooth, presenting details such as the projectile's shape features, surface curvature, seam positions, and surface markings.

[0070] This application further proposes methods for generating two-dimensional image features by performing nonlocal mean denoising on two-dimensional image sequences, using CLAHE technology to enhance the contrast of marked regions, and correcting image distortion through perspective transformation, including:

[0071] Nonlocal mean denoising includes: selecting multiple candidate pixel regions within a search window based on each pixel in the image data of the two-dimensional image sequence; calculating the gray-level similarity between the candidate pixel regions and the central pixel region; determining weight coefficients based on the gray-level similarity; and performing a weighted average of the candidate pixel regions to obtain the pixel value.

[0072] The CLAHE technique for enhancing the contrast of marked regions includes: dividing the image data within a two-dimensional image sequence into multiple regularly distributed local region grids; calculating a histogram of pixel gray-level distribution for each local region grid; redistributing the frequency of gray levels exceeding the contrast clipping threshold based on the histogram; performing an adaptive equalization transformation on the histogram to generate a locally contrast-enhanced image; and identifying the location of marked regions using an image segmentation algorithm and applying optimized contrast enhancement parameters to the marked regions.

[0073] Perspective transformation correction of image distortion includes: extracting feature points from image data in a two-dimensional image sequence based on a corner detection algorithm; matching feature point descriptors to obtain feature point pairs between the image and a standard plane; selecting interior feature point pairs that conform to the homography model based on the RANSAC algorithm; calculating the homography matrix based on the interior feature point pairs; and using the homography matrix to perform bilinear interpolation resampling on the image data of the two-dimensional image sequence to correct geometric distortion caused by the shooting angle.

[0074] Image data from a two-dimensional image sequence that has undergone nonlocal mean denoising, CLAHE contrast enhancement, and perspective transformation correction are integrated into two-dimensional image features.

[0075] Specifically, during feature extraction from two-dimensional image sequences, non-local mean denoising is used to precisely suppress image noise. For each pixel in the two-dimensional image sequence, multiple candidate pixel regions are selected within a set search window. Weighting coefficients are determined by calculating the gray-level similarity between the candidate regions and the central pixel region. Then, a weighted average is applied to the candidate pixel regions, effectively reducing random noise while preserving image edges and texture details. Contrast enhancement is achieved using CLAHE (Contrast Limited Adaptive Histogram Equalization). This method divides the image into multiple local grid regions, calculates the pixel gray-level histogram for each region, and evenly distributes gray-level frequencies exceeding a set contrast clipping threshold. An adaptive equalization transformation is then performed on the histogram to generate a locally contrast-enhanced image. Image segmentation algorithms accurately identify the location of the marked regions on the projectile surface, and optimized contrast enhancement parameters are applied to these key areas to highlight the projectile's identification features and improve recognition accuracy. To address issues related to shooting angle and lens distortion, perspective transformation is used to correct image geometric deformation. Image feature points are extracted using a corner detection algorithm and matched with feature point descriptors to obtain feature point pairs between the image and a standard plane. Then, the RANSAC algorithm is used to select interior feature point pairs that conform to the homography model, the homography matrix is ​​calculated, and bilinear interpolation resampling is used to perform geometric correction on the image, thereby eliminating distortions such as tilt and stretching caused by the shooting angle. The resulting two-dimensional image sequence, after nonlocal mean denoising, CLAHE contrast enhancement, and perspective transformation correction, retains the surface texture and marking information of the projectile.

[0076] The contrast cropping threshold is determined based on the pixel gray-level distribution statistically analyzed by the gray-level histogram of each local region grid in the two-dimensional image sequence.

[0077] As a preferred embodiment, the solution of this application is implemented as follows: When processing an image of a 75mm caliber artillery shell left over from World War II, multiple candidate pixel regions are selected within a set search window for each pixel in the image. The gray-level similarity between these regions and the central pixel is calculated, and weights are assigned according to the similarity. A new pixel value is obtained by weighted averaging, thereby effectively removing interference caused by uneven lighting or sensor noise in the shooting environment, while preserving the fine engravings and marking textures on the shell surface. CLAHE technology is used to enhance the local contrast of the shell surface markings. The image is divided into several local grid regions, and a gray-level histogram is calculated for each grid. The frequency of gray levels exceeding a set contrast clipping threshold is redistributed, and an adaptive equalization transformation is performed to generate a locally enhanced image. The shell number, production mark, and batch number are identified using an image segmentation algorithm, and optimized contrast enhancement parameters are applied to these key marking regions to make them more prominent and clear in subsequent identification stages. To eliminate geometric distortion caused by different camera shooting angles, feature points in the image are extracted based on a corner detection algorithm. Feature point descriptors are used to match and obtain feature point pairs with the standard plane. Outliers are removed using the RANSAC algorithm, the homography matrix is ​​calculated, and the image is corrected by bilinear interpolation resampling, so that the proportions and angles of the projectile surface markings and overall shape can be accurately restored.

[0078] Through the above technical solutions, this application improves the recognizability and data quality of shell surface features by performing nonlocal mean denoising, CLAHE contrast enhancement, and perspective transformation correction on two-dimensional image sequences. Nonlocal mean denoising can suppress environmental noise and sensor errors while preserving image details, making minute engravings, marking textures, and surface defects on the shell surface clearly visible, thus providing more accurate basic information for subsequent feature extraction. CLAHE technology enhances the contrast of key marked areas through adaptive histogram equalization of local regions. Even under uneven lighting or shadow occlusion, marking text, batch numbers, and production markings remain clearly prominent, improving the sensitivity of the classification model to fine-grained features. Perspective transformation correction can eliminate geometric distortions caused by differences in shooting angle and camera position, ensuring that the proportions and angles of the shell shape and marked areas are highly consistent with the actual physical dimensions, thereby guaranteeing accurate spatial feature matching during the training and recognition stages of the deep convolutional neural network.

[0079] This application further proposes methods for generating roughness features characterizing the surface properties of a projectile by voxelizing and downsampling the final 3D point cloud data and extracting curvature features, including:

[0080] The final 3D point cloud data is divided into multiple cubic voxel units; the coordinates of representative points in the 3D point cloud data within each cubic voxel unit are calculated, and the 3D point cloud data is downsampled; the point cloud set within the neighborhood of each downsampled 3D point cloud data is determined; the covariance matrix is ​​calculated based on the point cloud set within the neighborhood and decomposed into eigenvalues; the Gaussian curvature and mean curvature are calculated based on the eigenvalues; the variation characteristics of the Gaussian curvature and mean curvature are statistically analyzed to generate roughness features characterizing the surface properties of the projectile.

[0081] Specifically, in the implementation of voxel downsampling and curvature extraction to construct surface roughness features from the final 3D point cloud, the point cloud is divided into several cubic voxel units according to a preset voxel size. Within each voxel, the centroid or density-weighted centroid is used as the representative point coordinates to complete the downsampling, thereby preserving the geometric contour while significantly reducing the number of points. For each representative point, its local point cloud set is collected based on a given neighborhood definition (e.g., fixed radius r or k nearest neighbors). Outliers are removed from this neighborhood point set, and the point coordinates are weighted using distance weights to enhance robustness. The covariance matrix of this neighborhood point set relative to the representative point is calculated and its eigenvalues ​​are decomposed to obtain the eigenvalues ​​λ0≤λ1≤λ2 and the corresponding eigenvectors sorted by size. The principal curvatures k1 and k2 are estimated using a combination of covariance spectrum and local quadratic surface fitting. Initially, the eigenvector corresponding to the smallest eigenvalue is used as the normal vector for preliminary estimation. Then, a weighted least squares quadratic surface is fitted in the local coordinate system of the normal vector to accurately solve for the principal curvatures. Alternatively, the curvature measure can be approximated using the spectral method with the λ coefficient. The resulting principal curvatures are used to calculate commonly used curvature descriptors—Gaussian curvature K=k1·k2, mean curvature H=(k1+k2) / 2, and curvature direction information (principal curvature direction). For each representative point, the distribution characteristics of curvature in the neighborhood (mean, variance, skewness, kurtosis), extreme values, percentiles, and local curvature gradients are also statistically analyzed. These statistics are aggregated at the voxel grid scale to form a roughness description for each voxel (e.g., curvature energy, curvature texture histogram, surface variability λ0 / (λ0+λ1+λ2)). To obtain a robust multi-scale characterization, the above calculations can be repeated under different voxel sizes and neighborhood radii to generate multi-scale roughness vectors, which are then normalized to eliminate the influence of size differences between projectiles of different calibers. These curvatures and statistical features are then concatenated in a predetermined order to form a roughness feature vector that reflects the geometric details of the projectile's microscopic irregularities, rust pits, seams, and weld marks. This vector serves as the 3D input feature for subsequent cross-modal fusion and deep learning classification.

[0082] As a preferred embodiment, the specific implementation of this application is as follows: The complete point cloud data is divided into cubic voxel units with a side length of 5 mm. The centroid coordinates of all points within each voxel unit are calculated and used as representative points, thereby downsampling the original million-level point cloud to a sparse representation of approximately 100,000 points while preserving the main geometric shape of the projectile surface. A neighborhood with a radius of 10 mm is defined centered on each representative point. All point clouds within this neighborhood are collected, and outliers that are too far from the center point are removed. Then, the covariance matrix of the neighborhood point set is calculated and decomposed into eigenvalues ​​λ0, λ1, and λ2. Using these eigenvalues, local curvature is calculated using K = λ0·λ1 (Gaussian curvature) and H = (λ0 + λ1) / 2 (mean curvature). The mean, variance, and extreme values ​​of curvature within each neighborhood are statistically analyzed to reflect minute surface irregularities and roughness variations. The curvature statistics of each voxel are integrated to form a complete projectile surface roughness feature vector.

[0083] Through the above technical solution, this application improves the representation accuracy and processing efficiency of projectile surface features by voxelizing and downsampling the final 3D point cloud data and extracting curvature features. The high-density point cloud is divided into multiple cubic voxel units. Within each voxel unit, the centroid of the point cloud is calculated as a representative point, achieving downsampling while preserving the overall geometry and key details of the projectile surface. A neighborhood point set is constructed for each representative point, and the covariance matrix of the neighborhood point cloud is calculated and decomposed into eigenvalues, thereby obtaining the local Gaussian curvature and mean curvature. These curvatures reflect the changes in microscopic features such as surface unevenness, weld seams, scratches, or corrosion. By statistically analyzing the mean, variance, and distribution characteristics of the curvature of each voxel, a roughness feature vector is generated, providing high-quality 3D representation data for subsequent fusion with 2D image features.

[0084] This application further proposes a method for fusing two-dimensional image features and roughness features through an attention mechanism to construct a unified feature tensor, including:

[0085] The feature responses of the two-dimensional image features and roughness features are calculated separately in the channel dimension to generate channel attention weights; the two-dimensional image features and roughness features are weighted according to the channel attention weights, and the feature responses of the spatial dimension are calculated to generate spatial attention weights.

[0086] Two-dimensional image features are adjusted to the same feature dimension as roughness features through convolution transformation; two-dimensional image features and roughness features are weighted and fused according to channel attention weight and spatial attention weight to generate fused features; high-order representation of fused features is extracted through three-dimensional convolution operation to construct a unified feature tensor.

[0087] Specifically, in the process of fusing 2D image features and roughness features through an attention mechanism to construct a unified feature tensor, feature responses are calculated for both 2D image features and roughness features in the channel dimension. Statistical information for each channel is obtained through global average pooling or global max pooling, and then channel attention weights are generated through a fully connected layer to reflect the importance of each channel in the overall feature representation. The original 2D image features and roughness features are weighted according to the channel attention weights, strengthening the features of key channels while suppressing interference from irrelevant channels. Feature responses are calculated in the spatial dimension, and spatial attention weights are generated through convolution operations and normalization to highlight important regions sensitive to target surface features in both the 2D plane and 3D space, ensuring the integrity of spatial information during feature fusion. To achieve dimensional alignment, the 2D image features are adjusted to the same feature dimension as the roughness features through convolution transformation, ensuring the feasibility of subsequent fusion. Based on the calculated channel attention weights and spatial attention weights, the 2D image features and roughness features are weighted and fused to generate a fused feature tensor. Building upon this, a high-order representation of the fused features is extracted through 3D convolution operations, capturing the complex relationships and local spatial structure information between multimodal features, ultimately forming a unified feature tensor. This unified feature tensor contains not only the texture, marking, and color information of the 2D image, but also the curvature and roughness information of the 3D point cloud.

[0088] As a preferred embodiment, the specific implementation of this application's solution is as follows: For example, when performing multimodal feature fusion on a projectile to be identified, the two-dimensional image features extracted from the two-dimensional image sequence of the projectile surface after nonlocal mean denoising, CLAHE enhancement, and perspective correction, along with the roughness features obtained through three-dimensional point cloud voxelization downsampling and curvature calculation, are input into the channel attention module to calculate the importance weight of each channel. For the marker texture channel in the two-dimensional image features and the high curvature channel in the roughness features, the channel attention module assigns higher weights to highlight their recognition value. Based on the channel attention weights, the respective features are weighted, and the feature response is calculated in the spatial dimension to generate spatial attention weights, thereby highlighting the image marker region and surface regions with significant changes in point cloud curvature. In the dimension alignment stage, the two-dimensional image features are adjusted to the same feature dimension as the roughness features through convolution transformation to ensure consistency between different modal features during fusion. The two-dimensional image features and roughness features are weighted and fused according to the channel and spatial attention weights to generate a fused feature tensor. Regions on the projectile surface with obvious engravings or uneven structures are enhanced in the fused tensor. By extracting high-order representations of fused features through 3D convolution operations, we can capture the complex relationship between 2D texture information and 3D curvature distribution and construct a unified feature tensor.

[0089] Through the above technical solutions, the channel attention mechanism of this application can automatically evaluate the importance of different feature channels, assigning higher weights to key texture information in two-dimensional image features and high curvature regions in roughness features, thereby enhancing the responsiveness to key details of the projectile surface. The spatial attention mechanism further emphasizes the most valuable regions for identification in spatial location, effectively strengthening key surface features such as marked textures, concave-convex structures, or abrupt curvature changes in the fused feature tensor, thus improving the discriminativeness of the feature representation. By dimensionally aligning two-dimensional image features and roughness features through convolutional transformation, different modal data can be fused in the same feature space, avoiding information loss and modal conflicts. The fused high-order feature representation captures the complex relationship between two-dimensional texture and three-dimensional surface geometry through three-dimensional convolutional operations, achieving deep complementarity of multimodal information.

[0090] This application further proposes methods for inputting a unified feature tensor into a large-scale visual analysis model for feature extraction and outputting a classification probability distribution, including:

[0091] The large-scale visual analysis model is a deep convolutional neural network. A unified feature tensor is input into the initial processing layer of the deep convolutional neural network, and local feature extraction is performed in the spatial-channel dimension through deep convolutional kernels to capture multi-scale features of the shell shape and surface markings.

[0092] After multiple feature extraction stages, each stage includes a 3D convolutional layer, a batch normalization layer, and a non-linear activation layer to extract high-level abstract feature representations. An attention mechanism module is embedded in the feature extraction stage to dynamically calculate the importance weights of each spatial location and feature channel in the unified feature tensor. Global average pooling is performed on the extracted abstract feature representations to compress the feature space dimension and retain discriminative information.

[0093] Abstract feature representations are input into fully connected layers, and the classification probability distribution is output through the fully connected network.

[0094] Specifically, the process of inputting a unified feature tensor into a large-scale visual analysis model for feature extraction and outputting a classification probability distribution is highly complex and hierarchical. The unified feature tensor, as a multimodal fusion result, contains texture information from the two-dimensional image of the projectile and roughness features from the three-dimensional point cloud; this information is highly rich in both spatial and channel dimensions. After being input into the initial processing layer of the DCNN, a set of three-dimensional convolutional kernels performs sliding scans and convolution operations on local features in the spatial-channel dimension, capturing projectile outlines, markings, and surface details at different scales. Each convolutional operation is followed by a batch normalization layer to stabilize the training process and suppress internal covariate shifts. Simultaneously, nonlinear mapping capabilities are introduced through nonlinear activation functions (such as ReLU or Leaky ReLU), enabling the network to learn complex nonlinear feature relationships. In multiple feature extraction stages, the network progressively extracts higher-level abstract features, with each stage embedding an attention mechanism module. The channel attention module dynamically adjusts the importance of each feature channel, amplifying key texture and curvature variation features. The spatial attention module analyzes the spatial distribution of the feature map, highlighting areas with surface markings or geometric abrupt changes, thus improving the discriminative power of the features. At the end of each stage, global average pooling compresses the spatial dimension of the feature map while preserving global discriminative information, generating a compact and discriminative high-dimensional feature vector. This high-dimensional abstract feature representation is input into a fully connected layer, and after weight matrix operations and nonlinear activation, it outputs a classification probability distribution. Each probability value corresponds to the prediction confidence of the projectile type, applicable weapon type, caliber, and ammunition type.

[0095] As a preferred embodiment, the solution of this application is implemented as follows: When detecting the projectile to be identified, a unified feature tensor integrating two-dimensional image features and three-dimensional point cloud roughness features is input into the initial processing layer of the large-scale visual analysis model. At this time, the network performs local scanning and feature extraction on the tensor in the spatial-channel dimension through a three-dimensional convolutional kernel, capturing multi-scale features of the projectile's outline, markings, and surface texture. For projectiles with obvious engravings or sprayed markings, the convolutional kernel can detect these subtle shape and texture differences at different spatial scales. After the initial convolution, the feature map enters multiple consecutive feature extraction stages, each stage including a three-dimensional convolutional layer, a batch normalization layer, and a nonlinear activation layer, gradually extracting high-level abstract features of the projectile's morphology and surface markings, and enhancing the discriminative power of the feature representation. During this process, the embedded attention mechanism module dynamically adjusts the weights of the feature tensor: the channel attention module amplifies important channel features related to the projectile's material and marking type, while the spatial attention module emphasizes key areas of the projectile's surface markings or prominent structures, enabling the network to focus on the most discriminative features. The extracted high-level abstract features are compressed into a single feature vector through global average pooling, while retaining the core discriminative features of the shell's shape and surface information. This feature vector is input into a fully connected layer, where matrix operations and a Softmax activation function are used to generate a classification probability distribution. For example, the layer predicts whether the shell is a 105mm howitzer, suitable for the M101 howitzer, or a standard high-explosive shell, and provides probability values ​​for each category.

[0096] Through the above technical solution, this application inputs the fused unified feature tensor into a large-scale visual analysis model for feature extraction, achieving high-precision, multi-scale recognition of projectile surface morphology, texture, and marking information. The initial processing layer performs local scanning in the spatial-channel dimension using a 3D convolutional kernel, capturing detailed features of the projectile's outline, marking symbols, and surface microstructure, ensuring sensitivity to projectiles from different angles and materials. Multiple feature extraction stages, each consisting of a 3D convolutional layer, a batch normalization layer, and a nonlinear activation layer, continuously abstract and strengthen the feature representation, enabling the network to extract high-level shape patterns, marking positions, and surface roughness features. The embedded attention mechanism dynamically calculates the importance weights of spatial positions and feature channels, allowing the network to focus on key areas of the projectile and marking features, thereby enhancing discriminative ability. The extracted high-order features undergo global average pooling to compress the spatial dimension into a compact vector while retaining the most discriminative feature information. The compressed abstract features are input into the fully connected layer, and the classification probability distribution is output through the fully connected network, which can accurately identify the shell type, applicable weapon style, caliber and shell type.

[0097] This application further proposes methods for determining initial identification results and confidence levels based on classification probability distributions, including:

[0098] The ammunition types in the classification probability distribution are sorted according to their probability values, and the ammunition type with the highest probability value is selected as the initial identification result.

[0099] Based on the initial identification results of the ammunition type, the probability distribution of each dimension corresponding to the applicable weapon type, caliber and ammunition type in the classification probability distribution is classified and determined to identify the applicable weapon type, caliber and ammunition type.

[0100] The highest probability value corresponding to the ammunition type in the initial identification results is determined as the confidence level.

[0101] Specifically, the process of determining the initial identification result and confidence level based on the classification probability distribution sorts the multidimensional probability vector output by the deep network according to the ammunition type dimension, and selects the category with the highest probability value as the ammunition type of the initial identification result. According to this initial ammunition type, the corresponding probability distributions are extracted from other relevant dimensions output by the network (applicable weapon style, caliber, ammunition type), and maximum value judgment or threshold filtering is performed independently for each dimension to determine the specific category (for example, in the caliber dimension, the caliber value with the highest probability is selected, and if the maximum value is lower than the preset sub-threshold, it is marked as "to be reviewed"). The maximum probability value of the selected ammunition type is used as the confidence level of the initial identification result, and the probabilities of all relevant dimensions are recorded as auxiliary decision evidence for expert review or subsequent probability correction. This process also includes checking for cross-dimensional inconsistencies (for example, if there is a physical mismatch between the ammunition type and caliber, it is marked as abnormal and triggers review or automatic weight reduction).

[0102] As a preferred embodiment, the specific implementation of this application's solution is as follows: In a certain field test, the classification probability distribution of an unknown shell output is as follows: Ammunition type—high explosive shell 0.52, chemical shell 0.28, practice shell 0.10, fragmentation shell 0.10; Applicable weapon type—M101 howitzer 0.60, M114 howitzer 0.25, others 0.15; Caliber—105mm 0.55, 155mm 0.30, 75mm 0.15; Ammunition type subdivision—ordinary shell 0.50, armor-piercing shell 0.20, chemical loading 0.30. According to the rules, "high explosive shell" (probability 0.52) is taken as the initial ammunition type, the caliber is determined to be 105mm (probability 0.55), the applicable weapon is determined to be M101 (probability 0.60), and the initial ammunition type is determined to be ordinary shell (probability 0.50). The initial confidence level is taken as the maximum probability of the ammunition type, 0.52. Verification of dimensional consistency: If historical databases or dimensional measurements show that this shape typically corresponds to 155mm or that this caliber is rarely combined with "high-explosive shells," the initial result will be marked as "low consistency," and (if the confidence level is below a preset threshold, such as 0.85) auxiliary identification (e.g., LIBS spectral detection or manual verification) will be triggered. All original probability distributions, dimensional parameters, and consistency judgments will be saved along with the identification report.

[0103] Through the above technical solutions, this application adopts an initial judgment and confidence calculation based on probability distribution to quantify uncertainty in the automatic identification process, avoiding the blindness caused by a single hard decision; the probability of each relevant dimension (ammunition type, applicable weapon, caliber, ammunition type) is recorded simultaneously, which not only provides input for subsequent Bayesian correction or auxiliary evidence fusion, but also facilitates human experts to quickly understand the judgment basis and weaknesses of the model; through cross-dimensional consistency checks and threshold strategies, potential conflicts or low-confidence scenarios can be identified in a timely manner and a safety-first auxiliary identification process can be triggered, thereby significantly reducing the risk of misjudgment and missed judgment.

[0104] This application further proposes that when the confidence level is less than a confidence threshold, an auxiliary recognition mechanism is triggered to perform probability correction, and when generating the final recognition report, the following is included:

[0105] When the confidence level is less than the confidence threshold, the laser-induced breakdown spectral analysis module is triggered to perform micro-area elemental composition detection on the surface of the projectile to be identified; the spectral signal of the projectile material is acquired and converted into elemental feature information; the elemental feature information and the projectile size parameters are input into a Bayesian network to calculate the posterior probability distribution of each type of ammunition; the initial identification result is corrected according to the posterior probability distribution, and the final identification report is generated.

[0106] Specifically, when the maximum confidence level obtained from the initial identification is lower than a preset threshold, an auxiliary identification module is automatically activated, typically implemented as a laser-induced breakdown spectroscopy (LIBS) analysis module. The operation process includes: under multimodal acquisition posture, a robotic arm or positioning device positions the LIBS probe to the micro-region to be measured on the surface of the projectile; a series of pulsed lasers are emitted to break down the micro-region, generating plasma and acquiring instantaneous spectral signals; the obtained spectrum is preprocessed (baseline correction, noise filtering, spectral registration and normalization), peak detection and spectral line assignment are performed, and the relative intensity, spectral line ratio, and several statistical features (such as peak height, peak area, signal-to-noise ratio, etc.) of key elements are extracted to form elemental feature vectors (e.g., the intensity and ratio of elements such as Cl, S, P, N, Cu, and Fe). These elemental feature vectors, along with evidence such as on-site measured size parameters and density estimation, are used as observation inputs and fed into a trained Bayesian network (or a Bayesian model based on a probability condition table / empirical likelihood function). The Bayesian network uses the initial classification probability as the prior distribution and the conditional probabilities of observed elemental features and size (obtained from an offline sample library or experimental calibration) as the likelihood to calculate the posterior probability distribution for each candidate ammunition type. Based on the posterior probabilities, the probabilities are reordered, and the corrected probability values ​​and confidence intervals are output. If the highest posterior category exceeds the final judgment threshold, it is taken as the final identification conclusion; otherwise, it is marked as "requires manual review / further testing." The identification report also includes the original two-dimensional / three-dimensional evidence, LIBS spectral curves, elemental feature vectors, a comparison of prior and posterior probabilities, the rationale for the judgment, and recommended actions.

[0107] As a preferred embodiment, the specific implementation of this application's scheme is as follows: A severely corroded shell is detected on-site. The initial visual-geometric recognition output classification probability is: high explosive 0.48, chemical 0.38, practice 0.08, and others 0.06 (initial confidence level 0.48, below the threshold 0.85), triggering LIBS detection. LIBS detects significant Cl and S spectral lines in a certain micro-region of the shell. After processing, the elemental feature vector shows that the Cl intensity ratio is abnormally high, and the Cl / S ratio is highly consistent with the typical distribution of known chemical agent-filled shells in the database. Based on the likelihood function constructed from offline sample statistics, the likelihood of "chemical shell" is calculated to be much higher than that of "high explosive shell". Substituting the initial prior (high explosive 0.48, chemical 0.38) and this likelihood into a Bayesian update, the posterior probabilities are obtained: chemical 0.92, high explosive 0.05, practice 0.02, and others 0.01. Because the probability of a chemical bomb in the posterior is greater than 0.9, the final identification conclusion is set as "historical legacy munition containing chemical agents (posterior probability 0.92)". The generated report includes LIBS spectrum, element peak position and intensity table, posterior calculation details, recommended isolation radius and disposal procedures (such as activating a dedicated chemical defense working group, neutralization and dedicated destruction measures), and the credibility of the evidence is marked.

[0108] Through the above technical solutions, this application introduces an auxiliary identification mechanism of LIBS and Bayesian posterior correction, which can provide independent and complementary material composition evidence when the initial visual / geometric decision is uncertain, reducing the danger of misclassifying chemical bombs as conventional bombs or missing the judgment; the Bayesian framework can effectively fuse prior (model output) with observation (spectrum and size), quantify the contribution of evidence and output interpretable posterior probabilities, thereby improving the interpretability and credibility of the final judgment.

[0109] Based on the other preferred method described above, see [link / reference]. Figure 2 As shown, this embodiment provides a deep learning-based projectile type identification system for applying the aforementioned deep learning-based projectile type identification method, including:

[0110] The acquisition unit is configured to acquire multimodal data of the projectile to be identified based on an industrial camera array. The multimodal data includes a two-dimensional image sequence of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is also registered using an ICP algorithm to form the final three-dimensional point cloud data.

[0111] The processing unit is configured to perform nonlocal mean denoising on the two-dimensional image sequence, enhance the contrast of the marked region using CLAHE technology, and correct image distortion through perspective transformation to generate two-dimensional image features; perform voxelization downsampling on the final three-dimensional point cloud data and extract curvature features to generate roughness features characterizing the surface properties of the projectile; and fuse the two-dimensional image features and roughness features through an attention mechanism to construct a unified feature tensor.

[0112] The output unit is configured to input a uniform feature tensor into a large visual analysis model for feature extraction and output a classification probability distribution.

[0113] The generation unit is configured to determine the initial recognition result and confidence level based on the classification probability distribution. The confidence level is the maximum probability value in the classification probability distribution. The initial recognition result includes ammunition type, applicable weapon style, caliber, and ammunition type. When the confidence level is less than the confidence threshold, an auxiliary recognition mechanism is triggered to perform probability correction and generate the final recognition report.

[0114] In summary, by performing high-order fusion of 2D image texture features and 3D point cloud geometric features (including curvature / roughness) under an attention mechanism, the model can simultaneously utilize texture and shape discrimination information, improving the accuracy of ammunition type, caliber, and ammunition type discrimination. Preprocessing steps such as nonlocal mean denoising, CLAHE local contrast enhancement, and perspective correction can restore surface marker information degraded by corrosion, dirt, or shooting angle, reducing the misclassification rate caused by surface degradation. Based on stepwise ICP registration of local point clouds and convergence assessment and overlapping area fusion, the accuracy and consistency of 3D reconstruction are ensured, providing a reliable foundation for subsequent geometric feature extraction (such as Gaussian curvature, mean curvature, etc.), and enhancing the ability to distinguish ammunition with similar shapes but different geometric details. Channel and spatial level attention weights adaptively adjust the importance of each modal feature, achieving complementary use of information, enhancing the model's attention to key local markers and subtle geometric features, and improving discrimination performance in complex scenes. By setting a confidence threshold for classification probabilities, material composition detection, such as laser-induced breakdown spectroscopy (LIBS), is automatically triggered when confidence is insufficient. Posterior probability correction based on a Bayesian network reduces the risk of misjudgment and missed judgment, improving the credibility and security of the final decision. Automated identification and report generation significantly reduce reliance on manual experience in judgment, providing rapid decision-making suggestions to on-site personnel, shortening judgment time, and reducing the probability of safety accidents caused by human error. The synchronous triggering acquisition scheme of a ring camera array and helical laser scanning, combined with hardware triggering, ensures the temporal consistency and integrity of multimodal data, facilitating rapid and stable data acquisition and processing in on-site or assembly line conditions.

[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A deep learning-based method for identifying projectile types, characterized in that, include: The projectile to be identified is acquired using an industrial camera array. The multimodal data includes a sequence of two-dimensional images of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is further registered using the ICP algorithm to form the final three-dimensional point cloud data; The two-dimensional image sequence is subjected to nonlocal mean denoising, the contrast of the marked region is enhanced by CLAHE technology, and the image distortion is corrected by perspective transformation to generate two-dimensional image features. The final three-dimensional point cloud data is voxelized and downsampled, and curvature features are extracted to generate roughness features characterizing the surface properties of the projectile; the two-dimensional image features and the roughness features are fused through an attention mechanism to construct a unified feature tensor; The unified feature tensor is input into the large visual analysis model for feature extraction, and the classification probability distribution is output. The initial identification result and confidence level are determined based on the classification probability distribution. The confidence level is the maximum probability value in the classification probability distribution. The initial identification result includes ammunition type, applicable weapon style, caliber, and ammunition type. When the confidence level is less than the confidence threshold, an auxiliary identification mechanism is triggered to perform probability correction and generate a final identification report. When the unified feature tensor is input into a large-scale visual analysis model for feature extraction and the classification probability distribution is output, the following steps are included: The large-scale visual analysis model is a deep convolutional neural network. The unified feature tensor is input into the initial processing layer of the large-scale visual analysis model. Local features are extracted in the spatial-channel dimension through deep convolutional kernels to capture the multi-scale features of the shell shape and surface markings. After multiple feature extraction stages, each stage includes a 3D convolutional layer, a batch normalization layer, and a non-linear activation layer to extract high-level abstract feature representations. An attention mechanism module is embedded in the feature extraction stage to dynamically calculate the importance weights of each spatial location and feature channel in the unified feature tensor. Global average pooling is performed on the extracted abstract feature representations to compress the feature space dimension and retain discriminative information. Abstract feature representations are input into fully connected layers, and the classification probability distribution is output through the fully connected network.

2. The deep learning-based shell type identification method according to claim 1, characterized in that, When acquiring multimodal data of a projectile to be identified using an industrial camera array, the following are included: The industrial camera array is a ring array that surrounds and acquires circumferential images of the projectile to be identified, generating the two-dimensional image sequence and three-dimensional point cloud data.

3. The deep learning-based shell type identification method according to claim 2, characterized in that, When the 3D point cloud data is further registered using the ICP algorithm to form the final 3D point cloud data, it includes: Multiple local point cloud data of the three-dimensional point cloud data are input into the ICP algorithm to determine the corresponding point relationship of each adjacent local point cloud data. Based on the corresponding point relationship, the rigid body transformation matrix between local point clouds is calculated, and the spatial position of the local point cloud data is adjusted according to the rigid body transformation matrix. Determine whether the local point cloud registration error meets the convergence condition. If the convergence condition is not met, update the corresponding point relationship and recalculate the rigid body transformation matrix. Continue until the local point cloud registration error meets the convergence condition. Perform overlapping region data fusion on the converged local point cloud data to generate the final three-dimensional point cloud data.

4. The deep learning-based shell type identification method according to claim 3, characterized in that, When generating two-dimensional image features by performing nonlocal mean denoising on the two-dimensional image sequence, enhancing the contrast of the marked region using CLAHE technology, and correcting image distortion through perspective transformation, the process includes: The nonlocal mean denoising process includes: selecting multiple candidate pixel regions within a search window based on each pixel of the image data in the two-dimensional image sequence; calculating the gray-level similarity between the candidate pixel regions and the central pixel region; determining weight coefficients based on the gray-level similarity; and performing a weighted average of the candidate pixel regions to obtain the pixel value. The CLAHE technique for enhancing the contrast of marked regions includes: dividing the image data within the two-dimensional image sequence into multiple regularly distributed local region grids; calculating a histogram of pixel grayscale distribution for each local region grid; redistributing the frequency of grayscale levels exceeding the contrast clipping threshold based on the histogram; performing an adaptive equalization transformation on the histogram to generate a locally contrast-enhanced image; and identifying the location of marked regions using an image segmentation algorithm and applying optimized contrast enhancement parameters to the marked regions. The perspective transformation correction of image distortion includes: extracting feature points from the image data in the two-dimensional image sequence based on a corner detection algorithm; matching feature point descriptors to obtain feature point pairs between the image and a standard plane; selecting interior feature point pairs that conform to the homography model based on the RANSAC algorithm; calculating the homography matrix based on the interior feature point pairs; and using the homography matrix to perform bilinear interpolation resampling on the image data of the two-dimensional image sequence to correct geometric distortion caused by the shooting angle. The image data in the two-dimensional image sequence after nonlocal mean denoising, CLAHE contrast enhancement, and perspective transformation correction are integrated into the two-dimensional image features.

5. The deep learning-based shell type identification method according to claim 4, characterized in that, When performing voxelization downsampling and extracting curvature features on the final 3D point cloud data to generate roughness features characterizing the surface properties of the projectile, the process includes: The final 3D point cloud data is divided into multiple cubic voxel units; the coordinates of representative points of the 3D point cloud data within each cubic voxel unit are calculated, and the 3D point cloud data is downsampled; the point cloud set within the neighborhood of each downsampled 3D point cloud data is determined; the covariance matrix is ​​calculated based on the point cloud set within the neighborhood and decomposed into eigenvalues; the Gaussian curvature and mean curvature are calculated based on the eigenvalues; the variation characteristics of the Gaussian curvature and mean curvature are statistically analyzed to generate the roughness feature characterizing the surface properties of the projectile.

6. The deep learning-based shell type identification method according to claim 5, characterized in that, When fusing the two-dimensional image features and the roughness features through an attention mechanism to construct a unified feature tensor, the following steps are included: The feature responses of the two-dimensional image features and roughness features are calculated in the channel dimension to generate channel attention weights; the two-dimensional image features and roughness features are weighted according to the channel attention weights, and the feature responses in the spatial dimension are calculated to generate spatial attention weights. The two-dimensional image features are adjusted to the same feature dimension as the roughness features through convolution transformation; the two-dimensional image features and roughness features are weighted and fused according to channel attention weight and spatial attention weight to generate fused features; the high-order representation of the fused features is extracted through three-dimensional convolution operation to construct the unified feature tensor.

7. The deep learning-based shell type identification method according to claim 6, characterized in that, When determining the initial identification result and confidence level based on the classification probability distribution, the following are included: The ammunition types in the classification probability distribution are sorted according to their probability values, and the ammunition type with the highest probability value is selected as the ammunition type of the initial identification result; Based on the initial identification results of the ammunition type, the probability distribution of each dimension corresponding to the applicable weapon type, caliber and ammunition type in the classification probability distribution is classified and determined to identify the applicable weapon type, caliber and ammunition type. The maximum probability value corresponding to the ammunition type in the initial identification result is determined as the confidence level.

8. The deep learning-based shell type identification method according to claim 7, characterized in that, When the confidence level is less than the confidence threshold, the auxiliary recognition mechanism is triggered to perform probability correction. When generating the final recognition report, the following is included: When the confidence level is less than the confidence threshold, the laser-induced breakdown spectral analysis module is triggered to perform micro-area elemental composition detection on the surface of the projectile to be identified; the spectral signal of the projectile material is acquired and converted into elemental feature information; the elemental feature information and size parameters are input into a Bayesian network to calculate the posterior probability distribution of each type of ammunition; the initial identification result is corrected according to the posterior probability distribution, and the final identification report is generated.

9. A deep learning-based projectile type identification system, used to apply the deep learning-based projectile type identification method as described in any one of claims 1-8, characterized in that, include: The acquisition unit is configured to acquire multimodal data of the projectile to be identified based on an industrial camera array. The multimodal data includes a two-dimensional image sequence of the projectile surface, three-dimensional point cloud data, and size parameters. The three-dimensional point cloud data is further registered using an ICP algorithm to form the final three-dimensional point cloud data. The processing unit is configured to perform nonlocal mean denoising on the two-dimensional image sequence, enhance the contrast of the marked region using CLAHE technology, and correct image distortion through perspective transformation to generate two-dimensional image features. The final three-dimensional point cloud data is voxelized and downsampled, and curvature features are extracted to generate roughness features that characterize the surface properties of the projectile. The two-dimensional image features and the roughness features are fused using an attention mechanism to construct a unified feature tensor; The output unit is configured to input the unified feature tensor into a large visual analysis model for feature extraction and output a classification probability distribution. The generation unit is configured to determine an initial identification result and confidence level based on the classification probability distribution, wherein the confidence level is the maximum probability value in the classification probability distribution, and the initial identification result includes ammunition type, applicable weapon style, caliber, and ammunition type; when the confidence level is less than the confidence threshold, an auxiliary identification mechanism is triggered to perform probability correction and generate a final identification report.

Images