Target recognition method, device and storage medium

Abstract

The application provides a target identification method and device and a storage medium, and relates to the technical field of target identification based on material characteristics. The method comprises the following steps: acquiring a hyperspectral image, a polarization spectrum and an infrared image of a target; inputting the hyperspectral image into a double-flow identification network model after spectral offset correction to obtain a static true-false identification result; calculating an inter-frame difference feature according to the infrared image, and determining a dynamic true-false identification result according to the inter-frame difference feature; determining a hyperspectral feature according to the hyperspectral image, determining a polarization material feature according to the polarization spectrum, determining an infrared feature according to the infrared image, and determining a fusion true-false identification result based on the hyperspectral feature, the polarization material feature and the infrared feature; and determining a final true-false identification result based on the dynamic true-false identification result, the static true-false identification result and the fusion true-false identification result. The application can solve the problems of low dynamic scene precision and poor adaptability to complex and harsh environments in the existing identification technology, and improve the true-false target discrimination ability.

Description

Technical Field

[0001] This invention relates to the field of target identification technology based on material properties, and in particular to a target identification method, device and storage medium. Background Technology

[0002] Military camouflage target spectral identification is a core technology in the fields of battlefield optoelectronic reconnaissance and border defense. Its core principle is to accurately distinguish real combat equipment from passive camouflage false targets such as paint and fabrics based on the refined spectral texture characteristics of the target. It is widely used in various reconnaissance and combat equipment and is a core means to improve the ability to identify battlefield targets and break through camouflage interference.

[0003] Current technologies generally employ a combination of hyperspectral imaging and deep learning recognition networks. This involves acquiring visible or near-infrared hyperspectral images and using deep learning networks to extract two-dimensional spatial contour features and one-dimensional spectral features of the target. After feature fusion, the camouflaged target is classified and identified. However, this existing approach has significant drawbacks. Firstly, it is only suitable for identifying static, low-speed camouflaged targets. For high-speed, maneuvering camouflaged military vehicles and dynamically released decoys, severe inter-frame motion blur and pixel shift occur during imaging. This directly leads to shifts and distortions in the spectral features extracted by the network, significantly reducing feature representation capabilities. Ultimately, this results in a sharp drop in the accuracy of distinguishing real combat equipment from camouflaged targets in dynamic scenarios, failing to meet the precise identification requirements of dynamic battlefields. On the other hand, existing technologies rely solely on a single hyperspectral imaging acquisition mode. In complex battlefield interference scenarios such as sandstorms, dense smoke, strong backlight, and low visibility at night, they are prone to problems such as loss of effective spectral information and sharp decline in image signal-to-noise ratio. This leads to weakening of target spectral features and reduced recognizability. The spectral features of metal equipment and camouflage coatings are easily confused, resulting in high false alarm and false detection rates. The stability and robustness of the algorithm are extremely poor, making it difficult to adapt to the needs of all-weather, complex interference, and dynamic battlefield identification operations. Summary of the Invention

[0004] This invention provides a target recognition method, device, and storage medium to address the problems of low accuracy in dynamic scenes and poor adaptability to complex and harsh environments in existing spectral recognition technologies for camouflaged targets, thereby improving the ability to distinguish between genuine and fake targets.

[0005] In a first aspect, embodiments of the present invention provide a target recognition method, comprising: Acquire hyperspectral, polarization, and infrared images of the target; The hyperspectral image is subjected to spectral shift correction. The corrected hyperspectral image is then input into a preset dual-stream recognition network model to extract spatial-spectral joint features, and the static authenticity recognition result of the target is obtained from the model output. Calculate inter-frame difference features based on the infrared image, and determine the dynamic authenticity recognition result of the target based on the inter-frame difference features; Hyperspectral features are determined based on the hyperspectral image, polarization material features are determined based on the polarization spectrum, infrared features are determined based on the infrared image, and the fusion authenticity recognition result of the target is determined based on the hyperspectral features, the polarization material features, and the infrared features. Based on the dynamic authenticity identification results, static authenticity identification results, and fused authenticity identification results of the target, the final authenticity identification result of the target is determined.

[0006] In one possible implementation, the step of calculating inter-frame difference features based on the infrared image and determining the dynamic authenticity recognition result of the target based on the inter-frame difference features includes: Calculate the difference in infrared emissivity between adjacent frames of infrared images, and determine the inter-frame differential feature based on the difference in infrared emissivity. Based on the magnitude of the difference in infrared emissivity, the confidence level of the target as a fake target is determined, and the dynamic authenticity identification result of the target is obtained; The confidence level is positively correlated with the magnitude of the difference in infrared emissivity.

[0007] In one possible implementation, determining the fused authenticity identification result of the target based on the hyperspectral features, the polarization material features, and the infrared features includes: Obtain the weighting coefficients of the hyperspectral features, the polarization material features, and the infrared features; Based on the weighting coefficients, the hyperspectral features, the polarization material features, and the infrared features are weighted and fused to obtain the fused features; The fused features are input into a preset classifier, and the fused authenticity recognition result of the target is obtained based on the confidence level of the target as a fake target output by the classifier.

[0008] In one possible implementation, obtaining the weighting coefficients for the hyperspectral features, the polarization material features, and the infrared features includes: Obtain the preset initial weighting coefficients of the hyperspectral features, the polarization material features, and the infrared features, as well as the current environmental data; Determine the environmental signal-to-noise ratio based on the current environmental data; Based on the environmental signal-to-noise ratio, the preset initial weighting coefficients are adjusted to obtain the weighting coefficients of the hyperspectral feature, the polarization material feature, and the infrared feature; wherein, the weighting coefficient of the hyperspectral feature is positively correlated with the environmental signal-to-noise ratio, and the weighting coefficients of the polarization material feature and the infrared feature are negatively correlated with the environmental signal-to-noise ratio.

[0009] In one possible implementation, the dynamic authenticity identification result, the static authenticity identification result, and the fused authenticity identification result of the target are all the confidence scores of the target for faking a false target; The determination of the final authenticity identification result of the target based on the dynamic authenticity identification result, static authenticity identification result, and fused authenticity identification result includes: The dynamic authenticity recognition results, static authenticity recognition results, and fused authenticity recognition results are weighted and fused according to preset weights to obtain the final confidence level of the target as a disguised fake target; If the final confidence level is greater than or equal to the preset confidence level threshold, then the target is determined to be a disguised false target; If the final confidence level is less than the confidence threshold, then the target is determined to be a real target.

[0010] In one possible implementation, the spectral shift correction of the hyperspectral image includes: The horizontal and vertical optical flow displacements of each pixel in the hyperspectral image are determined using an optical flow algorithm. Based on the horizontal optical flow displacement and the vertical optical flow displacement, the hyperspectral image is remapped and aligned at the pixel level to obtain a hyperspectral image after spectral shift correction.

[0011] In one possible implementation, the infrared image includes a mid-wave infrared image and a long-wave infrared image, wherein the wavelength range of the long-wave infrared image is greater than the wavelength range of the mid-wave infrared image. The inter-frame difference feature is the average of the difference in infrared emissivity between adjacent frames of the mid-wave infrared image and the difference in infrared emissivity between adjacent frames of the long-wave infrared image; The infrared features include the infrared temperature gradient features and emissivity features of the mid-wave infrared image, as well as the infrared temperature gradient features and emissivity features of the long-wave infrared image.

[0012] In one possible implementation, before inputting the corrected hyperspectral image into a preset dual-stream recognition network model to extract spatial-spectral joint features, the method further includes: The scene complexity score is obtained by calculating the sum of the spatial gradients of the entire hyperspectral image using a preset scene complexity evaluation function. If the scene complexity score is less than a preset score threshold, the spectral channels of the hyperspectral image are cropped to obtain a cropped hyperspectral image. The cropped hyperspectral image is input into the dual-stream recognition network model.

[0013] In a second aspect, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation thereof.

[0014] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.

[0015] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows: The multimodal joint authenticity recognition scheme of this invention performs spectral shift correction on hyperspectral images to effectively suppress motion blur and spectral feature shift distortion caused by inter-frame pixel misalignment in high-speed maneuvering camouflage target imaging. It also mines spatial-spectral joint features through a dual-stream recognition network to ensure camouflage target recognition capability under static conditions. Furthermore, it utilizes infrared images to construct inter-frame difference features to obtain dynamic authenticity recognition results, compensating for the inadequacy of hyperspectral modalities in characterizing target motion attributes. It further integrates hyperspectral features, polarization material features, and infrared multi-dimensional features to fully leverage the complementary advantages of different imaging modalities, overcoming the shortcomings of single hyperspectral modalities in complex battlefield environments such as sandstorms, smoke, backlighting, and low visibility at night, including information loss, signal-to-noise ratio degradation, and weak anti-interference capabilities. Finally, it synthesizes static authenticity recognition results, dynamic authenticity recognition results, and multimodal fusion authenticity recognition results to output the final authenticity recognition result, achieving a combination of static and dynamic elements and mutual corroboration of multi-source information. This reduces the probability of false alarms and missed detections, significantly improving the accuracy and robustness of camouflage target authenticity judgment in complex environments. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the implementation of the target recognition method provided in this embodiment of the invention; Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] This application provides a target recognition method aimed at solving the problems of spectral feature shift and distortion caused by motion blur of high-speed maneuvering camouflaged targets, low recognition accuracy in dynamic scenes, and weak anti-interference ability of a single hyperspectral mode in complex environments. This application can be applied to: (1) Military reconnaissance field: portable reconnaissance equipment for individual soldiers, vehicle-mounted spectral reconnaissance system, UAV airborne spectral detection payload, etc., used for battlefield camouflage target detection, false target identification, and position reconnaissance.

[0019] (2) Border security field: intelligent monitoring system for border lines, perimeter security of important military facilities, etc., for detecting disguised intrusion of personnel and vehicles.

[0020] (3) Individual combat system: embedded individual soldier terminal, intelligent helmet display system, etc., to provide infantry with real-time camouflage target identification assistance.

[0021] (4) Potential civilian applications: scenarios that require hyperspectral identification and have requirements for real-time performance and environmental adaptability, such as mineral spectral identification in geological exploration, pollutant monitoring in environmental protection, and crop pest and disease identification in agricultural remote sensing.

[0022] See Figure 1 The flowchart illustrating the implementation of the target recognition method provided in this embodiment of the invention is described in detail below: Step S101: Acquire the hyperspectral image, polarization spectrum, and infrared image of the target.

[0023] In this embodiment, the target to be identified is a battlefield military camouflage target, including real military combat equipment and passive camouflage decoys composed of camouflage paint, camouflage fabric, and inflatable simulation models. Hyperspectral, polarization, and infrared images of the target can be acquired using a multi-mode integrated optoelectronic reconnaissance payload mounted on a reconnaissance platform. The payload includes a hyperspectral imaging module, a polarization imaging module, and an infrared thermal imaging module. An external trigger signal is used to achieve simultaneous exposure of multiple sensors, and geometric registration is performed on the three types of images acquired to align the spatial positions of target pixels in different modalities.

[0024] Among them, the hyperspectral image is a three-dimensional data cube containing two-dimensional spatial dimensions and multi-band spectral dimensions, used to obtain fine reflectance spectral information of the target material; the polarization spectrum is the polarization response image of the target under each spectral band, outputting the degree of polarization and polarization angle to characterize the polarization characteristics of the target surface material; the infrared image collects the target's own thermal radiation information, reflecting the target's temperature and infrared emissivity distribution, which can be used to extract inter-frame difference dynamic features and support moving target detection.

[0025] Following step S101, a candidate target pre-screening step may also be included. To address the large amount of environmental noise, highly reflective clutter, and non-threatening thermal radiation sources that may exist in infrared images, this embodiment constructs a rule-based candidate target pre-screening mechanism to filter out invalid interference at extremely low computational cost, ensuring that core computing power accurately serves truly threatening candidate targets.

[0026] The candidate target pre-screening mechanism includes three criteria: First, a geometric criterion, which calculates the area and aspect ratio of the target's connected regions. The area is preset to range from 200 to 43,000 pixels, and the aspect ratio is preset to range from 0.4 to 2.5. Regions outside these ranges are considered environmental clutter and are removed. Second, a radiation intensity criterion, which detects the maximum value of the 16-bit original ADC sampled value within the target region. If the value is lower than a preset radiation intensity threshold (e.g., 5800), it is considered low-radiation interference and is removed. Third, a boundary validity criterion, which ensures that the candidate target region does not exceed the image boundary to avoid truncating the target for subsequent recognition. After joint screening based on these three criteria, target regions that meet the conditions are included in the candidate target set. Subsequent recognition processes only target the targets in the candidate target set.

[0027] The aforementioned candidate target pre-screening mechanism can eliminate more than 90% of environmental noise and invalid reflective interference at extremely low computing power cost, reducing the number of candidate targets entering subsequent deep learning inference from hundreds per frame to less than 10 on average, significantly reducing the computational load of the core engine and system power consumption, while also significantly reducing the false alarm rate caused by high environmental reflective clutter.

[0028] Step S102: Perform spectral shift correction on the hyperspectral image, input the corrected hyperspectral image into a preset dual-stream recognition network model to extract spatial-spectral joint features, and obtain the static authenticity recognition result of the target output by the model.

[0029] To address the motion blur problem of high-speed maneuvering camouflage targets, this embodiment first performs spectral shift correction on the hyperspectral image to compensate for pixel shifts caused by inter-frame motion of the target, eliminating spectral curve distortion caused by motion blur and ensuring that pixel spectral information truly reflects the spectral reflectance characteristics of the target material itself. Then, the corrected hyperspectral image is input into a pre-defined two-stream recognition network model. This model includes parallel spatial feature branches and spectral feature branches. The spatial feature branch extracts two-dimensional spatial texture and contour appearance features from the hyperspectral image, while the spectral feature branch extracts one-dimensional material spectral features at the pixel level. The two types of features are fused to obtain spatial-spectral joint features. After classification calculation by a classification head, the static authenticity recognition result of the target is output. This result, based on the inherent properties of the target's appearance and material spectrum, represents the target authenticity judgment conclusion under static assumptions. It should be noted that the two-stream recognition network model can use conventional deep learning models in this field. For example, the two-stream recognition network model adopts a parallel two-stream convolutional neural network architecture, including a spatial feature branch, a spectral feature branch, a spatial-spectral feature fusion module, and a classification output head. The hyperspectral image cube, after spectral shift correction, is simultaneously input into both the spatial and spectral feature branches. The spatial feature branch extracts the target's two-dimensional contour and texture appearance information through multi-layer 2D convolution, batch normalization, activation functions, and pooling operations, and outputs a one-dimensional spatial feature vector after global pooling. The spectral feature branch extracts spectral-dimensional material features for each pixel's one-dimensional spectral curve using multi-layer 1D convolution, and then obtains a one-dimensional spectral feature vector through global spatial aggregation. The spatial-spectral feature fusion module concatenates the spatial and spectral feature vectors, and then completes feature interaction through a fully connected layer to obtain joint spatial-spectral features. The classification output head, composed of a fully connected layer, a Dropout layer, and a Softmax activation layer, receives the joint spatial-spectral features and outputs the confidence score of the target as a camouflaged false target. The static authenticity recognition result is determined based on the confidence score. This dual-stream recognition network model simultaneously mines the spatial appearance information and intrinsic material spectral information of the hyperspectral image, enabling it to distinguish between real combat equipment and camouflaged false targets.

[0030] Step S103: Calculate the inter-frame difference features based on the infrared image, and determine the dynamic authenticity recognition result of the target based on the inter-frame difference features.

[0031] This embodiment can calculate the difference in infrared emissivity between adjacent frames of infrared images as an inter-frame difference feature, as shown in the following formula: Δε(λ,t)=ε(λ,t+1)-ε(λ,t); Where ε is the infrared spectral emissivity of the target, Δε is the difference in infrared emissivity, t is the time, and λ is the wavelength.

[0032] Metallic targets exhibit specific transient emissivity variation patterns due to thermal inertia; however, the emissivity temporal variation characteristics of passive camouflage decoys (such as those made of paint) differ significantly from those of real metallic combat targets. Therefore, the difference in infrared emissivity can be used to distinguish between real metallic combat equipment and passive camouflage decoys. For example, based on the magnitude of the infrared emissivity difference, the confidence level of whether a target is a camouflage decoy can be calculated, thus obtaining the dynamic authenticity identification result of the target; where the confidence level is positively correlated with the magnitude of the infrared emissivity difference.

[0033] Furthermore, to improve the physical interpretability and discrimination accuracy of dynamic authenticity recognition results, a physical white-box engine based on thermodynamic intrinsic indices can be constructed. This engine extracts three physical fingerprint indices from continuous frame infrared image time-series data, building a spectral evidence chain for material determination. The physical white-box engine establishes a stateful evidence library for each tracked target, caching a preset number of frames (e.g., 8 frames) of time-series data. Each update calculates the following three indices: (1) Spectral emissivity contrast C_rad, characterizing the difference in radiation intensity between the target region and the background region, is expressed by the following formula: C_rad = |μ_target - μ_bg| / (μ_bg + ε) Where μ_target is the average ADC sampling value of pixels in the target region, μ_bg is the average ADC sampling value of pixels in the background region, and ε is a minimal constant to prevent division by zero. Metallic targets have low emissivity due to their infrared specular properties, resulting in minimal difference from the background; polymeric camouflage materials, due to their high-emissivity blackbody nature, have high emissivity, resulting in significant difference from the background.

[0034] (2) Thermal conductivity uniformity T_std, which characterizes the uniformity of temperature distribution on the target surface, is expressed by the following formula: T_std = σ(target_adc) Where σ is the standard deviation of the ADC sampling values ​​within the target area. Due to their high thermal conductivity, metals have a uniform surface temperature distribution, resulting in a lower standard deviation. However, polymer materials such as plastics have poor thermal conductivity, leading to localized heat accumulation and a severe thermal gradient, resulting in a significantly higher standard deviation.

[0035] (3) Transient thermal relaxation slope, which characterizes the temperature change trend after the target is removed from the heat source, reflects the specific heat capacity and thermal inertia of the material. The formula is: Slope = (M_0 - M_N) / N Where M_0 is the average ADC value of the target region in the first frame of the time window, M_N is the average ADC value of the target region in the last frame, and N is the number of frames in the time window (e.g., 8 frames). Metals have low thermal inertia and cool down rapidly with a positive slope; polymer materials have high specific heat capacity, and the heat accumulated inside continuously diffuses to the surface, so the surface radiation value increases instead of decreasing with a negative slope.

[0036] Once the evidence database is full for a preset number of frames, the three indicators are compared with their corresponding thresholds and scored normally. The final material determination score is obtained by weighted summation, using the following formula: S = w_std · S_std + w_contrast · S_contrast + w_slope · S_slope Wherein, S_std is the thermal conductivity uniformity score, and w_std is its weighting coefficient (e.g., 0.65); S_contrast is the spectral emissivity contrast score, and w_contrast is its weighting coefficient (e.g., 0.25); S_slope is the transient thermal relaxation slope score, and w_slope is its weighting coefficient (e.g., 0.10). Thermal conductivity uniformity, as the most fundamental physical criterion, is given the highest weight. The final material judgment score S is compared with a preset judgment threshold, and the output is the confidence level of the target as a disguised false target, serving as a physical white-box supplement to the dynamic authenticity recognition results.

[0037] Step S104: Determine hyperspectral features based on the hyperspectral image, determine polarization material features based on the polarization spectrum, determine infrared features based on the infrared image, and determine the fusion authenticity recognition result of the target based on the hyperspectral features, polarization material features, and infrared features.

[0038] The hyperspectral features are extracted from the hyperspectral image after spectral shift correction. They include the spectral reflectance curve features of each pixel position of the target, local spectral statistical features, etc., reflecting the spectral reflectance properties of the target surface material, and are used to distinguish the spectral differences between real metal equipment and camouflage coatings and camouflage fabrics.

[0039] The characteristics of polarized materials are obtained from polarization spectral analysis, mainly the degree of polarization P=(I max -I min ) / (I max +I min ), where I max I min These represent the maximum and minimum light intensities of the polarized light path. The polarization response characteristics of real metal equipment surfaces and the coatings and fabrics of camouflaged decoys inherently differ. Real metal targets have high polarization degrees, while fabric / coating camouflage targets have low polarization degrees. This characteristic characterizes the intrinsic polarization properties of the target surface material and is not easily simulated through surface camouflage techniques.

[0040] Infrared features are extracted from infrared images and reflect the target's thermal radiation and temporal thermal evolution patterns, highlighting the differences between real equipment and passive camouflage decoys in terms of thermal inertia and thermal radiation characteristics. For example, infrared temperature gradient features and emissivity features can be extracted as infrared features.

[0041] Furthermore, weighting coefficients corresponding to hyperspectral features, polarization material features, and infrared features are obtained respectively. These weighting coefficients can be preset or adaptively adjusted based on the imaging signal-to-noise ratio and environmental interference level of the current scene. The hyperspectral features, polarization material features, and infrared features are weighted and fused according to each weighting coefficient to obtain fused features. Weighted fusion can allocate the contribution of different modal features, highlighting effective discriminative information. The fused features are input into a pre-trained classifier, which calculates the confidence level of the output target as a disguised false target. Combined with a preset threshold, the authenticity determination is completed, thus obtaining the fused authenticity recognition result of the target.

[0042] For example, obtaining the weighting coefficients for hyperspectral features, polarization material features, and infrared features may include: Obtain preset initial weighting coefficients for hyperspectral features, polarization material features, and infrared features, as well as current environmental data; Based on current environmental data, determine the environmental signal-to-noise ratio; specifically, obtain the illuminance. L Atmospheric transmittance T stray light intensity S The environmental signal-to-noise ratio is calculated based on the pre-calibrated mapping relationship (obtained through numerous calibration experiments to create a mapping table / fitting function). For example, based on... K env =w L *L+w T *T+w S *S Constructing environmental attenuation factor K env And clamp it to the range of 0-1. w L It is 0.35. w T It is 0.5. w S The value is 0.15; the environmental signal-to-noise ratio is obtained by further mapping: SNR env = SNR min + K env *( SNR max - SNR min );in, SNRmin Limits are set for harsh environments. SNR max Values ​​for ideal observation environments; Based on the environmental signal-to-noise ratio, the preset initial weight coefficients are adjusted to obtain the weight coefficients of hyperspectral features, polarization material features, and infrared features. Among them, the weight coefficient of hyperspectral features is positively correlated with the environmental signal-to-noise ratio, while the weight coefficients of polarization material features and infrared features are negatively correlated with the environmental signal-to-noise ratio.

[0043] This embodiment can increase the contribution ratio of hyperspectral features under good observation conditions with high signal-to-noise ratio, reduce the weight of hyperspectral features under low signal-to-noise ratio conditions caused by sandstorms, dense smoke, and backlighting, and correspondingly increase the weight of polarization material features and infrared features, thereby realizing cross-modal weighted fusion of environmental perception and ensuring the reliability of identification in complex battlefield environments.

[0044] Step S105: Based on the dynamic authenticity recognition results, static authenticity recognition results, and fused authenticity recognition results of the target, determine the final authenticity recognition result of the target.

[0045] In this embodiment, the dynamic authenticity identification result, static authenticity identification result, and fused authenticity identification result of the target are all considered as confidence levels of the target being a fake target. The dynamic authenticity identification result, static authenticity identification result, and fused authenticity identification result are weighted and fused according to preset weights to obtain the final confidence level of the target being a fake target. If the final confidence level is greater than or equal to a preset confidence level threshold, the target is determined to be a fake target; if the final confidence level is less than the confidence level threshold, the target is determined to be a real target. The preset weights can be obtained through pre-training experiments. The confidence level threshold can be set as needed, but it must be greater than 50%.

[0046] In one possible implementation, before determining the final authenticity identification result, multi-target tracking and state management of candidate targets can be included. A six-dimensional Kalman filter is used to estimate the motion state of each target, with the state vector being [cx, cy, w, h, vx, vy], where cx and cy are the target center coordinates, w and h are the target bounding box width and height, and vx and vy are the horizontal and vertical velocities, respectively. The Hungarian algorithm is used to perform a global optimal matching between the detection results and the tracking trajectory based on the intersection-over-union ratio (IoU), and a persistent identity identifier is assigned to each target.

[0047] Each tracked target is associated with an independent evidence library, a temporal buffer queue, and a recognition result cache, enabling stateful tracing. The evidence library caches a preset number of frames of physical indicator temporal data, the temporal buffer queue provides continuous frame input for the deep learning network, and the recognition result cache avoids wasted computational power due to repeated inference. Through multi-target tracking and state management, the evolution of the physical characteristics of the same target can be continuously monitored, and a high-confidence judgment result can be output after accumulating sufficient evidence, while effectively suppressing single-frame false detections and transient interference.

[0048] The multimodal joint authenticity recognition scheme of this invention performs spectral shift correction on hyperspectral images to effectively suppress motion blur and spectral feature shift distortion caused by inter-frame pixel misalignment in high-speed maneuvering camouflage target imaging. It also mines spatial-spectral joint features through a dual-stream recognition network to ensure camouflage target recognition capability under static conditions. Furthermore, it utilizes infrared images to construct inter-frame difference features to obtain dynamic authenticity recognition results, compensating for the inadequacy of hyperspectral modalities in characterizing target motion attributes. It further integrates hyperspectral features, polarization material features, and infrared multi-dimensional features to fully leverage the complementary advantages of different imaging modalities, overcoming the shortcomings of single hyperspectral modalities in complex battlefield environments such as sandstorms, smoke, backlighting, and low visibility at night, including information loss, signal-to-noise ratio degradation, and weak anti-interference capabilities. Finally, it synthesizes static authenticity recognition results, dynamic authenticity recognition results, and multimodal fusion authenticity recognition results to output the final authenticity recognition result, achieving a combination of static and dynamic elements and mutual corroboration of multi-source information. This reduces the probability of false alarms and missed detections, significantly improving the accuracy and robustness of camouflage target authenticity judgment in complex environments.

[0049] In one possible implementation, the spectral shift correction of hyperspectral images can be performed using the Lucas-Kanade optical flow algorithm.

[0050] The Lucas-Kanade optical flow algorithm is designed for temporally continuous hyperspectral image sequences. It selects adjacent reference frames and the current frame, and solves the horizontal optical flow displacement u and vertical optical flow displacement v for each pixel. These displacements reflect the changes in pixel position between frames caused by the high-speed maneuvering of the target.

[0051] Hyperspectral images contain tens to hundreds of spectral bands. When a target moves, the same target material can experience pixel misalignment in images of different bands, directly causing pixels to read spectral values ​​in the wrong band, resulting in spectral shift distortion. Based on the solved horizontal and vertical optical flow displacements of each pixel, a pixel-level remapping and alignment operation is performed on the two-dimensional image of each hyperspectral band. Pixels in each band of the current frame are back-projected and corrected according to the optical flow displacement, compensating for the pixel position shift caused by target motion, so that the moved target pixels are realigned to their corresponding spatial positions in the reference frame. After pixel remapping and alignment, the spectral shift problem caused by inter-frame motion blur is eliminated. The spectral data corresponding to pixels at the same spatial position in the corrected image still represents the true spectral reflectance information of the target material, thus completing spectral shift correction and overcoming the defect of spectral feature distortion caused by the movement of high-speed maneuvering camouflage targets.

[0052] In one possible implementation, target detection employs Oriented Bounding Box (OBB) detection, outputting the coordinates of the target's four vertices and their rotation angles, ensuring the detection box is strictly aligned with the target's actual orientation. Compared to traditional horizontal rectangular boxes, rotating boxes can significantly reduce the proportion of the background area, especially when battlefield targets are tilted at large angles, increasing the effective target ratio from less than 40% to over 85%.

[0053] Based on the four vertices of the rotated bounding box, a precise polygonal mask is generated. Only the pixels inside the polygon are extracted as the target region. Simultaneously, the mask is morphologically dilated and then XORed with the original mask to obtain the background annular region, achieving accurate separation of target pixels from background pixels. This method overcomes the shortcomings of the traditional Otsu thresholding method in terms of poor segmentation results in complex backgrounds or blurred target edges, significantly improving the pixel extraction accuracy of the physics engine and almost eliminating background contamination.

[0054] Furthermore, affine correction is performed on the infrared image based on the target rotation angle. The rotated target is flattened, and then a normalized size region is cropped before being fed into the deep learning network. The affine correction uses a nearest neighbor interpolation algorithm to ensure that the underlying 16-bit absolute temperature value is not tampered with by interpolation. After affine correction, the proportion of effective targets in the input of the deep learning network is significantly improved, and the accuracy of temporal feature extraction is guaranteed.

[0055] In one possible implementation, the infrared images can include mid-wave infrared (MWI) and long-wave infrared (LWI) images, with the LWI image having a wider wavelength range than the MWI image. A reconnaissance payload simultaneously acquires MWI and LWI images with aligned fields of view, and performs pixel-level geometric registration of the two types of images. When calculating inter-frame difference features, the infrared emissivity of each frame can be obtained by inverting the MWI and LWI images respectively. The difference in infrared emissivity between adjacent frames of the MWI and LWI images is calculated, and the average of the emissivity differences between the two bands at the same pixel location is used as the inter-frame difference feature. This inter-frame difference feature is used to output dynamic authenticity recognition results. The infrared features can include the infrared temperature gradient and emissivity features of the MWI and LWI images, as well as the infrared temperature gradient and emissivity features of the LWI images.

[0056] In one possible implementation, considering that existing dual-stream identification networks have a large number of parameters, high computing power overhead, high inference latency and high power consumption when deployed on individual soldier embedded reconnaissance terminals, and cannot meet the hard requirements of rapid battlefield identification and real-time alarm, this embodiment designs a lightweight solution that combines scene-aware dynamic channel pruning and hierarchical knowledge distillation.

[0057] Specifically: This embodiment designs a scene complexity-aware dynamic channel pruning mechanism to adaptively reduce redundant network channels. First, a scene complexity evaluation function is constructed, which quantifies the complexity of the current battlefield scene by calculating the global gradient sum of the spectral spatial image. The scene complexity scoring formula is as follows: ; in, H , W These represent the height and width of the spectral spatial image, respectively. This represents the spatial gradient at the pixel location in the image. The smaller the sum of gradients, the less detail in the image texture and the less edge interference, indicating a simpler battlefield scene and higher redundancy in the hyperspectral channels. Conversely, the larger the sum of gradients, the more complex the scene texture and the richer the interference information, requiring the preservation of complete spectral channel information to ensure recognition accuracy.

[0058] This embodiment presets a scene complexity threshold. When the scene complexity score calculated in real time is lower than the preset threshold, the current scene is determined to be a simple scene, and a network spectral channel pruning strategy is automatically triggered to remove redundant spectral channels with high correlation and low signal-to-noise ratio, thereby reducing the amount of ineffective computation in the network. The channel pruning process follows the constraint of controllable accuracy, ensuring that the accuracy loss is less than... The network parameter count is minimized under constraints. When the scene complexity score calculated in real time is greater than or equal to a preset threshold, the network spectral channel pruning strategy is not triggered, and all channel data is used for identification.

[0059] Meanwhile, this embodiment can employ a hierarchical knowledge distillation training strategy to achieve lightweight iterative optimization of the model. A high-precision, complete dual-stream model is used as the teacher network, and a pruned, lightweight network is used as the student network. The spatial-spectral joint feature discrimination knowledge of the teacher network is transferred through hierarchical distillation to achieve accuracy compensation for the lightweight model. The overall loss function of the lightweight student model is constructed as follows: in, The loss for classifying the true and false targets is used to ensure the basic recognition ability of the student model; Hierarchical knowledge distillation loss is used to fit the feature distribution and discrimination logic of the teacher network; The distillation weight coefficients are used to balance the training proportions of classification loss and distillation loss. This strategy compensates for the slight accuracy loss caused by channel pruning through knowledge transfer, ensuring that the lightweight model's recognition performance approaches that of the original high-precision large model while maximally compressing the model size. This adaptive lightweight inference module is fully compatible with the original dual-stream recognition network architecture, requiring no modification to the hardware structure or imaging acquisition equipment of embedded soldier terminals, and can be directly deployed and adapted.

[0060] To quantify the lightweight inference performance, this embodiment constructs a quantification model for inference time and frame rate improvement. The model's single-frame inference time satisfies the following: in, This refers to the number of effective network parameters after channel pruning. This is the inherent computing power of embedded chips for individual soldiers.

[0061] The frame rate improvement ratio after lightweight processing meets the following requirements: in, The reasoning time for the original large model is relatively long. The inference time for the lightweight model.

[0062] Verified through actual embedded terminal deployment, this module can improve the model recognition frame rate by more than 70%, while significantly reducing the terminal's operating power consumption. It effectively solves the problems of large number of parameters, long inference time, difficulty in embedded terminal deployment, and poor real-time performance of traditional hyperspectral camouflage recognition models, thus meeting the operational needs of individual soldier terminals for real-time reconnaissance and recognition in the field.

[0063] To verify the actual effectiveness of the method of this invention, system integration testing and third-party evaluation were conducted on an embedded edge computing platform (peak computing power of approximately 20 TOPS). The tests used an industrial-grade uncooled long-wave infrared detector with a resolution of 640×512, a frame rate of 30Hz, and output 16-bit raw ADC radiation data. The test samples included six core materials: armor steel, polymer composite coatings, camouflage netting, fabrics, and typical backgrounds, totaling over 2000 time-series samples.

[0064] Material identification intrinsic accuracy test results show that, under isothermal heterogeneous conditions, the present invention achieves a 96.8% identification accuracy for armor steel and a 94.5% identification accuracy for polymer decoys, with an average identification accuracy of 95.1% for the six core materials. In contrast, traditional infrared identification methods, under heat source interference, achieve only 45.2% identification accuracy for armor steel and only 38.5% for polymer decoys. The present invention extracts the target's physical intrinsic characteristics through a pseudo-hyperspectral unmixing network, achieving an endmember similarity of over 0.94, demonstrating extremely high physical interpretability and robustness under strong adversarial conditions.

[0065] System performance and real-time test results show that, after acceleration by the TensorRT inference engine and INT8 quantization, the average end-to-end latency is 22.4ms at 640×512 resolution, with a stable frame rate of 44.6FPS; and the average latency is 18.5ms at 320×256 resolution, with a stable frame rate of 54.1FPS. The model size was compressed from 12.8MB to 3.2MB, achieving a compression rate of 75%, while the target detection accuracy decreased by only 1.7%. System power consumption was reduced from 15W to 4.5W, a 70% reduction, fully meeting the practical deployment requirements of handheld terminals for individual soldiers and micro-UAV platforms.

[0066] All-weather, complex environment adaptability tests show that in an artificially simulated smoke chamber environment (transmittance reduced to 15%-20%), the recognition confidence of this invention remains at 86.5%; in a strong fog environment (visibility limited), the recognition confidence remains at 89.2%; and at the thermal equilibrium point (temperature difference between ambient background and target ΔT ≤ 0.8K), the recognition confidence reaches 94.8%. Traditional infrared recognition methods, however, essentially fail in these environments due to lack of contrast. This invention does not rely on spatial texture but on temporal physical characteristics, maintaining a recognition confidence of over 85% even in smoke and thermal equilibrium environments where traditional infrared systems are blinded by lack of contrast.

[0067] The survivability and deployment cost comparison test results show that, compared with the hardware hyperspectral solution, the present invention achieves quasi-hyperspectral material identification capability by defining the perception path purely in software, with an overall weight increase of 0kg (relying on the existing infrared payload), and a single unit cost reduction of more than 95%. Moreover, it adopts a completely passive detection mode with no optical or electromagnetic signal leakage, which greatly improves the stealth and survivability of the individual soldier reconnaissance platform.

[0068] In summary, this application includes the following main improvements: (1) The accuracy of dynamic scene recognition is significantly improved. By correcting the spectral shift through optical flow motion compensation and capturing transient emissivity features by combining temporal spectral difference, the motion blur problem of high-speed maneuvering camouflaged targets is effectively solved. The target recognition accuracy in dynamic scenes is significantly improved compared with the original algorithm, and it can simultaneously cover two types of battlefield scenes: static positions and high-speed maneuvering.

[0069] (2) Enhanced anti-interference capability in complex environments. Through multi-modal synchronous acquisition of polarization spectrum, mid-wave infrared and long-wave infrared, combined with environmental perception adaptive weighted fusion algorithm, it can automatically switch to high confidence mode for decision-making in harsh environments such as sandstorms, smoke, backlight, and low visibility at night. The recognition stability and environmental adaptability are significantly enhanced, meeting the needs of all-weather operations.

[0070] (3) The real-time performance of embedded deployment is greatly improved. Through a lightweight solution that combines scene-aware dynamic channel pruning and hierarchical knowledge distillation, the number of network parameters is significantly reduced while keeping the accuracy loss to a very small range. The inference frame rate of the embedded soldier terminal is increased by more than 70%, and the power consumption of the device is reduced accordingly, which fully meets the tactical requirements of real-time and rapid identification on the battlefield.

[0071] (4) Strong architectural compatibility and flexible deployment. This solution can be seamlessly integrated with the existing dual-stream identification network. Upgrades can be achieved without overturning the existing technical architecture. At the hardware level, only an integrated optical path acquisition module needs to be added to complete the multimodal upgrade, which is convenient for the iterative transformation of existing equipment.

[0072] (5) Efficient candidate pre-screening with low computational cost. By using a candidate target pre-screening mechanism based on geometric features and radiation intensity as the criteria, more than 90% of environmental clutter and invalid interference are eliminated at extremely low computational cost. The number of candidate targets entering the core recognition engine is reduced from hundreds per frame to less than 10 on average. Peak memory usage is reduced by 69.1%, and system standby power consumption is reduced by 70%, which significantly improves the resource utilization efficiency of embedded terminals.

[0073] (6) Strong physical interpretability and reliable decision-making. A physical white-box engine is constructed by using three thermodynamic intrinsic indicators: spectral emissivity contrast, thermal conductivity uniformity, and transient thermal relaxation slope. It outputs an interpretable physical fingerprint evidence chain, which breaks through the limitation of the uninterpretability of pure deep learning black-box decision-making. Combined with the conflict rejection mechanism, when the physical white-box and deep learning judgments conflict, the confidence level is reset to zero and a verification mark is triggered, which ensures the reliability of multimodal fusion decision-making from the physical level.

[0074] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0075] Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. For example... Figure 2 As shown, the electronic device 2 in this embodiment includes a processor 20 and a memory 21. The memory 21 stores a computer program 22. When the processor 20 executes the computer program 22, it implements the steps in the various method embodiments described above.

[0076] For example, computer program 22 may be divided into one or more modules / units, which are stored in memory 21 and executed by processor 20 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 22 in electronic device 2.

[0077] Electronic device 2 may include, but is not limited to, processor 20 and memory 21. Those skilled in the art will understand that... Figure 2 This is merely an example of electronic device 2 and does not constitute a limitation on electronic device 2. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 2 may also include input / output devices, network access devices, buses, etc.

[0078] The processor 20 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0079] The memory 21 can be an internal storage unit of the electronic device 2, such as a hard disk or RAM. The memory 21 can also be an external storage device of the electronic device 2, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 21 can include both internal and external storage units of the electronic device 2. The memory 21 is used to store the computer program 22 and other programs and data required by the electronic device 2. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0080] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0081] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0082] This invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0083] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0084] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0085] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A target recognition method, characterized in that, include: Acquire hyperspectral, polarization, and infrared images of the target; The hyperspectral image is subjected to spectral shift correction. The corrected hyperspectral image is then input into a preset dual-stream recognition network model to extract spatial-spectral joint features, and the static authenticity recognition result of the target is obtained from the model output. Calculate inter-frame difference features based on the infrared image, and determine the dynamic authenticity recognition result of the target based on the inter-frame difference features; Hyperspectral features are determined based on the hyperspectral image, polarization material features are determined based on the polarization spectrum, infrared features are determined based on the infrared image, and the fusion authenticity recognition result of the target is determined based on the hyperspectral features, the polarization material features, and the infrared features. Based on the dynamic authenticity identification results, static authenticity identification results, and fused authenticity identification results of the target, the final authenticity identification result of the target is determined.

2. The target recognition method according to claim 1, characterized in that, The step of calculating inter-frame difference features based on the infrared image and determining the dynamic authenticity recognition result of the target based on the inter-frame difference features includes: Calculate the difference in infrared emissivity between adjacent frames of infrared images, and determine the inter-frame differential feature based on the difference in infrared emissivity. Based on the magnitude of the difference in infrared emissivity, the confidence level of the target as a fake target is determined, and the dynamic authenticity identification result of the target is obtained; The confidence level is positively correlated with the magnitude of the difference in infrared emissivity.

3. The target recognition method according to claim 1, characterized in that, The determination of the fused authenticity identification result of the target based on the hyperspectral features, the polarization material features, and the infrared features includes: Obtain the weighting coefficients of the hyperspectral features, the polarization material features, and the infrared features; Based on the weighting coefficients, the hyperspectral features, the polarization material features, and the infrared features are weighted and fused to obtain the fused features; The fused features are input into a preset classifier, and the fused authenticity recognition result of the target is obtained based on the confidence level of the target as a fake target output by the classifier.

4. The target recognition method according to claim 3, characterized in that, The step of obtaining the weighting coefficients for the hyperspectral features, the polarization material features, and the infrared features includes: Obtain the preset initial weighting coefficients of the hyperspectral features, the polarization material features, and the infrared features, as well as the current environmental data; Determine the environmental signal-to-noise ratio based on the current environmental data; Based on the environmental signal-to-noise ratio, the preset initial weighting coefficients are adjusted to obtain the weighting coefficients of the hyperspectral feature, the polarization material feature, and the infrared feature; wherein, the weighting coefficient of the hyperspectral feature is positively correlated with the environmental signal-to-noise ratio, and the weighting coefficients of the polarization material feature and the infrared feature are negatively correlated with the environmental signal-to-noise ratio.

5. The target recognition method according to claim 1, characterized in that, The dynamic authenticity recognition result, static authenticity recognition result, and fused authenticity recognition result of the target are all confidence scores of the target being a fake target. The determination of the final authenticity identification result of the target based on the dynamic authenticity identification result, static authenticity identification result, and fused authenticity identification result includes: The dynamic authenticity recognition results, static authenticity recognition results, and fused authenticity recognition results are weighted and fused according to preset weights to obtain the final confidence level of the target as a disguised fake target; If the final confidence level is greater than or equal to the preset confidence level threshold, then the target is determined to be a disguised false target; If the final confidence level is less than the confidence threshold, then the target is determined to be a real target.

6. The target recognition method according to any one of claims 1 to 5, characterized in that, The spectral shift correction of the hyperspectral image includes: The horizontal and vertical optical flow displacements of each pixel in the hyperspectral image are determined using an optical flow algorithm. Based on the horizontal optical flow displacement and the vertical optical flow displacement, the hyperspectral image is remapped and aligned at the pixel level to obtain a hyperspectral image after spectral shift correction.

7. The target recognition method according to any one of claims 1 to 5, characterized in that, The infrared image includes a mid-wave infrared image and a long-wave infrared image, wherein the wavelength range of the long-wave infrared image is greater than that of the mid-wave infrared image. The inter-frame difference feature is the average of the difference in infrared emissivity between adjacent frames of the mid-wave infrared image and the difference in infrared emissivity between adjacent frames of the long-wave infrared image; The infrared features include the infrared temperature gradient features and emissivity features of the mid-wave infrared image, as well as the infrared temperature gradient features and emissivity features of the long-wave infrared image.

8. The target recognition method according to any one of claims 1 to 5, characterized in that, Before inputting the corrected hyperspectral image into a preset dual-stream recognition network model to extract spatial-spectral joint features, the method further includes: The scene complexity score is obtained by calculating the sum of the spatial gradients of the entire hyperspectral image using a preset scene complexity evaluation function. If the scene complexity score is less than a preset score threshold, the spectral channels of the hyperspectral image are cropped to obtain a cropped hyperspectral image. The cropped hyperspectral image is input into the dual-stream recognition network model.

9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images