Multi-modal sensing method based on armored vehicle shooting simulation evaluation

Abstract

The invention discloses a multi-modal sensing method based on armored vehicle shooting simulation evaluation. The method comprises the steps that physical sensing data of a visual image sequence and a vibration signal sequence and digital simulation data of a shooting data parameter sequence are obtained; performing motion amplification processing on the visual image sequence, and performing time-frequency transformation processing on the vibration signal sequence; inputting the processed physical data and digital data into a double-flow neural network model, and outputting binary actual measurement and theoretical feature codes; and calculating the Hamming distance of the feature codes, and when the distance is greater than a threshold value, calculating a physical compensation parameter according to the coding difference by using a decoder network, and writing the physical compensation parameter into a fire control system error compensation unit. Through cross-modal feature fusion and Hash mapping, quantitative comparison of physical world responses and digital world instructions in a unified space is achieved, a correction link from deviation evaluation to parameter inversion is constructed, and the accuracy of shooting simulation evaluation and the intelligent level of error correction of a fire control system are improved.

Description

Technical Field

[0001] This invention relates to the field of shooting simulation evaluation technology, and in particular to a multimodal perception method based on armored vehicle shooting simulation evaluation. Background Technology

[0002] Comprehensive performance evaluation of armored vehicle fire control systems is a crucial step in ensuring the combat effectiveness and accuracy of weaponry. With the rapid evolution of modern optoelectronic detection technology, embedded computing, and digital signal processing algorithms, testing and evaluation methods for live-fire exercises have gradually shifted from traditional target impact point statistics and manual observation to data backtracking and digital simulation analysis based on multi-source sensors. Current mainstream technologies typically rely on external ballistic measurement, high-speed camera systems, and vehicle-mounted bus recorders to acquire projectile flight trajectories, muzzle dynamic images, and fire control calculation logs, respectively. These methods attempt to construct virtual simulation evaluation environments through hardware-in-the-loop (HIL) or digital twin technologies to achieve quantitative analysis of firing accuracy.

[0003] However, existing technologies still have significant limitations in processing multimodal heterogeneous data during live-fire exercises. Traditional evaluation methods often treat physically sensed data (such as mechanical vibration waveforms and muzzle vision images) separately from digital simulation data (such as fire control calculation commands), making it difficult to bridge the "semantic gap" between unstructured visual / signal data and structured bus parameters. This results in an inability to accurately quantify the consistency deviation between the physical execution system and the digital calculation logic. In particular, for the minute deformation of the barrel and high-frequency mechanical vibrations at the moment of firing, conventional simulation monitoring methods lack feature enhancement mechanisms based on motion amplification and time-frequency transformation, making it difficult to capture key transient features. Furthermore, existing systems generally lack feedback mechanisms based on deep hash mapping, making it impossible to calculate the difference between virtual and real data and automatically invert physical compensation parameters within a unified Hamming space, thus limiting the real-time performance and intelligence level of dynamic error correction in actual fire control systems. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a multimodal perception method based on armored vehicle firing simulation evaluation to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a multimodal perception method based on armored vehicle firing simulation evaluation, comprising: The physical perception data and digital simulation data of the armored vehicle during the live-fire period are acquired. The physical perception data includes a sequence of visual images of the muzzle and barrel area and a sequence of vibration signals of the turret mechanical structure. The digital simulation data includes a sequence of firing parameters extracted from the fire control system bus. The visual image sequence is subjected to motion amplification processing based on spatial multi-scale decomposition to generate an enhanced visual image sequence, and the vibration signal sequence is subjected to time-frequency transformation processing to generate a two-dimensional time-frequency distribution matrix. The enhanced visual image sequence and the two-dimensional time-frequency distribution matrix are input into the physical perception branch of the pre-constructed two-stream neural network model, and the binary measured feature code is output. The firing parameter sequence is input into the digital simulation branch of the two-stream neural network model, and the binary theoretical feature code is output. A bitwise XOR operation is performed on the measured feature code and the theoretical feature code. The number of non-zero bits in the operation result is counted to obtain the Hamming distance. When the Hamming distance is greater than a preset threshold, the physical compensation parameters are calculated using a decoder network based on the difference vector between the measured feature code and the theoretical feature code, and the physical compensation parameters are written into the solution unit of the fire control system.

[0007] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, after acquiring the physical perception data and digital simulation data, a data time alignment step is further included, which includes: The moment when the artillery firing is triggered by the control console in the fire control system is the common time zero point; The first peak moment in the vibration signal sequence whose amplitude exceeds the noise threshold is retrieved and marked as the physical reference point; Retrieve the frame moments in the visual image sequence where the average pixel brightness undergoes a step change, and mark them as visual reference points; Calculate the time difference between the physical reference point and the visual reference point relative to the common time zero point, and perform sliding alignment and truncation on the visual image sequence, vibration signal sequence and shooting parameter sequence based on the time difference.

[0008] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the obtained firing parameter sequence includes: Data on azimuth, elevation, propellant temperature, air temperature, crosswind, and turret angular velocity during artillery firing.

[0009] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the step of performing motion amplification processing based on spatial multi-scale decomposition includes: Each frame of the visual image sequence is decomposed using Laplacian pyramids to obtain sub-band signals with different spatial frequencies; Select a sub-band signal corresponding to the inherent frequency range of the tube and perform time-domain bandpass filtering; The filtered subband signal is multiplied by an amplification factor and then superimposed on the original Laplace pyramid decomposition signal. An inverse pyramid transform is performed on the superimposed signals to reconstruct an enhanced visual image sequence containing magnified displacement features.

[0010] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the step of performing time-frequency transformation processing includes: A complex mother wavelet function is selected to perform continuous wavelet transform on the one-dimensional vibration signal sequence; By changing the scale factor and translation factor, the wavelet coefficients of the vibration signal at different time points and frequency scales can be calculated. The modulus of the wavelet coefficients is extracted and mapped into a two-dimensional matrix format, which is used as the two-dimensional time-frequency distribution matrix.

[0011] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the dual-stream neural network model has the following structural configuration: The physical perception branch includes a convolutional neural network layer for extracting fusion features of the enhanced visual image sequence and the two-dimensional time-frequency distribution matrix; The digital simulation branch includes a multilayer perceptron and a recurrent neural network layer, used to extract the temporal features of the firing parameter sequence; Both the physical perception branch and the digital simulation branch are connected to a hash mapping layer at their ends. The hash mapping layer contains a fully connected operation unit and a hyperbolic tangent activation unit, which are used to map features into continuous numerical vectors.

[0012] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the step of outputting the measured feature code and the theoretical feature code in binary format includes: Obtain the continuous numerical vector output by the hash mapping layer; The continuous numerical vector is discretized using a sign function; Values ​​greater than zero are mapped to binary bits 1, and values ​​less than or equal to zero are mapped to binary bits 0, generating a fixed-length binary encoding sequence.

[0013] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the network parameters of the dual-stream neural network model are obtained through training, and the training process includes: Input pairs of sample data, which include measured-simulation data pairs labeled as matched and measured-simulation data pairs labeled as mismatched; Calculate the distance between the feature codes output by the model; The loss function is minimized by the backpropagation algorithm. The loss function constrains the feature encoding distance of the matching measured-simulated data pairs to approach zero, and constrains the feature encoding distance of the mismatched measured-simulated data pairs to be greater than a set marginal value.

[0014] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the step of calculating physical compensation parameters using a decoder network includes: Calculate the difference vector obtained by subtracting the theoretical feature code from the measured feature code; The difference vector is input into a multilayer regression network; The nonlinear mapping of the multi-layer regression network outputs a multi-dimensional vector containing azimuth, elevation, and angular velocity correction values ​​as the physical compensation parameters.

[0015] As a preferred embodiment of the multimodal perception method based on armored vehicle firing simulation evaluation described in this invention, the calculation unit written into the fire control system includes: The calculated physical compensation parameters are converted into floating-point numbers in the format specified by the fire control system communication protocol. Write commands are sent via the vehicle-mounted data bus to update the converted parameters to the calculation unit of the fire control computer.

[0016] Compared with existing technologies, the beneficial effects of this solution are: 1. This invention constructs a dual-stream neural network model and hash mapping to project unstructured visual / signal data of physical perception and structured parameter sequences of digital simulation into Hamming space to generate directly comparable binary feature codes. This solves the problem of fragmented processing of heterogeneous data and the existence of a "semantic gap" in the prior art, and realizes the quantification and evaluation of physical to digital simulation.

[0017] 2. In response to the minute vibrations of the barrel and the transient impacts on the mechanical structure of armored vehicles during firing, this invention employs motion amplification processing based on spatial multi-scale decomposition and time-frequency analysis based on continuous wavelet transform. This visualizes key dynamic information that is difficult to detect in the original data, thereby improving the model's ability to capture the core features of the physical process. This solves the problem of insufficient evaluation accuracy caused by insufficient feature extraction in traditional simulation monitoring methods.

[0018] 3. Furthermore, this invention also constructs a "simulation monitoring and evaluation" to "parameter correction" method. When the Hamming distance, which symbolizes the difference between virtual and real, exceeds a threshold, it can not only identify the deviation, but also use a specially trained decoder network to invert the abstract feature difference vector into compensation parameters (azimuth, elevation, and angular velocity correction values) with clear physical meaning, and output them to the fire control system. This solves the problem that existing technologies cannot automatically generate physical compensation parameters and rely on manual intervention, realizes adaptive learning and dynamic correction of actual fire control system errors, and improves the intelligence level of the fire control system and the firing accuracy of armored vehicles. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the overall process of a multimodal perception method based on armored vehicle firing simulation evaluation according to an embodiment of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0024] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] Example 1 Reference Figure 1 This is the first embodiment of the present invention, which provides a multimodal perception method based on armored vehicle firing simulation evaluation, including: S1. Acquire physical perception data and digital simulation data of armored vehicles during live-fire shooting. The physical perception data includes visual image sequences of the muzzle and barrel area and vibration signal sequences of the turret mechanical structure. The digital simulation data includes firing parameter sequences extracted from the fire control system bus.

[0027] It should be noted that, because live-fire exercises of armored vehicles are a complex dynamic process involving high temperature and pressure, strong impact and vibration, and millisecond-level rapid response, traditional single data sources cannot fully characterize the system state. Therefore, the core of this step is to simultaneously collect measured responses from the physical world and theoretical commands from the digital world through a deployed heterogeneous sensor network, thereby establishing a comprehensive and high-fidelity multimodal dataset for subsequent consistency assessment of virtual and real states.

[0028] Specifically, the multimodal dataset consists of physical sensing data and digital simulation data. The physical sensing data is characterized by visual image sequences and vibration signal sequences, while the digital simulation data is characterized by firing parameter sequences.

[0029] Furthermore, using external high-speed cameras deployed at the test site, continuous filming was conducted on the muzzle brake and barrel area of ​​the armored vehicle. The purpose of continuous filming was to directly capture the "barrel whipping" effect caused by the enormous recoil energy and pressure wave transmission within the barrel at the moment of firing, as well as phenomena such as muzzle flash and gas combustion disturbance. It should be explained that the barrel whipping effect is the elastic vibration phenomenon of the barrel caused by recoil during firing. Its core principle is similar to the whipping of a whip; that is, as energy is transferred from the breech (proximal end) to the muzzle (far end), the vibration amplitude is amplified step by step due to the decrease in mass. However, the minute elastic deformation and high-frequency vibration of the barrel are the key physical factors affecting the initial attitude (i.e., initial disturbance) of the projectile after leaving the barrel, which directly relates to firing accuracy. Based on this, we define the sequence of visual images acquired through continuous filming as a set:

[0030] in, Indicates at time The captured single-frame image, It is a time variable. and These are the start and end times of the data collection cycle, respectively. It is the sampling time interval, and its reciprocal is the camera's frame rate.

[0031] Furthermore, to ensure distortion-free capture of the dynamic process of the tube vibration, the camera's sampling frame rate must satisfy the Nyquist sampling theorem. Considering that the first and higher natural frequencies of the tube are typically distributed in the range of several hundred hertz, in this embodiment, it is preferable that the camera's sampling frame rate be higher than twice the main vibration frequency of the tube, for example, set at 1000 fps or higher, to ensure that subsequent motion amplification processing can effectively recover minute displacements.

[0032] Furthermore, highly sensitive acceleration sensors are installed at key mechanical structural nodes of the turret, such as the trunnions, gun mounts, or stabilizer actuator connections. These sensors are used to record the structural dynamic response of the entire turret system under firing impact loads. Their vibration signals can reflect the energy release and conduction path from ignition to projectile exit, as well as the coupled vibration characteristics between components, from a mechanical transmission perspective, serving as supplementary data to the displacement observed in visual images.

[0033] Specifically, the vibration signal sequence is represented as a one-dimensional time series:

[0034] in, A vibration signal sequence in one-dimensional time series. Represents the moment The measured amplitude of the vibration signal (such as acceleration value) ).

[0035] It should be noted that in order to capture the steep leading edge and rich high-frequency components of the impact signal, the sampling frequency of the sensor needs to be set high enough, usually in the tens of kilohertz (kHz) range (e.g., ≥20 kHz). At the same time, the sensor should have a sufficiently wide dynamic range to avoid signal clipping distortion under simulated strong impact.

[0036] Furthermore, by using a data recorder or bus analyzer to access the fire control system data bus (such as the CAN bus) of the armored vehicle, the theoretical parameters calculated by the fire control computer and sent to each actuator in real time before firing are captured and recorded. These data represent the "ideal" firing parameters calculated by the fire control system based on sensor inputs and ballistic models, which is the "theoretical expectation" of the digital world for this firing.

[0037] Specifically, the acquired sequence of firing parameters includes: The azimuth and elevation command values ​​calculated by the fire control system: These command values ​​are the aiming angles that the artillery servo system needs to execute, and are the theoretical reference for the firing direction.

[0038] Setting propellant temperature and air temperature: This is the propellant temperature, which is a key parameter affecting its combustion rate and chamber pressure. The fire control system will use this temperature to perform theoretical calculations to correct the projectile's initial velocity.

[0039] Crosswind sensor readings: These are the wind speed and direction data measured by the crosswind sensor, which are the main external environmental inputs used by the fire control system to calculate wind deviation corrections.

[0040] Turret angular velocity: The turret rotation speed is measured in real time by an angular velocity sensor under dynamic conditions such as firing while the armored vehicle is moving or rapid firing. It is a key parameter used by the fire control system to calculate and compensate for lead.

[0041] It should be noted that this sequence of firing parameters can be formalized as a set of time-synchronized multidimensional vectors. , where each vector Defined as:

[0042] in, It is the azimuth command value. It is the elevation angle command value. It is the propellant temperature value. This is the reading from the crosswind sensor. It is the rotational speed of the turret.

[0043] Furthermore, to ensure a precise temporal correspondence between physical sensing data and digital simulation data, the acquisition of all data streams must be based on a unified synchronization reference. This invention uses the firing trigger pulse moment of the fire control system as the common time zero point (…). The electrical pulse signal generated by the firing trigger pulse is a clear instruction from the fire control system to activate the firing circuit, marking the logical starting point of the entire firing event. It possesses extremely high instantaneity and determinism. All data acquisition devices (high-speed cameras, vibration signal acquisition sensors, bus recorders, and other sensors) must record the arrival time of this trigger pulse, and subsequent data processing will also use this as a benchmark for time alignment. This ensures that the entire causal chain, from the generation of the theoretical instruction to the occurrence of the physical response, has continuity and comparability in the time dimension.

[0044] Specifically, for visual reference points ( The calibration method involves retrieving the acquired visual image sequence. When an artillery piece fires, the appearance of the muzzle flash causes a sharp change in the average brightness of pixels within the field of view. Therefore, the frame with the largest rate of change in brightness (i.e., the brightness difference between adjacent frames) is identified and marked as the visual reference point. Compared to simple step detection, this method is less sensitive to slow changes in brightness and can more accurately locate the abrupt change in the flash emission. It should be noted that this moment corresponds to the instant when the energy of the gunpowder combustion begins to be released outward.

[0045] Specifically, for physical reference points ( Calibration of the collected vibration signal sequence: Since the mechanical impact generated at the moment of artillery firing is the most intense and sudden, its signal characteristics are the most distinct. Therefore, by setting a noise threshold slightly higher than the background noise, the first peak moment in the signal where the amplitude significantly exceeds the noise threshold and is sustained above it can be marked as the physical reference point. To increase robustness, the time point at which the signal energy first shows a sharp increase can also be found as this physical reference point; for example, by calculating the short-time energy envelope of the signal and locating the maximum value of its first derivative. It should be noted that this peak moment clearly corresponds to the starting moment of the firing mechanical structure response, exhibiting extremely high positioning accuracy.

[0046] Furthermore, considering the inconsistent sampling frequencies of different modal data (e.g., visual sensors sample at video frame rates, while vibration sensors sample high-frequency analog signals), resampling of each modal data is necessary before time alignment. Specifically, this can be achieved by setting a uniform time step, downsampling or extracting frame-by-frame features from high-frequency vibration signals (e.g., calculating the root mean square value of vibration within each frame's time window), and interpolating low-frequency data. This ensures that the modal data subsequently input into the neural network have a corresponding mapping relationship in the time dimension.

[0047] It should be emphasized that, for this noise threshold, the vibration signal can be collected for a period of time in a non-firing state (e.g., only the fire control system power is turned on but no shot is fired), the peak value of the signal can be calculated, and two to three times the peak value can be used as the noise threshold to effectively filter out the interference of environmental and electronic noise.

[0048] Furthermore, the time difference between the physical reference point and the visual reference point relative to the common time zero point was calculated. and Based on these two time differences, the vibration signal sequence and the visual image sequence are shifted along the time axis so that their physical event start points are aligned with the theoretical common zero point. Simultaneously, based on the aligned timestamps, a unified time window is extracted from all three data sequences (visual image sequence, vibration signal sequence, and firing parameter sequence), for example, extracting a data segment from several milliseconds before to several milliseconds after zero.

[0049] S2. Perform motion amplification processing based on spatial multi-scale decomposition on the visual image sequence to generate an enhanced visual image sequence, and perform time-frequency transformation processing on the vibration signal sequence to generate a two-dimensional time-frequency distribution matrix.

[0050] It should be noted that the goal of this step is to enhance and unify the representation of two different modalities of raw physical perception data (i.e., visual image sequences and vibration signal sequences) to address the difficulty of existing simulation monitoring methods in capturing the transient characteristics of minute deformations of the tube and high-frequency mechanical vibrations. By explicitly amplifying and transforming the subtle physical phenomena, the key dynamic information hidden in the original signals is converted into salient features that can be efficiently learned by neural networks.

[0051] Furthermore, regarding visual image sequences, the barrel vibration at the moment of firing (i.e., the "barrel whipping" effect) is a minute and rapid motion, often imperceptible to the naked eye in raw high-speed video footage. If the raw image sequence is directly input into a neural network, the model may not be able to effectively learn this crucial motion feature carried by minute pixel displacements. Therefore, this invention employs a linear motion amplification technique based on spatial multi-scale decomposition to visually enhance this minute motion without introducing significant noise. The processing is as follows: S201.1, Visual image sequence Each frame of visual image in Perform Laplacian pyramid decomposition. It's important to explain that this decomposition process is an overcomplete image representation method that decomposes a visual image into a series of sub-band signals with different spatial frequencies. Its physical meaning lies in simulating the visual system, separating image information according to a "coarse-to-fine" scale. The decomposition process is as follows: Building the Gauss Pyramid ,in, , Yes The Laplacian pyramid is obtained by performing Gaussian blur and downsampling. layer The difference between two adjacent levels of the Gaussian pyramid is calculated as follows:

[0052] in, Indicates the first The sub-band image of the layer contains details and texture information at that spatial frequency scale. This represents an upsampling operation.

[0053] It should be noted that, through step S201.1, the original visual image is decomposed into a set of sub-band signals. And the top of the Pyramid of Gauss .

[0054] S201.2. Select a specific level of sub-band signal for time-domain filtering. This is significant because the vibration of the tube is mainly concentrated at its inherent frequencies, so we focus on pixel changes with specific frequencies in the time dimension. Based on this, for a selected level of the Laplace pyramid... At the same spatial location in all its frames The pixel values ​​constitute a one-dimensional time series. Then, an ideal bandpass filter is applied to the time series, the passband frequency range of which is... The frequency range is set to correspond to the natural frequency range of the barrel. This natural frequency range can be determined in advance through finite element modal analysis (FEM), by consulting weapon design data, or by conducting experimental modal analysis of the barrel in the early stages (such as the hammer impact method). For example, if the first-order natural frequency of the barrel is known to be 200Hz, the filter range can be set between 190Hz and 210Hz. The goal is to accurately extract the intensity variation component caused by the periodic vibration of the barrel from all pixel variations, denoted as... .

[0055] S201.3, The filtered sub-band signal containing only target motion information Multiply by a magnification factor And then superimpose it with the original Laplace pyramid decomposition signal:

[0056] in, This is the enhanced sub-band signal. The amplification factor... The value of this factor needs to strike a balance between clear visibility of motion and avoiding excessive image distortion. It can start with a small value (e.g., 10) and gradually increase, observing the enhanced visual image sequence to select a critical value that makes the tube vibration pattern visually clear without causing excessive distortion of the image background or significant amplification of noise. Alternatively, quantitative indicators, such as calculating the change in signal-to-noise ratio (SNR) or structural similarity index (SSIM) of the image sequences before and after enhancement, can be used to select the amplification factor that most significantly enhances motion while maintaining high image quality. In this embodiment, the value ranges from 10 to 50.

[0057] It should be emphasized that the core of step S201.3 is to amplify the phase changes of the target frequency components. When these amplified phase changes are reconstructed into an image, they will manifest as macroscopic displacement of spatial location.

[0058] S201.4, Using the enhanced Laplace pyramid layer and the top of the original Gaussian pyramid Perform inverse pyramid transform to reconstruct the enhanced image of a single frame. This process is the inverse operation of decomposition; that is, it adds the upsampled lower-level image to the detailed information of the higher-level image layer by layer to finally recover the full-resolution image. By performing the above process on each frame of the visual image sequence, the enhanced visual image sequence can be generated. In this visual image sequence, previously imperceptible tube vibrations become clearly visible, providing high-quality input for feature extraction by the neural network.

[0059] Furthermore, regarding vibration signal sequences, the mechanical vibrations generated by artillery firing are typical non-stationary, transient impact signals, and their frequency components change drastically over time. Traditional Fourier transforms can only obtain the global frequency distribution of the signal and cannot reveal the dynamic characteristics of frequency evolution over time. To solve this problem, the present invention employs a time-frequency analysis method to transform the vibration signal in a one-dimensional time series into a two-dimensional "time-frequency image," which can then be input into the neural network along with a visual image. The processing procedure is as follows: S202.1 Select a complex mother wavelet function (For example, complex Morlet wavelets), for one-dimensional vibration signal sequences Perform a continuous wavelet transform. It's important to explain that a complex mother wavelet is chosen because its transform result contains both amplitude and phase information, providing a more comprehensive description of the signal characteristics. The continuous wavelet transform (CWT) scales and translates the mother wavelet, then performs an inner product operation with the signal to be analyzed. Its mathematical definition is:

[0060] in, In scale and time translation The complex wavelet coefficients at the given location. It is the mother wavelet function . It is a scaling factor, inversely proportional to frequency, used to scale the mother wavelet to match different frequency components. It is a time shift factor used to slide the mother wavelet along the time axis to locate the time when the event occurred.

[0061] S202.2 A series of complex wavelet coefficients are calculated by continuously changing the scale factor and time shift factor within a preset scale range and the entire time domain. It is important to emphasize that this process is equivalent to scanning the entire signal using a "time-frequency analysis window" to obtain the frequency components and their intensities contained in the signal at each time point.

[0062] S202.3 Extract the magnitude (i.e., amplitude) of all calculated complex wavelet coefficients, denoted as... The magnitude of this modulus represents the signal's time shift. and scale The energy intensity at that location. Subsequently, these moduli are mapped into a two-dimensional matrix according to the correspondence between time shift and scale. In this two-dimensional matrix, the row index corresponds to the scale (or the transformed frequency), and the column index corresponds to the time shift (which can be represented as time in this case). The values ​​of the elements in this matrix are denoted as... It should be explained that this two-dimensional matrix is ​​the two-dimensional time-frequency distribution matrix, also known as the wavelet time spectrum or scale map. This matrix visually displays the distribution of vibration signals on the time-frequency plane in the form of an image, successfully transforming one-dimensional non-stationary signals into two-dimensional structured data suitable for image processing.

[0063] It should be noted that the processing steps for the visual image sequence and the vibration signal sequence are parallel. Through this processing step, the two heterogeneous physical perception data (visual image sequence and vibration signal sequence) can be uniformly converted into two image-based and feature-enhanced representations.

[0064] S3. Input the enhanced visual image sequence and the two-dimensional time-frequency distribution matrix into the physical perception branch of the pre-constructed two-stream neural network model, and output the binary measured feature code. Input the shooting parameter sequence into the digital simulation branch of the two-stream neural network model, and output the binary theoretical feature code.

[0065] It should be noted that the task of this step is to construct a two-stream neural network model to process heterogeneous data from the physical world and the digital world respectively, and finally project these heterogeneous data into the same low-dimensional Hamming Space to solve the "semantic gap" problem between different data types. It should be emphasized that in this process, the present invention also uses deep hashing technology to compress the complex system state into efficient and comparable binary codes as the basis for subsequent quantitative evaluation and difference inversion.

[0066] Furthermore, the constructed two-stream neural network model includes two specially designed parallel branches: a physical perception branch and a digital simulation branch.

[0067] It should be explained that the physical perception branch is mainly responsible for processing physical perception data that represents "what actually happened," and its input is the sequence of enhanced visual images generated in step S2. and two-dimensional time-frequency distribution matrix The processing flow is as follows: Since the input to the physics perception branch is all image data, its core is a convolutional neural network (CNN). Specifically, this physics perception branch contains two parallel CNN backbone networks: A 3D-CNN for processing augmented visual image sequences To extract the spatiotemporal characteristics of phenomena such as barrel vibration and muzzle flash.

[0068] A 2D-CNN for processing two-dimensional time-frequency distribution matrices To extract key mode features of vibration and shock energy in the time-frequency plane.

[0069] The high-dimensional features extracted from the two CNN backbone networks are concatenated after undergoing global average pooling or flattening operations to form a unified, high-dimensional fused feature vector. .

[0070] It should be noted that the physical perception branch allows the network to comprehensively learn and represent the physical process of a live-fire exercise involving an armored vehicle from both the macroscopic motion of vision and the microscopic vibration of mechanics. The fused feature vector... It is a highly condensed description of the physical state of the shooting event.

[0071] Accordingly, this digital simulation branch is responsible for processing digital simulation data representing "what should theoretically happen," and its input is the sequence of firing parameters obtained in step S1. Its processing flow is as follows: Since the input is structured time-series data, the structure of this digital simulation branch is designed as a combination of multilayer perceptron (MLP) and recurrent neural network (RNN) layers: Each multidimensional vector Both methods utilize an MLP network to learn the nonlinear combination relationships between different parameters (such as drug temperature, crosswind, and angular velocity) at a given time step. Subsequently, the feature sequence encoded by the MLP network is fed into a recurrent neural network (such as a GRU, where the number of hidden layer nodes can be set to 128 or 256) to capture the temporal characteristics of these parameters' evolution over time. The hidden state of the RNN at the last time step is then used as a summary of the entire time series data, forming a theoretical feature vector. .

[0072] It should be noted that this digital simulation branch allows the model to understand the dynamic evolution of a series of commands and parameters of the fire control system before firing, rather than viewing a single instantaneous value in isolation. Its final output feature vector... It is a deep semantic representation of the "theoretical expectation" of this shot.

[0073] Furthermore, high-dimensional feature vectors are extracted independently from the two parallel branches. and Next, they need to be mapped to a unified Hamming space.

[0074] Specifically, a hash mapping layer is connected to the end of both the physical perception branch and the digital simulation branch. This hash mapping layer, located at the end of these two parallel branches, consists of a fully connected layer and a tanh activation function, compressing high-dimensional features into continuous feature vectors of length 32 bits or 64 bits. Its function is to linearly project high-dimensional feature vectors of arbitrary dimensions onto a preset dimension (i.e., the length of the hash code, for example...). (bits), and then perform nonlinear activation using the tanh activation function. For the physics sensing branch, its output is a continuous numerical vector. The calculation is as follows:

[0075] For the digital simulation branch, its output is a continuous numerical vector. The calculation is as follows:

[0076] in, It is a length of A continuous numerical vector, where the range of each element is [-1, 1]. It is the weight matrix of the fully connected layer. It is the bias vector of the fully connected layer. The function's purpose is to introduce nonlinearity and compress the output value to the [-1,1] interval, thereby stabilizing the model training process and preparing for discretization.

[0077] Furthermore, in order to generate the final binary code, the aforementioned continuous numerical vector needs to be discretized. Specifically, numerical vectors greater than zero are mapped to binary 1s, and numerical vectors less than or equal to zero are mapped to binary 0s, generating a fixed-length binary encoded sequence. This discretization operation follows a threshold judgment rule: based on the above... For example, its measured binary feature encoding Each digit can be determined in the following way: if The If the nth element is greater than zero, then the nth element of the measured feature encoding is... If each element is 1; The If the nth element is less than or equal to zero, then the nth element of the measured feature code is... One element is 0. It is important to emphasize that... and Similarly, its binary theoretical feature encoding This also applies to the judgment rule.

[0078] It is important to emphasize that the parameters of this two-stream neural network model (such as...) And all learnable parameters in CNNs / RNNs are not randomly assigned, but obtained through pre-supervised training. This training process aims to teach the model to "cluster similar elements together," and the training process is as follows: For measured-simulation data pairs labeled as matched (i.e., physical sensing data and digital simulation data from the same successful firing), the model outputs binary codes with very close distances (ideally, a Hamming distance of 0). For data pairs labeled as mismatched (e.g., pairing physical data from firing A with digital data from firing B), the model outputs codes with larger distances (greater than a predetermined margin value). The encoding of ). Specifically, in actual training, these mismatched data pairs can be generated in batches using efficient strategies. For example, within a training batch, for any test-simulation data pair, assuming it is ). Then all other measured data within that batch or simulation data All can be with or This involves constructing mismatched data pairs (i.e., an "in-batch negative sampling" strategy) to efficiently build a large number of negative samples for contrastive learning. To achieve this goal, the present invention employs a contrastive loss function, where the loss for a real-simulation data pair is calculated as follows: Defined as:

[0079] in, It's a tag, used when the data pair doesn't match. When there is a mismatch . Represents two binary codes ( and The Hamming distance between them. This is a preset marginal value, a hyperparameter, that defines the minimum distance that feature codes of mismatched data pairs should maintain in Hamming space. Its purpose is to provide a clear "push-out" target for model learning, preventing all codes from collapsing to similar positions. The value of is usually determined through hyperparameter tuning methods such as grid search on the validation set. For example, in Several candidate values ​​were selected within the range for experiments. During the experiments, one value was chosen that enabled the model to achieve the best discrimination performance (e.g., the highest mean average precision, mAP) on the validation set. The value is used as the initial value for this marginal value.

[0080] The loss is calculated using the backpropagation algorithm. The gradients of the parameters in the two-stream neural network model are used to update the parameters, thereby minimizing the total loss across the entire training dataset. This process forces the two-stream neural network model to learn a mapping relationship where semantically similar (matching) multimodal inputs are also mapped to nearby positions in the Hamming space, while semantically unrelated inputs are pushed away.

[0081] It should be noted that by constructing a two-stream neural network model, it is possible to transform complex observations of the physical world and theoretical instructions of the digital world into measured feature codes that can be directly quantified and compared within the same compact space. and theoretical feature encoding This provides an efficient and meaningful feature representation for subsequent virtual-real consistency assessment.

[0082] S4. Perform a bitwise XOR operation on the measured feature code and the theoretical feature code, count the number of non-zero bits in the operation result to obtain the Hamming distance. When the Hamming distance is greater than the preset threshold, calculate the physical compensation parameters using the decoder network based on the difference vector between the measured feature code and the theoretical feature code, and write the physical compensation parameters into the solution unit of the fire control system.

[0083] It should be noted that the task of this step is to efficiently quantify the consistency between "theory" and "reality" using Hamming distance. When significant deviations are found, a specially trained decoder network is used to invert the abstract feature encoding differences into specific, physically meaningful error compensation values. These compensation values ​​are then written into the fire control system to achieve adaptive correction of future shooting accuracy. This solves the problem that traditional methods cannot automatically invert the difference between virtual and real data into physical compensation parameters, thus limiting the real-time performance of dynamic error correction.

[0084] Furthermore, since in step S3 we have mapped the complex physical measurement process and digital simulation instructions into two separate commands of length [length missing]. The binary encoding, i.e., the measured feature encoding. and theoretical feature encoding Therefore, based on this, the consistency between the two codes is quantitatively measured.

[0085] Specifically, a bitwise XOR operation is used to compare the two binary codes. The Hamming distance is then used. Defined as the number of distinct characters at corresponding positions in two equal-length encoded strings. In fact, in binary (represented by 0 / 1), it is equivalent to performing a bitwise XOR operation on the two codes and counting the number of "1"s in the result. Its mathematical form is:

[0086] in, The calculated Hamming distance is a non-negative integer. and These are two codes in the... The binary value of a bit. This represents the bitwise XOR operator.

[0087] It should be noted that the physical significance of the Hamming distance lies in its quantification of the execution fidelity of theoretical commands in a fire control system. When When the measured feature code is completely consistent with the theoretical feature code, it indicates that the entire chain from digital instruction to physical response is highly consistent, and the system is operating "as expected". The deviation of the physical world from the theoretical expectations of the digital world occurs in certain characteristic dimensions. The larger the distance, the more significant and prevalent this deviation is. The Hamming distance is chosen because its calculation involves only bitwise operations, resulting in extremely high hardware efficiency, making it suitable for embedded or real-time systems requiring rapid decision-making.

[0088] Furthermore, after calculating the Hamming distance, it is necessary to determine whether the deviation needs to be corrected.

[0089] Specifically, the calculated Hamming distance With a preset threshold Compare. If If the deviation between real and simulated fire in this armored vehicle firing is considered to be within an acceptable noise range or the system's normal tolerance, no correction is required, and the judgment process terminates. If the system exhibits a significant bias that requires correction, the compensation parameter calculation process is initiated.

[0090] It should be noted that this preset threshold The value of the threshold needs to be balanced between the system's "sensitivity" and "stability." A threshold that is too low will cause the system to overreact to minor noise, leading to frequent corrections and instability. Conversely, a threshold that is too high may affect early deviations in system accuracy. In this embodiment, the specific value can be determined empirically using the following method: A large number of shooting data pairs known to be "normal" (i.e., shooting accuracy within the allowable range) are collected, and their corresponding Hamming distances are calculated to form a normal sample Hamming distance distribution. Simultaneously, a small number of shooting data pairs known to be "abnormal" (e.g., artificially introduced fixed errors) are collected, and their Hamming distances are calculated. Based on this, the preset threshold can be selected as a value slightly higher than the upper bound of the normal sample Hamming distance distribution (e.g., the mean plus three standard deviations) and significantly lower than the Hamming distance of abnormal samples, to ensure effective differentiation between normal fluctuations and significant deviations, avoiding overreaction to normal system noise.

[0091] Furthermore, once it is determined that a correction is needed, the measured feature codes are calculated. With theoretical feature encoding The difference vector between :

[0092] It's important to explain that because the Hamming distance is a scalar, it only tells us "how large the deviation is," but not "where the deviation is located or what kind of deviation it is." The difference vector... It is a vector that preserves the "directional" information of the deviation. If the encoding is represented by {-1,1}, then... Each element can be {-2, 0, 2}, indicating which abstract feature dimension learned by the two-stream neural network model shows a flip from 1 to -1 or from -1 to 1. This difference vector... This constitutes a "diagnostic fingerprint" of the system's deviation state.

[0093] Furthermore, a decoder network is used to abstract the "diagnostic fingerprint". Translate into specific physical compensation parameters.

[0094] Specifically, in this embodiment, the decoder network is a multilayer regression network, which includes a multilayer perceptron (MLP) with several hidden layers. The input of the multilayer perceptron is... dimensional difference vector The output of this multilayer perceptron is a multidimensional vector containing azimuth correction, elevation correction, and initial velocity correction values. .

[0095]

[0096] in, These are the amounts of compensation required for the fire control system in terms of azimuth, elevation, and projectile initial velocity.

[0097] It should be noted that this decoder network is trained in pairs with the two-stream encoder network in step S3. The training objective of the decoder network is to establish a mapping between feature encoding differences and physical error quantities. Its training dataset is constructed as follows: A series of live-fire exercises involving armored vehicles were conducted under controlled conditions. Some of these exercises served as baseline firings (without additional errors), while others introduced known physical errors in a controlled, quantitative manner. For example, known azimuth deviations (e.g., -0.2 mils) or elevation deviations (e.g., +0.1 mils) were superimposed on the commands sent to the servo system by modifying fire control system parameters, or ammunition with known initial velocity deviations was used. Specifically, a precise, known offset (e.g., in 0.05 mil increments) could be directly superimposed on the digital commands sent from the fire control computer to the servo controller via a dedicated simulation testing software interface, rather than through physical adjustments. Besides using ammunition with known performance deviations, a key parameter used to calculate the initial velocity (e.g., propellant temperature) could be artificially modified in the fire control system's ballistic calculation module, and the corresponding theoretical initial velocity change could be recorded. Simultaneously, high-precision measuring equipment such as external ballistic radar was used to measure the actual initial velocity of the exercise to verify the effectiveness of the error simulation.

[0098] For each time a known error is introduced (e.g., azimuth error), Elevation / slope error Initial velocity error The system fires the data, collects its physical perception data, and pairs it with “ideal” digital simulation data under the same conditions (i.e., theoretical parameters without introducing errors).

[0099] The paired data is input into the pre-trained two-stream neural network model to obtain the measured feature encoding. and theoretical feature encoding And calculate its difference vector. .

[0100] The calculated difference vector is used as the input to the training samples, and the artificially introduced known physical error vector is used as the input. This serves as the label for the sample. By learning from a large number of such sample pairs and optimizing using regression loss functions such as mean squared error (MSE), the decoder network can establish a nonlinear mapping between the bias vector in Hamming space and the error quantity in the physical parameter space. This enables it to understand which combination of physical errors most likely corresponds to a particular type of coding difference.

[0101] Furthermore, the calculated physical compensation parameters are written into the fire control system.

[0102] Specifically, firstly, the floating-point compensation value (i.e., the multi-dimensional vector) output by the decoder network is... The data is converted into a floating-point number format specified by the fire control system communication protocol. This floating-point format includes data type (such as float or double), byte order (big-endian or little-endian), etc., to ensure that the fire control computer can correctly parse it. Then, a write command is sent through the vehicle data bus (such as the CAN bus). This command contains the target device address (fire control computer), the target memory address (system error table area), and the formatted compensation parameter data. When the command is received by the fire control computer, the system error table area in its internal non-volatile memory (such as Flash or EEPROM) is updated.

[0103] It should be noted that writing the compensation parameters into non-volatile memory means that the correction effect is persistent. Therefore, in the next armored vehicle firing mission, the fire control computer will automatically load these compensation values ​​when performing ballistic calculations, pre-correcting the theoretical calculation results. In this way, the system learns and compensates for its own system errors through self-perception and evaluation during one (or more) live-fire exercises, thereby dynamically improving the accuracy of subsequent armored vehicle firing.

[0104] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0108] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0109] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A multimodal perception method based on armored vehicle firing simulation evaluation, characterized in that, include: The physical perception data and digital simulation data of the armored vehicle during the live-fire period are acquired. The physical perception data includes a sequence of visual images of the muzzle and barrel area and a sequence of vibration signals of the turret mechanical structure. The digital simulation data includes a sequence of firing parameters extracted from the fire control system bus. The visual image sequence is subjected to motion amplification processing based on spatial multi-scale decomposition to generate an enhanced visual image sequence, and the vibration signal sequence is subjected to time-frequency transformation processing to generate a two-dimensional time-frequency distribution matrix. The enhanced visual image sequence and the two-dimensional time-frequency distribution matrix are input into the physical perception branch of the pre-constructed two-stream neural network model, and the binary measured feature code is output. The firing parameter sequence is input into the digital simulation branch of the two-stream neural network model, and the binary theoretical feature code is output. A bitwise XOR operation is performed on the measured feature code and the theoretical feature code. The number of non-zero bits in the operation result is counted to obtain the Hamming distance. When the Hamming distance is greater than a preset threshold, the physical compensation parameters are calculated using a decoder network based on the difference vector between the measured feature code and the theoretical feature code, and the physical compensation parameters are written into the solution unit of the fire control system.

2. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, After acquiring the physical sensing data and digital simulation data, a data time alignment step is also included, which includes: The moment when the artillery firing is triggered by the control console in the fire control system is the common time zero point; The first peak moment in the vibration signal sequence whose amplitude exceeds the noise threshold is retrieved and marked as the physical reference point; Retrieve the frame moments in the visual image sequence where the average pixel brightness undergoes a step change, and mark them as visual reference points; Calculate the time difference between the physical reference point and the visual reference point relative to the common time zero point, and perform sliding alignment and truncation on the visual image sequence, vibration signal sequence and shooting parameter sequence based on the time difference.

3. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 2, characterized in that, The obtained firing parameter sequence includes: Data on azimuth, elevation, propellant temperature, air temperature, crosswind, and turret angular velocity during artillery firing.

4. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, The motion amplification processing based on spatial multi-scale decomposition includes: Each frame of the visual image sequence is decomposed using Laplacian pyramids to obtain sub-band signals with different spatial frequencies; Select a sub-band signal corresponding to the inherent frequency range of the tube and perform time-domain bandpass filtering; The filtered subband signal is multiplied by an amplification factor and then superimposed on the original Laplace pyramid decomposition signal. An inverse pyramid transform is performed on the superimposed signals to reconstruct an enhanced visual image sequence containing magnified displacement features.

5. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, The time-frequency transformation process includes: A complex mother wavelet function is selected to perform continuous wavelet transform on the one-dimensional vibration signal sequence; By changing the scale factor and translation factor, the wavelet coefficients of the vibration signal at different time points and frequency scales can be calculated. The modulus of the wavelet coefficients is extracted and mapped into a two-dimensional matrix format, which is used as the two-dimensional time-frequency distribution matrix.

6. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, The two-stream neural network model has the following structure: The physical perception branch includes a convolutional neural network layer for extracting fusion features of the enhanced visual image sequence and the two-dimensional time-frequency distribution matrix; The digital simulation branch includes a multilayer perceptron and a recurrent neural network layer, used to extract the temporal features of the firing parameter sequence; Both the physical perception branch and the digital simulation branch are connected to a hash mapping layer at their ends. The hash mapping layer contains a fully connected operation unit and a hyperbolic tangent activation unit, which are used to map features into continuous numerical vectors.

7. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 6, characterized in that, The steps of outputting the measured feature code and the theoretical feature code of the binary representation include: Obtain the continuous numerical vector output by the hash mapping layer; The continuous numerical vector is discretized using a sign function; Values ​​greater than zero are mapped to binary bits 1, and values ​​less than or equal to zero are mapped to binary bits 0, generating a fixed-length binary encoding sequence.

8. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 6, characterized in that, The network parameters of the dual-stream neural network model are obtained through training, and the training process includes: Input pairs of sample data, which include measured-simulation data pairs labeled as matched and measured-simulation data pairs labeled as mismatched; Calculate the distance between the feature codes output by the model; The loss function is minimized by the backpropagation algorithm. The loss function constrains the feature encoding distance of the matching measured-simulated data pairs to approach zero, and constrains the feature encoding distance of the mismatched measured-simulated data pairs to be greater than a set marginal value.

9. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, The calculation of physical compensation parameters using the decoder network includes: Calculate the difference vector obtained by subtracting the theoretical feature code from the measured feature code; The difference vector is input into a multilayer regression network; The nonlinear mapping of the multi-layer regression network outputs a multi-dimensional vector containing azimuth, elevation, and angular velocity correction values ​​as the physical compensation parameters.

10. The multimodal perception method based on armored vehicle firing simulation evaluation as described in claim 1, characterized in that, The calculation unit written into the fire control system includes: The calculated physical compensation parameters are converted into floating-point numbers in the format specified by the fire control system communication protocol. Write commands are sent via the vehicle-mounted data bus to update the converted parameters to the calculation unit of the fire control computer.

Images