Training method and console training device

Abstract

The application discloses a training method and a console training device, and relates to the technical field of simulation training. The method comprises the following steps: loading a prefabricated training model; loading prefabricated training data, reading binary data frames in a fixed length, analyzing state information, and constructing structured data; continuously collecting operation data of a simulated console; interpolating and padding each physical quantity in the training data to generate a real-time training data set; fusing and calculating the training data and the operation data to generate real-time operation result data; and applying the real-time operation result data to posture updating of a three-dimensional model material to form an operation feedback image. The application solves the problem of feedback lag in the operation training process in the prior art and the problem of disconnection between images and operation, avoids the technical defects of low simulation degree in the training process and disconnection between the training process and the operation result of real equipment, and realizes the improvement of the integration effect of operation behavior and image feedback.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simulation training, in particular to a training method based on three-dimensional model and real-time data fusion and a console training device, which is particularly suitable for basic training and proficiency improvement of operating skills of armored vehicles, fire control weapons and other equipment. BACKGROUND

[0002] The console is a key interactive device of the fire control system of an armored vehicle, which is used to control the gun control subsystem, operate components such as the gun, turret and sighting telescope, and realize functions such as direction control, ranging, aiming and firing of the gun and parallel machine guns. The operation accuracy directly affects the task execution efficiency and combat performance. The existing console training is mostly completed by relying on actual equipment systems, which not only has boring and inefficient training content, but also occupies a large amount of resources, resulting in shortening of the service life of the equipment. At the same time, in the training field, some soldiers are in a state of no task for a long time, causing waste of training time and affecting skill mastery. Therefore, it is urgent to provide a console training device and method with high simulation degree, strong interactivity and the ability to achieve training goals. SUMMARY

[0003] The present application provides a training method and a console training device, which aims to solve the technical problems of loose data coupling, unrealistic posture response and poor real-time performance of the existing training system, and to realize high-precision and low-delay operation simulation training effect.

[0004] In a first aspect, the present application provides a training method for realizing the simulation effect of gunner operation, which comprises the following steps: presetting a sampling frequency according to the training accuracy requirement and the performance index of the target equipment; loading a pre-prepared training model to obtain three-dimensional model materials, wherein the three-dimensional model materials comprise a vehicle body model, a target model and a training scene model, and the vehicle body model internally contains a turret sub-model with updateable azimuth angle and elevation angle; loading pre-prepared training data to obtain a training data set composed of structured data, wherein the structured data contains timestamp, acceleration, yaw angle, pitch angle and roll angle information; continuously collecting data of the operation state of the simulated console based on the preset sampling frequency to obtain real-time operation data, wherein the real-time operation data contains azimuth angle change and elevation angle change; interpolating and supplementing the training data set based on the preset sampling frequency to generate a real-time training data set composed of structured data synchronized with the preset sampling frequency, wherein the structured data contains timestamp, acceleration, yaw angle, pitch angle and roll angle information; fusing the real-time training data set and the real-time operation data based on the preset sampling frequency to generate real-time operation result data, wherein the real-time operation result data contains timestamp, acceleration, yaw angle, pitch angle, roll angle, azimuth angle and elevation angle information; and updating the posture of the three-dimensional model materials with the real-time operation result data based on the preset sampling frequency to form an operation feedback image.

[0005] The second aspect of the present application provides a console training device, which includes: a sampling frequency preset module, which presets the sampling frequency according to the training accuracy requirements and the performance indicators of the target equipment for training; a three-dimensional model material acquisition module, which loads a pre-made training model and parses it to obtain a three-dimensional model material, wherein the three-dimensional model material includes a vehicle model, a target model, and a training scene model, and the vehicle model contains a turret sub-model with updateable direction angles and elevation angles; a training data set acquisition module, which loads pre-made training data to obtain a training data set composed of structured data, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle and roll angle information; a real-time operation data acquisition module, which continuously collects data on the operating status of the simulated console based on the preset sampling frequency to obtain real-time operation data. The module comprises a real-time manipulation data, wherein the real-time manipulation data includes a change in azimuth angle and a change in elevation angle; a real-time training data set acquisition module, wherein the real-time training data set acquisition module interpolates and fills the training data set based on a preset sampling frequency to generate a real-time training data set composed of structured data synchronized with the preset sampling frequency, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle and roll angle information; a real-time operation result data generation module, wherein the real-time operation result data generation module fuses the real-time training data set with the real-time manipulation data based on a preset sampling frequency to generate real-time operation result data, wherein the real-time operation result data includes timestamp, acceleration, yaw angle, pitch angle, roll angle, azimuth angle and elevation angle information; and an operation feedback image formation module, wherein the operation feedback image formation module updates the posture of the three-dimensional model material with the real-time operation result data based on the preset sampling frequency to form an operation feedback image.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] The training method and the console training device provided by the application relate to the field of simulation training, a preset sampling frequency is set according to a training precision requirement and a performance index of a target equipment to be trained; a pre-prepared training model is loaded, and three-dimensional model materials are obtained by analysis, the three-dimensional model materials including a vehicle body model, a target model and a training scene model, the vehicle body model internally including a turret sub-model with an updateable azimuth angle and an elevation angle; pre-prepared training data is loaded, and a training data set composed of structured data is obtained, the structured data including timestamp, acceleration, yaw angle, pitch angle and roll angle information; based on the preset sampling frequency, the operation state of a simulated console is continuously data-acquired, and real-time operation data is obtained, the real-time operation data including an azimuth angle change amount and an elevation angle change amount; based on the preset sampling frequency, the training data set is interpolated and supplemented, and a real-time training data set composed of structured data is generated, the structured data being synchronized with the preset sampling frequency and including timestamp, acceleration, yaw angle, pitch angle and roll angle information; based on the preset sampling frequency, the real-time training data set and the real-time operation data are fused, and real-time operation result data is generated, the real-time operation result data including timestamp, acceleration, yaw angle, pitch angle, roll angle, azimuth angle and elevation angle information; based on the preset sampling frequency, the real-time operation result data is used to update the posture of the three-dimensional model materials, so as to form an operation feedback image, effectively solving the problems of feedback lag in the operation training process and disconnection between the image and the operation in the prior art, effectively simulating the influence of vehicle body movement on the turret posture by introducing a dynamic compensation calculation model of multiple physical quantities, solving the problems of low simulation degree in the training process and disconnection between the training process and the real equipment operation result in the prior art, and realizing the integration of the operation behavior and the image feedback. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A training method flowchart provided by the embodiment of the present application;

[0010] Figure 2 A console training device structure diagram provided by the embodiment of the present application;

[0011] Figure 3 A frame structure diagram of training data in a training method provided by the embodiment of the present application;

[0012] Figure 4A frame structure diagram of training data supporting multi-level data verification in a training method provided by an embodiment of the present application is shown in the figure.

[0013] Reference signs: sampling frequency preset module 11, three-dimensional model material obtaining module 12, training data set obtaining module 13, real-time operation data obtaining module 14, real-time training data set obtaining module 15, real-time operation result data generating module 16, operation feedback image forming module 17. DETAILED DESCRIPTION

[0014] The present application provides a training method and a manipulation table training device, which are used to solve the technical problems of insufficient three-dimensional posture restoration precision, out-of-sync operation data and physical state, large operation feedback delay, and lack of real sense in the traditional training system.

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0016] It should be noted that the terms "first", "second", and the like in the specification and the above drawings of the present application are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product, or server including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices.

[0017] Embodiment one, as shown in the figure, the present application provides a training method, the method comprises: Figure 1

[0018] S000: preset the sampling frequency according to the training precision requirement and the performance index of the target equipment to be trained;

[0019] ​The sampling frequency, also known as the sampling rate, refers to the number of times a continuous signal or state is discretely sampled per unit time, usually measured in Hertz (Hz). The sampling frequency is used to control the time resolution of data acquisition and processing, and its size directly determines the speed of the training process and the response speed and precision of the operation action changes in the training process. In this application, the simulation console is used to simulate the operating equipment of the equipment, and the trainee generates operation control data by moving the simulation console up, down, left and right. The sampling frequency determines the response speed and operation precision of the system to the trainee's operation.

[0020] Specifically, a too low sampling frequency may not be able to capture rapid manipulation changes in the training process in time, causing posture restoration delay or motion process distortion, affecting the authenticity and immersion of the training. Although a higher sampling frequency can improve the precision of the posture data, it will significantly increase the computational load of data processing and rendering, which may cause processor overload, rendering frame drop, and reduce the stability of the training.

[0021] In this embodiment, the determination method of the preset sampling frequency is as follows:

[0022] First, determine the accuracy requirements of the training task, including the restoration accuracy of the posture change, the time delay constraint and the smoothness index of the training process. For example, if it is required to accurately reflect the state changes such as vehicle turning, acceleration and jolting within 0.1 seconds, the minimum time resolution of the training system should be ensured to be not less than 100Hz.

[0023] Secondly, confirm the performance indicators of the target equipment, including the refresh rate of the simulation console input device, the maximum frame rate of the three-dimensional model rendering engine, the maximum throughput rate of the data processing module, and the response speed of the sensor or interface module. The ideal sampling frequency setting should satisfy: the maximum frame rate of the rendering engine <= the sampling frequency <= the refresh rate of the simulation console input device <= the maximum throughput rate of the data processing module. For example, if the control signal refresh rate of the simulation console is 120Hz, the maximum frame rate of the rendering engine is 60fps, and the maximum throughput rate of the data processing module is 200Hz, the sampling frequency can be preferably set to 100Hz after considering the delay and system load.

[0024] S100: load the pre-prepared training model, and parse to obtain three-dimensional model materials, including a vehicle model, a target model, and a training scene model, wherein the vehicle model internally contains a turret sub-model with an updateable azimuth angle and an elevation angle.

[0025] Specifically, a prefabricated training model is loaded into the file system. The prefabricated training model is stored in a standard format (such as .fbx, .dae, etc.) and uses a right-handed coordinate system for modeling and spatial definition, where the Y axis is the vertical direction and X / Y / Z points to the right / up / forward, respectively. The 3D model file includes but is not limited to the vehicle model, target model, and training scene model.

[0026] in:

[0027] The vehicle body model is the core object of the training system. The vehicle body model contains a turret sub-model. The turret sub-model has multiple preset control binding points, such as the turret rotation axis and the barrel pitch axis, and supports skeletal animation or joint control. The turret sub-model has a posture control unit that can be updated independently of the vehicle body model, including: an azimuth control module, which can update the horizontal rotation angle of the turret according to the given azimuth data to achieve lateral aiming; an elevation control module, which can update the vertical rotation angle of the turret according to the given pitch angle data to achieve vertical aiming.

[0028] The target model is the tracking target object in the training task. It can be bound to a path animation controller to achieve the predetermined movement of the target in the training scene.

[0029] The training scene model is a three-dimensional training environment in the training task, which is used to carry the vehicle model and the target model, and provide the spatial reference and visual feedback required for simulation.

[0030] S200: Loading prefabricated training data to obtain a training data set consisting of structured data, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle, and roll angle information;

[0031] The prefabricated training data is a file consisting of a set of motion parameter data corresponding to the three-dimensional model. It is composed of multiple fixed-length binary data frames arranged in chronological order. The prefabricated training data only stores the key frames in the training process, which determine the main nodes of the vehicle model's movement changes during the training process. Each data frame is used to describe the state information of the vehicle model at a certain moment.

[0032] Specifically, if Figure 3 As shown in the figure, the frame structure of the training data is: the timestamp is a 3-byte unsigned integer, the acceleration is a 4-byte single-precision floating-point number, the yaw angle is a 4-byte single-precision floating-point number, the pitch angle is a 4-byte single-precision floating-point number, and the roll angle is a 4-byte single-precision floating-point number.

[0033] The following is a detailed description of each field.

[0034] Timestamp: used to indicate the number of milliseconds from the start time of training (0) to the time point corresponding to the current data frame based on the preset sampling frequency;

[0035] Acceleration: used to represent the instantaneous acceleration of the vehicle body model along the trajectory in three-dimensional space at this time point, unit: meter per square second

[0036] Yaw angle: used to represent the angle of rotation of the vehicle body model around the Y axis in the global coordinate system of the three-dimensional scene, using the right-hand coordinate system convention, the Y axis is vertically upward, reflecting the change of the vehicle body in the horizontal plane, unit: angle;

[0037] Pitch angle: used to represent the angle of rotation of the vehicle body model around the X axis in the global coordinate system, using the right-hand coordinate system convention, the X axis is horizontally to the right, reflecting the change of the vehicle body in the front and back direction, unit: angle;

[0038] Roll angle: represents the angle of rotation of the vehicle body model around the Z axis in the global coordinate system, using the right-hand coordinate system convention, the Z axis is horizontally forward, reflecting the change of the vehicle body in the left and right direction, unit: angle;

[0039] Each of the above attitude angles is referenced to the global three-dimensional world coordinate system, ensuring that the attitude state at each time is calculated and expressed under the unified spatial reference, thereby supporting the subsequent accurate restoration in the three-dimensional environment.

[0040] In actual operation, the system first reads the binary data frames from the training data one by one according to a fixed frame length, and performs field parsing on them, extracts each state parameter and encapsulates it as a structured data object. Subsequently, using the timestamp as the key, the structured data object is written into an ordered set in memory (such as a time series container based on a red-black tree or a skip list), forming a training data set arranged in chronological order, for subsequent processing.

[0041] In order to improve the data processing efficiency of the system, this process supports two modes: one is the full preloading mode, which loads all the training data into memory at one time during system initialization; the second is the incremental reading mode based on the buffer, which sets the buffer size according to the hardware resource configuration, and gradually parses and loads the data frames with the system clock advancing, to ensure the real-time and stability of the system under resource limited conditions.

[0042] S300: Based on the preset sampling frequency, the operation state of the simulation console is continuously data-acquired to obtain real-time operation data, which includes the direction angle change and the height angle change;

[0043] In order to realize real-time capture and delay characteristic simulation of the operation behavior, the system continuously acquires data based on the preset sampling frequency to generate real-time operation data.

[0044] ​Specifically, the system collects the change amount of operation instructions from the analog console within a sampling period based on a preset sampling frequency, wherein the higher the sampling frequency, the finer the granularity of the operation data obtained, and the more accurately the dynamic control characteristics of the operator can be reflected. By adjusting the sampling frequency parameter, the delay characteristics of different equipment in operation response can be simulated, thereby improving the adaptation capability.

[0045] In each sampling period, the system collects the operation change amount at the current time from the console, specifically including:

[0046] The direction angle change amount (unit: degree) represents the rotation angle of the analog console around the horizontal rotation axis (Y axis) within the current sampling period, corresponding to the change of the turret orientation;

[0047] The high-low angle change amount (unit: degree) represents the rotation angle of the analog console around the pitch rotation axis (X axis) within the current sampling period, corresponding to the change of the barrel pitch state;

[0048] The collected real-time operation data is saved in real time to a designated storage area in the memory, which can be pre-allocated or dynamically applied, for subsequent data processing, attitude compensation calculation and three-dimensional visualization update process. To ensure data consistency and sampling real-time, the system can cooperate with a ring buffer, queue cache or other memory structure to temporarily store and update the data.

[0049] S400: based on the preset sampling frequency, the training data set is interpolated and filled, and a real-time training data set composed of structured data is generated, which is synchronized with the preset sampling frequency, and the structured data includes timestamp, acceleration, yaw angle, pitch angle and roll angle information;

[0050] To realize strict alignment and real-time synchronization processing of the training data and the operation data generated by the analog console on the time axis, after the training data set is constructed, the sampling frequency alignment processing of the training data is performed, and a real-time training data set consistent with the operation data sampling frequency is generated, to ensure that each frame of control instruction corresponds to a frame of physical state data, facilitating unified execution of data fusion, attitude compensation and control operation;

[0051] Specifically, the system first calculates the sampling period (unit: ms) according to the set sampling frequency (unit: Hz), and generates an equidistant discrete sampling timestamp sequence based on the start time of the training data (timestamp 0):

[0052]

[0053] The timestamp sequence constitutes a unified time index for resampling and filling the training data;

[0054] At each sampling time point , the system performs time alignment retrieval in the original training data set, finds the two closest data points in time, the previous data point (data is ) and the next data point (data is ), which satisfies the condition , if there is an exact match (i.e. or ), the original structured data of the point is directly used as the corresponding training data; if it is not an exact match, the system performs linear interpolation calculation on each physical quantity (such as acceleration, yaw angle, pitch angle, roll angle) in the structured data that needs to be synchronized:

[0055]

[0056] That is, the interpolation estimated state value corresponding to the sampling time, the interpolation estimated state value of each physical quantity is used to construct interpolation structured data, and the interpolation structured data is as the timestamp;

[0057] For the case of insufficient training data, i.e. , a zero value filling strategy (representing the vehicle body at rest or uniform motion) is used to generate a filling data with all physical quantities set to zero as the training data corresponding to the sampling time point, where each field is set to: timestamp= , acceleration = 0, yaw angle = 0, pitch angle = 0, roll angle = 0.

[0058] Insert the generated interpolation data or filling data into the new ordered set in the order of sampling time to form a real-time training data set that is consistent with the real-time operation data sampling frequency and corresponds to each point on the time axis.

[0059] In order to improve the data processing efficiency of the system, this process supports two modes: one is the full pre-processing mode, which processes all training data at one time in the system initialization stage; the second is the incremental reading mode based on the buffer, which sets the buffer size according to the hardware resource configuration and processes step by step with the system clock to ensure the real-time performance and stability of the system under resource limited conditions.

[0060] S500: Based on the preset sampling frequency, the real-time training data set is fused with the real-time operation data to generate real-time operation result data, which contains timestamp, acceleration, yaw angle, pitch angle, roll angle, direction angle and height angle information;

[0061] At the beginning of training, the system clock is set based on the preset sampling frequency so that the overall system process advancement frequency is consistent with the control data collection frequency, ensuring that each data processing unit of the system runs synchronously under a unified time base, and initializing the global operation results to: timestamp = 0, acceleration = 0, yaw angle = 0, pitch angle = 0, roll angle = 0, heading angle = 0, and elevation angle = 0.

[0062] As the system clock continues to advance, the system uses the current timestamp as the keyword to extract structured data corresponding to the current moment from the real-time training data set. The structured data is the real-time training data. The real-time training data and the real-time control data are used to perform data fusion update on the global operation results. Specifically: the timestamp is updated to the current timestamp, the acceleration is updated to the acceleration in the real-time training data, the yaw angle is updated to the yaw angle in the real-time training data, the pitch angle is updated to the pitch angle in the real-time training data, the roll angle is updated to the roll angle in the real-time training data, the direction angle is updated to the current direction angle + the direction angle change in the real-time control data, the altitude angle is updated to the current altitude angle + the altitude angle change in the real-time control data, and the global operation result after the fusion update is completed is the real-time operation result.

[0063] S600: updating the posture of the three-dimensional model material using the real-time operation result data based on a preset sampling frequency to form an operation feedback image;

[0064] The real-time operation result data generated in step S500 is used to update the posture of the vehicle model loaded in step S100 and the turret sub-model contained in the vehicle model. After the posture update is completed, the system triggers the graphics rendering module to re-render the three-dimensional environment including the vehicle model, turret sub-model, target model and training scene model, and outputs the final rendering result to the simulation observation and aiming device (such as an LCD screen, projection device or VR headset) to achieve real-time image feedback of the operation results.

[0065] Furthermore, step S200 in the embodiment of the present application further includes:

[0066] S210: In the process of constructing structured data, multi-level verification is performed on the binary data frame of the training data to obtain data frame information that fails the verification. The multi-level verification includes frame header verification, field boundary verification, check bit comparison, and inter-frame consistency verification.

[0067] It should be understood that, in order to solve the structural anomaly problem caused by data corruption, frame misplacement or format deviation in the training data, the embodiment of the present application introduces a multi-level data checking mechanism. The mechanism performs multi-level checking and anomaly identification on the binary structure of the original training data in the data loading stage. Through field parsing rules, frame boundary alignment detection and redundant data segment consistency comparison, the system's ability to identify potential structural errors is enhanced, and the robustness and error recovery capability of the data parsing process are improved. For this purpose, as shown in Figure 4

[0068] Specifically, the processing process of multi-level data checking is as follows:

[0069] Frame header checking: the first level of checking, to determine whether it starts with the correct frame header identifier, if not, skip the frame and find the next valid frame;

[0070] Field boundary checking: the second level of checking, to confirm whether the byte length and offset position of each field are aligned to prevent misinterpretation caused by field misplacement;

[0071] Check bit comparison: the third level of checking, using the CRC32 checking algorithm to calculate the CRC check value of the data in the current data frame excluding the check bit and comparing it with the frame tail check field. If they are not consistent, it is marked as a damaged frame;

[0072] Inter-frame consistency checking: the fourth level of checking, to compare whether the time stamp interval of adjacent frames exceeds the preset fluctuation range (such as 1000ms±10%) to detect time jump.

[0073] For each data frame, multi-level data checking is performed, and the data range information of the failed checking is saved to the exception buffer area of the stack structure in the memory. The data structure of the checking failed data range information is as follows:

[0074] Serial number: 2-byte integer,

[0075] Start byte position: 4-byte integer,

[0076] End byte position: 4-byte integer.

[0077] Further, the step S210 of the embodiment of the present application further includes:

[0078] ​S210-1: data fitting reconstruction is performed on the data frame with failed verification based on the data of the two normal data frames before and after the data frame.

[0079] Specifically, whether there is data frame damage is confirmed by checking the abnormal cache area, when detecting data frame damage, data fitting reconstruction is performed based on the data of the two normal data frames before and after the failed verification data range using a linear interpolation method, and the fitting reconstruction process is as follows:

[0080] On the time sequence, the previous frame and the next frame of the failed verification data range are located, the state values such as acceleration, yaw angle, pitch angle and roll angle are extracted in the previous frame and the next frame of the failed verification data range, the fitting reconstructed state values are constructed based on the above state values by a linear interpolation method, and the corresponding state values in the abnormal data frame are replaced by the fitting reconstructed state values, when the time stamp difference in the previous frame and the next frame of the failed verification data range is greater than a preset threshold (for example: 2000) or the abnormal data frame is the first frame or the last frame, the data fitting reconstruction is not performed, and the current training process is terminated.

[0081] Further, the step S400 of the embodiment of the present application further includes:

[0082] S410: loading a posture change template to obtain posture change template data, the posture change template is pre-established, the data structure is the same as the pre-prepared training data, and is a template file for describing motion posture changes; S420: segmenting and analyzing the real-time training data set using a sliding window method to obtain a motion posture feature vector sequence, the motion posture feature vector sequence is a vector sequence obtained by using a motion posture feature extraction method based on the training data in the sliding window; S430: segmenting and analyzing the posture change template data using a sliding window method to obtain a motion posture feature vector sequence; S440: performing similarity scoring on the motion posture feature vector sequence of the training data set and the motion posture feature vector sequence of the posture change template, and obtaining the data with motion posture change abnormalities in the real-time training data set through scoring.

[0083] It should be understood that, in order to solve the "non-physical" posture jump problem caused by physical modeling errors or simulation instability in the training data, the step S400 of the embodiment of the present application further includes an abnormality detection and optimization mechanism based on the motion posture change template, which detects and optimizes the real-time training data set generated after interpolation and synchronization with the sampling frequency, improves the authenticity and stability of three-dimensional motion restoration, and specifically,

[0084] A plurality of posture change templates conforming to vehicle dynamics constraints are pre-established, including but not limited to:

[0085] Smooth running template: smooth acceleration change, slight yaw angle swing,

[0086] Linear acceleration template: acceleration increases continuously but pitch angle changes slowly.

[0087] Standard turning template: yaw angle changes smoothly, accompanied by slight swing of roll angle,

[0088] Deceleration braking template: acceleration jumps in the opposite direction, pitch angle sinks,

[0089] Bump response template: Pitch and roll angles fluctuate rapidly and alternately but with limited amplitude;

[0090] Template data can be obtained through simulation or real vehicle collection, and the data structure is the same as the real-time training data set.

[0091] A sliding window method is used, with a fixed window length (e.g., 200 frames), a fixed sliding step size (e.g., 100 frames), and overlapping sliding windows, to perform segmented analysis on the real-time training dataset. Specifically, the sliding window is gradually advanced, and the motion posture feature extraction method is used to extract motion posture feature vectors from the training data within the sliding window. Finally, the motion posture feature vectors of all windows are obtained, and the motion posture feature vectors of all windows are organized into a motion posture feature vector sequence in window order. This motion posture feature vector sequence is the feature vector sequence of the real-time training dataset.

[0092] A sliding window method is used, with a fixed window length (e.g., 200 frames), a fixed sliding step size (e.g., 100 frames), and an overlapping sliding window to perform segmented analysis on each posture change template. Specifically, the sliding window is gradually advanced, and the motion posture feature extraction method is used to extract the motion posture feature vector from the posture change template data in the sliding window. Finally, the motion posture feature vectors of all windows are obtained, and the motion posture feature vectors of all windows are organized into a motion posture feature vector sequence in window order. The above process is repeated for multiple posture change templates, and the motion posture feature vector sequences of multiple posture change templates are put into an array to obtain an array composed of the motion posture feature vector sequences of multiple posture change templates.

[0093] Traverse the feature vector sequence of the real-time training data set to obtain the feature vector ,in Represents the element subscript in the feature vector sequence of the real-time training dataset. Perform the following nested matching process: traverse the array composed of the motion posture feature vector sequences of multiple posture change templates to obtain the feature vector of the posture change template ,in Indicates the element subscript in the array. Indicates The element subscripts in the corresponding eigenvector sequence, for each set of eigenvector pairs using cosine similarity or dynamic time warping (DTW) algorithm for vector similarity scoring to obtain a score score Among all the obtained score scores, the maximum score score is extracted, and if the maximum score score is lower than a pre-set similarity threshold (for example: 0.4), the first data window of the training data set is determined as an abnormal behavior window, the training data in the abnormal behavior window is saved into an array, and the data with abnormal motion posture change in the real-time training data set is obtained.

[0094] It should be understood that the length of the sliding window and the sliding step should be set with reference to the pre-set sampling frequency, preferably, the length of the sliding window should be set to twice the sampling frequency, and the sliding step should be set to half of the length of the sliding window (for example: if the pre-set sampling frequency is 100 Hz, the length of the sliding window is preferably set to 200 frames, and the sliding step is preferably set to 100 frames).

[0095] Further, the step S440 of the embodiment of the present application further comprises:

[0096] S440-1: traversing the data with abnormal motion posture change in the real-time training data set, and calculating the continuous interval of the data with abnormal motion posture change; S440-2: sequentially processing each continuous interval, and if the number of data in the current continuous interval is less than or equal to a pre-set threshold, compensating and correcting the data in the current continuous interval by an abnormal behavior compensation method.

[0097] Specifically, the data with abnormal motion posture change in the real-time training data set is traversed, whether the adjacent data is continuous is determined according to whether the timestamp difference value of the adjacent data is equal to the sampling period value corresponding to the pre-set sampling frequency, a plurality of continuous data is allocated to the same data interval, the data interval is an array structure, and finally a plurality of arrays corresponding to a plurality of continuous data intervals are obtained, each array contains a plurality of timestamp-continuous abnormal data.

[0098] It should be understood that since the overlapping sliding window is used to determine the abnormal motion posture change, the identified abnormal data is repetitive, and after the above-mentioned continuous data interval identification is completed, each array containing a plurality of timestamp-continuous abnormal data needs to be de-duplicated, the elements with repeated timestamps in the array are removed, and only unique values are retained.

[0099] Traversing the plurality of arrays corresponding to the plurality of continuous data intervals, if the number of abnormal data in the array structure is less than or equal to a pre-set threshold (for example: 200), the data in the current continuous interval is compensated and corrected by an abnormal behavior compensation method, otherwise, it is not processed.

[0100] ​It should be understood that since the sliding window method is used to segment data when determining abnormal changes in motion posture, the identified abnormal data are all continuous data based on the size of the sliding window used. Therefore, the preset threshold should be adjusted as appropriate with reference to the size of the sliding window. Preferably, the preset threshold is set to the same as the sliding window size.

[0101] Furthermore, step S440-1 of the embodiment of the present application also includes: S440-1-1: using the motion posture feature extraction method to obtain the motion posture feature vector sequence corresponding to the data in the current continuous interval; S440-1-2: using the motion posture feature extraction method to obtain the motion posture feature vector sequence corresponding to all posture change template data; S440-1-3: performing similarity scoring on the motion posture feature vector sequence corresponding to the data in the current continuous interval and the motion posture feature vector sequence corresponding to all posture change template data, and obtaining the posture change template closest to the data in the current continuous interval through scoring; S440-1-4: based on the closest posture change template and the data in the current continuous interval, using the third-order Bezier curve fitting method to compensate and correct the data in the current continuous interval.

[0102] Specifically,

[0103] First, the multiple timestamp continuous abnormal data obtained in step S440-1 are extracted using the motion posture feature extraction method to obtain the corresponding feature vectors. , Secondly, obtain the data frame sequence of each posture change template in turn and use the motion posture feature extraction method to extract the corresponding feature vector , use cosine similarity or dynamic time warping (DTW) algorithm to and Perform similarity calculation, the one with the highest similarity score The corresponding attitude change template is the closest attitude template. Finally, the third-order Bezier curve fitting algorithm is used to fit and repair the direction angle, pitch angle, and roll angle in each data frame of the multiple timestamp continuous abnormal data obtained in step S440-1 with the closest attitude template as a reference.

[0104] Furthermore, the motion posture feature extraction method in the embodiment of the present application is:

[0105] For continuous multi-frame attitude data, calculate the average yaw angle change rate: ,in, is the change in yaw angle between two adjacent frames, in degrees. is the time interval between two adjacent frames, in seconds, and the rate of change of each adjacent frame is obtained. , for the rate of change Take the average value;

[0106] The Pearson correlation coefficient of acceleration and yaw angle is calculated for continuous multi-frame attitude data: wherein, is the acceleration sequence, unit (g), is the sequence of yaw angle change over time, unit (degree), is the covariance of and , indicating the degree of joint variation, is the standard deviation of the acceleration sequence, indicating the fluctuation degree, is the standard deviation of the yaw angle sequence, and the correlation coefficient ranges from ;

[0107] The Pearson correlation coefficient of acceleration and pitch angle is calculated for continuous multi-frame attitude data: wherein, is the acceleration sequence, unit (g), is the sequence of pitch angle change over time, unit (degree), is the covariance of and , indicating the degree of joint variation, is the standard deviation of the acceleration sequence, indicating the fluctuation degree, is the standard deviation of the pitch angle sequence, and the correlation coefficient ranges from ;

[0108] The Pearson correlation coefficient of acceleration and roll angle is calculated for continuous multi-frame attitude data: wherein, is the acceleration sequence, unit (g), is the sequence of roll angle change over time, unit (degree), is the covariance of and , indicating the degree of joint variation, is the standard deviation of the acceleration sequence, indicating the fluctuation degree, is the standard deviation of the roll angle sequence, and the correlation coefficient ranges from ;

[0109] The standard deviation of the pitch angle is calculated for continuous multi-frame attitude data, indicating the amplitude of the up / down head / sink fluctuation;

[0110] The maximum absolute value of the roll angle is calculated for continuous multi-frame attitude data, reflecting the degree of vehicle inclination;

[0111] ​​​The maximum yaw angle change rate is calculated for continuous multi-frame attitude data, representing the turning or rotation mutation degree;

[0112] The average yaw angle change rate, the Pearson correlation coefficient of acceleration and yaw angle, the Pearson correlation coefficient of acceleration and pitch angle, the Pearson correlation coefficient of acceleration and roll angle, the standard deviation of pitch angle, the maximum absolute value of roll angle, and the maximum value of yaw angle change rate are combined in a preset order to form the attitude change feature vector.

[0113] Further, the step S500 of the embodiment of the application further includes:

[0114] S510: using an inertial coupling calculation method to dynamically compensate the direction angle and the height angle in the real-time operation result data.

[0115] It should be understood that, in order to improve the accuracy and response of the training, effectively improve the training accuracy and system response efficiency, in the embodiment of the application, a real-time motion state model containing acceleration, yaw angle, pitch angle, roll angle and the like is constructed, and inertial coupling calculation is combined to dynamically compensate the direction angle and the height angle in the operation result data, the influence of the vehicle body dynamics on the turret attitude is simulated, and specifically,

[0116] Direction angle compensation calculation:

[0117]

[0118] Wherein:

[0119] is the current system clock,

[0120] is the direction angle of the turret that needs to be compensated (in the vehicle body coordinate system),

[0121] is the change amount of the vehicle body yaw angle at the moment relative to the moment,

[0122] is the inertial coupling adjustment coefficient, which is adjusted according to the actual situation, and different setting values can reflect the inertial characteristics of different equipment in the direction angle,

[0123] is the vehicle body instantaneous acceleration at the moment,

[0124] is the included angle between the vehicle body yaw angle and the turret pointing direction at the moment;

[0125] High-low angle compensation calculation:

[0126]

[0127] Wherein:

[0128] is the angle of the turret that needs to be compensated (in the vehicle body coordinate system),

[0129] is the change of the vehicle body pitch angle at time t relative to time t-1,

[0130] is the change of the vehicle body roll angle at time t relative to time t-1,

[0131] is the yaw angle of the turret at time t-1 (in the vehicle body coordinate system, the direction angle relative to the center axis of the vehicle body),

[0132] is the inertia coupling adjustment coefficient, which is adjusted according to the actual situation, and different set values can represent the inertia characteristics of different equipment in the high-low angle,

[0133] is the instantaneous acceleration of the vehicle body at time t,

[0134] is the vehicle body pitch angle at time t-1;

[0135] The direction angle result in the real-time operation result data obtained in step S500 is added by , the high-low angle result in the real-time operation result data obtained in step S500 is added by , and the dynamic compensation of the direction angle and the high-low angle in the operation result data is completed.

[0136] In the second embodiment, based on the same inventive concept as the training method in the foregoing embodiments, as shown in Figure 2 , the application provides a console training device, and the device and method embodiments in the application are based on the same inventive concept. Wherein, the device comprises:

[0137] A sampling frequency preset module 11, the sampling frequency preset module 11 is preset according to the training accuracy requirement and the performance index of the target equipment.

[0138] The three-dimensional model material obtaining module 12 loads a prefabricated training model, parses to obtain a three-dimensional model material, and the three-dimensional model material includes a vehicle body model, a target model, and a training scene model, and the vehicle body model internally contains a turret sub-model with an updateable direction angle and a height angle;

[0139] The training data set obtaining module 13 loads prefabricated training data, and obtains a training data set composed of structured data, and the structured data contains timestamp, acceleration, yaw angle, pitch angle, and roll angle information;

[0140] The real-time operation data obtaining module 14 continuously collects data of the operation state of the simulation console based on a preset sampling frequency, and obtains real-time operation data containing a direction angle change amount and a height angle change amount, and the determination method of the preset sampling frequency is as follows:

[0141] Firstly, the accuracy requirement of the training task is determined, including the restoration accuracy of the attitude change, the time delay constraint, and the smoothness index of the training process,

[0142] Secondly, the performance index of the target equipment is confirmed, including the refresh rate of the simulation console input device, the maximum frame rate of the three-dimensional model rendering engine, the maximum throughput rate of the data processing module, and the response speed of the sensor or interface module, and the ideal sampling frequency setting should satisfy: the maximum frame rate of the rendering engine <= the sampling frequency <= the refresh rate of the simulation console input device <= the maximum throughput rate of the data processing module;

[0143] The real-time training data set obtaining module 15 interpolates and fills the training data set based on the preset sampling frequency, generates a real-time training data set composed of structured data synchronized with the preset sampling frequency, and the structured data contains timestamp, acceleration, yaw angle, pitch angle, and roll angle information;

[0144] The real-time operation result data generating module 16 fuses the real-time training data set and the real-time operation data based on the preset sampling frequency, generates real-time operation result data containing timestamp, acceleration, yaw angle, pitch angle, roll angle, direction angle, and height angle information, and the generation of the real-time operation result data includes: using an inertial coupling calculation method to dynamically compensate the direction angle and the height angle in the real-time operation result data;

[0145] The operation feedback image forming module 17 updates the attitude of the three-dimensional model material with the real-time operation result data based on the preset sampling frequency, and is used for forming an operation feedback image.

[0146] It should be noted that the above-mentioned order of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0147] The above only describes the preferred embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0148] The present application and the drawings are only exemplary descriptions of the present application, and are considered to cover any and all modifications, changes, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the scope of the present application. Thus, if these modifications and changes of the present application belong to the scope of the present application and its equivalent technology, the present application intends to include these modifications and changes.

Claims

1. A training method, characterized in that: The method comprises: Preset the sampling frequency based on the training accuracy requirements and the performance indicators of the target equipment for training; Load the prefabricated training model and parse it to obtain a 3D model material. The 3D model material includes a vehicle model, a target model, and a training scene model. The vehicle model contains a turret sub-model with updateable azimuth and elevation angles. Loading prefabricated training data to obtain a training data set consisting of structured data, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle, and roll angle information; Continuously collect data on the operating status of the simulated control console based on a preset sampling frequency to obtain real-time control data, wherein the real-time control data includes the change in direction angle and the change in height angle. The preset sampling frequency is determined by: First, determine the accuracy requirements of the training task, including the restoration accuracy of posture changes, delay constraints, and smoothness indicators of the training process. Secondly, confirm the performance indicators of the target equipment, including the refresh rate of the analog console input device, the maximum frame rate of the 3D model rendering engine, the maximum throughput rate of the data processing module, and the response speed of the sensor or interface module. The ideal sampling frequency setting should meet the following requirements: maximum frame rate of the rendering engine <= sampling frequency <= refresh rate of the analog console input device <= maximum throughput rate of the data processing module; Interpolating and padding the training data set based on a preset sampling frequency to generate a real-time training data set composed of structured data synchronized with the preset sampling frequency, the structured data including timestamp, acceleration, yaw angle, pitch angle, and roll angle information; The real-time training data set is integrated with the real-time control data based on a preset sampling frequency to generate real-time operation result data, wherein the real-time operation result data includes a timestamp, acceleration, yaw angle, pitch angle, roll angle, azimuth angle, and elevation angle information. The generating of the real-time operation result data includes: dynamically compensating the azimuth angle and elevation angle in the real-time operation result data using an inertial coupling calculation method; The posture of the three-dimensional model material is updated with real-time operation result data based on a preset sampling frequency to form an operation feedback image.

2. A training method according to claim 1, characterized in that: The loading of prefabricated training data to obtain a training data set consisting of structured data further includes: In the process of constructing structured data, multi-level verification is performed on the binary data frame of the training data to obtain data frame information that fails the verification. The multi-level verification includes frame header verification, field boundary verification, check bit comparison, and inter-frame consistency verification.

3. A training method according to claim 2, characterized in that: The multi-level verification of the binary data frame of the training data during the process of constructing the structured data further includes: The data frame that failed the verification is reconstructed by data fitting based on the data of the two previous and next normal data frames through linear interpolation.

4. A training method according to claim 1, characterized in that: The interpolation and padding of the training data set based on the preset sampling frequency to generate a real-time training data set composed of structured data synchronized with the preset sampling frequency also includes: Loading a posture change template to obtain posture change template data, wherein the posture change template is pre-established, has the same data structure as the pre-made training data, and is a template file for describing motion posture changes; Using a sliding window method to perform segmented analysis on a real-time training data set to obtain a motion posture feature vector sequence, wherein the motion posture feature vector sequence is a vector sequence obtained by using a motion posture feature extraction method based on the training data within the sliding window; The sliding window method is used to segment the posture change template data to obtain the motion posture feature vector sequence; A similarity score is performed on the motion posture feature vector sequence of the training data set and the motion posture feature vector sequence of the posture change template, and the data of abnormal motion posture changes in the real-time training data set is obtained through scoring.

5. A training method according to claim 4, characterized in that: The method further comprises: performing similarity scoring on the motion posture feature vector sequence of the training data set and the motion posture feature vector sequence of the posture change template, and obtaining data of abnormal motion posture changes in the real-time training data set through scoring determination; Traverse the data with abnormal motion posture changes in the real-time training data set and calculate the continuous interval of the data with abnormal motion posture changes; Each continuous interval is processed in turn. If the number of data in the current continuous interval is less than or equal to the preset threshold, the data in the current continuous interval is compensated and corrected using the abnormal behavior compensation method.

6. A training method according to claim 5, characterized in that: The abnormal behavior compensation method includes: Using the motion posture feature extraction method to obtain the motion posture feature vector sequence corresponding to the data in the current continuous interval; Using the motion posture feature extraction method to obtain the motion posture feature vector sequence corresponding to all the posture change template data; Perform similarity scoring on the motion posture feature vector sequence corresponding to the data in the current continuous interval and the motion posture feature vector sequence corresponding to all posture change template data, and obtain the posture change template closest to the data in the current continuous interval through scoring; Based on the closest posture change template and the data in the current continuous interval, the data in the current continuous interval is compensated and corrected by using a third-order Bezier curve fitting method.

7. A training method according to claim 4 or 6, characterized in that: The motion posture feature extraction method comprises: For continuous multi-frame attitude data, calculate the average yaw angle change rate: ,in, is the change in yaw angle between two adjacent frames, in degrees, is the time interval between two adjacent frames, in seconds, and the rate of change of each adjacent frame is obtained. , for the rate of change Take the average value; For continuous multi-frame attitude data, calculate the Pearson correlation coefficient between acceleration and yaw angle: ; in, is the acceleration sequence, in units of , is the time series of yaw angle changes, in degrees, for and The covariance of , which indicates the degree of joint variation, is the standard deviation of the acceleration series, indicating its degree of fluctuation, is the standard deviation of the yaw angle series, and the correlation coefficient range is ; For continuous multi-frame attitude data, calculate the Pearson correlation coefficient between acceleration and pitch angle: ; in, is the acceleration sequence, in units of , is the time series of pitch angle changes, in degrees, for and The covariance of , which indicates the degree of joint variation, is the standard deviation of the acceleration series, indicating its degree of fluctuation, is the standard deviation of the pitch angle sequence, and the correlation coefficient range is ; For continuous multi-frame attitude data, calculate the Pearson correlation coefficient between acceleration and roll angle: ; in, is the acceleration sequence, in units of , is the time series of the roll angle, in degrees, for and The covariance of , which indicates the degree of joint variation, is the standard deviation of the acceleration series, indicating its degree of fluctuation, is the standard deviation of the roll angle series, and the correlation coefficient range is ; Calculate the standard deviation of the pitch angle for continuous multi-frame attitude data; Calculate the maximum absolute value of the roll angle for multiple frames of continuous attitude data; For multiple frames of continuous attitude data, find the maximum value of the yaw angle change rate; The average yaw angle change rate, the Pearson correlation coefficient between the acceleration and the yaw angle, the Pearson correlation coefficient between the acceleration and the pitch angle, the Pearson correlation coefficient between the acceleration and the roll angle, the standard deviation of the pitch angle, the maximum absolute value of the roll angle, and the maximum value of the yaw angle change rate are combined in a preset order to form a motion posture feature vector.

8. The console training device is characterized by: The device comprises: A sampling frequency preset module, wherein the sampling frequency preset module presets the sampling frequency according to the training accuracy requirements and the performance indicators of the target equipment to be trained; A 3D model material acquisition module, which loads a prefabricated training model and parses it to obtain 3D model material. The 3D model material includes a vehicle model, a target model, and a training scene model. The vehicle model contains a turret sub-model with updateable azimuth and elevation angles. A training data set acquisition module, wherein the training data set acquisition module loads prefabricated training data to obtain a training data set consisting of structured data, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle and roll angle information; A real-time control data acquisition module is configured to continuously collect data on the operating status of the simulated control console based on a preset sampling frequency to obtain real-time control data. The real-time control data includes azimuth angle changes and elevation angle changes. The preset sampling frequency is determined by: First, determine the accuracy requirements of the training task, including the restoration accuracy of posture changes, delay constraints, and smoothness indicators of the training process. Secondly, confirm the performance indicators of the target equipment, including the refresh rate of the analog console input device, the maximum frame rate of the 3D model rendering engine, the maximum throughput rate of the data processing module, and the response speed of the sensor or interface module. The ideal sampling frequency setting should meet the following requirements: maximum frame rate of the rendering engine <= sampling frequency <= refresh rate of the analog console input device <= maximum throughput rate of the data processing module; A real-time training data set acquisition module, which interpolates and fills the training data set based on a preset sampling frequency to generate a real-time training data set composed of structured data synchronized with the preset sampling frequency, wherein the structured data includes timestamp, acceleration, yaw angle, pitch angle, and roll angle information; A real-time operation result data generation module, wherein the real-time operation result data generation module fuses the real-time training data set with the real-time control data based on a preset sampling frequency to generate real-time operation result data, wherein the real-time operation result data includes timestamp, acceleration, yaw angle, pitch angle, roll angle, azimuth angle, and elevation angle information. Generating the real-time operation result data includes: dynamically compensating the azimuth angle and elevation angle in the real-time operation result data using an inertial coupling calculation method; The operation feedback image forming module updates the posture of the three-dimensional model material with real-time operation result data based on a preset sampling frequency to form an operation feedback image.

Images