Method and system for realizing real soldier confrontation by capturing weapon firing direction by using imaging equipment
By using imaging equipment to capture the weapon's shooting direction and perform 3D modeling and external ballistic solution, the problem that traditional laser simulation training systems cannot truly simulate bullet penetration and ballistic factors is solved, and the accuracy and practical value of training are improved.
Patent Information
- Application Number
- CN202411376814.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-03-11
AI Technical Summary
Traditional laser simulation training systems cannot truly simulate the penetration and ballistic factors of bullets, such as falling and wind deviation, resulting in the training effect being unable to be directly converted into actual combat capabilities.
Imaging equipment is used to capture the weapon's shooting direction, obtain environmental data through the server to generate 3D modeling, calculate the precise shooting direction of the weapon, and perform external ballistic solutions to obtain the hit result.
It improves the accuracy of hit judgment, enhances the authenticity and practical value of training, and makes the training effect more directly transformed into practical combat capabilities.
Smart Images
Figure CN119594794A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital ammunition, and in particular to a method and system for using imaging equipment to capture the direction of a weapon to achieve real-life confrontation. Background Art
[0002] Traditional live-fire confrontation systems simulate ballistics by using lasers instead of actual ammunition. In this system, each soldier participating in the training is equipped with a sensor and a simulated weapon equipped with a laser transmitter. When a soldier "fires", the laser transmitter emits a beam of laser with a specific code to simulate the ballistic path of an actual bullet. If this laser beam hits the sensor of another soldier, the sensor will detect the laser signal and determine the location and effect of the hit based on the received data, just like a live bullet hitting the target. This technology makes the training environment safe and effective, while accurately reflecting the shooting and being shot on the battlefield, greatly improving the authenticity and effectiveness of tactical training.
[0003] The use of lasers for live-fire confrontation training has obvious limitations. First, lasers cannot simulate the penetration of actual ammunition, which means that it is impossible to realistically simulate the situation of bullets penetrating walls or armor during training. In addition, laser systems generally do not take into account ballistic factors such as bullet drop and wind deflection, which makes it impossible to fully convert shooting technology training into actual combat capabilities. As the distance increases, the effective action area of the laser becomes wider, which is contrary to the increasing difficulty of shooting with increasing distance in actual combat. Therefore, although this training system provides a safe and cost-effective training method, its defects in simulating real battlefield environments and tactical decision-making may result in the inability to directly convert training results into actual combat capabilities, thereby weakening its value in training at the tactical and strategic levels.
[0004] In order to overcome the limitations of traditional laser simulation training, modern military training is turning to a more high-tech and precise solution, namely "digital ammunition" technology. This technology can simulate the flight trajectory and behavior of bullets in detail in a digital twin environment by collecting the specific position and precise posture of the weapon when firing, combined with advanced external ballistic simulation. In this environment, the system can not only perform collision detection, but also simulate potential damage effects. In this way, digital ammunition technology can accurately confirm the hit relationship, thereby providing a safe and economical way to realistically simulate shooting and tactical actions in actual combat, greatly enhancing the realism and practical value of training. The development of this technology marks the evolution of military training simulation technology to a higher level.
[0005] In the implementation of digital ammunition technology, collecting accurate data on the direction of the weapon has become the key to the success of the technology. This process involves the use of high-precision sensors and advanced data acquisition systems to record the exact position, posture and direction of the weapon at the moment of shooting in real time. These data are the basis for external ballistic simulation, ensuring that the shooting behavior in the simulation environment can highly restore the actual shooting conditions. Accurate weapon direction data collection not only improves the authenticity of the simulation, but also enhances the effectiveness and efficiency of training, allowing trainers to analyze and evaluate the effects of each shot in detail in a fully controlled digital environment. In order to ensure that the ballistic simulation of weapons (such as rifles) meets actual combat standards in digital training, the importance of direction acquisition accuracy cannot be ignored. Summary of the invention
[0006] Based on this, it is necessary to provide a method and system for using imaging equipment to capture the direction of weapons and realize real-life confrontation in order to address the above technical issues.
[0007] The method for using an imaging device to capture the direction of a weapon to achieve real-life combat is applied to a system for using an imaging device to capture the direction of a weapon to achieve real-life combat, which includes a shooting image acquisition device, a server, and a target device; the method includes the following steps:
[0008] The shooting image acquisition device acquires shooting images and sends the shooting images to the server;
[0009] The server acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features;
[0010] The server receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics calculation according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device;
[0011] The hitting device receives the hitting result and generates a response measure according to the hitting result.
[0012] In one embodiment, the server obtains environmental data, generates 3D modeling according to the environmental data, and obtains modeling features including:
[0013] Acquire environmental data, perform feature extraction on the environmental data, and obtain key features;
[0014] According to the key features, matching feature points are obtained through a feature matching algorithm;
[0015] According to the matching feature points, the feature three-dimensional coordinates are obtained by using the SfM algorithm;
[0016] Acquire RTK positioning system data and inertial navigation system data, and obtain estimated 3D feature point positions according to the RTK positioning system data, the inertial navigation system data and the feature three-dimensional coordinates;
[0017] Generate 3D modeling based on the estimated 3D feature point positions to obtain modeling features.
[0018] In one embodiment, the server receives the shooting image, and obtaining the shooting feature according to the shooting image includes:
[0019] The shooting image is received, a feature extraction algorithm is acquired, and a shooting feature is obtained according to the shooting image through the feature extraction algorithm.
[0020] In one embodiment, calculating the precise shooting direction of the weapon according to the modeling feature and the shooting feature includes:
[0021] Performing feature matching on the modeling feature and the shooting feature to obtain a matching feature;
[0022] The precise shooting direction of the weapon is calculated through the matching features.
[0023] In one embodiment, feature matching is performed on the modeling feature and the shooting feature to obtain matching features, including:
[0024] Acquire a matching algorithm, and use the matching algorithm to perform feature matching on the modeling feature and the shooting feature to obtain matching similarity points;
[0025] Acquire multiple approximate nearest neighbor search algorithms to perform performance evaluation on the matching similar points to obtain matching evaluation results;
[0026] According to the matching evaluation result, a best approximate nearest neighbor search algorithm is obtained, and feature matching is performed according to the best approximate nearest neighbor search algorithm to obtain a matching feature pair;
[0027] Perform matching verification on the matching feature pair to obtain matching features.
[0028] In one embodiment, calculating the precise shooting direction of the weapon through the matching features includes:
[0029] A three-dimensional map is obtained, and the camera posture is calculated based on the matching features and the three-dimensional map through a PNP algorithm to obtain the precise shooting direction of the weapon.
[0030] In one embodiment, performing external ballistic calculation according to the precise shooting direction of the weapon to obtain a hit result includes:
[0031] Constructing an exterior ballistics model;
[0032] Acquire initial information of the projectile, and obtain an external ballistic trajectory according to the initial information of the projectile and the external ballistic model;
[0033] Prepare external ballistic impact shooting tables;
[0034] Correcting the external ballistic trajectory according to the external ballistic influence shooting table to obtain a corrected external ballistic trajectory;
[0035] A hit determination is performed according to the corrected outer ballistic trajectory to obtain a hit result.
[0036] In one embodiment, performing hit determination according to the modified exterior ballistic trajectory to obtain a hit result includes:
[0037] Acquiring shooting parameters, and calculating the damage range according to the shooting parameters and the corrected exterior ballistic trajectory;
[0038] Obtaining an actual damage range, determining whether the damage range falls within the actual damage range, and performing a hit determination;
[0039] In response to the damage range falling into the actual damage range, determining that it is a hit, and obtaining a hit message;
[0040] In response to the damage range not falling within the actual damage range, determining that it is a miss, and obtaining a miss message;
[0041] The hit result includes the hit message and the miss message.
[0042] A system for using an imaging device to capture the direction of a weapon to achieve real-life combat, which is used to implement the method of using an imaging device to capture the direction of a weapon to achieve real-life combat as described above, comprising: a shooting image acquisition device, a server, and a hit-side device;
[0043] The server acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features;
[0044] The shooting image acquisition device acquires shooting images and sends the shooting images to the server;
[0045] The server receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics calculation according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device;
[0046] The hitting device receives the hitting result and generates a response measure according to the hitting result.
[0047] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: the method provided by the present invention for using imaging equipment to capture the direction of a weapon to achieve real-life confrontation is used for simulating trajectory estimation in a confrontation system, and a database of modeling features is established through the collected environmental data. It is not necessary to obtain information about the target to be attacked, and the shooting image is directly collected through a shooting image collection device. The collected shooting image is sent to a server, and the server can compare it with the database of modeling features, calculate the precise direction of the weapon, perform external ballistics solution, and obtain a hit result. While reducing the amount of simulated data acquisition, the accuracy of hit determination is improved, ensuring scenario training in various simulated environments and various tactical training. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of an application scenario of a method for using an imaging device to capture the direction of a weapon to achieve real-life combat in one embodiment;
[0049] Figure 2 A schematic diagram of the composition and interaction of a system for using an imaging device to capture the direction of a weapon to achieve real-life combat in one embodiment;
[0050] Figure 3 A flowchart of a method for using an imaging device to capture a weapon's firing direction to achieve real-life combat in one embodiment;
[0051] Figure 4 A schematic diagram of a modeling feature acquisition process in one embodiment;
[0052] Figure 5 Schematic diagram of the posture tracking process in one embodiment. DETAILED DESCRIPTION
[0053] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0054] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of this specification should be understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in one or more implementations of this specification do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0055] As described in the background technology section, since the key to high-quality rifle ballistic simulation training is to ensure the effectiveness of training and the accurate transmission of shooting skills, a method for collecting accurate data of weapon shooting direction for analysis is needed.
[0056] Based on the problems existing in the above-mentioned prior art, one or more embodiments of this specification provide a method and system for using imaging equipment to capture the direction of weapons to achieve real-life confrontation. The cloud server analyzes the acquired shooting images to obtain accurate hit results, and sends the hit results to the hitting center equipment. The hitting center equipment receives the hit results and generates corresponding measures based on the hit results to achieve the simulation of real-life confrontation.
[0057] The technical solution of the present disclosure is further described in detail below through specific embodiments.
[0058] First, one or more embodiments of this specification provide a method for using an imaging device to capture the direction of a weapon to achieve real-life combat. The method is applied to a system that uses an imaging device to capture the direction of a weapon to achieve real-life combat.
[0059] Figure 1 A schematic diagram of an application scenario of a method for using an imaging device to capture the direction of a weapon to achieve real-life combat in an embodiment of the present invention is shown.
[0060] refer to Figure 2, showing the composition and interaction mode of the system for using imaging devices to capture the direction of weapons to achieve real-life confrontation. The system for using imaging devices to capture the direction of weapons to achieve real-life confrontation includes: a shooting image acquisition device 101, a server 102, and a hitting device 103. Among them, the shooting image acquisition device 101, the server 102, and the hitting device 103 can be connected through a wired or wireless communication network. The shooting image acquisition device 101 uses a wireless communication network such as 4G, 5G, or WiFi to transmit the shooting image to the server 102 in real time, and the server 102 uses a wireless communication network such as 4G, 5G, or WiFi to send the hit result to the Chinese device 103.
[0061] The shooting image acquisition device 101 is set on each weapon system, especially at the front end of the barrel or gun body, a shooting image acquisition device 101 is installed. These shooting image acquisition devices 101 are not limited to specific device types, and need to have high durability to withstand the impact and vibration of the muzzle, and should have sufficient resolution and frame rate to clearly record the shooting process and shooting direction environment.
[0062] The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The Chinese equipment 103 includes but is not limited to individual equipment, vehicle systems or command and control centers, etc., depending on the configuration of training or actual combat operations.
[0063] The cloud server 102 is used to provide a hit result delivery service to users of the hitting device 103. The edge hitting device 103 can communicate with the server 102 in real time. After receiving the hit result, the hitting device 103 can take corresponding response measures, such as igniting pyrotechnics to indicate the hit status, or lighting up an indicator light and activating a buzzer to remind trainees of the hit status.
[0064] Combine the following Figure 1 Application scenarios and Figure 2 The data interaction mode of the present invention is used to describe the method of using an imaging device to capture the direction of a weapon to achieve real-life confrontation according to an exemplary embodiment of the present application. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.
[0065] First, one or more embodiments of this specification provide a method for using an imaging device to capture the direction of a weapon to achieve real-life combat. Figure 3, the method comprises the following steps:
[0066] Step S301: the server acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features.
[0067] Specifically, after acquiring the environmental data, the server processes the environmental data and then performs 3D modeling to obtain a database of modeling features.
[0068] On this basis, the server obtains environmental data, generates 3D modeling according to the environmental data, and obtains modeling features including:
[0069] Acquire environmental data, perform feature extraction on the environmental data, and obtain key features;
[0070] According to the key features, matching feature points are obtained through a feature matching algorithm;
[0071] According to the matching feature points, the feature three-dimensional coordinates are obtained by using the SfM algorithm;
[0072] Acquire RTK positioning system data and inertial navigation system data, and obtain estimated 3D feature point positions according to the RTK positioning system data, the inertial navigation system data and the feature three-dimensional coordinates;
[0073] Generate 3D modeling based on the estimated 3D feature point positions to obtain modeling features.
[0074] Specifically, Figure 4 As shown, data collection is first performed, and the server obtains the collected environmental data. The environmental data can be captured in the target environment using a device equipped with a high-resolution camera (such as a drone, a handheld device, or a camera system fixed at a specific location) to obtain a large number of images. The camera covers different angles and heights to ensure sufficient viewing angle coverage for subsequent processing.
[0075] Then, feature extraction is performed on the environmental data, and feature descriptors such as SIFT, SURF, FAST, and ORB are applied to extract feature descriptors from the collected images to obtain key features. These feature descriptors can identify unique points in the image, such as corners and edges, and extract local feature descriptors of these key points at different scales to obtain matching feature points. These matching feature points are invariant to rotation, scaling, and illumination changes, and are suitable for subsequent matching and 3D position estimation.
[0076] Then perform feature matching and tracking, and use the feature matching algorithm to find the same matching feature points between multiple images. By detecting the matching feature points in the image, locate and identify specific objects in the image. Then use the matched feature points to reconstruct the positions of these points in 3D space through the SfM algorithm. The SfM algorithm uses the parallax of feature points between two or more images to estimate their three-dimensional coordinates and obtain feature three-dimensional coordinates. Specifically, the feature matching algorithms include: 1. Classic feature matching algorithms: Brute-Force Matching / FLANN / KD Tree Matching / LSH / RANSAC; 2. Modern deep learning matching algorithms: DeepMatcher / LOFTR / SuperGlue. The feature matching algorithm includes but is not limited to the feature matching algorithms listed in this specification. All feature matching algorithms that can realize the function of finding the same matching feature points between multiple images are within the scope of this application.
[0077] After that, precise positioning and data fusion are performed, combining the high-precision position and orientation information provided by the RTK positioning system and the inertial navigation system (INS), and accurate geolocation of each image at the time of shooting is performed to obtain the estimated 3D feature point position. This step improves the accuracy of the position estimation of the feature's three-dimensional coordinates, especially in large-scale or open space environments.
[0078] After that, 3D model generation and optimization are performed to build a 3D model of the entire scene based on the estimated 3D feature point positions. At this stage, a variety of optimization techniques can be applied to improve the accuracy and authenticity of the model, such as mesh optimization, texture mapping, and model simplification.
[0079] Finally, a feature database is constructed, and a 3D model is generated based on the estimated 3D feature point positions. The obtained 3D model and its feature point positions are stored in a feature database.
[0080] Step S302: the shooting image acquisition device acquires a shooting image and sends the shooting image to the server.
[0081] Specifically, a shooting image acquisition device 101 is installed on each weapon system, especially at the front end of the barrel or gun body. These shooting image acquisition devices 101 need to have high durability to withstand the impact and vibration of the muzzle, and should have sufficient resolution and frame rate to clearly record the shooting process and shooting direction environment. When shooting, the image acquisition is triggered synchronously, and the shooting behavior of the weapon pulling the trigger is confirmed by an electrical device. The shooting trigger of the installed shooting image acquisition device 101 is synchronized with the firing mechanism of the weapon through a digital circuit to ensure that the shooting image acquisition device 101 automatically starts recording each time the weapon is fired, and the image capture is completely synchronized with the actual shooting behavior. The shooting image acquisition device 101 performs image acquisition in real time, captures the shooting moment and the subsequent continuous actions, records the field of view image in front of the muzzle, obtains the shooting image and uploads it to the server 102, and the shooting image includes the target area, possible obstacles and other environmental details. The captured shooting image is compressed according to the characteristics and needs of the scene to prepare for the next stage of data transmission.
[0082] Step S303, the server receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics solution according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device.
[0083] Specifically, the server 102 extracts features from the uploaded shooting image and matches them with known features in the modeling feature database to calculate the precise shooting direction of the weapon, performs exterior ballistics calculation based on the shooting direction, determines the hit and obtains the hit result, and sends the hit result to the hitting center device 103.
[0084] On this basis, the server receives the shooting image, and obtains the shooting features according to the shooting image, including:
[0085] The shooting image is received, a feature extraction algorithm is acquired, and a shooting feature is obtained according to the shooting image through the feature extraction algorithm.
[0086] Specifically, the server 102 first uses an advanced image processing feature extraction algorithm from the uploaded shooting image to obtain the shooting feature. Specifically, the feature extraction algorithm includes: 1. Classic feature extraction algorithm: SIFT / SURF / ORB / FAST / Harris Corner Detector; 2. Modern deep learning feature extraction algorithm: SuperPoint / D2-Net / R2D2 / LF-Net, etc. The feature extraction algorithm includes but is not limited to the feature extraction algorithms listed in this specification. All feature extraction algorithms that can obtain shooting features from the uploaded shooting image are within the scope of this application.
[0087] The following is an explanation of the shooting feature extraction process using the specific SIFT feature extraction algorithm as an example:
[0088] The shooting image is blurred using a Gaussian blur kernel to generate a series of processed images of different scales. The differential Gaussian images of the processed images of multiple scales are calculated, that is, two adjacent blurred processed images are subtracted at different scales to obtain the DoG pyramid. The positions of extreme points are detected in the DoG pyramid, which are potential key points. The positions of potential key points are accurately located by sub-pixel interpolation, and low-contrast points and edge response points are removed to obtain potential key points. The Taylor series expansion is used to calculate the exact position and scale of each potential key point. The integral image is used to accelerate the calculation. At any point in the processed image, the determinant of the Hessian matrix is used to detect the key point. The elements in the Hessian matrix are the second-order derivatives of the image. A box filter is used instead of a Gaussian filter to approximate the second-order derivative to accelerate the calculation using the integral image.
[0089] Find the extreme points of the Hessian matrix determinant in the processed images of different scales, and use these extreme points as key points. Calculate the gradient direction histogram of the area around the key point. Select the main peak direction in the histogram, and may select multiple directions as the direction attribute of the key point. Establish a direction and scale invariant coordinate system in the neighborhood of the key point. Calculate the gradient direction and amplitude of the area around the key point, generate a feature vector (descriptor), and obtain the shooting feature.
[0090] On this basis, calculating the precise shooting direction of the weapon according to the modeling features and the shooting features includes:
[0091] Performing feature matching on the modeling feature and the shooting feature to obtain a matching feature;
[0092] The precise shooting direction of the weapon is calculated through the matching features.
[0093] On this basis, feature matching is performed on the modeling features and the shooting features, and the matching features obtained include:
[0094] Acquire a matching algorithm, and use the matching algorithm to perform feature matching on the modeling feature and the shooting feature to obtain matching similarity points;
[0095] Acquire multiple approximate nearest neighbor search algorithms to perform performance evaluation on the matching similar points to obtain matching evaluation results;
[0096] According to the matching evaluation result, a best approximate nearest neighbor search algorithm is obtained, and feature matching is performed according to the best approximate nearest neighbor search algorithm to obtain a matching feature pair;
[0097] Perform matching verification on the matching feature pair to obtain matching features.
[0098] Specifically, the extracted shooting features are then compared and matched with the known modeling features pre-stored in the feature database. In this process, the server 102 uses a feature matching algorithm, such as FLANN or BFMatcher, to find similar feature points between different images and obtain matching similar points. Combining a variety of approximate nearest neighbor search algorithms and automatic selection mechanisms, including KD tree, random kd tree, hierarchical clustering tree, K-means tree, etc., the performance of different approximate nearest neighbor search algorithms and parameters is evaluated, including construction time, query time and accuracy. Through test queries, the effect of each configuration is evaluated to obtain a matching evaluation result. According to the matching evaluation result, the feature matching algorithm automatically selects the best approximate nearest neighbor search algorithm and parameter configuration to find the best balance between construction time, query time and search accuracy (the construction time and query time are the shortest when the exploration accuracy requirements are met), and obtain the best approximate nearest neighbor search algorithm.
[0099] The obtained best approximate nearest neighbor search algorithm finds matching feature points by comparing the modeling features and the shooting features. Common matching strategies include calculating the distance between descriptors, such as the Euclidean distance, and finding the point pair with the smallest distance. In order to improve the accuracy of the matching, a ratio test is used, in which the best matching distance and the second best matching distance are compared. Only when the best match is much better than the second best match, it is considered to be a good match and a matching feature pair is obtained.
[0100] In order to further eliminate incorrect matching feature pairs, geometric constraints are used for matching verification. This usually involves geometric relationships such as epipolar geometry or homography matrix, and the best geometric model is found through RANSAC (Random Sample Consensus) or other robust estimation techniques, so that only matching feature pairs that meet these geometric constraints are retained to obtain matching features.
[0101] On this basis, calculating the precise shooting direction of the weapon through the matching features includes:
[0102] A three-dimensional map is obtained, and the camera posture is calculated based on the matching features and the three-dimensional map through a PNP algorithm to obtain the precise shooting direction of the weapon.
[0103] Specifically, Figure 5 As shown, once feature matching is completed and matching features are obtained, the server 102 uses the geometric relationship between the matching features to calculate the shooting direction of the weapon. By matching with the pre-established high-precision three-dimensional map and matching features, using the matching feature point pairs (3D-2D), by solving the PnP problem, the camera pose is estimated, and accurate positioning and posture tracking of the equipment or weapon is achieved.
[0104] Specifically, two-dimensional image feature points (2D): matching features from the current camera image, with image coordinates (u, v).
[0105] Three-dimensional map feature points (3D): Feature points from high-precision three-dimensional maps with spatial coordinates (X, Y, Z).
[0106] Formation of feature point pairs: Through the matching process, a set of corresponding 3D-2D feature point pairs are formed, providing a data basis for subsequent pose estimation.
[0107] The PnP (Perspective-n-Point) problem is to solve the pose (position and attitude) of the camera when n three-dimensional points and their corresponding two-dimensional projection points are known. First, the PnP algorithm (such as EPnP, UPnP, RPnP) is used in combination with the RANSAC (random sampling consensus) method to robustly solve the camera's external parameter matrix (rotation matrix R and translation vector t), thereby obtaining the camera's pose in the world coordinate system. Then the camera pose is estimated and the pose parameters are obtained: the rotation matrix R and translation vector t obtained by solving the solution accurately describe the spatial position and orientation of the camera relative to the three-dimensional map. Finally, the coordinate system is transformed. According to the installation relationship between the camera and the weapon (for example, the fixed offset and angle of the camera relative to the weapon), the camera pose is converted to the weapon pose, and then the weapon's shooting direction is determined.
[0108] Calculate the weapon's firing direction and direction vector calculation: Use the weapon's position information and the weapon's own structural parameters to calculate the unit vector of the firing direction. Consider environmental factors (such as wind speed, gravity) and weapon characteristics, make necessary compensation and corrections to the firing direction, and improve the accuracy of the calculation.
[0109] In this embodiment, advanced image processing and pattern recognition algorithms are used to quickly identify corresponding features in real-time images and digital twin models, thereby achieving high-precision spatial positioning and posture measurement.
[0110] On this basis, the external ballistics is calculated according to the precise shooting direction of the weapon, and the hit results include:
[0111] Constructing an exterior ballistics model;
[0112] Acquire initial information of the projectile, and obtain an external ballistic trajectory according to the initial information of the projectile and the external ballistic model;
[0113] Prepare external ballistic impact shooting tables;
[0114] Correcting the external ballistic trajectory according to the external ballistic influence shooting table to obtain a corrected external ballistic trajectory;
[0115] A hit determination is performed according to the corrected outer ballistic trajectory to obtain a hit result.
[0116] Specifically, to obtain the precise shooting direction of the weapon, the server 102 uses the exterior ballistic model to calculate the flight trajectory of the bullet, and then corrects the exterior ballistic trajectory through the exterior ballistic influence shooting table. The exterior ballistic influence shooting table takes into account environmental factors such as gravity, wind, and temperature to ensure that the simulated shooting trajectory is as close to the actual situation as possible.
[0117] Construct an external ballistic model. The external ballistic model includes:
[0118]
[0119] Among them, x, y represent the horizontal component and vertical component of the coordinates (x, y) of the projectile at time t after leaving the muzzle, u represents the horizontal component velocity, p represents the vertical component velocity, v represents the flight speed of the projectile, C represents the ballistic coefficient, H() represents the air density function, G() represents the air resistance function, g represents the acceleration of gravity, and P represents the engine thrust.
[0120] Obtain the initial information of the projectile, including the projectile range, projectile flight speed, etc. Input the initial information of the projectile into the exterior ballistic model, use MATLAB simulation software to simulate the exterior ballistic model, and obtain the exterior ballistic trajectory.
[0121] The external ballistic influence shooting table is compiled through a variety of existing external ballistic influence shooting tables, including: the influence of terrain on shooting hits and correction data, the influence of temperature on shooting hits and correction data, the influence of air pressure on shooting hits and correction data, and the influence of wind speed and wind direction on shooting hits and correction data, etc.
[0122]
[0123]
[0124] Table 1 External ballistics impact table
[0125] On this basis, the hit determination is performed according to the modified exterior ballistic trajectory, and the hit results obtained include:
[0126] Obtaining the type of ammunition, and calculating the damage range according to the type of ammunition and the corrected exterior ballistic trajectory;
[0127] Obtaining an actual damage range, determining whether the damage range falls within the actual damage range, and performing a hit determination;
[0128] In response to the damage range falling into the actual damage range, determining that it is a hit, and obtaining a hit message;
[0129] In response to the damage range not falling within the actual damage range, determining that it is a miss, and obtaining a miss message;
[0130] The hit result includes the hit message and the miss message.
[0131] Specifically, based on the corrected exterior ballistic trajectory solved and corrected by the exterior ballistics, the server 102 evaluates the positional relationship between the end point of the bullet and the preset target, calculates the damage range, obtains the actual damage range, determines whether the damage range falls within the actual damage range, and performs a hit judgment to determine whether the target is hit.
[0132] The server 102 obtains the shooting parameters from the ballistic simulation, and the shooting parameters include: ammunition type, damage radius corresponding to the ammunition, and explosion effect. According to the type of ammunition used in the ballistic simulation, the server 102 queries the corresponding damage radius and other parameters in the built-in database. At the same time, the server 102 processes the RTK high-precision positioning data from the training ground target in real time to ensure the accuracy of the location information. The server 102 calculates the expected damage range based on the ballistic simulation results and the damage radius of the ammunition type. Finally, the server 102 compares the damage range with the actual damage range. If the damage range falls within the actual damage range, it is determined to be a hit, and a hit message is obtained. If the damage range does not fall within the actual damage range, it is determined to be a miss, and a miss message is obtained.
[0133] The calculation of the damage range includes the calculation of the damage radius and the judgment of the spatial position relationship. The calculation of the damage radius and the judgment of the spatial position relationship can be calculated by various existing algorithms. According to the type of ammunition and the explosion effect, the damage radius is calculated by using a geometric or physical model. The geometric principle is used to determine whether the target position is within the damage radius, including but not limited to the calculation of the position relationship between a point and a circle (sphere).
[0134] Step S304: the hitting device receives the hitting result and generates a response measure according to the hitting result.
[0135] Specifically, after the server 102 determines the hit and obtains the hit result, the hit result is sent in real time to the hit center device 103 through the communication network. The hit center device 103 may be a single soldier device, a vehicle system or a command and control center, depending on the configuration of the training or actual combat operation.
[0136] After receiving the hit result, the hitting device 103 can take corresponding response measures, such as igniting pyrotechnics to indicate the hit status, or lighting an indicator light, activating a buzzer to remind the trainees of the hit status.
[0137] The method provided by the present invention for using an imaging device to capture the direction of a weapon to realize real-life confrontation is used for trajectory estimation in a simulated confrontation system. A database of modeling features is established through the collected environmental data. No information of the target to be attacked is required. The shooting image is directly collected through a shooting image collection device. The collected shooting image is sent to a server. The server can compare it with the database of modeling features, calculate the precise direction of the weapon, perform external ballistics solution, and obtain a hit result. While reducing the amount of simulated data acquisition, the accuracy of hit determination is improved, and scenario training in various simulated environments and various tactical training are guaranteed.
[0138] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.
[0139] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0140] On this basis, the present invention also provides a system for using an imaging device to capture the direction of a weapon to achieve real-life combat, including: a shooting image acquisition device 201, a server 202, and a hitting device 203;
[0141] The server 202 acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features;
[0142] The shooting image acquisition device 201 acquires shooting images and sends the shooting images to the server 202;
[0143] The server 202 receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics calculation according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device 203;
[0144] The hit-targeting device 203 receives the hit result and generates a response measure according to the hit result.
[0145] The system for using imaging equipment to capture the direction of weapons and achieve real-life combat in the above-mentioned embodiment is used to implement the corresponding method for using imaging equipment to capture the direction of weapons and achieve real-life combat in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0146] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0147] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0148] While specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present invention, it will be apparent to those skilled in the art that the embodiments of the present invention may be implemented without or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0149] While the invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art in light of the foregoing description.
[0150] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for using an imaging device to capture the direction of a weapon to achieve real-life combat, characterized in that: Applicable to a system including shooting image acquisition equipment, server and hit-side equipment that uses imaging equipment to capture the direction of weapons and realize real-life confrontation; The method comprises: The server acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features; The shooting image acquisition device acquires shooting images and sends the shooting images to the server; The server receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics calculation according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device; The hitting device receives the hitting result and generates a response measure according to the hitting result.
2. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 1, characterized in that: The server obtains environmental data, generates 3D modeling according to the environmental data, and obtains modeling features including: Acquire environmental data, perform feature extraction on the environmental data, and obtain key features; According to the key features, matching feature points are obtained through a feature matching algorithm; According to the matching feature points, the feature three-dimensional coordinates are obtained by using the SfM algorithm; Acquire RTK positioning system data and inertial navigation system data, and obtain estimated 3D feature point positions according to the RTK positioning system data, the inertial navigation system data and the feature three-dimensional coordinates; Generate 3D modeling based on the estimated 3D feature point positions to obtain modeling features.
3. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 1, characterized in that: The server receives the shooting image, and obtaining the shooting feature according to the shooting image includes: The shooting image is received, a feature extraction algorithm is acquired, and a shooting feature is obtained through the feature extraction algorithm according to the shooting image.
4. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 1, characterized in that: The calculating the precise shooting direction of the weapon according to the modeling feature and the shooting feature comprises: Performing feature matching on the modeling feature and the shooting feature to obtain a matching feature; The precise shooting direction of the weapon is calculated through the matching features.
5. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 4, characterized in that: The performing feature matching on the modeling feature and the shooting feature to obtain the matching feature comprises: Acquire a matching algorithm, and use the matching algorithm to perform feature matching on the modeling feature and the shooting feature to obtain matching similarity points; Acquire multiple approximate nearest neighbor search algorithms to perform performance evaluation on the matching similar points to obtain matching evaluation results; According to the matching evaluation result, a best approximate nearest neighbor search algorithm is obtained, and feature matching is performed according to the best approximate nearest neighbor search algorithm to obtain a matching feature pair; Perform matching verification on the matching feature pair to obtain matching features.
6. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 4, characterized in that: Calculating the precise shooting direction of the weapon by using the matching features includes: A three-dimensional map is obtained, and the camera posture is calculated based on the matching features and the three-dimensional map through a PNP algorithm to obtain the precise shooting direction of the weapon.
7. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 1, characterized in that: The performing of external ballistic calculation according to the precise shooting direction of the weapon to obtain the hit result comprises: Constructing an exterior ballistics model; Acquire initial information of the projectile, and obtain an external ballistic trajectory according to the initial information of the projectile and the external ballistic model; Prepare external ballistic impact shooting tables; Correcting the external ballistic trajectory according to the external ballistic influence shooting table to obtain a corrected external ballistic trajectory; A hit determination is performed according to the corrected outer ballistic trajectory to obtain a hit result.
8. The method for using an imaging device to capture the direction of a weapon to achieve real-life combat according to claim 7, characterized in that: The performing hit determination according to the corrected outer ballistic trajectory to obtain a hit result includes: Acquiring shooting parameters, and calculating the damage range according to the shooting parameters and the corrected exterior ballistic trajectory; Obtaining an actual damage range, determining whether the damage range falls within the actual damage range, and performing a hit determination; In response to the damage range falling into the actual damage range, determining that it is a hit, and obtaining a hit message; In response to the damage range not falling within the actual damage range, determining that it is a miss, and obtaining a miss message; The hit result includes the hit message and the miss message.
9. A system for using imaging equipment to capture the direction of weapons and achieve real-life combat, characterized in that: A method for realizing real-life combat by using an imaging device to capture the direction of a weapon as described in claims 1 to 8, comprising: a shooting image acquisition device, a server, and a hit-side device; The server acquires environmental data, generates 3D modeling according to the environmental data, and obtains modeling features; The shooting image acquisition device acquires shooting images and sends the shooting images to the server; The server receives the shooting image, obtains shooting features according to the shooting image; calculates the precise shooting direction of the weapon according to the modeling features and the shooting features; performs exterior ballistics calculation according to the precise shooting direction of the weapon to obtain a hit result, and sends the hit result to the hit center device; The hitting device receives the hitting result and generates a response measure according to the hitting result.