An indoor grenade throwing simulation system and method

Abstract

The application discloses an indoor hand grenade throwing simulation system and method, and the system comprises a first high-speed camera, a second high-speed camera, a high-definition projector, a fusion calculation module, a scene rendering module, a hit-resistant projection curtain and a simulation hand grenade. The first high-speed camera and the second high-speed camera are used for acquiring a first image sequence and a second image sequence; the fusion calculation module is used for acquiring a landing point coordinate; the scene rendering module is used for acquiring a first virtual environment, a first scene rendering image and a second scene rendering image; the high-definition projector is used for projecting the first scene rendering image and the second scene rendering image to the hit-resistant projection curtain; and the hit-resistant projection curtain is used for blocking the thrown simulation hand grenade and displaying the first scene rendering image and the second scene rendering image. It can be seen that the system is favorable for improving the authenticity, safety and accuracy of hand grenade throwing training.

Description

Technical Field

[0001] This invention belongs to the field of military training assessment technology, specifically an indoor grenade throwing simulation system and method. Background Technology

[0002] Hand grenade throwing is an important part of modern military training. Traditionally, training with real hand grenades in outdoor training fields effectively improves trainees' throwing skills, but absolute safety cannot be guaranteed during the throwing process. While using simulated hand grenades can solve the safety issue, this method cannot be used for training in special terrains and built environments.

[0003] Constructing training grounds with diverse scenarios can enhance the realism of grenade throwing training, allowing trainees to experience a realistic battlefield environment. However, these training grounds require significant land area and constant updates based on actual needs, necessitating substantial financial investment. On the other hand, virtual simulation can create realistic virtual battlefield environments indoors at a lower cost, allowing trainees to experience real combat scenarios within a confined space. This approach has gained favor among researchers worldwide. However, effectively constructing a simulated indoor environment, guiding trainees to throw grenades, and accurately aiming for the target spot presents a new challenge.

[0004] (1) Binocular stereo recognition technology

[0005] Binocular stereo vision is a method based on the principle of parallax, which uses multiple images to acquire the three-dimensional geometric information of an object. In machine vision systems, binocular vision typically involves two cameras simultaneously acquiring two digital images of the surrounding scene from different angles, or a single camera acquiring two digital images of the surrounding scene from different angles at different times. Based on the principle of parallax, the three-dimensional geometric information of the object can be recovered, reconstructing the three-dimensional shape and position of the surrounding scene. However, the accuracy of 3D recognition results using binocular vision technology is greatly affected by the target distance; when the target is too far or too close, the recognition accuracy will decrease significantly.

[0006] (2) Industrial high-speed camera shooting technology

[0007] Industrial high-speed cameras are characterized by high frame rate, stable operation, small size, and rich interfaces, and have wide applications in machine vision, medical and health care, industrial production, sports and other fields. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide an indoor grenade throwing simulation system and method, which is beneficial to improving the realism, safety and accuracy of grenade throwing training.

[0009] To address the aforementioned technical problems, the first aspect of this invention discloses an indoor grenade throwing simulation system, the system comprising a first high-speed camera, a second high-speed camera, a high-definition projector, a fusion computing module, a scene rendering module, an impact-resistant projection screen, and a simulated grenade;

[0010] Both the first high-speed camera and the second high-speed camera are data-connected to the fusion computing module and are used to be set in front of the impact-resistant projection screen to synchronously collect images, acquire the first image sequence and the second image sequence respectively, and send them to the fusion computing module.

[0011] The fusion computing module is data-connected to the scene rendering module and is used to process the first image sequence and the second image sequence based on the calibration parameter list obtained from the calibration to obtain the landing point coordinates.

[0012] The scene rendering module is connected to the high-definition projector and is used to obtain a first virtual environment and a first scene rendering image based on the preset virtual three-dimensional environment and target throwing point coordinates, and to obtain a second scene rendering image based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, and to send the first scene rendering image and the second scene rendering image to the high-definition projector.

[0013] The high-definition projector is used to be placed in front of the impact-resistant projection screen to project the first scene rendering image and the second scene rendering image onto the impact-resistant projection screen.

[0014] The impact-resistant projection screen is used to be placed in front of the first high-speed camera, the second high-speed camera and the high-definition projector to block the thrown simulated hand grenade and to display the first scene rendering image and the second scene rendering image projected by the high-definition projector.

[0015] As an optional implementation, in the first aspect of the present invention, the fusion calculation module includes a data fusion unit, a filtering and smoothing unit, a data fitting unit, and a landing point calculation unit;

[0016] The data fusion unit is data-connected to the first high-speed camera, the second high-speed camera, and the filtering and smoothing unit. It is used to perform fusion processing on the first image sequence and the second image sequence to obtain a three-dimensional coordinate sequence and an acquisition time sequence, and send them to the filtering and smoothing unit.

[0017] The filtering and smoothing unit is data-connected to the data fitting unit and the landing point calculation unit. It is used to process the acquired time series and the three-dimensional coordinate series to obtain a set of filtered time series and smoothed series, and send the set of filtered time series and smoothed series to the data fitting unit and the set of filtered time series to the landing point calculation unit.

[0018] The set of smoothed sequences includes a first smoothed sequence, a second smoothed sequence, and a third smoothed sequence;

[0019] The data fitting unit is connected to the landing point calculation unit and is used to perform fitting processing based on the filtered time series and smoothed series set to obtain a set of fitting curves, which are then sent to the landing point calculation unit; the set of fitting curves includes a first fitting curve, a second fitting curve, a third fitting curve, and a fourth fitting curve.

[0020] The landing point calculation unit is connected to the scene rendering module for calculating the landing point coordinates based on the filtered time series and the set of fitted curves, and then sending them to the scene rendering module.

[0021] A second aspect of this invention discloses an indoor grenade throwing simulation method, the method comprising:

[0022] S1, the first high-speed camera, the second high-speed camera, and the high-definition projector are all positioned in front of the impact-resistant projection screen; the distance between the first high-speed camera and the second high-speed camera and the impact-resistant projection screen is a preset first distance;

[0023] S2, calibrate the first high-speed camera and the second high-speed camera to obtain the calibration parameter list; the calibration parameter list includes a first intrinsic parameter matrix, a first distortion parameter, a second intrinsic parameter matrix, a second distortion parameter, a rotation matrix, and a translation matrix;

[0024] S3, using the scene rendering module, based on the preset virtual 3D environment and target throwing point coordinates, obtain the first virtual environment and the first scene rendering image; using the high-definition projector, project the first scene rendering image onto the impact-resistant projection screen;

[0025] S4, using the first high-speed camera and the second high-speed camera to synchronously acquire data, the first image sequence and the second image sequence are obtained respectively;

[0026] S5, using the fusion calculation module, based on the calibration parameter list, the first image sequence and the second image sequence are processed to obtain the landing point coordinates;

[0027] S6, using the scene rendering module, based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, obtain the second scene rendering image; using the high-definition projector, project the second scene rendering image onto the impact-resistant projection screen.

[0028] As an optional implementation, in a second aspect of the present invention, the step of processing the first image sequence, the second image sequence, and the acquisition time sequence based on the calibration parameter list to obtain the landing point coordinates includes:

[0029] S51, using the data fusion unit, based on the calibration parameter list, the first image sequence and the second image sequence are fused to obtain the three-dimensional coordinate sequence and the acquisition time sequence;

[0030] S52, using the filtering and smoothing unit, the acquired time series and the three-dimensional coordinate series are processed to obtain the set of the filtered time series and the smoothed sequence;

[0031] S53, using the data fitting unit, the set of filtered time series and smoothed series is fitted to obtain the set of fitted curves;

[0032] The set of fitted curves includes a first fitted curve, a second fitted curve, a third fitted curve, and a fourth fitted curve;

[0033] S54, using the landing point calculation unit, the filtered time series and the set of fitted curves are processed to obtain the landing point coordinates; the landing point coordinates include the landing point offset and the landing point depth.

[0034] As an optional implementation, in a second aspect of the present invention, the step of fusing the first image sequence and the second image sequence based on the calibration parameter list to obtain a three-dimensional coordinate sequence and an acquisition time sequence includes:

[0035] S511, initialize the intermediate coordinate sequence to an empty sequence, and initialize the loop count i to 0;

[0036] S512, process the calibration parameter list, the i-th first image in the first image sequence, and the i-th second image in the second image sequence to obtain three-dimensional coordinates;

[0037] S513, increment the loop count i by 1;

[0038] S514, insert the three-dimensional coordinates at the end of the intermediate coordinate sequence;

[0039] S515, repeat M to execute S512~S514;

[0040] S516, determine the three-dimensional coordinate sequence as the intermediate coordinate sequence;

[0041] S517, Obtain the timestamp of the first image sequence or the second image sequence to obtain the acquisition time sequence.

[0042] As an optional implementation, in the second aspect of the present invention, the processing of the acquired time series and the three-dimensional coordinate sequence to obtain a set of filtered time series and smoothed sequences includes:

[0043] S521. Preset a second distance and a third distance; the first distance, the second distance, and the third distance increase sequentially;

[0044] S522. Based on the second distance and the third distance, the three-dimensional coordinate sequence and the acquisition time sequence are filtered to obtain a three-dimensional filtered sequence and the filtered time sequence.

[0045] S523. The three-dimensional screening sequence is smoothed to obtain a three-dimensional smoothed sequence;

[0046] S524. The three-dimensional smooth sequence is split to obtain the smooth sequence set.

[0047] As an optional implementation, in a second aspect of the present invention, the step of filtering the three-dimensional coordinate sequence and the acquisition time sequence based on the second distance and the third distance to obtain a three-dimensional filtered sequence and the filtered time sequence includes:

[0048] S5221. Obtain the index of the three-dimensional coordinate whose z-component is closest to the second distance in the three-dimensional coordinate sequence, and obtain the first index;

[0049] S5222. Obtain the index of the three-dimensional coordinate whose z-component is closest to the third distance in the three-dimensional coordinate sequence, and obtain the second index;

[0050] S5223. Obtain the three-dimensional coordinates in the three-dimensional coordinate sequence that are numbered between the first sequence number and the second sequence number to obtain a three-dimensional filtering sequence;

[0051] S5224. Obtain the collection time numbered between the first sequence number and the second sequence number in the collection time series to obtain the filtered time series.

[0052] As an optional implementation, in a second aspect of the present invention, the set of filtered time series and smoothed series is fitted to obtain a set of fitted curves, including:

[0053] S531. Perform a first fitting process on the first smoothed sequence and the filtered time series to obtain the first fitting curve;

[0054] S532. Perform a second fitting process on the second smoothed sequence and the filtered time series to obtain the second fitting curve;

[0055] S533. Perform a third fitting process on the third smoothed sequence and the filtered time series to obtain the third fitting curve;

[0056] S534. Perform a fourth fitting process on the first smoothed sequence and the third smoothed sequence to obtain the fourth fitting curve.

[0057] As an optional implementation, in the second aspect of the present invention, processing the selected time series and the set of fitted curves to obtain the landing point coordinates includes:

[0058] S541, Obtain the first and last elements of the filtered time series respectively to obtain the first time and the second time;

[0059] S542. Using the analysis time calculation model, process the first time and the second time to obtain the analysis time;

[0060] The analysis time calculation model is as follows:

[0061] t = w(t2 - t1) + t1

[0062] In the formula, t, t1, and t2 are the analysis time, the first time, and the second time, respectively, and w is a preset analysis coefficient;

[0063] S543. Calculate the slopes of the first fitting curve, the third fitting curve, the second fitting curve, and the fourth fitting curve at the analysis time to obtain the first velocity, the second velocity, the third velocity, and the pitch slope.

[0064] S544. Using the initial motion parameter calculation model, process the first velocity, the second velocity, and the pitch slope to obtain the initial motion parameters; the initial motion parameters include forward velocity and pitch angle;

[0065] The initial motion parameter calculation model is as follows:

[0066]

[0067] In the formula, v0, v1 and v2 are the forward velocity, the first velocity and the second velocity, respectively, and k0 and β are the pitch slope and the pitch angle, respectively.

[0068] S545. The forward speed, pitch angle, pitch slope and first distance are processed to obtain the landing depth;

[0069] S546, Based on the second fitting curve, the third fitting curve, and the landing point depth, obtain the landing point offset.

[0070] As an optional implementation, in a second aspect of the present invention, processing the forward velocity, the pitch angle, the pitch slope, and the first distance to obtain the landing depth includes:

[0071] S5451, using the distance parameter value calculation model, the forward speed and the pitch angle are processed to obtain the distance parameter value;

[0072] The distance parameter value calculation model is as follows:

[0073]

[0074] In the formula, k1 is the distance parameter value, g is the gravitational acceleration, v0 is the forward velocity, and β is the pitch angle;

[0075] S5452, using the landing depth calculation model, the pitch slope, the distance parameter value, and the first distance are processed to obtain the landing depth;

[0076] The landing depth calculation model is as follows:

[0077]

[0078] In the formula, d is the landing depth, H is the first distance, and k0 is the pitch slope.

[0079] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0080] (1) The first scene rendering image and the second scene rendering image are projected onto the impact-resistant projection screen through a high-definition projector, so that the trainees can avoid wearing heavy virtual reality equipment, and the virtual scene can still be accurately presented after the simulated grenade hits the impact-resistant projection screen, thus improving the realism of the training.

[0081] (2) The first high-speed camera and the second high-speed camera are used to acquire images simultaneously to obtain the first image sequence, the second image sequence and the acquisition time sequence. After fusion processing, the three-dimensional coordinate sequence is obtained, which can accurately obtain the three-dimensional trajectory of the simulated grenade in the initial stage after the grenade is thrown.

[0082] (3) Based on the second distance and the third distance, the three-dimensional coordinate sequence and the acquisition time sequence are filtered to obtain the three-dimensional filtered sequence and the filtered time sequence. The most accurate part of the binocular vision recognition result can be extracted, thereby improving the accuracy of subsequent throwing trajectory prediction. The three-dimensional filtered sequence is smoothed to remove the spikes in the data and further improve the prediction accuracy.

[0083] (4) By fitting the set of selected time series and smoothed series, a set of fitted curves is obtained. The set of selected time series and fitted curves is then processed to obtain the landing coordinates. Thus, in the case of a simulated grenade being blocked by an anti-blow curtain, the accurate landing coordinates are obtained through trajectory prediction, thereby overcoming the limitations of indoor training venues and improving realism and accuracy.

[0084] (5) Using simulated grenades instead of real grenades and using anti-blowout curtains to block the thrown simulated grenades improved the safety of the training process and avoided the impact on surrounding facilities. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0086] Figure 1 This is a schematic diagram of the structure of an indoor grenade throwing simulation system proposed in this invention.

[0087] Figure 2 This is a schematic diagram of the fusion calculation module of an indoor grenade throwing simulation system proposed in this invention.

[0088] Figure 3 This is a flowchart illustrating an indoor grenade throwing simulation method proposed in this invention. Detailed Implementation

[0089] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0091] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "data connection" should be interpreted broadly. For example, they can refer to a fixed data connection, a detachable data connection, or an integral data connection; they can refer to a mechanical data connection or an electrical data connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0092] Example 1

[0093] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the structure of an indoor grenade throwing simulation system disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the fusion calculation module of the indoor grenade throwing simulation system disclosed in an embodiment of the present invention. Figure 1 The described indoor grenade throwing simulation system is applied in the field of military training assessment, such as indoor grenade throwing training, etc., and the embodiments of the present invention are not limited thereto. Figure 1 As shown, the indoor grenade throwing simulation system includes a first high-speed camera, a second high-speed camera, a high-definition projector, a fusion computing module, a scene rendering module, an impact-resistant projection screen, and a simulated grenade.

[0094] Both the first high-speed camera and the second high-speed camera are data-connected to the fusion computing module and are used to be set in front of the impact-resistant projection screen to synchronously collect images, acquire the first image sequence and the second image sequence respectively, and send them to the fusion computing module.

[0095] Preferably, the operating parameters of the first high-speed camera and the second high-speed camera are exactly the same;

[0096] It should be noted that since the first high-speed camera and the second high-speed camera acquire data synchronously, the timestamps of the first image sequence and the second image sequence are completely consistent, and both are captured at the same time each time.

[0097] The fusion computing module is data-connected to the scene rendering module and is used to process the first image sequence and the second image sequence based on the calibration parameter list obtained from the calibration to obtain the landing point coordinates.

[0098] The scene rendering module is connected to the high-definition projector and is used to obtain a first virtual environment and a first scene rendering image based on the preset virtual three-dimensional environment and target throwing point coordinates, and to obtain a second scene rendering image based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, and to send the first scene rendering image and the second scene rendering image to the high-definition projector.

[0099] The high-definition projector is used to be placed in front of the impact-resistant projection screen to project the first scene rendering image and the second scene rendering image onto the impact-resistant projection screen.

[0100] The impact-resistant projection screen is used to be placed in front of the first high-speed camera, the second high-speed camera and the high-definition projector to block the thrown simulated hand grenade and to display the first scene rendering image and the second scene rendering image projected by the high-definition projector.

[0101] It should be noted that the aforementioned impact-resistant projection screen is made of a soft, impact-resistant material, ensuring that it will not deform, detach, or affect the projection effect after being hit by a thrown simulated grenade. Furthermore, during training, participants throw simulated grenades at the impact-resistant projection screen from a pre-designated throwing area, based on the target throwing point indicated by the first scene rendered image on the screen.

[0102] Preferably, the aforementioned impact-resistant projection screen is white.

[0103] Preferably, the size and weight of the simulated hand grenade are consistent with those of a real hand grenade.

[0104] In an optional embodiment, such as Figure 2 As shown, the above-mentioned fusion calculation module includes a data fusion unit, a filtering and smoothing unit, a data fitting unit, and a landing point calculation unit;

[0105] The data fusion unit is data-connected to the first high-speed camera, the second high-speed camera, and the filtering and smoothing unit. It is used to perform fusion processing on the first image sequence and the second image sequence to obtain a three-dimensional coordinate sequence and an acquisition time sequence, and send them to the filtering and smoothing unit.

[0106] The filtering and smoothing unit is data-connected to the data fitting unit and the landing point calculation unit. It is used to process the acquired time series and the three-dimensional coordinate series to obtain a set of filtered time series and smoothed series, and send the set of filtered time series and smoothed series to the data fitting unit and the set of filtered time series to the landing point calculation unit.

[0107] The set of smoothed sequences includes a first smoothed sequence, a second smoothed sequence, and a third smoothed sequence;

[0108] The data fitting unit is connected to the landing point calculation unit and is used to perform fitting processing based on the filtered time series and smoothed series set to obtain a set of fitting curves, which are then sent to the landing point calculation unit; the set of fitting curves includes a first fitting curve, a second fitting curve, a third fitting curve, and a fourth fitting curve.

[0109] The landing point calculation unit is connected to the scene rendering module for calculating the landing point coordinates based on the filtered time series and the set of fitted curves, and then sending them to the scene rendering module.

[0110] In another alternative embodiment, the system further includes a first gimbal and a second gimbal:

[0111] The first high-speed camera is fixed to the first gimbal by screws;

[0112] The second high-speed camera is fixed to the second gimbal by screws.

[0113] It is evident that implementing the indoor grenade throwing simulation system described in the embodiments of the present invention is beneficial to improving the realism, safety, and accuracy of grenade throwing training.

[0114] Example 2

[0115] Please see Figure 3 , Figure 3 This is a flowchart illustrating an indoor grenade throwing simulation method disclosed in an embodiment of the present invention. Figure 3 The described indoor grenade throwing simulation method is applied in the field of military training assessment, such as indoor grenade throwing training, etc., and the embodiments of this invention are not limited thereto. Figure 3 As shown, the indoor grenade throwing simulation method includes:

[0116] S1, the first high-speed camera, the second high-speed camera, and the high-definition projector are all positioned in front of the impact-resistant projection screen; the distance between the first high-speed camera and the second high-speed camera and the impact-resistant projection screen is a preset first distance;

[0117] Preferably, the first high-speed camera and the second high-speed camera are symmetrically arranged on both sides of the high-definition projector;

[0118] Preferably, the aforementioned impact-resistant projection screen has a throwing area in the center of the front;

[0119] Preferably, the aforementioned high-definition projector is suspended in front of an impact-resistant projection screen;

[0120] S2, calibrate the first high-speed camera and the second high-speed camera to obtain the calibration parameter list; the calibration parameter list includes a first intrinsic parameter matrix, a first distortion parameter, a second intrinsic parameter matrix, a second distortion parameter, a rotation matrix, and a translation matrix;

[0121] Optionally, the above calibration is achieved using Zhang Zhengyou's calibration method;

[0122] S3, using the scene rendering module, based on the preset virtual 3D environment and target throwing point coordinates, obtain the first virtual environment and the first scene rendering image; using the high-definition projector, project the first scene rendering image onto the impact-resistant projection screen;

[0123] It should be noted that the aforementioned virtual 3D environment is a 3D model built based on the real training scene. It includes a 3D terrain model and a 3D building model, which are used to simulate the terrain and building distribution of the training site, respectively.

[0124] S4, using the first high-speed camera and the second high-speed camera to synchronously acquire data, the first image sequence and the second image sequence are obtained respectively;

[0125] It should be noted that the first image sequence and the second image sequence mentioned above each include the same number of first images and second images, and each first image and each second image is accompanied by a timestamp; the timestamp is the acquisition time of the corresponding first image or second image.

[0126] S5, using the fusion calculation module, based on the calibration parameter list, the first image sequence and the second image sequence are processed to obtain the landing point coordinates;

[0127] S6, using the scene rendering module, based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, obtain the second scene rendering image; using the high-definition projector, project the second scene rendering image onto the impact-resistant projection screen.

[0128] As can be seen, the indoor grenade throwing simulation method described in this embodiment of the invention projects the first and second scene rendering images onto an impact-resistant projection screen using a high-definition projector. This allows trainees to avoid wearing heavy virtual reality equipment, and the virtual scene can still be accurately presented after the simulated grenade hits the impact-resistant projection screen, improving the realism of the training. Furthermore, by simultaneously acquiring data using a first and second high-speed camera, a first image sequence, a second image sequence, and an acquisition time sequence are obtained. After fusion processing, a three-dimensional coordinate sequence is obtained, which can accurately capture the initial three-dimensional trajectory of the simulated grenade after it is thrown. Simultaneously, using a simulated grenade instead of a real grenade and using an impact-resistant screen to block the thrown simulated grenade improves the safety of the training process and avoids impact on surrounding facilities.

[0129] In another optional embodiment, the above-mentioned acquisition of the first virtual environment and the first scene rendering image based on the preset virtual 3D environment and target throwing point coordinates includes:

[0130] S31, In the virtual three-dimensional environment, the coordinates of the target throwing point are marked to obtain the first virtual environment;

[0131] It should be noted that the above-mentioned marking of the preset target throwing point coordinates is achieved by placing a preset first marker model at the target throwing point coordinates in the three-dimensional environment; the first marker model can be a 3D model such as a flag or a three-dimensional arrow.

[0132] S32, render the first virtual environment from a preset angle to obtain the first rendered image.

[0133] In another optional embodiment, the above-mentioned processing of the first image sequence, the second image sequence, and the acquisition time sequence based on the calibration parameter list to obtain the landing point coordinates includes:

[0134] S51, using the data fusion unit, based on the calibration parameter list, the first image sequence and the second image sequence are fused to obtain the three-dimensional coordinate sequence and the acquisition time sequence;

[0135] S52, using the filtering and smoothing unit, the acquired time series and the three-dimensional coordinate series are processed to obtain the set of the filtered time series and the smoothed sequence;

[0136] S53, using the data fitting unit, the set of filtered time series and smoothed series is fitted to obtain the set of fitted curves;

[0137] The set of fitted curves includes a first fitted curve, a second fitted curve, a third fitted curve, and a fourth fitted curve;

[0138] S54, using the landing point calculation unit, the filtered time series and the set of fitted curves are processed to obtain the landing point coordinates; the landing point coordinates include the landing point offset and the landing point depth.

[0139] In another optional embodiment, the above-mentioned fusion processing of the first image sequence and the second image sequence based on the calibration parameter list to obtain a three-dimensional coordinate sequence and an acquisition time sequence includes:

[0140] S511, initialize the intermediate coordinate sequence to an empty sequence, and initialize the loop count i to 0;

[0141] S512, process the calibration parameter list, the i-th first image in the first image sequence, and the i-th second image in the second image sequence to obtain three-dimensional coordinates;

[0142] S513, increment the loop count i by 1;

[0143] S514, insert the three-dimensional coordinates at the end of the intermediate coordinate sequence;

[0144] S515, repeat M to execute S512~S514;

[0145] S516, determine the three-dimensional coordinate sequence as the intermediate coordinate sequence;

[0146] S517, Obtain the timestamp of the first image sequence or the second image sequence to obtain the acquisition time sequence.

[0147] In another optional embodiment, the above-described processing of the calibration parameter list, the i-th first image in the first image sequence, and the i-th second image in the second image sequence to obtain three-dimensional coordinates includes:

[0148] S511, based on the calibration parameter list, perform stereo correction operation on the i-th first image and the i-th second image to obtain a first corrected image, a second corrected image and a mapping matrix;

[0149] Optionally, the above stereo calibration operation is implemented using the OpenCV library, and the relevant code is as follows:

[0150] left_K = config.cam_matrix_left # First intrinsic parameter matrix

[0151] right_K = config.cam_matrix_right # First distortion parameter

[0152] left_distortion = config.distortion_l # Second intrinsic parameter matrix

[0153] right_distortion = config.distortion_r # Second distortion parameter

[0154] R = config.R # Rotation matrix

[0155] T = config.T # Translation matrix

[0156] R1,R2,P1,P2,Q,roi1,roi2=cv2.stereoRectify(left_K,left_distortion,right_K,right_distortion,(width,height),R,T,alpha=0) #img1 and img2 are the first and second images, width and height are their width and height, and Q is the mapping matrix

[0157] map1x,map1y=cv2.initUndistortRectifyMap(left_K,left_distortion,R1,P1,(width,height),cv2.CV_

[0158] 32FC1)

[0159] map2x,map2y=cv2.initUndistortRectifyMap(right_K,right_distortion,R2,P2,(width,height),cv2.C

[0160] V_32FC1)

[0161] `rectified_img1 = cv2.remap(img1, map1x, map1y, cv2.INTER_AREA)` # First corrected image `rectified_img2 = cv2.remap(img2, map2x, map2y, cv2.INTER_AREA)` # Second corrected image

[0162] S512, Perform stereo matching processing on the first corrected image and the second corrected image to obtain a stereo matching image;

[0163] Optionally, the above stereo matching process is implemented using the OpenCV library, and the relevant code is as follows:

[0164]

[0165] left_matcher=cv2.StereoSGBM_create(**paraml)

[0166] size=(left_image.shape[1],left_image.shape[0])

[0167] disparity_left=left_matcher.compute(left_image,right_image)

[0168] `trueDisp_left = disparity_left.astype(np.float32) / 16` # Stereo matching image

[0169] S513, Based on the mapping matrix, perform depth calculation processing on the stereo matching image to obtain a depth matrix;

[0170] It should be noted that the depth matrix mentioned above is a P×Q×3 matrix; P and Q are the number of rows and columns of the stereo matching image, respectively.

[0171] Optionally, the above depth calculation processing is implemented using the OpenCV library, and the relevant code is as follows:

[0172] points_3d=cv2.reprojectImageTo3D(trueDisp_left,Q)

[0173] S514, Using the YOLOv5 model, the stereo matching images are processed to obtain the recognition regions;

[0174] S515, calculate the row and column coordinates of the center pixel of the recognition area in the stereo matching image to obtain the first coordinate n1 and the second coordinate n2;

[0175] S516, Obtain the sub-vector in the n1-th row and n2-th column of the depth matrix to obtain the three-dimensional coordinates.

[0176] It should be noted that the x, y, and z components of the above three-dimensional coordinates represent spatial offset coordinates, spatial height coordinates, and spatial depth coordinates, respectively.

[0177] In another optional embodiment, the above-described processing of the acquired time series and the three-dimensional coordinate sequence to obtain a set of filtered time series and smoothed sequences includes:

[0178] S521. Preset a second distance and a third distance; the first distance, the second distance, and the third distance increase sequentially;

[0179] S522. Based on the second distance and the third distance, the three-dimensional coordinate sequence and the acquisition time sequence are filtered to obtain a three-dimensional filtered sequence and the filtered time sequence.

[0180] S523. The three-dimensional screening sequence is smoothed to obtain a three-dimensional smoothed sequence;

[0181] S524. The three-dimensional smooth sequence is split to obtain the smooth sequence set.

[0182] It should be noted that the first smoothing sequence, the second smoothing sequence, and the third smoothing sequence mentioned above are sequences composed of the y, x, and z components of all three-dimensional coordinates in the three-dimensional smoothing sequence, respectively.

[0183] As can be seen, by implementing the indoor grenade throwing simulation method described in this embodiment of the invention, the three-dimensional coordinate sequence and the acquisition time sequence are filtered based on the second distance and the third distance to obtain the three-dimensional filtered sequence and the filtered time sequence. This can extract the most accurate part of the binocular vision recognition result, thereby improving the accuracy of subsequent throwing trajectory prediction. Smoothing the three-dimensional filtered sequence can remove glitch in the data and further improve the prediction accuracy.

[0184] In another optional embodiment, the above-described filtering process, based on the second distance and the third distance, of the three-dimensional coordinate sequence and the acquisition time series to obtain a three-dimensional filtered sequence and the filtered time series includes:

[0185] S5221. Obtain the index of the three-dimensional coordinate whose z-component is closest to the second distance in the three-dimensional coordinate sequence, and obtain the first index;

[0186] S5222. Obtain the index of the three-dimensional coordinate whose z-component is closest to the third distance in the three-dimensional coordinate sequence, and obtain the second index;

[0187] S5223. Obtain the three-dimensional coordinates in the three-dimensional coordinate sequence that are numbered between the first sequence number and the second sequence number to obtain a three-dimensional filtering sequence;

[0188] S5224. Obtain the collection time numbered between the first sequence number and the second sequence number in the collection time series to obtain the filtered time series.

[0189] It can be seen that the z-component, i.e. the depth, of the three-dimensional filtered sequence obtained after filtering the above three-dimensional coordinate sequence is within the optimal recognition distance of the first high-speed camera and the second high-speed camera.

[0190] In another optional embodiment, the above-described fitting process of the selected time series and smoothed series set to obtain a set of fitted curves includes:

[0191] S531. Perform a first fitting process on the first smoothed sequence and the filtered time series to obtain the first fitting curve;

[0192] Preferably, the first fitting process described above is a quadratic polynomial fitting, and the first fitting curve described above is a quadratic curve.

[0193] S532. Perform a second fitting process on the second smoothed sequence and the filtered time series to obtain the second fitting curve;

[0194] Preferably, the second fitting process described above is a linear fitting, and the second fitting curve described above is a straight line;

[0195] S533. Perform a third fitting process on the third smoothed sequence and the filtered time series to obtain the third fitting curve;

[0196] Preferably, the third fitting process described above is a quadratic polynomial fitting, and the third fitting curve described above is a quadratic curve.

[0197] S534. Perform a fourth fitting process on the first smoothed sequence and the third smoothed sequence to obtain the fourth fitting curve.

[0198] Preferably, the fourth fitting process described above is a quadratic polynomial fitting, and the fourth fitting curve described above is a quadratic curve.

[0199] It should be noted that in the first, second, and third fitting processes described above, the time series was used as the x-axis. Therefore, the x-axis of the first, second, and third fitting curves is the acquisition time.

[0200] As can be seen, by implementing the indoor grenade throwing simulation method described in the embodiments of the present invention, by fitting the selected time series and smoothed series sets to obtain a set of fitted curves, and by processing the selected time series and the set of fitted curves to obtain the landing point coordinates, the accurate landing point coordinates can be obtained by trajectory prediction when the grenade is blocked by the anti-impact curtain, thereby overcoming the limitations of indoor training venues and improving realism and accuracy.

[0201] In another optional embodiment, the above-described processing of the selected time series and the set of fitted curves to obtain the landing point coordinates includes:

[0202] S541, Obtain the first and last elements of the filtered time series respectively to obtain the first time and the second time;

[0203] S542. Using the analysis time calculation model, process the first time and the second time to obtain the analysis time;

[0204] The analysis time calculation model is as follows:

[0205] t = w(t2 - t1) + t1

[0206] In the formula, t, t1, and t2 are the analysis time, the first time, and the second time, respectively, and w is a preset analysis coefficient;

[0207] Optionally, the above analytical coefficients are greater than 0 and less than 1;

[0208] Preferably, the above analytical coefficient is 0.5;

[0209] S543. Calculate the slopes of the first fitting curve, the third fitting curve, the second fitting curve, and the fourth fitting curve at the analysis time to obtain the first velocity, the second velocity, the third velocity, and the pitch slope.

[0210] S544. Using the initial motion parameter calculation model, process the first velocity, the second velocity, and the pitch slope to obtain the initial motion parameters; the initial motion parameters include forward velocity and pitch angle;

[0211] The initial motion parameter calculation model is as follows:

[0212]

[0213] In the formula, v0, v1 and v2 are the forward velocity, the first velocity and the second velocity, respectively, and k0 and β are the pitch slope and the pitch angle, respectively.

[0214] S545. The forward speed, pitch angle, pitch slope and first distance are processed to obtain the landing depth;

[0215] S546, Based on the second fitting curve, the third fitting curve, and the landing point depth, obtain the landing point offset.

[0216] In another optional embodiment, the above-described processing of the forward velocity, the pitch angle, the pitch slope, and the first distance to obtain the landing depth includes:

[0217] S5451, using the distance parameter value calculation model, the forward speed and the pitch angle are processed to obtain the distance parameter value;

[0218] The distance parameter value calculation model is as follows:

[0219]

[0220] In the formula, k1 is the distance parameter value, g is the gravitational acceleration, v0 is the forward velocity, and β is the pitch angle;

[0221] S5452, using the landing depth calculation model, the pitch slope, the distance parameter value, and the first distance are processed to obtain the landing depth;

[0222] The landing depth calculation model is as follows:

[0223]

[0224] In the formula, d is the landing depth, H is the first distance, and k0 is the pitch slope.

[0225] In yet another optional embodiment, obtaining the landing point offset based on the second fitted curve, the third fitted curve, and the landing point depth includes:

[0226] S5461, calculate the acquisition time when the third fitted curve value is the landing depth, and obtain the landing time.

[0227] S5462, calculate the value of the second fitted curve at the landing time to obtain the landing offset.

[0228] In another optional embodiment, obtaining the second scene rendering image based on the first virtual environment, the target throwing point coordinates, and the landing point coordinates includes:

[0229] S61, In the first virtual environment, the coordinates of the landing point are marked to obtain the second virtual environment;

[0230] Optionally, the above-mentioned marking of the landing point coordinates is achieved by placing a preset second marker model at the location of the landing point coordinates in the three-dimensional environment; the second marker model can be a 3D model of an object such as a flag or a three-dimensional arrow.

[0231] Preferably, the first marker model and the second marker model are different in shape, color or size, so as to facilitate differentiation;

[0232] S62, render the second virtual environment from a preset angle to obtain an intermediate rendered image;

[0233] S63, calculate the distance between the target throwing point coordinates and the landing point coordinates to obtain the landing point distance;

[0234] S64, the landing point distance is converted into text information and added to the intermediate rendering image to obtain the second scene rendering image.

[0235] It is evident that implementing the indoor grenade throwing simulation method described in the embodiments of the present invention is beneficial to improving the realism, safety, and accuracy of grenade throwing training.

[0236] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0237] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0238] Finally, it should be noted that the indoor grenade throwing simulation system and method disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for simulating indoor grenade throwing, characterized in that, An indoor grenade throwing simulation system is applied, the system including a first high-speed camera, a second high-speed camera, a high-definition projector, a fusion computing module, a scene rendering module, an impact-resistant projection screen, and a simulated grenade; Both the first high-speed camera and the second high-speed camera are data-connected to the fusion computing module and are used to be set in front of the impact-resistant projection screen to synchronously collect images, acquire the first image sequence and the second image sequence respectively, and send them to the fusion computing module. The fusion computing module is data-connected to the scene rendering module and is used to process the first image sequence and the second image sequence based on the calibration parameter list obtained from the calibration to obtain the landing point coordinates. The scene rendering module is connected to the high-definition projector and is used to obtain a first virtual environment and a first scene rendering image based on a preset virtual three-dimensional environment and target throwing point coordinates, and to obtain a second scene rendering image based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, and to send the first scene rendering image and the second scene rendering image to the high-definition projector. The high-definition projector is used to be placed in front of the impact-resistant projection screen to project the first scene rendering image and the second scene rendering image onto the impact-resistant projection screen. The impact-resistant projection screen is used to be placed in front of the first high-speed camera, the second high-speed camera and the high-definition projector to block the thrown simulated hand grenade and to display the first scene rendering image and the second scene rendering image projected by the high-definition projector. The fusion computing module includes a data fusion unit, a filtering and smoothing unit, a data fitting unit, and a landing point calculation unit; The data fusion unit is data-connected to the first high-speed camera, the second high-speed camera, and the filtering and smoothing unit. It is used to perform fusion processing on the first image sequence and the second image sequence to obtain a three-dimensional coordinate sequence and an acquisition time sequence, and send them to the filtering and smoothing unit. The filtering and smoothing unit is data-connected to the data fitting unit and the landing point calculation unit. It is used to process the acquired time series and the three-dimensional coordinate series to obtain a set of filtered time series and smoothed series, and send the set of filtered time series and smoothed series to the data fitting unit and the set of filtered time series to the landing point calculation unit. The set of smoothed sequences includes a first smoothed sequence, a second smoothed sequence, and a third smoothed sequence; The data fitting unit is connected to the landing point calculation unit and is used to perform fitting processing based on the filtered time series and smoothed series set to obtain a set of fitting curves, which are then sent to the landing point calculation unit; the set of fitting curves includes a first fitting curve, a second fitting curve, a third fitting curve, and a fourth fitting curve. The landing point calculation unit is connected to the scene rendering module for calculating the landing point coordinates based on the filtered time series and the set of fitted curves, and sending them to the scene rendering module. The method includes: S1, the first high-speed camera, the second high-speed camera, and the high-definition projector are all positioned in front of the impact-resistant projection screen; the distance between the first high-speed camera and the second high-speed camera and the impact-resistant projection screen is a preset first distance; S2, calibrate the first high-speed camera and the second high-speed camera to obtain a calibration parameter list; the calibration parameter list includes a first intrinsic parameter matrix, a first distortion parameter, a second intrinsic parameter matrix, a second distortion parameter, a rotation matrix, and a translation matrix; S3, using the scene rendering module, based on the preset virtual 3D environment and target throwing point coordinates, obtain the first virtual environment and the first scene rendering image; using the high-definition projector, project the first scene rendering image onto the impact-resistant projection screen; S4, using the first high-speed camera and the second high-speed camera to synchronously acquire images, respectively, to obtain the first image sequence and the second image sequence; S5, using the fusion calculation module, based on the calibration parameter list, the first image sequence and the second image sequence are processed to obtain the landing point coordinates; S6, using the scene rendering module, based on the first virtual environment, the target throwing point coordinates and the landing point coordinates, obtain a second scene rendering image; using the high-definition projector, project the second scene rendering image onto the impact-resistant projection screen; S5 includes: S51, using the data fusion unit, based on the calibration parameter list, the first image sequence and the second image sequence are fused to obtain a three-dimensional coordinate sequence and an acquisition time sequence; S52, using the filtering and smoothing unit, the acquired time series and the three-dimensional coordinate series are processed to obtain the set of the filtered time series and the smoothed sequence; S53, using the data fitting unit, the set of filtered time series and smoothed series is fitted to obtain a set of fitted curves; The set of fitted curves includes a first fitted curve, a second fitted curve, a third fitted curve, and a fourth fitted curve; S54, using the landing point calculation unit, the filtered time series and the set of fitted curves are processed to obtain the landing point coordinates; the landing point coordinates include landing point offset and landing point depth; S52 includes: S521. Preset a second distance and a third distance; the first distance, the second distance, and the third distance increase sequentially; S522. Based on the second distance and the third distance, the three-dimensional coordinate sequence and the acquisition time sequence are filtered to obtain a three-dimensional filtered sequence and the filtered time sequence. S523. The three-dimensional screening sequence is smoothed to obtain a three-dimensional smoothed sequence; S524. The three-dimensional smooth sequence is split to obtain the smooth sequence set.

2. The indoor grenade throwing simulation method according to claim 1, characterized in that, The process of fusing the first image sequence and the second image sequence based on the calibration parameter list to obtain a three-dimensional coordinate sequence and an acquisition time sequence includes: S511, initialize the intermediate coordinate sequence to an empty sequence, and set the loop count. i Initialize to 0; S512, for the calibration parameter list, the first image sequence in the first image sequence... i The first image, and the second image sequence in the second image sequence i The second image is processed to obtain three-dimensional coordinates; S513, the number of loops i Add 1; S514, insert the three-dimensional coordinates at the end of the intermediate coordinate sequence; S515, repeat S512~S514; S516, determine the three-dimensional coordinate sequence as the intermediate coordinate sequence; S517, Obtain the timestamp of the first image sequence or the second image sequence to obtain the acquisition time sequence.

3. The indoor grenade throwing simulation method according to claim 1, characterized in that, The step of filtering the three-dimensional coordinate sequence and the acquisition time sequence based on the second distance and the third distance to obtain a three-dimensional filtered sequence and the filtered time sequence includes: S5221. Obtain the three-dimensional coordinate sequence. z The first index is obtained by finding the index of the three-dimensional coordinate that is closest to the second distance of the component; the three-dimensional coordinate... z Components represent spatial depth coordinates; S5222. Obtain the three-dimensional coordinate sequence. z The index of the three-dimensional coordinate that is closest to the third distance of the component is used to obtain the second index; S5223. Obtain the three-dimensional coordinates in the three-dimensional coordinate sequence that are numbered between the first sequence number and the second sequence number to obtain a three-dimensional filtering sequence; S5224. Obtain the collection time numbered between the first sequence number and the second sequence number in the collection time series to obtain the filtered time series.

4. The indoor grenade throwing simulation method according to claim 1, characterized in that, The process of fitting the selected time series and smoothed series sets to obtain a set of fitted curves includes: S531. Perform a first fitting process on the first smoothed sequence and the filtered time series to obtain the first fitting curve; S532. Perform a second fitting process on the second smoothed sequence and the filtered time series to obtain the second fitting curve; S533. Perform a third fitting process on the third smoothed sequence and the filtered time series to obtain the third fitting curve; S534. Perform a fourth fitting process on the first smoothed sequence and the third smoothed sequence to obtain the fourth fitting curve.

5. The indoor grenade throwing simulation method according to claim 1, characterized in that, The process of processing the selected time series and the set of fitted curves to obtain the landing point coordinates includes: S541, Obtain the first and last elements of the filtered time series respectively to obtain the first time and the second time; S542. Using the analysis time calculation model, process the first time and the second time to obtain the analysis time; The analysis time calculation model is as follows: In the formula, t , and These refer to the analysis time, the first time, and the second time, respectively. w These are the preset analysis coefficients; S543. Calculate the slopes of the first fitting curve, the third fitting curve, the second fitting curve, and the fourth fitting curve at the analysis time to obtain the first velocity, the second velocity, the third velocity, and the pitch slope. S544. Using the initial motion parameter calculation model, process the first velocity, the second velocity, and the pitch slope to obtain the initial motion parameters; the initial motion parameters include forward velocity and pitch angle; The initial motion parameter calculation model is as follows: In the formula, , and These are the forward speed, the first speed, and the second speed, respectively. and These are the pitch slope and the pitch angle, respectively. S545. The forward speed, pitch angle, pitch slope and first distance are processed to obtain the landing depth; S546, Based on the second fitting curve, the third fitting curve, and the landing point depth, obtain the landing point offset.

6. The indoor grenade throwing simulation method according to claim 5, characterized in that, The process of processing the forward velocity, the pitch angle, the pitch slope, and the first distance to obtain the landing depth includes: S5451, using the distance parameter value calculation model, the forward speed and the pitch angle are processed to obtain the distance parameter value; The distance parameter value calculation model is as follows: In the formula, The distance parameter value, g It is the acceleration due to gravity. The forward speed is... β The pitch angle is mentioned; S5452, using the landing depth calculation model, the pitch slope, the distance parameter value, and the first distance are processed to obtain the landing depth; The landing depth calculation model is as follows: In the formula, d The landing depth, H For the first distance, The pitch slope is the given value.

Images