Land battle scene data set generation method and device based on Unity engine

Abstract

The invention discloses a land battle scene data set generation method and device based on a Unity engine, and the method comprises the steps: S1, building at least one virtual local land battle battlefield environment in the Unity engine based on the Unity engine; s2, in each virtual local land battle battlefield environment, setting of a preset actual combat scene is completed, multiple preset target parameters are correspondingly set, and the multiple target parameters at least comprise environment parameters, equipment parameters and camera shooting parameters; and S3, controlling the Unity engine to run a preset actual combat scene, and generating an environment reconstruction data set. On the premise that a scene and an equipment model with relatively high authenticity can be obtained, the method has the advantages that a large amount of image data which can be used for training a neural network can be generated in a virtual land battlefield environment; the difficulty of acquiring land battlefield data in reality and the huge workload of marking massive image data are avoided, so that the efficiency of training related neural networks is improved.

Description

Technical Field

[0001] The present application relates to the field of virtual image data generation, including a method and device for generating a land battle scene dataset based on the Unity engine. Background Art

[0002] With the improvement of camera performance and the development of machine vision technology, computer vision technology is becoming increasingly widely used in the military. Battlefield situational awareness technology based on bionic vision has gradually become a research focus. This technology aims to combine inertial measurement unit (IMU) data, typical battlefield target detection and segmentation based on Mask R-CNN, and simultaneous localization and mapping (SLAM) technology to achieve autonomous positioning, environmental reconstruction, and target recognition for unmanned detection platforms. However, due to the dangerous nature of battlefield environments, obtaining training data in real-world situations is difficult and unfeasible.

[0003] In view of the above problems existing in this field, no effective solution has been proposed. Summary of the Invention

[0004] This paper addresses the problem of generating a virtual land battle scene dataset and provides a method for generating a land battle scene dataset based on the Unity engine. The method can be used to generate a target detection and segmentation dataset and an environment reconstruction dataset. This method includes the following:

[0005] 1. A method for generating a land battle scene dataset based on the Unity engine, characterized in that the method comprises the following steps:

[0006] S1: Based on the Unity engine, build at least one virtual local land battle battlefield environment in the Unity engine;

[0007] S2: In each virtual local land battle battlefield environment, complete the setting of a preset actual combat scene and set a plurality of preset target parameters accordingly, wherein the plurality of target parameters at least include environmental parameters, equipment parameters, and camera shooting parameters;

[0008] S3: Controls the Unity engine to run the preset actual combat scenario and generates a land combat scenario dataset, which includes an equipment detection and segmentation dataset and a land combat scenario environment reconstruction dataset.

[0009] 2. The method according to 1, characterized in that, based on the Unity engine, at least one virtual local land battle battlefield environment is constructed in the Unity engine, comprising:

[0010] Selecting a terrain model and a ground object model that match the virtual local land battle battlefield environment, wherein the scaling ratios of the terrain model and the ground object model are both a first preset ratio;

[0011] Screening a scene equipment model that matches the virtual local land battle battlefield environment, wherein the scaling model ratio of the scene equipment model is a second preset ratio;

[0012] Control the terrain model, ground object model and scene equipment model to be imported into the Unity engine, and adjust the position of the scene equipment model to build a virtual local land battle battlefield environment.

[0013] 3. The method according to item 1 is characterized in that, in each virtual local land battle battlefield environment, a preset actual combat scene is set, and a plurality of preset target parameters are set accordingly, including:

[0014] S3-1: Set environmental parameters by presetting Unity components. The environmental parameters include at least lighting parameters and weather parameters.

[0015] S3-2: Add a mesh collision box for each target model and adjust the label corresponding to each target model. The target model can be any of the terrain model, object model, and scene equipment model.

[0016] S3-3: Add animation driver scripts to each scene equipment model;

[0017] S3-4: Add the animation master control script to each scene equipment model and obtain the equipment prefab corresponding to the scene equipment model;

[0018] S3-5: Construct multi-angle images to generate corresponding empty object combinations, and attach corresponding control scripts and interfaces to each component in the empty object combination;

[0019] S3-6: Add a shooting drive script to the empty object combination, and set multiple driving parameters corresponding to the shooting drive script. The multiple driving parameters at least include model parameters, camera parameters, environment parameters, and data saving parameters.

[0020] 4. The method according to 3, characterized in that, in each virtual local land battle battlefield environment, after completing the preset actual combat scene setting and correspondingly setting a plurality of preset target parameters, the method further comprises:

[0021] S4-1: Set all terrain models and object models to static, and use the Unity engine's navigation tool to bake the navigation mesh corresponding to the preset actual combat scene;

[0022] S4-2: Set environmental parameters by presetting Unity components. The environmental parameters include at least lighting parameters and weather parameters.

[0023] S4-3: Import the control equipment prefab into the Unity engine, add a navigation mesh proxy component to the scene equipment model, and set at least one control parameter in the navigation mesh proxy component;

[0024] S4-4: Control the navigation mesh proxy path control script to add to the scene equipment model and arrange the path point sequence in the preset actual combat scene. The path point sequence is used to allow the scene equipment model to circulate in the preset path point group;

[0025] S4-5: Select the target scene equipment model as the shooting vehicle in the preset actual combat scene operation process, add the script that generates simulated IMU data to the target scene equipment, and set the corresponding IMU parameters;

[0026] S4-6: constructing a multi-view camera group and setting corresponding shooting parameters of the multi-view camera group, where the multi-view camera group is used to shoot the target scene equipment;

[0027] S4-7: Add transformation matrix generation script and distance data truth value script to the scene equipment model;

[0028] S4-8: Determine the shooting master control script of the preset actual combat scene and set multiple operating parameters of the shooting master control script.

[0029] 5. The method according to 4, characterized in that the Unity engine is controlled to run a preset actual combat scene and generate an environment reconstruction data set, including:

[0030] Control the navigation grid agent path control script and the shooting control script to run independently and generate images accordingly. The shooting control script calls the IMU simulation at a preset frequency during operation.

[0031] When the number of generated images reaches the preset number, the operation of the preset actual combat scenario is terminated and the target detection segmentation dataset is generated;

[0032] Based on the target detection segmentation dataset, an environment reconstruction dataset is generated.

[0033] 6. The method according to 5, characterized in that the navigation grid proxy path control script and the shooting control script are controlled to run independently and generate images accordingly, including:

[0034] S6-1: Determine whether the target scene equipment model has been shot;

[0035] S6-2: If the target scene equipment model completes shooting, destroy the target scene equipment model, and instantiate another target scene equipment model as the current target scene equipment model according to a predetermined order, and determine the animation control interface corresponding to the current target scene equipment model;

[0036] S6-3: adjusting multiple postures of the equipment model of the current target scene multiple times, controlling the multi-view camera component to perform posture shooting in each posture, and completing the posture shooting;

[0037] S6-4: After confirming that the shooting is completed in multiple postures, the current target scene equipment model is adjusted multiple times to multiple positions corresponding to the preset actual combat scene, and the shooting in multiple positions is completed;

[0038] S6-5: After completing the shooting at multiple positions, the position of the multi-view camera assembly is changed, and the positions of the camera views corresponding to the multi-view camera assembly are determined, and rays are generated for collision detection to obtain a mask image of the shooting equipment;

[0039] S6-6: Repeat the process of S6-1-S6-5 until all scene equipment models have completed shooting, exit the shooting process, and obtain all original pictures.

[0040] 7. The method according to 1, characterized in that after controlling the Unity engine to run the preset actual combat scene, the method further includes:

[0041] During the Unity engine operation, check whether there are any runtime errors and / or runtime estimation deviations.

[0042] If there are runtime errors and / or runtime estimation deviations, pause the runtime process and adjust the Unity engine and / or preset target parameters corresponding to the actual scenario.

[0043] 8. A device for generating a land battle scene dataset based on the Unity engine, characterized in that the device comprises the following parts:

[0044] A construction module, based on the Unity engine, constructs at least one virtual local land battle battlefield environment in the Unity engine;

[0045] A setting module is used to complete the preset actual combat scene setting in each virtual local land battle battlefield environment, and set a plurality of preset target parameters accordingly, wherein the plurality of target parameters at least include environmental parameters, equipment parameters and camera shooting parameters;

[0046] The generation module is used to control the Unity engine to run the preset actual combat scene and generate the environment reconstruction data set.

[0047] 9. An electronic computer device comprising a memory, a processor, a bus, and an executable program stored in the memory and run by the processor, wherein when the processor executes the program, it can implement any one of 1-6, a method for generating a land battle scene dataset based on the Unity engine.

[0048] 10. A computer-readable storage medium, characterized in that the computer-executable instructions, when executed by a computer processor, are used to execute a method for generating a land battle scene dataset based on a Unity engine as described in any one of 1-6. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. Those skilled in the art can derive other drawings based on these drawings without inventive effort. Among them:

[0050] Figure 1 A flow chart of a method for generating a land battle scene dataset based on the Unity engine provided by an embodiment of the present invention;

[0051] Figure 2 A flowchart for arranging a local land battle battlefield environment provided by an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of some scenarios and land combat equipment provided by embodiments of the present invention;

[0053] Figure 4 A flowchart of parameter settings for generating a target detection segmentation dataset provided by an embodiment of the present invention;

[0054] Figure 5 A flowchart of parameter settings for generating an environmental reconstruction dataset according to an embodiment of the present invention;

[0055] Figure 6 A flowchart of the script for generating the original image and mask image provided by an embodiment of the present invention;

[0056] Figure 7 Flowchart of the IMU simulation data generation script provided in an embodiment of the present invention;

[0057] Figure 8 Flowchart of the position and posture true value recording script provided by an embodiment of the present invention;

[0058] Figure 9 A schematic diagram of an orthophoto image of a ground scene in a target area provided by an embodiment of the present invention. Figure 10 A schematic diagram of a target detection segmentation dataset that can be generated according to an embodiment of the present invention; Figure 11 A schematic diagram of a generative environment reconstruction dataset provided by an embodiment of the present invention; Figure 12 A structural diagram of an electronic computer device provided in an embodiment of the present invention; Specific implementation methods

[0059] Unity3D is a professional 3D virtual scene development engine. It integrates numerous resource systems and offers a user-friendly editing interface, making it a powerful tool for creating virtual battlefield environments. 3D real-world modeling software (such as ContextCapture) can perform aerial triangulation, dense reconstruction, and texture mapping on a series of oblique photographic images, rapidly generating high-precision, wide-ranging 3D scene models. By importing 3D reconstructed environment models and existing military equipment models into the Unity3D engine, the basic environment of a virtual land warfare battlefield can be constructed.

[0060] Using virtual environments for battlefield simulation has long been a hot topic in the simulation field. Traditional applications primarily involve virtual military exercises and wargaming. However, this patent addresses the needs of neural network training, focusing on image data acquisition and data annotation. Specifically, the invention designs methods for generating two types of datasets.

[0061] The first type of dataset is a target detection and segmentation dataset, used to train the Mask R-CNN network for target recognition and segmentation. The generated dataset must include the original image, pixel-level instance segmentation masks, and target labels. This data requires specific targets to be captured in different background environments, with different poses, and from different distances and angles. The second type of dataset is an environment reconstruction dataset, used to test the reconstruction and position estimation performance of visual SLAM. The generated data must include the following: simulated binocular camera frame sequences, depth map sequences, simulated IMU data sequences, and viewpoint and target position sequences.

[0062] In order to make the technical solution of the present invention clearer, the present invention is further described below with reference to the accompanying drawings. The overall flow chart is as follows: Figure 1 As shown:

[0063] Specifically, a method for generating a land battle scene dataset based on the Unity engine includes the following steps:

[0064] S1: Based on the Unity engine, build at least one virtual local land battle battlefield environment in the Unity engine;

[0065] Import the required environment and vehicle models into the Unity engine, arrange the corresponding models according to generation needs, and form a local land battle battlefield environment scene.

[0066] S2: In each virtual local land battle battlefield environment, complete the setting of a preset actual combat scene and set a plurality of preset target parameters accordingly, wherein the plurality of target parameters at least include environmental parameters, equipment parameters, and camera shooting parameters corresponding to the preset actual combat scene;

[0067] Set a series of environment, target, and camera driver script parameters. Perform any necessary preprocessing as needed. The parameters that need to be set can be divided into three categories: environment-related parameters, such as lighting and weather; equipment-related parameters, such as position, pose, movement, and rotation speed; and capture-related parameters, such as the total number of images, camera position, movement, and rotation speed.

[0068] S3: Control the Unity engine to run the preset actual combat scene and generate the environment reconstruction data set.

[0069] Run the completed scene in the Unity engine, observe the running status, and check the generated results in the preset save path after completion.

[0070] like Figure 2 As shown in the flowchart, the specific process of constructing a local land battle battlefield environment scene is as follows:

[0071] S1-1: Import a local land battle environment and select suitable terrain and object models. Object models can be obtained through the Unity Asset Store or other channels. Large-scale terrain can be generated through satellite or aerial photography or using terrain rendering plug-ins. When scaling, the unit length in Unity can be treated as one meter in reality, and reasonable scaling can be achieved by comparing it with objects of known length in the scene. For example, a city lane is 3.5 meters wide, and a playground runway is 1.25 meters wide.

[0072] S1-2: Import and filter the required land battle scene equipment models. These models can also be obtained through the Unity Asset Store, or by looking for unpacked models from various war games. When scaling the models, refer to the actual parameters of each piece of equipment.

[0073] S1-3: Adjust the scene as needed to avoid occlusion and other issues during the subsequent generation of photo data. Specifically, generating the target detection and segmentation dataset requires ensuring that there is sufficient space around the equipment model; generating the environment reconstruction dataset requires ensuring that there is at least one navigable path in the scene and sufficient frame rate during movement;

[0074] The local land battle scene and equipment model finally constructed and obtained are as follows: Figure 3 As shown, the model preparation is completed, and the parameters adjustment and overall preparation steps are entered.

[0075] like Figure 4 As shown in the flowchart, the specific process to be taken in step S2 to generate the target detection segmentation dataset is:

[0076] S3-1: Use Unity's corresponding components to set lighting and weather conditions. Specifically, you can use Unity's Light component to adjust the lighting direction and color, and change the weather by replacing the skybox texture. You can also achieve better effects by using various weather components available in the Unity Asset Store, such as the UniStorm component, which creates dynamic clouds and various rain and snow effects.

[0077] S3-2: Add mesh collision boxes to all models and adjust their labels to facilitate subsequent instance segmentation mask generation. Specifically, the equipment model in the land battle scene needs to be labeled "Tank," while the labels of other models should be adjusted to "Default." If you need to capture multiple pieces of equipment simultaneously and distinguish them, you need to assign different labels to each piece of equipment.

[0078] S3-3: Add animation driver scripts for land battle scene equipment models. If the model you obtain already contains animations, you can use Unity's own animation component driver; if not, refer to the following methods to solve the problem:

[0079] Equipment pose change animation can be broken down into a combination of three basic animation types: continuous rotation, fixed-angle rotation, and fixed-distance displacement. Scripts are written to implement these three types of motion. These scripts are then added to the components of the equipment model that require these motions, and parameters are set according to actual needs. Continuous rotation scripts require the rotation axis and speed, fixed-angle rotation scripts require the axis, rotation angle step, and rotation duration, and fixed-distance displacement scripts require the translation axis, translation distance, and translation duration. These scripts are then combined in subsequent master control scripts to implement pose changes, for example, combining wheel rotation animations to create a vehicle driving animation.

[0080] S3-4: Add a master control script to each equipment model. After testing, save it as a prefab for future use. The equipment master control script must provide interfaces for controlling the equipment's movement, rotation, and specific poses, so that the data generation control script can call it. For example, an animated effect for equipment movement can be achieved by continuously rotating each wheel and rotating the orientation of the first pair of wheels at a fixed angle.

[0081] S3-5: Construct the empty object assembly required for multi-angle image generation. This assembly consists of two empty objects, a parent and a child, with the camera set on the child. During subsequent capture, the parent object will be positioned at the center of the device being captured. The parent object only allows rotation, which moves the camera around its center; the child object only allows translation along the line connecting it to the parent object, which changes the camera's distance from the center. Attach the required control scripts to each component of the empty object and set the corresponding interfaces to facilitate the call of the data generation control script, enabling multi-angle capture of the camera around the device.

[0082] S3-6: Set various parameters required for the shooting driver script. Specifically, model-related parameters include which models to use, the number of model poses, model position, and angle offset; camera-related parameters include shooting angle, shooting distance, and number of shots; environment-related parameters include time, weather, etc.; and data storage paths and tags.

[0083] like Figure 5 As shown in the flowchart, the specific process to be taken in step S2 to generate the environment reconstruction dataset is:

[0084] S4-1: Set all terrain models to static and bake the scene's navigation mesh using Unity's navigation tools. When baking, be sure to set the baking parameters based on the actual length, width, height, slope angle, and obstacle clearance of the equipment. If you need to allow equipment to pass through low bushes, be sure not to set them to static and set their collision boxes so that they cannot interact with other collision boxes of the same type.

[0085] S4-2: Set natural environmental conditions such as light and weather. The specific operation is the same as S3-1;

[0086] S4-3: Import the equipment prefab saved in S3-4, add a navigation mesh proxy component to the equipment model, and set the corresponding control parameters in the component. Parameters include equipment length, width, height, movement and rotation speed, and acceleration limit.

[0087] S4-4: Add a navigation mesh proxy path control script to the device and arrange a waypoint sequence in the scene. The navigation mesh proxy component added in the previous step drives the device to a specific waypoint. The control script saves a waypoint sequence and switches to the next waypoint when the device is sufficiently close to the current one, allowing the device to cycle through the pre-set set of waypoints. When arranging the waypoint sequence, be sure to ensure that the waypoints are within the reach of the navigation mesh.

[0088] S4-5: Select a device to serve as the vehicle for subsequent filming and attach a script to it to generate simulated IMU data. This script measures the device's velocity and angular velocity differences at fixed time intervals to obtain the true values of three-axis acceleration and angular velocity. Based on the error characteristics of real-world IMU measurement devices, it simulates six-axis IMU errors using normally distributed random numbers. Parameters to be set include acceleration random error, acceleration wander error, angular velocity random error, and angular velocity wander error. These parameters should be set according to the manual for the real-world IMU measurement chip.

[0089] S4-6: Build a camera group. The camera group consists of three viewpoints: two simulated binocular cameras and one depth camera placed in the center of the binocular cameras. Set the camera group parameters and the shooting script parameters, setting it to shoot the vehicle sub-object. Required parameters include the camera group position (typically set in front of the cab, ensuring that the vehicle is not in the picture), binocular baseline length (i.e., the distance between the two eyes), image resolution, camera focal length, and sensor size. These parameters can be set based on the parameters of a real-world binocular camera device.

[0090] S4-7: Build a depth map shooting component. Mount the script for the depth camera in S4-6 to enable depth rendering; create a plane and an orthographic projection camera, adjust the relative position of the plane and the camera so that the orthographic camera can just capture the entire plane; create a texture renderer, use the grayscale image converted from the depth data as the plane texture, and replace the material of the above plane with this depth material; at this point, intercept the picture of the orthographic camera to get the depth map. The script and renderer code required are as follows Figure 6 As shown:

[0091] S4-8: Add a script to generate the transformation matrix and true distance data for each device model. The parameters to be set are the world coordinate system origin (typically set to the camera's starting position) and the data save path. If you need to generate the IMU-to-camera coordinate system transformation matrix, add a transformation matrix generation script for the IMU's location and set the origin to the camera's real-time position.

[0092] S4-9: Set parameters for the scene's master control script, including running frame rate, total number of generated images, etc.

[0093] The entire scenario preparation is now complete, and the execution and result checking steps are now underway.

[0094] Click the Unity Run button to run the scene prepared in the previous steps and observe whether the effect meets your expectations. Specifically, you should observe whether there are any errors during the run and whether the motion trajectory meets the preset requirements.

[0095] After the run is complete, check the generated data in the preset save path. When generating data for object detection and segmentation, check the captured images for large areas of occlusion that do not meet the requirements. When generating data for visual SLAM, check the generated frame sequence for problems such as excessive bumps or rapid turns.

[0096] The above description does not explain the specific process of each script used in detail. The following is a detailed explanation of each:

[0097] Generate the original image and mask image generation script used by the target detection segmentation dataset. The process is as follows Figure 7 As shown:

[0098] S6-1: Determine whether the current equipment model has completed shooting. If the equipment model has completed shooting, destroy the current model, instantiate a new model according to the predetermined order, obtain the corresponding animation control interface, and prepare for the next shooting; if the shooting has not yet been completed, proceed directly to the next step;

[0099] S6-2: Determine whether the current posture has been completed for shooting. If the current equipment model posture has been completed for shooting, change the model posture through the animation control interface to prepare for the next shooting; otherwise, proceed directly to the next step;

[0100] S6-3: Determine whether the current position has been captured. If so, change the model position and prepare for the next capture. Otherwise, proceed directly to the next capture. Specifically, the model position here refers to the three-axis offset and rotation of the equipment model relative to the predetermined capture point.

[0101] S6-4: Change the camera position; specifically, the camera position here refers to the three-axis offset and rotation of the camera relative to the predetermined shooting point;

[0102] S6-5: Prepare for shooting and check whether the conditions for shooting are met. If not, wait for a while and repeat this step; if yes, proceed to the next step. If you want to ensure that the mask image has no ghosting as much as possible, you can adjust the scene time flow rate to zero.

[0103] S6-6: Take an image of the equipment mask, enumerate each position in the camera view, generate rays for collision detection, set the color of the corresponding position based on the label of the returned object, and save it as a mask image. For example, if there is only one type of target, if the returned label is an equipment model, the mask image of that position is white; if the label is not an equipment model, it is black. After enumerating all positions in the image, save the mask image. The flowchart shows the case of only one type of target. If necessary, a similar method can be used to set different colors for different targets.

[0104] S6-7: Use the Unity camera component to render the original image of the device and save the camera view shooting data as the original image;

[0105] S6-8: Generate target labels. Generate corresponding labels based on the preset and model names. Set the names of the saved images and record the number of images taken. If the time flow was paused before, it needs to be restored here.

[0106] S6-9: Determine whether all the shooting is completed. If the shooting of all equipment models is completed, exit the entire process; otherwise, return to S6-1;

[0107] The IMU module is analyzed and modeled. The analysis process is as follows:

[0108] The state of the inertial measurement unit IMU is shown in formula 1:

[0109]

[0110] Where I and W represent the IMU coordinate system and the world coordinate system, respectively. Indicates the IMU rotation attitude (expressed in unit quaternion), W p I Represents the IMU three-dimensional coordinates, W v I represents the translation speed, b g and b a are the random errors of the gyroscope and accelerometer, respectively.

[0111] According to the above model, the IMU motion equation is as follows:

[0112]

[0113] in, W a represents the translational acceleration, n wg and n wa It obeys a Gaussian distribution with a mean of zero and covariances of Q wg and Q wa .

[0114] From the above motion equations, we can know the IMU's gyroscope and accelerometer measurement model as follows:

[0115] ω m = I ω+b g +n g (8)

[0116]

[0117] Among them, ω m and a m denote the measured angular velocity and linear acceleration, respectively. I ω and W a represents the true angular velocity and linear acceleration, n g and n a Represents Gaussian white noise with zero mean, and g represents the acceleration of gravity. In ground vehicle systems, there is no need to subtract the acceleration of gravity, and this value will be removed in subsequent formulas.

[0118] Based on the modeling process above, we can conclude that the IMU error can be divided into two parts: the rapidly fluctuating white noise error n and the slowly varying random walk error b. To facilitate a unified representation, this paper makes the following assumptions.

[0119]

[0120] Where w follows a standard normal distribution and the four coefficients represent the strength of each of these noises.

[0121] The above expressions are all continuous-time descriptions. To facilitate simulation in the program, this article converts the expressions into discrete forms. The discretized expressions are as follows.

[0122]

[0123] where w[k] follows a standard normal distribution. The four coefficients are discretized coefficients, and their relationship with the original coefficients is as follows.

[0124]

[0125] Where Δt is the system sampling time, which is the time interval for generating simulated IMU data.

[0126] To generate IMU data errors using the above formula, it is necessary to obtain normally distributed random numbers. The boxmuller algorithm can be used to generate normally distributed random numbers. It can be mathematically proved that the two-dimensional standard normal distribution can be regarded as the product of two distributions. A polar coordinate system is established to observe a two-dimensional normal distribution. It obeys a uniform distribution in terms of angle and an exponential distribution with λ=1 in terms of radius. The boxmuller algorithm is based on this principle. The algorithm is divided into three steps: (1) Generate a uniform distribution u1 in [0,1] and use the inverse transform sampling method to convert it into exp(1) samples to represent the radius r of a two-dimensional plane point. (2) Generate a uniform distribution u2 in [0,1] and multiply it by 2π to represent the angle θ of the sample point. (3) Convert the polar coordinates into rectangular coordinates to obtain a two-dimensional standard normal distribution.

[0127] The above is the error generation method. The true value of the IMU can be obtained by periodically recording the three-axis velocity and angle of the object and performing differential calculations.

[0128] Based on the above analysis, an IMU data generation script can be written. The specific process is as follows: Figure 8 As shown:

[0129] S8-1: After being called, the system time is recorded, the difference between the system time and the last time it was called is calculated, the sampling interval Δt is obtained, and the last time of the call is updated.

[0130] S8-2: Generate the discretized parameters according to the above formula 12. The original continuous parameters, namely the white noise coefficient and the random walk noise coefficient, can be obtained from the manual of the IMU chip to be simulated.

[0131] S8-3: Use the interface provided by the transformer in Unity to read the target's speed and angle, and subtract them from the speed and angle obtained in the last call;

[0132] S8-4: Divide the above-mentioned difference result by the sampling time interval to calculate the actual three-axis acceleration and angular velocity, and update the stored velocity and angle;

[0133] S8-5: Generate normally distributed random numbers using the boxmuller algorithm;

[0134] S8-6: Update the error value according to formula 11;

[0135] S8-7: According to formulas 8 and 9, the error is added to the true value to generate IMU data with error;

[0136] The calculation and analysis of the true position value are as follows:

[0137] Assume that the standard orthogonal basis used by the world system is (e1, e2, e3), the standard orthogonal basis used by the viewpoint system itself is (e1′, e2′, e3′), and the coordinates of the object in the world system are [a1, a2, a3] T , the coordinates of the object in the viewpoint's own system are [a1′, a2′, a3′] T , then the formula is as follows:

[0138]

[0139] The above formula is for the case where the viewpoint and the world origin coincide with each other, with only the positive directions of the three axes being different. If they do not coincide, translation is required, using the following formula:

[0140]

[0141] By outputting the transformation matrix in Formula 14, we can obtain the position and posture at a specific moment.

[0142] Based on the above analysis, a script for outputting the true value of position and attitude can be written. The specific process is as follows: Figure 9 As shown:

[0143] S9-1: After being called, read the preset coordinate origin position and orientation (usually the initial position of the left camera of the binocular camera is selected as the world system origin) and the position and orientation of the object mounted with the script at the time of calling;

[0144] S9-2: Calculate the value of each element of the matrix according to formula 14;

[0145] S9-3: Output the value of each element line by line and record other necessary information (such as the object name of the mounting script, timestamp, etc.);

[0146] When generating an environment reconstruction dataset: the navigation mesh proxy path control script runs independently of the shooting master control script; the master control script calls the IMU simulation, pose generation, image generation and other scripts at regular intervals according to the preset frequency to generate a series of data, and ends the scene operation when the number of images reaches the preset value;

[0147] Finally, the target detection segmentation dataset can be generated, and some pictures and masks are as follows Figure 10 As shown:

[0148] Finally, the environment reconstruction dataset can be generated, and some binocular frame sequences and IMU data are as follows: Figure 11 As shown:

[0149] The present invention also provides another embodiment, which provides a land combat scene dataset generation device based on the Unity engine, characterized in that the device includes the following parts: a construction module, which is based on the Unity engine and constructs at least one virtual local land combat battlefield environment in the Unity engine; a setting module, which is used to complete the preset actual combat scene setting in each virtual local land combat battlefield environment, and set multiple preset target parameters accordingly, wherein the multiple target parameters include at least environmental parameters, equipment parameters and camera shooting parameters; a generation module, which is used to control the Unity engine to run the preset actual combat scene and generate an environment reconstruction dataset.

[0150] An electronic computer device provided by an embodiment of the present invention has a structure as follows: Figure 12 As shown:

[0151] Specifically, an electronic computer device comprises a memory 31, an executable program 32, a processor 33, and a system bus 34. Memory 31 stores the computer software system and can be primarily divided into two areas: program and data storage, which respectively store at least one operating system and at least one application program, as well as cached data generated by the operating system and program. Memory 31 contains an executable program 32 for a network request method. The program is divided into multiple modules, including functional program instruction segments for detection, processing, and positioning. For example, the computer program can be divided into functional modules such as source detection, target detection, self-positioning, fusion results, and command. Processor 33 is the control center of the electronic computer, executing the executable program 32 stored in memory 31 and reading and writing data in the data area of memory 31. Processor 33 may include one or more processing units. System bus 34 connects the various functional components of the electronic computer and transmits information such as data, addresses, and instructions. System bus 34 can also connect to other peripherals such as network interfaces and display terminals.

[0152] An embodiment of the present invention provides a computer-readable storage medium storing a computer program.

[0153] The computer-readable storage medium in the embodiment of the present invention stores a computer program, which is read by an electronic computer device and processed by a processor to execute the real-time battlefield three-dimensional scene and target perception method and system based on binocular vision according to the embodiment of the present invention. Through the above description of the embodiment of the present invention, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary hardware. The part of the present invention that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory, random access memory, flash memory, hard disk, optical disk, etc., using a number of instructions to enable one or more electronic computer devices to execute the method of the embodiment of the present invention.

[0154] As described above, the above embodiments of the invention are intended only to illustrate the technical solutions and system module architecture, and are not intended to limit them. Those skilled in the art will appreciate that the aforementioned embodiments may be simplified, modified, replaced, combined, etc., but such modifications do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and such replacements are included within the scope of protection of the present invention.

Claims

1. A method for generating a land battle scene dataset based on the Unity engine, characterized in that: The method comprises the following steps: S1: Based on the Unity engine, construct at least one virtual local land battle battlefield environment in the Unity engine; S2: In each of the virtual local land battle battlefield environments, completing the setting of a preset actual combat scene and correspondingly setting a plurality of preset target parameters, wherein the plurality of target parameters at least include environmental parameters, equipment parameters, and camera shooting parameters; S3: Control the Unity engine to run the preset actual combat scene and generate a land combat scene dataset, where the land combat scene dataset includes an equipment detection segmentation dataset and a land combat scene environment reconstruction dataset.

2. The method according to claim 1, characterized in that Based on the Unity engine, at least one virtual local land battle battlefield environment is constructed in the Unity engine, including: Screening a terrain model and a ground object model that match the virtual local land battle battlefield environment, wherein the scaling ratios of the terrain model and the ground object model are both a first preset ratio; Screening a scene equipment model that matches the virtual local land battle battlefield environment, wherein the scaled model ratio of the scene equipment model is a second preset ratio; The terrain model, the object model and the scene equipment model are controlled to be imported into the Unity engine, and the position of the scene equipment model is adjusted to construct the virtual local land battle battlefield environment.

3. The method according to claim 1, characterized in that In each of the virtual local land battle battlefield environments, a preset actual combat scenario is set, and a plurality of preset target parameters are set accordingly, including: S3-1: setting the environmental parameters by presetting Unity components, where the environmental parameters include at least lighting parameters and weather parameters; S3-2: adding a grid collision box for each target model, and adjusting a label corresponding to each target model, wherein the target model is any one of the terrain model, the object model, and the scene equipment model; S3-3: Adding animation driving scripts to each of the scene equipment models; S3-4: Adding the animation master control script to each of the scene equipment models, and obtaining the equipment prefab corresponding to the scene equipment model; S3-5: Constructing a multi-angle image to generate a corresponding empty object combination, and attaching a corresponding control script and interface to each component in the empty object combination; S3-6: Add a shooting drive script to the empty object combination, and set multiple driving parameters corresponding to the shooting drive script, where the multiple driving parameters at least include model parameters, camera parameters, environment parameters, and data saving parameters.

4. The method according to claim 3, characterized in that After completing the preset actual combat scene setting in each of the virtual local land combat battlefield environments and correspondingly setting a plurality of preset target parameters, the method further includes: S4-1: setting all the terrain models and feature models to static, and using the navigation tool of the Unity engine to bake the navigation mesh corresponding to the preset actual combat scene; S4-2: Setting the environmental parameters by presetting Unity components, where the environmental parameters include at least lighting parameters and weather parameters; S4-3: controlling the equipment prefab to be imported into the Unity engine, adding a navigation mesh proxy component to the scene equipment model, and setting at least one control parameter in the navigation mesh proxy component; S4-4: Controlling the navigation mesh proxy path control script to be added to the scene equipment model, and arranging a path point sequence in the preset actual combat scene, wherein the path point sequence is used to allow the scene equipment model to cyclically move in a preset path point group; S4-5: Selecting the target scene equipment model as the shooting vehicle during the shooting process of the preset actual combat scene, adding the script for generating simulated IMU data to the target scene equipment, and setting the corresponding IMU parameters; S4-6: constructing a multi-view camera group and correspondingly setting a plurality of shooting parameters of the multi-view camera group, wherein the multi-view camera group is used to shoot the target scene equipment; S4-7: Add the transformation matrix generation script and the distance data true value script to the scene equipment model; S4-8: Determine the shooting master control script of the preset actual combat scene, and set multiple operating parameters of the shooting master control script.

5. The method according to claim 4, characterized in that Controlling the Unity engine to run the preset combat scenario and generate an environment reconstruction data set includes: Controlling the navigation grid proxy path control script and the shooting master control script to run independently and generate images accordingly. The shooting master control script calls the IMU simulation at a preset frequency during operation. When the number of the generated images reaches a preset number, the operation of the preset actual combat scenario is terminated, and a target detection segmentation data set is generated; The environment reconstruction dataset is generated based on the target detection segmentation dataset.

6. The method according to claim 5, characterized in that Control the navigation grid proxy path control script and the shooting control script to run independently and generate pictures accordingly, including: S6-1: Determine whether the target scene equipment model has been photographed; S6-2: If the target scene equipment model completes the shooting, destroy the target scene equipment model, instantiate another target scene equipment model as the current target scene equipment model according to a predetermined order, and determine the animation control interface corresponding to the current target scene equipment model; S6-3: adjusting multiple postures of the current target scene equipment model multiple times respectively, controlling the multi-view camera assembly to perform posture shooting in each posture, and completing the posture shooting; S6-4: After confirming that the shooting is completed in the multiple postures, adjusting the multiple positions of the current target scene equipment model corresponding to the preset actual combat scene multiple times, and completing the shooting in the multiple positions; S6-5: After completing the shooting at the plurality of positions, changing the position of the multi-view camera assembly, determining the positions of the camera views corresponding to the multi-view camera assembly, generating rays for collision detection, and obtaining a mask image of the shooting equipment; S6-6: Repeat the process of S6-1-S6-5 until all the scene equipment models are photographed, exit the photographing process, and obtain all the original pictures.

7. The method according to claim 1, characterized in that After controlling the Unity engine to run the preset actual combat scenario, the method further includes: During the operation of the Unity engine, confirm whether there are any operation errors and / or operation estimation deviations; If the operation error and / or the operation estimation deviation occurs, the operation process is paused, and the Unity engine and / or the target parameters corresponding to the preset actual combat scenario are adjusted.

8. A land battle scene dataset generation device based on Unity engine, characterized in that: The apparatus comprises the following parts: a construction module, based on a Unity engine, for constructing at least one virtual local land battle battlefield environment in the Unity engine; A setting module is used to complete the setting of a preset actual combat scene in each of the virtual local land combat battlefield environments, and to set a plurality of preset target parameters accordingly, wherein the plurality of target parameters at least include environmental parameters, equipment parameters, and camera shooting parameters; The generation module is used to control the Unity engine to run the preset actual combat scene and generate an environment reconstruction data set.

9. An electronic computer device comprising a memory, a processor, a bus, and an executable program stored in the memory and run by the processor, characterized in that: When the processor executes the program, it can implement a method for generating a land combat scene dataset based on the Unity engine as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that When executed by a computer processor, the computer executable instructions are used to execute a method for generating a land battle scene dataset based on a Unity engine as described in any one of claims 1 to 6.

Images