A basic data generation method for unmanned system intelligence level assessment

Abstract

The present invention belongs to the field of unmanned system intelligence level assessment, and in particular, is a basic data generation method for unmanned system intelligence level assessment. The present invention constructs a basic model database, and then based on these data, automatically generates diversified model data through data expansion and derivative technologies. Specifically, it includes the following types of key technologies and approaches for basic data generation: building model data production and expansion, ground, vegetation, road, river, landmark and other model production and expansion technology, terrain scene data generation, urban scene data generation, meteorological environment data superposition and sensor data generation. The present invention can construct a large amount of evaluation benchmark data, and form assessment data on demand that can meet the graded assessment of intelligence levels of different difficulty levels.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned system intelligence level assessment, and in particular to a basic data generation method for unmanned system intelligence level assessment. Background Art

[0002] At present, a new round of world scientific and technological revolution represented by artificial intelligence is emerging, and the intelligent development of unmanned systems has emerged. Relying on the unmanned system intelligent promotion project, a large number of unmanned system intelligent projects have made positive progress, and the intelligent empowerment and efficiency-enhancing effect has initially emerged, but problems such as insufficient quantification of project intelligent goals and insufficient traction have gradually emerged. In order to promote the accelerated development of unmanned system intelligence, scientifically plan the development path of unmanned system intelligence, and promote the innovation of unmanned system hierarchical and classified management mode, it is urgent to carry out the hierarchical evaluation of the intelligence level of unmanned systems. Evaluating the intelligence level of unmanned systems has important theoretical and practical significance for accelerating the development of unmanned systems intelligence.

[0003] To carry out the intelligent level assessment of unmanned systems, a large amount of benchmark data covering various unmanned system fields, scenarios and intelligent levels is needed to evaluate the intelligence levels of unmanned systems such as autonomy, collaboration and learning, and provide standardized third-party data input for the intelligent level assessment of unmanned systems based on simulation. However, the current benchmark data for the intelligent level assessment of unmanned systems has not yet formed a systematic and standardized scale. There is a large gap in the quantity and quality of the benchmark data for the intelligent level assessment of unmanned systems. There are few high-quality data samples and extremely limited labeled data. It is difficult to meet the basic requirements of the intelligent level assessment of unmanned systems. In addition, the existing data of various units cannot be directly used as benchmark data. On the one hand, the evaluation data has a clear mapping relationship with the evaluation scenario, evaluation subject, and evaluation indicator, which needs to be further clarified and established according to the evaluation method. On the other hand, the resolution and annotation focus of data in various fields are different, which makes it difficult to ensure the fairness and objectivity of the evaluation. Therefore, in order to effectively carry out the intelligent level assessment of unmanned systems and ensure the fairness and scientificity of the intelligent level assessment of unmanned systems, it is urgent to conduct research on the generation and application of benchmark data for the intelligent level assessment of unmanned systems.

[0004] Building complex and diverse evaluation scenarios requires a large amount of 3D model data, including various buildings, ground, roads, rivers, vehicles, pedestrians, aircraft, landscapes, landmarks, etc. A complex urban scene often contains more than 100,000 or even millions of models, which are completely dependent on manual production or procurement, requiring a lot of manpower, material resources or financial resources.

[0005] Therefore, it is urgent to build a basic data generation method for unmanned system intelligence level assessment to overcome the above technical problems. Summary of the invention

[0006] The evaluation method and system of the present invention provide a comprehensive performance evaluation framework for intelligent models, which helps to ensure the effectiveness and reliability of models in practical applications. This method is not only applicable to the model development and optimization stage, but also to the continuous monitoring and evaluation after the model is deployed.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A basic data generation method for unmanned system intelligence level assessment includes the following steps:

[0009] Step 1: Architectural model data production and extension production

[0010] Constructing a building model database: The building model is divided into three building structures: the base, the theme, and the top. Each building structure has several texture maps of different styles and resolutions. The texture map sets the texture UV coordinates to support reuse and splicing according to the building structure. The UV coordinate system represents the texture mapping of the model surface through the U and V axes. The coordinate system is defined based on the surface topology and geometry of the model. Each vertex has a corresponding UV coordinate to determine the position of the vertex in the texture image.

[0011] There are two generation modes for building model data production and expansion: generating building models through free combination or generating building models of different sizes according to requirements;

[0012] Generate architectural models through free combination Based on the three-segment architectural structure, different styles of architectural models are randomly combined by the algorithm through different combinations and different texture maps;

[0013] Generate architectural models of different sizes according to requirements. Modelers can manually select the size and texture of architectural models and generate architectural models according to their needs.

[0014] Step 2: Irregular model making and expansion

[0015] The irregular model includes ground, vegetation, roads, rivers, and landmarks. Due to the irregular shape of the model itself, it is impossible to use modeling software to establish a single general model, and all models need to be generated according to requirements;

[0016] The irregular model making and extension basic library is provided with a number of texture maps of different styles and resolutions; the texture map texture UV coordinates support the style of repeated use and splicing; the irregular model making and extension module generates a road model three-dimensional grid according to the input irregular model data boundary, calculates the texture UV coordinates according to the irregular model data boundary, and maps according to the calculated coordinate values;

[0017] Step 3: Terrain scene data generation

[0018] Generate terrain scene data by any one or a combination of two or more of the three terrain scene generation technical methods, the three methods being: manual modeling, oblique photography, and superposition of DOM and DEM data;

[0019] The manual modeling method involves the designer manually constructing a terrain grid, editing the height of each grid point, and then making a surface image for mapping;

[0020] Oblique photography technology acquires the texture of the top and side views of the terrain scene by simultaneously collecting images from five different viewing angles, no less than one vertical and four oblique ones;

[0021] DOM and DEM data overlay modeling constructs real three-dimensional terrain by using digital orthophoto DOM data and digital elevation model DEM data;

[0022] Step 4: Generate basic scene data

[0023] Based on the basic data generated by the building model data production and expansion module, the irregular model production and expansion module and the terrain scene data, the basic data are combined and conditional parameters are set to generate basic scene data;

[0024] Step 5: Overlay of meteorological and environmental data

[0025] Meteorological data is superimposed on the basic scene data, and the meteorological data includes illumination, rain, snow, and fog data; the environmental data is superimposed based on a rendering engine provided by a graphics platform and combined with a particle system and shading language technology to complete the superposition;

[0026] The particle system implementation steps are as follows:

[0027] S1: Create a particle emitter

[0028] The particle emitter is the starting point of the particle system and defines where and how particles are generated;

[0029] S2: Set particle properties

[0030] Particle properties include particle size, shape, color, transparency, speed, direction, and life cycle; these properties determine the appearance and behavior of the particle system;

[0031] S3: Update particle status

[0032] In each frame of three-dimensional graphics rendering, the position, velocity and direction of the particles are updated according to the properties of the particles and physical rules; the physical rules include gravity and wind force;

[0033] S4: Drawing particles

[0034] Drawing the updated particles is achieved through graphics rendering technology;

[0035] S5: Particle Lifecycle Management

[0036] Set the time from when particles are emitted to when they disappear, to ensure that particles are generated and disappear at the desired time;

[0037] Use shading language to achieve different lighting effects at different time periods. The steps are as follows:

[0038] S1: Define the light source

[0039] Define the position and color of the sun: This is achieved by defining a light source vector, which points to the position of the sun and contains the color information of the sun;

[0040] S2: Calculate the light direction

[0041] For each object in the scene, calculate the lighting direction, which is the direction between the sun position and the object's surface point; this is obtained by subtracting the object's surface point position vector from the sun position vector.

[0042] S3: Calculate light intensity

[0043] The light intensity is calculated by multiplying the light direction and the normal vector of the object surface. The dot product result represents the intensity of the light. The larger the value, the stronger the light.

[0044] S4: Calculate diffuse reflection

[0045] Diffuse reflection is the effect of light scattering on the surface of an object. The color of diffuse reflection is calculated using the light intensity and the diffuse reflection coefficient of the object surface. This is achieved through the multiplication operator in the shading language.

[0046] S5: Calculate specular reflections

[0047] Highlight reflection is the specular reflection effect produced by light on the surface of an object. The color of the high light reflection is calculated based on the light direction, the sight direction and the normal vector of the object surface. The high light reflection is calculated using power operations and dot multiplication operations.

[0048] S6: Synthetic lighting results

[0049] The colors of diffuse and specular reflections are synthesized with the base color of the object surface to obtain the final lighting effect, and the addition operator in the shading language is used to achieve color synthesis;

[0050] Step 6: Sensor data generation

[0051] Generate sensor data in real-time simulation based on sensors carried by the unmanned system, wherein the sensors include visible light camera sensors, infrared camera sensors, and lidar sensors;

[0052] The visible light camera sensor and infrared camera sensor data are generated based on Render To Texture (RTT) technology;

[0053] Simulate the perception results of lidar by generating simulated lidar point cloud data.

[0054] Preferably, the steps of generating model data by using the oblique photography technology are as follows:

[0055] Step 1: using several sets of aerial photography data or satellite high-definition image sensor data at different angles and orientations as input, wherein the angles and orientations of the input data can cover the top and surrounding areas of the terrain;

[0056] Step 2: Perform computational processing on a high-performance workstation to fuse and stitch the input data into a terrain scene model.

[0057] Preferably, the steps of generating model data by the DOM and DEM data superposition method are as follows:

[0058] Step 1: First, obtain the DOM and DEM data of the specified area;

[0059] Step 2: After acquiring the data, preprocessing work is performed, including data cleaning, format conversion, and coordinate system unification to facilitate subsequent processing.

[0060] Step 3: Generate TIN using an algorithm using DEM data; the algorithm includes Delaunay triangulation; TIN is a method for representing a terrain surface, which is composed of a series of irregular triangles;

[0061] Step 4: Map the DOM data as a texture onto the TIN so that each triangle has a corresponding texture.

[0062] Preferably, the steps of generating urban scene data are as follows:

[0063] Step 1: Factor decomposition

[0064] Decomposing the three-dimensional model constituting the basic scene into independent, spliced ​​or repeatedly usable single elements, wherein the elements include texture maps or single three-dimensional models;

[0065] The decomposed elements need to be stored by type, and a description file needs to be configured to describe their attributes for subsequent scheduling; the attributes include type, size, resolution, and storage location;

[0066] Step 2: Region Segmentation

[0067] Divide the scene of a specified size into several areas, and set model labels with different characteristics for each area, including residential areas, commercial areas, transportation hubs, etc. Subsequently, different models will be randomly selected from the feature library or automatically generated according to conditions for filling;

[0068] Step 3: Area filling

[0069] According to the characteristics of each region, several feature models are randomly selected from the feature library and placed in the region;

[0070] For example, commercial areas are filled with commercial models, residential areas are filled with residential models, and green areas are filled with plants. Conditional parameters such as distribution density and building height will affect the filling distance and model height. Different conditional parameters ensure that the production scenarios meet the test requirements.

[0071] Step 4: Scene modification

[0072] Use street lights, green plants, and landscape models to fill and modify the perimeter of each area to form a complete basic scene.

[0073] Preferably, the generation of visible light camera sensor data in step six comprises the following steps:

[0074] Step 1: Create the texture

[0075] Predefine the texture in the computer and adjust the size and resolution of the texture as needed;

[0076] Step 2: Render to texture

[0077] By using the function in the rendering engine to render the scene into a texture, the final result will be saved to the texture; when rendering, use the render target (that is, the texture) as the render output target and ensure that the rendering result matches the texture size and resolution;

[0078] Step 3: Applying the texture

[0079] By building a virtual camera in the scene, rendering the scene image taken by the virtual camera into the prepared two-dimensional texture, and then compressing the texture data and sending it out according to the interface requirements, the visible light sensor data simulation process is realized;

[0080] After the scene is rendered into the texture, the rendered texture can be applied to other objects or scenes; when applying the texture, the texture is used as a map of the material and the texture is ensured to match the geometry of the object surface;

[0081] By modifying the FOV (field of view), Near plane (near clipping plane), and Far plane (far clipping plane) of the virtual camera, you can simulate visible light sensors with different lens parameters;

[0082] By using RTT technology, more complex effects can be achieved, such as dynamic monitor images and real-time updated sights; at the same time, RTT technology can also improve rendering efficiency because some complex rendering calculation results can be saved to textures and then reused when needed.

[0083] Preferably, the infrared camera sensor data generation steps are as follows:

[0084] Step 1: Infrared scene modeling

[0085] Infrared scene modeling mainly includes the following three aspects:

[0086] 101. Establish a geometric model library:

[0087] Provide geometric models and texture features of infrared targets and backgrounds to serve subsequent scene modeling;

[0088] 102. Establish spectral characteristics library:

[0089] The spectral characteristic library contains the spectral characteristics of various materials in the geometric model library, including spectral reflection, absorption and emissivity parameters, providing material spectral data for subsequent infrared radiation calculations;

[0090] 103.Scene Modeling:

[0091] Establish 3D models of static and dynamic targets and backgrounds in the scene; set specific weather conditions, ephemeris models, and the position and status of the observer; set the material and texture of objects in the scene based on the geometric model library and spectral characteristic library;

[0092] Step 2: Infrared radiation modeling

[0093] According to the established scene model and the radiation characteristics of various materials, the infrared radiation of the target and background in the scene is analyzed, and the infrared radiation intensity distribution of the target and background reaching the detector is calculated;

[0094] Step 3: Target background synthesis

[0095] For the entire infrared scene, infrared synthesis is performed on the target and the background, and the mutual influence between the target and the background is calculated by combining the radiation transfer theory with the boundary conditions;

[0096] Step 4: Atmospheric transport modeling

[0097] There is atmosphere on the path from the scene to the infrared detector. The infrared radiation of the object is attenuated by the selective absorption and scattering of the atmosphere and suspended particles. Weather factors, including wind, rain, fog, haze, and snow, directly affect the atmospheric attenuation.

[0098] Calculate atmospheric attenuation by empirical formula method or atmospheric model method based on atmospheric transmittance, atmospheric radiation, solar radiation, lunar radiation, infrared window, geographical location, season, time, climate conditions, atmospheric humidity, visibility and atmospheric temperature parameters;

[0099] Step 5: Sensor Modeling

[0100] Model scanning, staring and micro-scanning infrared sensors to form infrared thermal images;

[0101] Includes modeling of AC coupling, optical transmittance, defocus error, detector tilt, detector geometry model, noise (various types), non-uniformity, saturation, sampling artifacts and halo; noise includes thermal noise, shot noise, 1 / f noise, generation-recombination noise, photon noise, dark current noise and CCD transfer noise.

[0102] Furthermore, there are three main methods for calculating the scene temperature and radiation characteristics:

[0103] 201. According to the classical radiation principle, the radiation transfer equation is established, and the scene temperature and radiation distribution are obtained by solving the equation;

[0104] 202. Based on the measured data, the feature distribution of the scene is obtained by using methods such as feature matching and pasting;

[0105] 203. Combine theory with experiment to establish a semi-empirical model.

[0106] Method 201 is based on classical radiation transfer theory and can obtain accurate infrared radiation distribution of the scene. However, due to the need to solve a large number of equations, its calculation process is relatively complex and requires a large amount of calculation, and requires support from high-performance software and hardware platforms. Method 202 is much simpler than method a in terms of both modeling process and calculation amount, but due to the possible differences between the simulated scene and the actual image, its simulation accuracy and flexibility are poor.

[0107] Method 203 combines the advantages and disadvantages of the previous two methods. It does not need to rely too much on the support of large-scale hardware and software platforms, and takes into account both fidelity and flexibility. It is the preferred method when the hardware and software platforms are not perfect.

[0108] Furthermore, in step 3, the target scene synthesis method is one of the following two methods:

[0109] 301. Model the target and background radiation separately, embed the target into the background radiation image, and calculate the infrared radiation interaction in the overlapping area of ​​the target and background;

[0110] 302. Based on the theoretical model of target and background infrared radiation, combined with the relative position of the target and background, a coupled infrared feature theoretical model is established to directly calculate the target and background infrared radiation and contrast features; for dynamic infrared scenes, the high dynamic range (HDR) change of the radiation intensity of the entire infrared scene is calculated.

[0111] Preferably, when the radar is a single-line laser radar, the steps of generating radar sensor data based on the constructed three-dimensional scene simulation are as follows:

[0112] Step (a) Based on the radar device scanning plane, with the radar device as the center point, a line segment is emitted at every n° interval, the length of the line segment is the same as the radar scanning distance, and the total number of line segments is 360 / n

[0113] Step (b) calculates the intersection of each line segment and the object in the scene, saves the coordinate values ​​of all intersection points, and outputs them in radar data format.

[0114] In a second aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0115] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] Figure 1 Schematic diagram of the basic scene generation technology process described in Example 1.

[0117] Figure 2 Schematic diagram of the infrared camera sensor data generation technology process described in Example 1. DETAILED DESCRIPTION

[0118] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0119] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0120] Example 1

[0121] A basic data generation technology is implemented through the following six ways: building model data production and expansion, ground, vegetation, road, river, landmark and other model production and expansion technology, terrain scene data generation, urban scene data generation, meteorological environment data overlay and sensor data generation.

[0122] 1. Architectural model data production and expansion

[0123] The architectural model data needs to be modeled using 3D modeling software, such as Autodesk Maya and 3D Studio Max.

[0124] Models created by modeling software are exported into the ".FBX" 3D data format. To meet the needs of model data expansion, the following rules need to be followed when using modeling software to create models:

[0125] The model is divided into three architectural structures: base, theme, and top;

[0126] For each building structure, multiple sets of textures and materials with different styles and resolutions are prepared.

[0127] Texture mapping sets the texture UV coordinates to support reuse and splicing according to the building structure. The UV coordinate system is a two-dimensional coordinate system that represents the texture mapping of the model surface through two axes, U and V. Usually, this coordinate system is defined based on the surface topology and geometry of the model. Each vertex has a corresponding UV coordinate that is used to determine the position of the vertex in the texture image.

[0128] Generate architectural models through free combination: Based on the three-section architectural structure, different styles of architectural models can be freely synthesized through different combinations and different texture maps.

[0129] Generate architectural models of different sizes according to requirements: In addition to the free combination method, architectural models of different sizes can also be generated according to requirements. While redefining the model size, the texture UV coordinate value is calculated according to the new size information.

[0130] (II) Modeling and extension technology for ground, vegetation, roads, rivers, landmarks, etc.

[0131] For ground, vegetation, road, and river models, due to the irregular shapes of the models themselves, it is not possible to use modeling software to build a single general model, and they need to be automatically generated according to requirements.

[0132] Similar to the requirements for building model expansion, the expansion of models such as ground, vegetation, roads, rivers, and landmarks requires the preparation of multiple sets of texture maps with different styles and resolutions. At the same time, the texture UV coordinates of these texture maps must support reuse and splicing. Taking roads as an example, the steps to generate model data are as follows:

[0133] Step 1: Evaluation data preparation Generate a 3D grid of the road model based on the road boundary

[0134] Step 2: Calculate the texture UV coordinates according to the road width, and map according to the calculated coordinate values

[0135] 3. Terrain scene data generation

[0136] Common terrain scene generation techniques include manual modeling, oblique photography, and DOM and DEM data overlay.

[0137] 1. Manual Modeling

[0138] The manual modeling method of terrain scenes is similar to that of ordinary models such as buildings, vehicles, and aircraft. 3Dmax and Maya modeling tools are used to build terrain grids, edit the height of each grid point, and finally make surface images and perform mapping.

[0139] 2. Oblique photography modeling

[0140] Oblique photography technology acquires high-resolution textures of the top and side views by synchronously collecting images from one vertical, four oblique, and five different perspectives. Through positioning, fusion, and modeling technology, a real three-dimensional city model is generated. The steps to generate model data are as follows:

[0141] Step 1: Use multiple sets of aerial photography data or satellite high-definition image sensor data from different angles and directions as input and perform computational processing on a high-performance workstation.

[0142] Step 2: Fusion and splicing into terrain scene model.

[0143] Step 3: The automatically generated oblique photography model is manually processed later.

[0144] 3. DOM and DEM data overlay modeling

[0145] The DOM and DEM data overlay method uses DOM (digital orthophoto) data and DEM (digital elevation model) data to construct a real three-dimensional terrain. The steps of the DOM and DEM data overlay method to generate model data are as follows:

[0146] Step 1: First, obtain the DOM and DEM data of the specified area. The data is obtained from various public geographic data websites, including USGS (United States Geological Survey), NASA (National Aeronautics and Space Administration), and Geospatial Data Cloud.

[0147] Step 2: After acquiring the data, preprocessing work is performed, including data cleaning, format conversion, and coordinate system unification to facilitate subsequent processing.

[0148] Step 3: Use DEM data to generate TIN through Delaunay triangulation. TIN is a method of representing terrain surface, which consists of a series of irregular triangles.

[0149] Step 4: Map the DOM data as a texture onto TIN, so that each triangle has a corresponding texture, making the terrain surface more realistic.

[0150] 4. Urban scene data generation

[0151] Urban scene data is customized based on various conditional parameters by making reasonable use of basic data on the basis of 3D model production and model data expansion, terrain scene data generation. The steps to generate urban scene data are as follows:

[0152] Step 1: Factor decomposition

[0153] Decompose the 3D models that constitute the basic scene into independent, spliced ​​or repeatedly used single elements, mainly texture maps or single 3D models.

[0154] The decomposed elements need to be stored by type, and a description file should be configured to describe their properties (type, size, resolution, storage location, etc.) in detail to facilitate subsequent scheduling.

[0155] Step 2: Region Segmentation

[0156] The scene of a specified size is divided into several areas using features such as roads and rivers. Different characteristics are set for each area, such as residential areas, commercial areas, transportation hubs, etc. Different models will be randomly selected from the feature library or the corresponding models will be automatically generated according to the conditions for filling.

[0157] Step 3: Area filling

[0158] According to the characteristics of each area, several element models are randomly selected from the element library and placed in the area. For example, commercial areas are filled with commercial models, residential areas are filled with residential models, and green areas are filled with plants. Conditional parameters such as distribution density and building height will affect the filling distance and model height. Different conditional parameters ensure that the production scene meets the test requirements.

[0159] Step 4: Scene modification

[0160] Finally, use models such as street lights, green plants, and landscapes to fill in and modify the surroundings of each area to form a complete basic scene.

[0161] (V) Meteorological and environmental data superposition

[0162] Meteorological environment data overlay is to overlay meteorological data on the basic scene data to simulate the effects of real-world lighting, rain, snow, fog, etc. Environmental data overlay is usually based on the rendering engine provided by the graphics platform and combined with particle systems and shading language technology to achieve.

[0163] Particle system is a computer graphics technology used to simulate specific blur phenomena. The key steps to implement particle system are as follows:

[0164] Step 1: Create a particle emitter

[0165] The particle emitter is the starting point of the particle system, which defines where and how particles are generated.

[0166] Step 2: Set particle properties

[0167] Particle properties include particle size, shape, color, transparency, speed, direction, life cycle, etc. These properties will determine the appearance and behavior of the particle system.

[0168] Step 3: Update particle status

[0169] In each frame of 3D graphics rendering, the position, velocity, and direction of the particles need to be updated according to the properties of the particles and physical rules (such as gravity, wind, etc.).

[0170] Step 4: Draw particles

[0171] Draw the updated particles to the screen. This is done using graphics rendering technology, such as OpenGL or DirectX.

[0172] Step 5: Particle life cycle management

[0173] The process of particles being emitted and then disappearing needs to be carefully managed to ensure that particles are created and disappear at the appropriate times.

[0174] Shading language is a programming language used to write shaders, also known as shader language. Shading languages ​​include HLSL (High Level Shading Language) of DirectX, GLSL of OpenGL,

[0175] (OpenGL Shading Language) and NVIDIA's CG (C for Graphics). Shading languages ​​are designed specifically for graphics programming, and they contain many features and functions specifically for graphics processing and rendering. Shading languages ​​are usually designed to be executed in parallel, and many graphics processing tasks need to be executed in parallel to improve performance. Taking sunlight as an example, the steps to use shading languages ​​to achieve different lighting effects at different time periods are as follows:

[0176] Step 1: Define the light source

[0177] Define the position and color of the sun. This is done by defining a light source vector that points to the sun's position and contains the sun's color information.

[0178] Step 2: Calculate the light direction

[0179] For each object in the scene, the lighting direction needs to be calculated, that is, the vector between the sun position and the object surface point. This can be obtained by subtracting the position vector of the object surface point from the sun position vector.

[0180] Step 3: Calculate light intensity

[0181] The light intensity can be calculated by multiplying the light direction and the normal vector of the object surface. The dot product result can represent the intensity of the light, and the larger the value, the stronger the light.

[0182] Step 4: Calculate diffuse reflection

[0183] Diffuse reflection is the effect of light scattering on the surface of an object. The diffuse color can be calculated using the light intensity and the diffuse reflection coefficient of the object surface. This calculation can be implemented using the multiplication operator in the shading language.

[0184] Step 5: Calculate specular reflections

[0185] Specular reflection is the specular reflection effect produced by light on the surface of an object. The color of the specular reflection can be calculated based on the light direction, the view direction and the normal vector of the object surface. Specular reflection is calculated using power operations and dot multiplication.

[0186] Step 6: Synthesize the lighting results

[0187] The diffuse and specular colors are combined with the base color of the object surface to get the final lighting effect. Use the addition operator in the shading language to achieve color synthesis.

[0188] 6. Sensor Data Generation

[0189] Sensor data generation is the real-time simulation of sensor data based on sensor type and performance. The sensors currently carried on unmanned systems mainly include visible light camera sensors, infrared camera sensors, and lidar sensors.

[0190] Visible light camera sensor data generation

[0191] The simulation and generation of visible light camera sensor and infrared camera sensor data in Unity and Unreal Engine graphics engines requires the Render To Texture (RTT) technology. The texture rendering process in RTT technology includes the following steps:

[0192] Step 1: Create the texture

[0193] Predefine the texture in your computer and adjust the size and resolution of the texture as needed.

[0194] Step 2: Render to texture

[0195] By using the function in the rendering engine to render the scene into a texture, the final result will be saved in the texture. When rendering, you need to use the render target (that is, the texture) as the render output target and ensure that the rendering result matches the texture size and resolution.

[0196] Step 3: Applying the texture

[0197] Once the scene has been rendered into a texture, you can apply this texture to other objects or scenes. This can be done by using the texture as part of a material. When applying a texture, you need to use the texture as a texturemap for the material and make sure that the texture matches the geometry of the object's surface.

[0198] By using RTT technology, more complex effects can be achieved, such as dynamic monitor screens, real-time updated sights, etc. At the same time, RTT technology can also improve rendering efficiency, because some complex rendering calculation results can be saved to textures and then reused when needed.

[0199] By building a virtual camera in the scene, rendering the scene image captured by the virtual camera into the prepared two-dimensional texture, and then compressing the texture data and sending it out according to the interface requirements, the visible light sensor data simulation process is simply realized. By modifying the virtual camera's FOV (field of view), Near plane (near clipping plane), and Far plane (far clipping plane), visible light sensors with different lens parameters can be simulated.

[0200] Infrared camera sensor data generation

[0201] The infrared camera sensor data generation technology is similar to the visible light camera sensor data generation technology, and both can generate image data with the help of graphics engine combined with RTT technology. Unlike the visible light camera sensor, the infrared camera sensor data needs to consider factors such as scene model, atmospheric transmission, infrared radiation, etc. when rendering the three-dimensional scene. The steps of infrared camera sensor data generation technology are as follows:

[0202] Step 1: Infrared scene modeling

[0203] Infrared scene modeling mainly includes the following three aspects:

[0204] Create a geometric model library:

[0205] Provide geometric models and texture features of common targets and backgrounds to serve subsequent scene modeling;

[0206] Build a spectral characteristics library:

[0207] The spectral characteristic library contains the spectral characteristics of various materials in the geometric model library, such as spectral reflectance, absorption and emissivity parameters, etc., providing material spectral data for subsequent infrared radiation calculations;

[0208] Scene Modeling:

[0209] Build 3D models of various static and dynamic targets and backgrounds in the scene. Set specific weather conditions, ephemeris models, and the position and status of the observer. Set the material and texture of objects in the scene based on the geometric model library and spectral characteristic library.

[0210] Step 2: Infrared radiation modeling

[0211] According to the established scene model and the radiation characteristics of various materials, the infrared radiation of the target and background in the scene is analyzed, and the infrared radiation intensity distribution of the target and background reaching the detector is calculated. There are three main methods for calculating scene temperature and radiation characteristics:

[0212] According to the classical radiation principle, the radiation transfer equation is established, and the scene temperature and radiation distribution are obtained by solving the equation;

[0213] Based on the measured data, the feature distribution of the scene is obtained by using methods such as feature matching and pasting;

[0214] A semi-empirical model is established by combining theory with experiment.

[0215] Step 3: Target background synthesis

[0216] For the entire infrared scene, the target and the background need to be synthesized. The mutual influence between the target and the background needs to be considered during the synthesis, which is usually calculated with the help of radiation transfer theory combined with boundary conditions. There are currently two scene synthesis methods:

[0217] The target and background radiation are modeled separately, the target is embedded in the background radiation image, and only the infrared radiation interaction in the overlapping area of ​​the target and background is considered;

[0218] Based on the theoretical model of target and background infrared radiation, combined with the relative position of the target and background, a coupled infrared feature theoretical model is established to directly calculate the target and background infrared radiation and contrast features. For dynamic infrared scenes, the high dynamic range (HDR) changes of the radiation intensity of the entire infrared scene must also be considered during synthesis.

[0219] Step 4: Atmospheric transport modeling

[0220] There is atmosphere on the path from the scene to the infrared detector, and the infrared radiation of the object is attenuated by the selective absorption and scattering of the atmosphere and suspended particles. Many weather factors, such as wind, rain, fog, haze, snow, etc., directly affect atmospheric attenuation. The main methods for calculating atmospheric attenuation are empirical formula method and atmospheric model method. For infrared imaging, the establishment of an atmospheric transmission model requires the knowledge of atmospheric transmittance, atmospheric radiation, solar (or lunar) radiation, and infrared window, etc., and should include parameters such as geographical location, season, time, climate conditions, atmospheric humidity, visibility, and atmospheric temperature.

[0221] Step 5: Sensor Modeling

[0222] The modeling of infrared sensors includes the modeling of scanning, staring and micro-scanning infrared sensors. Furthermore, the modeling of AC coupling, optical transmittance, defocus error, detector tilt, detector geometric arrangement model, noise (various types), non-uniformity, saturation, sampling artifacts and halo is also included. Noise mainly includes thermal noise, shot noise, 1 / f noise, generation-recombination noise, photon noise, dark current noise and CCD transfer noise.

[0223] After obtaining the synthetic infrared radiation distribution of the scene reaching the detector, the infrared sensor effect should also be considered. The infrared sensor performs a series of conversions on the incident radiation energy and finally forms an infrared thermal image.

[0224] LiDAR sensor data generation

[0225] Using software to simulate LiDAR mainly simulates the perception results of LiDAR by generating simulated LiDAR point cloud data. Taking a single-line LiDAR as an example, the steps to simulate and generate radar sensor data based on the constructed 3D scene are as follows:

[0226] Step (a) is based on the radar device scanning plane, with the radar device as the center point, and a line segment is emitted at every n° interval (the smaller the value of n, the higher the data accuracy and the larger the simulation calculation amount), and the length of the line segment is connected to the radar scanning distance, and the total number of line segments is 360 / n

[0227] Step (b) calculates the intersection of each line segment and the object in the scene, saves the coordinate values ​​of all intersection points, and outputs them in radar data format.

Claims

1. A basic data generation method for unmanned system intelligence level assessment, characterized in that: The steps include: Step 1: Architectural model data production and extension production Constructing a building model database: the building model is divided into three building structures: the base, the main body, and the top. Each building structure has several texture maps of different styles and resolutions; Texture maps are generated based on the building structure, and the texture UV coordinates are set to support reuse and splicing. The UV coordinate system represents the texture mapping of the model surface through two axes, U and V. The coordinate system is defined based on the surface topology and geometry of the model. Each vertex of the texture map has a corresponding UV coordinate, which is used to determine the position of the vertex in the texture image. There are two generation modes for building model data production and expansion: generating building models through free combination or generating building models of different sizes according to requirements; Generate architectural models through free combination Based on the three-segment architectural structure, different styles of architectural models are randomly combined by the algorithm through different combinations and different texture maps; Generate architectural models of different sizes according to requirements. Modelers can manually select the size and texture of architectural models and generate architectural models according to their needs. Step 2: Irregular model making and expansion The irregular model includes ground, vegetation, roads, rivers, and landmarks. Due to the irregular shape of the model itself, it is impossible to use modeling software to establish a single general model, and all models need to be generated according to requirements; The irregular model making and extension basic library is provided with a number of texture maps of different styles and resolutions; the texture map texture UV coordinates support the style of repeated use and splicing; the irregular model making and extension module generates a road model three-dimensional grid according to the input irregular model data boundary, calculates the texture UV coordinates according to the irregular model data boundary, and maps according to the calculated coordinate values; Step 3: Terrain scene data generation Generate terrain scene data by any one or a combination of two or more of the three terrain scene generation technical methods, the three methods being: manual modeling, oblique photography, and superposition of DOM and DEM data; The manual modeling method involves the designer manually constructing a terrain grid, editing the height of each grid point, and then making a surface image for mapping; Oblique photography technology acquires the texture of the top and side views of the terrain scene by simultaneously collecting images from five different viewing angles, no less than one vertical and four oblique ones; DOM and DEM data overlay modeling constructs real three-dimensional terrain by using digital orthophoto DOM data and digital elevation model DEM data; Step 4: Generate basic scene data Based on the basic data generated by the building model data production and expansion module, the irregular model production and expansion module and the terrain scene data, the basic data are combined and conditional parameters are set to generate basic scene data; specifically, the following steps are included: 401: Factor decomposition Decomposing the three-dimensional model constituting the basic scene into independent, spliced ​​or repeatedly usable single elements, wherein the elements include texture maps or single three-dimensional models; The decomposed elements need to be stored according to type, and a description file is configured to describe their attributes; the attributes include type, size, resolution, and storage location; 402: Region Split Divide the scene of a specified size into several areas, and set model labels with different characteristics for each area, including residential areas, commercial areas, transportation hubs, etc. Subsequently, different models will be randomly selected from the feature library or automatically generated according to conditions for filling; 403: Area Fill According to the characteristics of each region, several feature models are randomly selected from the feature library and placed in the region; 404: Scene modification Fill and modify the surroundings of each area with models to form a complete basic scene Step 5: Overlay of meteorological and environmental data Meteorological data is superimposed on the basic scene data, and the meteorological data includes illumination, rain, snow, and fog data; the environmental data is superimposed based on a rendering engine provided by a graphics platform and combined with a particle system and shading language technology to complete the superposition; The particle system implementation steps are as follows: S1: Create a particle emitter The particle emitter is the starting point of the particle system and defines where and how particles are generated; S2: Set particle properties Particle properties include particle size, shape, color, transparency, speed, direction, and life cycle; these properties determine the appearance and behavior of the particle system; S3: Update particle status In each frame of three-dimensional graphics rendering, the position, velocity and direction of the particles are updated according to the properties of the particles and physical rules; the physical rules include gravity and wind force; S4: Drawing particles Drawing the updated particles is achieved through graphics rendering technology; S5: Particle Lifecycle Management Set the time from when particles are emitted to when they disappear, to ensure that particles are generated and disappear at the desired time; Use shading language to achieve different lighting effects at different time periods. The steps are as follows: S1: Define the light source Define the position and color of the sun: This is achieved by defining a light source vector, which points to the position of the sun and contains the color information of the sun; S2: Calculate the light direction For each object in the scene, calculate the lighting direction, which is the direction of the sun position and the object's surface point; get it by subtracting the object's surface point position vector from the sun's position vector; S3: Calculate light intensity The light intensity is calculated by multiplying the light direction and the normal vector of the object surface. The dot product result represents the intensity of the light. The larger the value, the stronger the light. S4: Calculate diffuse reflection Diffuse reflection is the effect of light scattering on the surface of an object. The color of diffuse reflection is calculated using the light intensity and the diffuse reflection coefficient of the object surface. This is achieved through the multiplication operator in the shading language. S5: Calculate specular reflections Highlight reflection is the specular reflection effect produced by light on the surface of an object. The color of the high light reflection is calculated based on the light direction, the sight direction and the normal vector of the object surface. The high light reflection is calculated using power operations and dot multiplication operations. S6: Synthetic lighting results The colors of diffuse and specular reflections are synthesized with the base color of the object surface to obtain the final lighting effect, and the addition operator in the shading language is used to achieve color synthesis; Step 6: Sensor data generation Generate sensor data in real-time simulation based on sensors carried by the unmanned system, wherein the sensors include visible light camera sensors, infrared camera sensors, and lidar sensors; The visible light camera sensor and infrared camera sensor data are generated based on Render To Texture (RTT) technology; Simulate lidar perception results by generating simulated lidar point cloud data.

2. The basic data generation method for unmanned system intelligence level assessment according to claim 1 is characterized in that: The steps of generating model data by using the oblique photography technology are as follows: Step 1: using several sets of aerial photography data or satellite high-definition image sensor data at different angles and orientations as input, wherein the angles and orientations of the input data can cover the top and surrounding areas of the terrain; Step 2: Perform computational processing on a high-performance workstation to fuse and stitch the input data into a terrain scene model.

3. The basic data generation method for unmanned system intelligence level assessment according to claim 1 is characterized in that: The steps of generating model data by superimposing DOM and DEM data are as follows: Step 1: First, obtain the DOM and DEM data of the specified area; Step 2: After acquiring the data, preprocessing is performed, including data cleaning, format conversion, and coordinate system unification to facilitate subsequent processing; Step 3: Generate TIN using an algorithm using DEM data; the algorithm includes Delaunay triangulation; TIN is a method for representing a terrain surface, which is composed of a series of irregular triangles; Step 4: Map the DOM data as a texture onto the TIN so that each triangle has a corresponding texture.

4. The basic data generation method for unmanned system intelligence level assessment according to claim 1 is characterized in that: The visible light camera sensor data generation in step 6 includes the following steps: Step 1: Create the texture Predefine the texture in the computer and adjust the size and resolution of the texture as needed; Step 2: Render to texture By using the functions in the rendering engine to render the scene into a texture, the final result will be saved to the texture; when rendering, use the render target as the render output target and ensure that the rendering result matches the texture size and resolution; Step 3: Applying the texture By building a virtual camera in the scene, rendering the scene image taken by the virtual camera into the prepared two-dimensional texture, and then compressing the texture data and sending it out according to the interface requirements, the visible light sensor data simulation process is realized; After the scene is rendered into the texture, the rendered texture can be applied to other objects or scenes; when applying the texture, the texture is used as a map of the material and the texture is ensured to match the geometry of the object surface; By modifying the FOV (field of view), Near plane (near clipping plane), and Far plane (far clipping plane) of the virtual camera, visible light sensors with different lens parameters can be simulated.

5. The basic data generation method for unmanned system intelligence level assessment according to claim 1 is characterized in that: The infrared camera sensor data generation steps are as follows: Step 1: Infrared scene modeling Infrared scene modeling mainly includes the following three aspects:

101. Establish a geometric model library: Provide geometric models and texture features of infrared targets and backgrounds to serve subsequent scene modeling; 102. Establish spectral characteristics library: The spectral characteristic library contains the spectral characteristics of various materials in the geometric model library, including spectral reflectance, absorption and emissivity parameters, providing material spectral data for subsequent infrared radiation calculations; 103.Scene Modeling: Establish 3D models of static and dynamic targets and backgrounds in the scene; set specific weather conditions, ephemeris models, and the position and status of the observer; set the material and texture of objects in the scene based on the geometric model library and spectral characteristic library; Step 2: Infrared radiation modeling According to the established scene model and the radiation characteristics of various materials, the infrared radiation of the target and background in the scene is analyzed, and the infrared radiation intensity distribution of the target and background reaching the detector is calculated; Step 3: Target background synthesis For the entire infrared scene, infrared synthesis is performed on the target and the background, and the mutual influence between the target and the background is calculated by combining the radiation transfer theory with the boundary conditions; Step 4: Atmospheric transport modeling There is atmosphere on the path from the scene to the infrared detector. The infrared radiation of the object is attenuated by the selective absorption and scattering of the atmosphere and suspended particles. Weather factors, including wind, rain, fog, haze, and snow, directly affect the atmospheric attenuation. Calculate atmospheric attenuation by empirical formula method or atmospheric model method based on atmospheric transmittance, atmospheric radiation, solar radiation, lunar radiation, infrared window, geographical location, season, time, climate conditions, atmospheric humidity, visibility and atmospheric temperature parameters; Step 5: Sensor Modeling Model scanning, staring, and micro-scanning infrared sensors to form infrared thermal images.

6. The basic data generation method for unmanned system intelligence level assessment according to claim 5 is characterized in that: There are three main methods for calculating the scene temperature and radiation characteristics:

201. According to the classical radiation principle, the radiation transfer equation is established, and the scene temperature and radiation distribution are obtained by solving the equation; 202. Based on the measured data, the feature distribution of the scene is obtained by using methods such as feature matching and pasting; 203. Combine theory with experiment to establish a semi-empirical model.

7. The basic data generation method for unmanned system intelligence level assessment according to claim 5 is characterized in that: Step 3: The target scene synthesis method is one of two:

301. Model the target and background radiation separately, embed the target into the background radiation image, and calculate the infrared radiation interaction in the overlapping area of ​​the target and background; 302. Based on the theoretical model of target and background infrared radiation, combined with the relative position of target and background, a coupled infrared feature theoretical model is established to directly calculate the infrared radiation and contrast features of target and background; For dynamic infrared scenes, the high dynamic range (HDR) variation of the radiation intensity of the entire infrared scene is calculated.

8. The basic data generation method for unmanned system intelligence level assessment according to claim 1 is characterized in that: When the radar is a single-line laser radar, the steps of generating radar sensor data based on the constructed three-dimensional scene simulation are as follows: Step (a) based on the radar device scanning plane, with the radar device as the center point, a line segment is emitted at every n° interval, the length of the line segment is the same as the radar scanning distance, and the total number of line segments is 360 / n; Step (b) calculates the intersection of each line segment and the object in the scene, saves the coordinate values ​​of all intersection points, and outputs them in radar data format.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 8 is implemented.

Images