Method, device and equipment for navigation algorithm simulation test and storage medium

Abstract

The invention relates to the technical field of simulation testing, and discloses a navigation algorithm simulation testing method, device and equipment and a storage medium, and the method comprises the steps: loading a three-dimensional model generated based on three-dimensional surveying and mapping data of a target area to a graphic rendering engine; based on the position and the current motion state of the carrier, calculating the force and the torque of the carrier, and based on the force and the torque, generating numerical sensor simulation data; rendering and generating image type sensor simulation data based on the position, a view field range of an image acquisition device of the carrier and a graphic rendering engine; and navigation calculation and simulation test are carried out based on the numerical sensor simulation data and the image sensor simulation data. According to the method, the defect that image data rendering corresponding to the real world cannot be realized by a numerical simulation method in related technologies is overcome, digital twinning of the real world is established through mapping and sampling of the real world, the effect which is almost the same as that of real machine testing can be achieved, and the requirements of accuracy verification and reliability testing of a navigation algorithm are met.

Description

Technical Field

[0001] The present invention relates to the technical field of simulation testing, and in particular to a method, a device, a equipment and a storage medium for simulation testing of a navigation algorithm. Background Art

[0002] In related technologies, the test of multi-source fusion navigation algorithm adopts numerical simulation method, which can well simulate pure numerical measurements such as inertia or ranging through dynamic and kinematic modeling, but cannot realize the rendering of image data corresponding to the real world. Summary of the invention

[0003] In view of this, the present invention provides a method, apparatus, device and storage medium for simulation testing of a navigation algorithm to solve the problem that the numerical simulation method cannot realize the rendering of image data when testing a multi-source fusion navigation algorithm.

[0004] In a first aspect, the present invention provides a method for simulation testing of a navigation algorithm, the method comprising: loading a three-dimensional model generated based on three-dimensional surveying and mapping data of a target area into a graphics rendering engine; calculating the forces and moments acting on the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the forces and moments; rendering and generating image sensor simulation data based on the position, the field of view of an image acquisition device of the vehicle and the graphics rendering engine; and performing navigation solution and simulation testing based on the numerical sensor simulation data and the image sensor simulation data.

[0005] In an optional embodiment, based on the position and current motion state of the vehicle, the forces and torques applied to the vehicle are calculated, and numerical sensor simulation data are generated based on the forces and torques, including: calculating the environmental forces and torques applied to the vehicle based on the position and the current motion state; acquiring the control signal received by the vehicle based on the actuator modeling of the vehicle, converting the control signal into an execution action, and solving the control force and control torque applied to the vehicle based on the actuator model and the execution action, wherein the control signal is generated based on the deviation between the command trajectory of the vehicle and the current motion state; generating the numerical sensor simulation data based on Newton's laws of motion, the environmental forces, the environmental torques, the control forces and the control torques, and updating the motion state at the next moment.

[0006] In an optional embodiment, the numerical sensor simulation data is generated based on Newton's laws of motion, the environmental force, the environmental torque, the control force and the control torque, including: adding zero bias and / or random walk to the true value of the sensor simulation data obtained based on Newton's laws of motion, the environmental force, the environmental torque, the control force and the control torque, and obtaining the numerical sensor simulation data based on the true value of the sensor simulation data after adding the zero bias and / or random walk.

[0007] In an optional embodiment, based on the graphics rendering engine, the position and the vehicle posture, image sensor simulation data is rendered and generated, including: based on the position and the installation position, orientation, and internal parameters of the image acquisition device carried by the vehicle, determining the field of view range of the image acquisition device; based on the field of view range, determining the three-dimensional terrain range and the level of the three-dimensional terrain that need to be loaded; calling the graphics rendering engine to render and generate image sensor simulation data based on the three-dimensional terrain range and the level of the three-dimensional terrain.

[0008] In an optional embodiment, the graphics rendering engine is called to render and generate image sensor simulation data based on the three-dimensional terrain range to be loaded and the level of the three-dimensional terrain, including: based on preset lighting parameters, the three-dimensional terrain range and the level of the three-dimensional terrain, calling the graphics rendering engine to capture an image, adjusting the exposure effect of the captured image, or adding a motion blur effect to the captured image to obtain the image sensor simulation data.

[0009] In an optional embodiment, the loading of the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine includes: using a three-dimensional geographic information data loading module to load the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine; the calculating of the forces and moments exerted on the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the forces and moments includes: using a dynamics and control module to calculate the forces and moments exerted on the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the forces and moments; the rendering and generating an image-type sensor simulation data packet based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine The invention comprises: using a graphics rendering engine module to render and generate image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine; performing navigation solution and simulation test based on the numerical sensor simulation data and the image sensor simulation data comprises: using a sensor simulation data publishing module to obtain the numerical sensor simulation data and the image sensor simulation data, and publishing the numerical sensor simulation data and the image sensor simulation data; the method further comprises: using a navigation algorithm end to receive the numerical sensor simulation data and the image sensor simulation data from the sensor simulation data publishing module based on the middleware, and performing navigation solution and simulation test based on the received data.

[0010] In an optional embodiment, loading a three-dimensional model generated based on three-dimensional surveying and mapping data of a target area into a graphics rendering engine includes: acquiring the three-dimensional surveying and mapping data, converting the three-dimensional surveying and mapping data through a data format to obtain the three-dimensional model, and loading the three-dimensional model into the graphics rendering engine.

[0011] In a second aspect, the present invention provides a device for simulation testing of navigation algorithms, the device comprising: a three-dimensional geographic information data loading module, for loading a three-dimensional model generated based on three-dimensional surveying and mapping data of a target area into a graphics rendering engine; a dynamics and control module, for calculating the forces and moments acting on the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the forces and moments; a graphics rendering engine module, for rendering and generating image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine; and a testing module, for performing navigation solution and simulation testing based on the numerical sensor simulation data and the image sensor simulation data.

[0012] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for navigation algorithm simulation testing of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for simulation testing of a navigation algorithm according to the first aspect or any corresponding embodiment thereof.

[0014] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for simulating and testing a navigation algorithm according to the first aspect or any corresponding embodiment thereof.

[0015] Through the position of the vehicle, the field of view of the vehicle's image acquisition device and the graphics rendering engine, images and other numerical sensor simulation data that are almost identical to the real world can be rendered in real time or at a periodic or regular time interval to perform navigation solution and simulation testing. This solves the deficiency that numerical simulation methods in related technologies cannot achieve rendering of image data corresponding to the real world. Through real-world mapping and sampling, a digital twin of the real world is established, and software-in-the-loop testing is implemented, which can achieve results that are almost the same as actual machine testing, meet the needs of navigation algorithm accuracy verification and reliability testing, reduce testing costs and shorten the iteration cycle of navigation algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic flow chart of a method for simulating and testing a navigation algorithm provided by an embodiment of the present invention is shown;

[0018] Figure 2 A schematic diagram of a process of loading a three-dimensional model into a graphics rendering engine in an embodiment of the present invention is shown;

[0019] Figure 3 A schematic flow chart of another method for simulation testing of a navigation algorithm provided by an embodiment of the present invention is shown;

[0020] Figure 4A schematic flow chart of another method for simulating and testing a navigation algorithm provided by an embodiment of the present invention is shown;

[0021] Figure 5 A schematic flow chart showing another method for simulating and testing a navigation algorithm according to an embodiment of the present invention is shown;

[0022] Figure 6 A schematic diagram of the structure of a device for navigation algorithm simulation testing provided by an embodiment of the present invention is shown;

[0023] Figure 7 A schematic diagram showing the structure of another device for navigation algorithm simulation testing provided by an embodiment of the present invention;

[0024] Figure 8 A schematic diagram of the structure of the dynamics and control module in an embodiment of the present invention is shown;

[0025] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] In the related technologies, the multi-source fusion navigation algorithm with visual odometer and intelligent image matching as the core, integrating inertial measurement, laser ranging and geomagnetic information, overcomes the shortcomings of weak and easily interfered traditional satellite navigation signals, and becomes a hot research direction in the field of navigation. Among them, the principle of the visual navigation part is to collect images in the current environment through the image sensor carried by the vehicle, and perform matching between the previous and next frames, or with existing satellite remote sensing image data, to determine the motion state of the vehicle. In order to meet the needs of accuracy verification and reliability testing of the corresponding navigation algorithm, reduce testing costs and shorten the algorithm iteration cycle, virtual simulation technology is required to generate corresponding sensor simulation data and realize software-in-the-loop testing (Software-in-the-Loop, referred to as SIL). Traditional numerical simulation methods can well simulate pure numerical measurements such as inertia or ranging through dynamics and kinematic modeling, but cannot achieve rendering of image data.

[0028] To make up for the above shortcomings, virtual simulation testing technology based on high-performance graphics rendering engine has become a new technical path. The carrier moves in the pre-built "virtual world", and the simulated image is rendered in real time according to the position of the image sensor and the direction of the lens. Compared with pure numerical simulation, this technology can achieve intuitive visual demonstration and provide a good human-computer interaction experience. At present, the method of building a virtual environment is mainly based on the "building block" arrangement and stacking of various man-made objects, which is still significantly different from the real physical world, and it is impossible to verify the operation effect of navigation algorithms such as intelligent image matching in the real world.

[0029] In order to test the operation effect of the navigation algorithm in the real world, it is necessary to establish a digital twin environment similar to the real world based on the three-dimensional mapping data, so as to achieve an effect that is almost the same as the actual machine test.

[0030] According to an embodiment of the present invention, a method embodiment for simulation testing of a navigation algorithm is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] In this embodiment, a method for simulating and testing a navigation algorithm is provided, which can be used in a terminal with an operating system, such as a mobile phone, a desktop computer, or a tablet computer. Figure 1 A schematic diagram of a method for simulating and testing a navigation algorithm according to an embodiment of the present invention is shown. Figure 1 As shown, the process includes the following steps:

[0032] Step S101: loading a three-dimensional model generated based on three-dimensional surveying and mapping data of a target area into a graphics rendering engine.

[0033] In this step, the 3D mapping data can be an open scene graph binary file (OSGB) obtained by drone oblique photography, or a digital surface model (DSM) and digital orthophoto map (DOM) obtained by satellite remote sensing, or a manually made FBX / OBJ model, etc. FBX (FilmBox) is a file format for 3D models, animations, and scenes, used to share and transmit data between different 3D software; OBJ (Wavefront OBJ) is used for mutual guidance between 3D software models.

[0034] Figure 2 A schematic diagram of the process of loading a three-dimensional model into a graphics rendering engine in an embodiment of the present invention is shown.

[0035] like Figure 2 As shown:

[0036] Step S201, obtaining model data in the format of OSGB, DSM+DOM, FBX or OBJ;

[0037] Step S202, converting the three-dimensional surveying and mapping data acquired in S201 into a three-dimensional model;

[0038] Step S203, the three-dimensional surveying and mapping data can be transmitted by distributing services based on the local data Hypertext Transfer Protocol (HTTP), so that the three-dimensional geographic information data loading module can obtain local terrain data;

[0039] Step S204, the 3D geographic information data loading module obtains the local terrain data and then loads the 3D model into a graphics rendering engine, wherein the graphics rendering engine may be Unity 3D or Unreal Engine.

[0040] Step S102, based on the position and current motion state of the vehicle, the forces and moments acting on the vehicle are calculated, and numerical sensor simulation data is generated based on the forces and moments.

[0041] In this step, the vehicle may include various transportation tools, such as satellites, cars, trains, aircraft, drones, ships, or some industrial equipment or mechanical equipment. The vehicle may be used to provide convenient transportation and transportation tools, and the industrial equipment and mechanical equipment may be cranes or excavators.

[0042] The position of the vehicle may be the coordinates of the vehicle in three-dimensional space. The current motion state of the vehicle may include speed, direction, acceleration, angular velocity or angular acceleration, etc.

[0043] The forces and moments on a vehicle can be calculated based on the vehicle's state of motion and the forces acting on it. The forces on the vehicle can be calculated based on Newton's second law, using the mass and acceleration of the vehicle, and can also include external forces on the vehicle, such as gravity and aerodynamics. The moment is the product of the force and the lever arm, which describes the effect of the force on the vehicle's rotation. The total moment can be calculated by all the forces that have a rotational effect on the vehicle and the corresponding lever arms.

[0044] After obtaining the forces and moments acting on the vehicle, numerical sensor simulation data can be generated based on the forces and moments. Numerical sensor simulation data may include sensor simulation data describing the specific position and direction of the vehicle in three-dimensional space; sensor simulation data describing the motion state of the vehicle and its changes; sensor simulation data describing the vehicle's ability to perceive the environment; sensor simulation data describing the performance characteristics of the sensor, and other sensor simulation data that needs to be simulated.

[0045] Among them, the sensor simulation data describing the specific position and direction of the vehicle in three-dimensional space may include sensor simulation data such as the latitude and longitude, altitude, pitch angle, yaw angle or roll angle of the vehicle. The sensor simulation data describing the motion state of the vehicle and its changes may include sensor simulation data such as the speed, acceleration or angular velocity of the vehicle. The sensor simulation data describing the vehicle's ability to perceive the environment may include sensor simulation data obtained by simulating cameras, laser radars, millimeter-wave radars, etc. The sensor simulation data describing the performance characteristics of the sensor may include sensor simulation data such as the range, sensitivity, linearity or resolution of the sensor. Other sensor simulation data that need to be simulated can determine sensor simulation data such as temperature, pressure, humidity or state parameters of the vehicle's internal system according to the specific type of the vehicle and the simulation requirements.

[0046] Step S103 , based on the position, the field of view of the image acquisition device of the vehicle and the graphics rendering engine, rendering and generating image sensor simulation data.

[0047] In this step, the initial position and orientation of the vehicle in the virtual environment can be determined based on the position of the vehicle. After the initial position and orientation are determined, the field of view of the vehicle can be determined based on the image acquisition device carried by the vehicle. After the field of view is determined, the image can be captured through the graphics rendering engine to obtain image sensor simulation data.

[0048] The image sensor simulation data can be generated by rendering in real time, can be generated at preset time intervals, or can be generated by rendering at regular time intervals.

[0049] Step S104, performing navigation solution and simulation test based on the numerical sensor simulation data and the image sensor simulation data.

[0050] In this step, after the navigation algorithm under test obtains the numerical sensor simulation data and the image sensor simulation data, it preprocesses the two sensor simulation data. Then, the navigation algorithm under test estimates the position, speed, direction and other states of the carrier based on the preprocessed data to evaluate the performance of the navigation algorithm under test.

[0051] In the embodiment of the present invention, the object of the software-in-the-loop test is a vision-based multi-source fusion navigation algorithm, and various elements of the real test are simulated in a simulation environment. The simulation software simulates various test elements, interacts with the vision-based multi-source fusion navigation algorithm through the middleware, and provides an operating environment for the vision-based multi-source fusion navigation algorithm.

[0052] The method for simulation testing of navigation algorithms provided in this embodiment can render images and other numerical sensor simulation data that are almost identical to the real world in real time through the position of the vehicle, the field of view of the vehicle's image acquisition device and the graphics rendering engine, and perform navigation solution and simulation testing, which solves the deficiency that numerical simulation methods in related technologies cannot achieve real-time rendering of image data corresponding to the real world. Through real-world mapping and sampling, a digital twin of the real world is established, and software-in-the-loop testing is implemented, which can achieve results that are almost identical to actual machine testing, meet the needs of navigation algorithm accuracy verification and reliability testing, and can reduce testing costs and shorten the iteration cycle of navigation algorithms.

[0053] In this embodiment, a method for simulating and testing a navigation algorithm is provided, which can be used for the above-mentioned terminal with an operating system, such as a mobile phone, a desktop computer, or a tablet computer, etc. Figure 3 A schematic flow chart of another method for simulating and testing a navigation algorithm provided by an embodiment of the present invention is shown. Figure 3 As shown, the process includes the following steps:

[0054] Step S301, loading the 3D model generated based on the 3D mapping data of the target area into the graphics rendering engine. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0055] Step S302, based on the position and current motion state of the vehicle, the forces and moments acting on the vehicle are calculated, and numerical sensor simulation data is generated based on the forces and moments.

[0056] Specifically, the above step S302 includes:

[0057] Step S3021, based on the position and current motion state, calculate the environmental force and environmental torque acting on the vehicle.

[0058] In this step, define the coordinates and motion state of the vehicle in three-dimensional space. The motion state includes: velocity and acceleration, angle and angular acceleration. To calculate the environmental force, you can use aerodynamic formulas based on the shape, velocity and medium density of the vehicle. To calculate the environmental torque, you can calculate the torque caused by the uneven pressure distribution generated by air flow on the surface of the vehicle. Integrate the calculated environmental forces and environmental torques into the dynamic model of the vehicle, considering the direction and point of action of the force, as well as the rotation axis of the torque. To update the vehicle state, you can use Newton's second law and Euler's equations to update the position, velocity, acceleration, angular velocity, and angular acceleration of the vehicle. Perform iterative calculations and repeat the above steps in the simulation loop to simulate the continuous movement of the vehicle in the environment.

[0059] Step S3022, based on the modeling of the vehicle's actuator, obtain the control signal received by the vehicle, convert the control signal into an execution action, and calculate the control force and control torque exerted on the vehicle based on the model of the actuator and the execution action, wherein the control signal is generated based on the deviation between the vehicle's command trajectory and the current motion state.

[0060] In this step, the actuator of the vehicle can be the engine or brake of a car, or the propeller of a drone. Obtain the command trajectory of the vehicle, that is, the desired motion trajectory or state, and obtain the current motion state of the vehicle, including: position, speed, acceleration, angular velocity or angular acceleration, etc. Calculate the deviation between the command trajectory and the current motion state as the basis for generating the control signal. The deviation is converted into a control signal through a control algorithm, and the control signal is distributed to each actuator to ensure that each actuator can receive the corresponding control instruction. According to the mathematical model of the actuator and the received control signal, the output of the actuator is settled, and the control force and control torque of the vehicle are calculated according to the output of the actuator and the dynamic model of the vehicle. Or the motion state of the vehicle is obtained according to a preset period as feedback for the next control, and the above steps are repeated continuously to form a closed-loop control so that the vehicle can move according to the command trajectory.

[0061] Step S3023, based on Newton's laws of motion, environmental forces, environmental torques, control forces and control torques, generate numerical sensor simulation data and update the motion state at the next moment.

[0062] In this step, after obtaining all the environmental forces, environmental moments, control forces and control moments of the vehicle, the acceleration and angular acceleration of the vehicle are calculated according to Newton's laws of motion, and the position, attitude and velocity of the next moment are obtained by integrating over time. The motion state of the next moment can be updated by numerical integration.

[0063] Step S303, based on the position, the field of view of the image acquisition device of the vehicle and the graphics rendering engine, render and generate image sensor simulation data. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0064] Step S304: Perform navigation calculation and simulation test based on numerical sensor simulation data and image sensor simulation data. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0065] The method for simulation testing of navigation algorithms provided in this embodiment can significantly reduce costs and shorten development cycles by simulating the operating environment of the vehicle's vision-based multi-source fusion navigation algorithm, allowing multiple iterations and tests in the generated digital twin virtual environment without the need to build and test physical prototypes each time.

[0066] In some optional embodiments, numerical sensor simulation data is generated based on Newton's laws of motion, environmental forces, environmental torques, control forces and control torques, including: adding zero bias and / or random walk to the true value of the sensor simulation data obtained based on Newton's laws of motion, environmental forces, environmental torques, control forces and control torques, and obtaining numerical sensor simulation data based on the true value of the sensor simulation data after adding zero bias and / or random walk.

[0067] Specifically, zero bias, random walk, or zero bias and random walk can be added to the true value of sensor simulation data obtained based on Newton's laws of motion, environmental force, environmental torque, control force and control torque. In this embodiment, zero bias and random walk can be added to the calculated true values ​​of acceleration and angular velocity according to the inertial sensor characteristics and error model as the simulation data of inertial measurement. Other numerical sensor simulation data can also be generated in accordance with the above method.

[0068] In this way, the authenticity of the simulation data can be enhanced, making the simulation results closer to the output of the actual sensor, which can improve the authenticity of the simulation.

[0069] In this embodiment, a method for simulating and testing a navigation algorithm is provided, which can be used for the above-mentioned terminal with an operating system, such as a mobile phone, a desktop computer, or a tablet computer, etc. Figure 4 A flowchart of another method for simulating and testing a navigation algorithm provided by an embodiment of the present invention is shown. Figure 4 As shown, the process includes the following steps:

[0070] Step S401: Load the 3D model generated based on the 3D mapping data of the target area into the graphics rendering engine. Figure 1Step S101 of the illustrated embodiment will not be described in detail here.

[0071] Step S402: Calculate the force and torque on the vehicle based on the position and current motion state of the vehicle, and generate numerical sensor simulation data based on the force and torque. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0072] Step S403 , based on the position, the field of view of the image acquisition device of the vehicle and the graphics rendering engine, rendering and generating image sensor simulation data.

[0073] Specifically, the above step S403 includes:

[0074] Step S4031, determining the field of view of the image acquisition device based on the position and the installation position, orientation, and internal parameters of the image acquisition device carried by the vehicle.

[0075] In this step, the specific installation position of the image acquisition device carried by the vehicle on the vehicle is obtained, including its coordinates in the vehicle coordinate system, and the orientation of the image acquisition device, that is, its rotation angle relative to the vehicle coordinate system, is determined. Based on the internal parameters and orientation of the image acquisition device carried by the vehicle, the field of view of the image acquisition device at a specific distance can be calculated. Since the scene facing the image acquisition device will also change as the vehicle moves, the current position of the vehicle and possible movement trajectory can be considered when determining the field of view. The true value of the current position of the vehicle can be calculated by the dynamics and control module, and the field of view of the current image acquisition device of the vehicle can be determined based on the true value of the current position and the installation position, orientation and internal parameters of the image acquisition device.

[0076] Step S4032: Based on the field of view, determine the range of the three-dimensional terrain and the level of the three-dimensional terrain that needs to be loaded.

[0077] In this step, the range and level of the 3D terrain that needs to be loaded and rendered can be determined based on the field of view and the set accuracy threshold. The range of the 3D terrain that needs to be loaded can be calculated based on the field of view and the current position of the vehicle. In order to improve rendering efficiency, the 3D terrain can be divided into multiple levels, and the terrain data of which levels needs to be loaded can be determined based on the field of view and the characteristics of the terrain.

[0078] Step S4033, calling a graphics rendering engine to render and generate image sensor simulation data based on the three-dimensional terrain range and the level of the three-dimensional terrain.

[0079] In this step, the graphics rendering engine can be initialized, and the corresponding terrain data can be loaded according to the determined three-dimensional terrain range and the level of the three-dimensional terrain, and the rendering operation can be performed, and the rendering result can be obtained from the rendering engine to obtain the image sensor simulation data.

[0080] Step S404: Perform navigation calculation and simulation test based on numerical sensor simulation data and image sensor simulation data. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0081] In this way, based on the graphics rendering engine, the position and posture of the vehicle, real-time rendering can be performed to generate realistic image-based sensor simulation data, providing high-quality image input for simulation testing, which can enhance the realism and credibility of the simulation.

[0082] In some optional embodiments, a graphics rendering engine is called to render and generate image sensor simulation data based on the three-dimensional terrain range and the level of the three-dimensional terrain that need to be loaded, including: based on preset lighting parameters, the three-dimensional terrain range and the level of the three-dimensional terrain, the graphics rendering engine is called to capture the image, the exposure effect of the captured image is adjusted, or a motion blur effect is added to the captured image to obtain the image sensor simulation data.

[0083] In this embodiment, the illumination parameters may be set according to the requirements of the simulation scene, and the illumination parameters may include the type, position, intensity, color, and illumination model of the light source. The exposure effect may be adjusted by changing the brightness and contrast of the image or applying a specific exposure algorithm. The motion blur effect may be generated by simulating the movement of the camera relative to the scene during the exposure time or by using a specific image processing algorithm.

[0084] In this way, the image performance under different lighting conditions can be simulated, making the simulated image closer to the real scene, enhancing the realism and dynamics of the image, and improving the simulation effect.

[0085] In this embodiment, a method for simulating and testing a navigation algorithm is provided, which can be used for the above-mentioned terminal with an operating system, such as a mobile phone, a desktop computer, or a tablet computer, etc. Figure 5 A flow chart of another method for simulating and testing a navigation algorithm according to an embodiment of the present invention is shown. Figure 5 As shown, the process includes the following steps:

[0086] Step S501, loading the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine includes: using a three-dimensional geographic information data loading module to load the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine;

[0087] Step S502, calculating the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the force and torque, includes: using a dynamics and control module to calculate the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the force and torque;

[0088] Step S503, based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine, rendering and generating the image sensor simulation data includes: using the graphics rendering engine module to render and generate the image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine;

[0089] Step S504, performing navigation solution and simulation test based on numerical sensor simulation data and image sensor simulation data includes: using a sensor simulation data publishing module to obtain the numerical sensor simulation data and the image sensor simulation data, and publishing the numerical sensor simulation data and the image sensor simulation data;

[0090] The aforementioned method for simulation testing of navigation algorithms also includes: step S505, using the navigation algorithm end to receive numerical sensor simulation data and image sensor simulation data from the sensor simulation data publishing module based on the middleware, and performing navigation solution and simulation testing based on the received data.

[0091] In step S505, the numerical sensor simulation data and the image sensor simulation data generated by the graphics rendering module can be encapsulated into corresponding data structures, and the simulation data can be published through middleware such as remote procedure call (RPC) or robot operating system (ROS), and received by the multi-source fusion navigation algorithm end.

[0092] In some optional embodiments, a three-dimensional model generated based on the three-dimensional mapping data of the target area is loaded into a graphics rendering engine, including: acquiring the three-dimensional mapping data, converting the three-dimensional mapping data into a batched three-dimensional model (Batched 3D Mode, B3DM for short) file, obtaining the three-dimensional model, and loading the three-dimensional model into the graphics rendering engine.

[0093] In this embodiment, B3DM is a data format for batch processing of 3D models, allowing multiple 3D models to be combined into a single optimized data set. By using the functions of B3DM, the 3D mapping data can be optimized, compressed and batch processed to improve the performance and loading speed of the 3D model. The 3D model obtained after conversion to B3DM format has better performance and visualization effect, and is more suitable for simulation testing of navigation algorithms.

[0094] Figure 6 The structural schematic diagram of the device for navigation algorithm simulation test provided by the embodiment of the present invention is shown. In this embodiment, a device for navigation algorithm simulation test is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0095] This embodiment provides a device for simulating and testing a navigation algorithm, such as Figure 6 As shown, including:

[0096] The three-dimensional geographic information data loading module 601 is used to load the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine.

[0097] The dynamics and control module 602 is used to calculate the forces and torques acting on the vehicle based on the position and current motion state of the vehicle, and to generate numerical sensor simulation data based on the forces and torques.

[0098] The graphics rendering engine module 603 is used to render and generate image sensor simulation data based on the position, the field of view of the vehicle's image sensor, and the graphics rendering engine.

[0099] The testing module 604 is used to perform navigation solution and simulation test based on numerical sensor simulation data and image sensor simulation data.

[0100] Figure 7 FIG. 2 shows a schematic diagram of the structure of another device for simulating and testing a navigation algorithm provided by an embodiment of the present invention. Figure 7As shown, the three-dimensional geographic information data loading module 601 obtains three-dimensional mapping data, and transmits the obtained batch three-dimensional model to the graphics rendering engine module 602; the dynamics and control module 602 obtains the command motion trajectory, transmits the calculated numerical sensor simulation data to the sensor simulation data publishing module 6041, and transmits the obtained vehicle current position and posture to the graphics rendering engine module 602; after the graphics rendering engine module 602 obtains the batch three-dimensional model data, the vehicle current position and posture, it obtains the image sensor simulation data, and transmits the image sensor simulation data to the sensor simulation data publishing module 6041; after the sensor simulation data publishing module 6041 obtains the numerical sensor simulation data and the image sensor simulation data, it publishes the numerical sensor simulation data and the image sensor simulation data through the middleware; the multi-source fusion navigation algorithm under test 6042 receives the numerical sensor simulation data and the image sensor simulation data. Among them, the test module 604 includes: the sensor simulation data publishing module 6041 and the multi-source fusion navigation algorithm under test 6042.

[0101] Figure 8 FIG. 2 shows a schematic diagram of the structure of the dynamics and control module in an embodiment of the present invention. Figure 8 As shown, in some optional embodiments, the dynamics and control module 602 includes:

[0102] The first dynamics and control unit 6021 is used to calculate the environmental forces and environmental moments acting on the vehicle based on the position and current motion state.

[0103] The second dynamics and control unit 6022 is used to model the vehicle's actuator, obtain the control signal received by the vehicle, convert the control signal into an execution action, and calculate the control force and control torque exerted on the vehicle based on the actuator model and the execution action, wherein the control signal is generated based on the deviation between the vehicle's command trajectory and the current motion state.

[0104] The third dynamics and control unit is used to generate numerical sensor simulation data based on Newton's laws of motion, environmental forces, environmental torques, control forces and control torques, and update the motion state at the next moment.

[0105] In some optional embodiments, the third dynamics and control unit includes:

[0106] The third dynamics and control subunit is used to add zero bias and / or random walk to the true value of sensor simulation data obtained based on Newton's laws of motion, environmental forces, environmental torques, control forces and control torques, and obtain numerical sensor simulation data based on the sensor simulation data after adding zero bias and / or random walk.

[0107] In some optional implementations, the graphics rendering engine module 603 includes:

[0108] The first graphics rendering engine unit is used to determine the field of view range of the image acquisition device based on the position and installation position, orientation, and internal parameters of the image acquisition device carried by the vehicle; based on the field of view range, determine the three-dimensional terrain range and the level of the three-dimensional terrain that need to be loaded; call the graphics rendering engine to render and generate image sensor simulation data based on the three-dimensional terrain range and the level of the three-dimensional terrain.

[0109] In some optional implementations, the first graphics rendering engine unit includes:

[0110] The first graphics rendering engine subunit is used to call the graphics rendering engine to capture images based on preset lighting parameters, three-dimensional terrain range and three-dimensional terrain levels, adjust the exposure effect of the captured image, or add motion blur effect to the captured image to obtain image sensor simulation data.

[0111] In some optional implementations, the three-dimensional geographic information data loading module 601 includes:

[0112] The three-dimensional geographic information data loading unit is used to obtain three-dimensional surveying and mapping data, convert the three-dimensional surveying and mapping data into batch three-dimensional model files, obtain the three-dimensional model, and load the three-dimensional model into the graphics rendering engine.

[0113] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0114] The device for simulation testing of navigation algorithms in this embodiment can render images and other numerical sensor simulation data that are almost identical to the real world in real time or at periodic or regular time intervals through the position of the vehicle, the field of view of the vehicle's image acquisition device, and the graphics rendering engine, to perform navigation solution and simulation testing, thereby solving the problem that numerical simulation methods in related technologies cannot render image data. Through real-world surveying and sampling, a digital twin of the real world is established to implement software-in-the-loop testing, which can achieve results that are almost identical to actual machine testing, meet the needs of navigation algorithm accuracy verification and reliability testing, reduce testing costs, and shorten the iteration cycle of navigation algorithms.

[0115] The device for navigation algorithm simulation testing in this embodiment is presented in the form of a functional unit, where the unit refers to an application specific integrated circuit (ASIC) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0116] The embodiment of the present invention also provides a computer device having the above Figure 6 The device shown is used for simulation testing of navigation algorithms.

[0117] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0118] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0119] The aforementioned memory 20 stores instructions executable by at least one processor 10, so that the aforementioned at least one processor 10 executes the method shown in the above embodiment.

[0120] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0121] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0122] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0123] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (such as a light emitting diode) and a tactile feedback device (such as a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0124] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0125] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0126] By using the method, apparatus, equipment, storage medium and program product for simulation testing of navigation algorithms provided in the embodiments of the present invention, simulation tests have been successfully carried out on computer vision navigation and positioning algorithms such as visual-inertial odometer and satellite remote sensing map matching, with good results.

[0127] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for simulation testing of a navigation algorithm, characterized in that: The method comprises: Loading a three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into a graphics rendering engine; Calculate the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generate numerical sensor simulation data based on the force and torque; Rendering and generating image sensor simulation data based on the position, the field of view of the image acquisition device of the vehicle, and the graphics rendering engine; Navigation solution and simulation test are performed based on the numerical sensor simulation data and the image sensor simulation data.

2. The method according to claim 1, characterized in that Based on the position and current motion state of the vehicle, the force and torque applied to the vehicle are calculated, and numerical sensor simulation data is generated based on the force and torque, including: Calculate the environmental force and the environmental moment on the vehicle based on the position and the current motion state; Based on the modeling of the actuator of the vehicle, a control signal received by the vehicle is obtained, the control signal is converted into an execution action, and the control force and control torque exerted on the vehicle are calculated based on the model of the actuator and the execution action, wherein the control signal is generated based on the deviation between the command trajectory of the vehicle and the current motion state; Based on Newton's laws of motion, the environmental force, the environmental torque, the control force and the control torque, the numerical sensor simulation data is generated, and the motion state at the next moment is updated.

3. The method according to claim 2, characterized in that Generating the numerical sensor simulation data based on Newton's laws of motion, the environmental force, the environmental torque, the control force and the control torque includes: Zero bias and / or random walk are added to the true value of sensor simulation data obtained based on Newton's laws of motion, the environmental force, the environmental torque, the control force and the control torque, and the numerical sensor simulation data is obtained based on the true value of the sensor simulation data after adding the zero bias and / or random walk.

4. The method according to claim 1, characterized in that: Rendering and generating image sensor simulation data based on the graphics rendering engine, the position and the vehicle posture includes: Determine the field of view of the image acquisition device based on the position and the installation position, orientation, and internal parameters of the image acquisition device carried by the vehicle; Based on the field of view, determine the range of the three-dimensional terrain and the level of the three-dimensional terrain that needs to be loaded; The graphic rendering engine is called to render and generate image sensor simulation data based on the three-dimensional terrain range and the level of the three-dimensional terrain.

5. The method according to claim 4, characterized in that Calling the graphics rendering engine to render and generate image sensor simulation data based on the three-dimensional terrain range to be loaded and the level of the three-dimensional terrain, including: Based on preset lighting parameters, the three-dimensional terrain range and the level of the three-dimensional terrain, the graphics rendering engine is called to capture images, the exposure effect of the captured image is adjusted, or a motion blur effect is added to the captured image to obtain the image sensor simulation data.

6. The method according to claim 1, characterized in that The loading of the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine comprises: using a three-dimensional geographic information data loading module to load the three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into the graphics rendering engine; The calculating the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the force and torque comprises: using a dynamics and control module to calculate the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the force and torque; The rendering and generating image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine includes: using a graphics rendering engine module to render and generate image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine; The performing navigation solution and simulation test based on the numerical sensor simulation data and the image sensor simulation data comprises: using a sensor simulation data publishing module to obtain the numerical sensor simulation data and the image sensor simulation data, and publishing the numerical sensor simulation data and the image sensor simulation data; The method also includes: using the navigation algorithm end to receive the numerical sensor simulation data and the image sensor simulation data from the sensor simulation data publishing module based on the middleware, and performing navigation solution and simulation test based on the received data.

7. The method according to claim 1, characterized in that Loading the 3D model generated based on the 3D mapping data of the target area into the graphics rendering engine includes: The three-dimensional surveying and mapping data is acquired, the three-dimensional surveying and mapping data is converted into a data format to obtain the three-dimensional model, and the three-dimensional model is loaded into the graphics rendering engine.

8. A device for simulation testing of navigation algorithms, characterized in that: The device comprises: A three-dimensional geographic information data loading module is used to load a three-dimensional model generated based on the three-dimensional surveying and mapping data of the target area into a graphics rendering engine; A dynamics and control module, used for calculating the force and torque applied to the vehicle based on the position and current motion state of the vehicle, and generating numerical sensor simulation data based on the force and torque; A graphics rendering engine module, used for rendering and generating image sensor simulation data based on the position, the field of view of the image sensor of the vehicle, and the graphics rendering engine; A testing module is used to perform navigation solution and simulation test based on the numerical sensor simulation data and the image sensor simulation data.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for navigation algorithm simulation testing described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for navigation algorithm simulation testing according to any one of claims 1 to 7.

Images