Three-dimensional scene reconstruction method, method for intelligent driving analog simulation, computer equipment, storage medium and program product

Abstract

The invention discloses a three-dimensional scene reconstruction method, a method for intelligent driving analog simulation, computer equipment, a storage medium and a program product. The three-dimensional scene reconstruction method comprises the steps that an earth space is divided into a plurality of blocks, and the earth space comprises a spherical space in a preset range above the earth surface; acquiring scene data of the earth space for three-dimensional reconstruction to obtain a three-dimensional reconstruction result, the three-dimensional reconstruction result comprising a reconstruction model and reconstruction model information; and dividing the reconstruction model into corresponding blocks according to the reconstruction model information. According to the embodiment of the invention, the spherical space in the preset range above the earth surface is divided into the plurality of blocks, the three-dimensional reconstruction is carried out according to the acquired scene data in the spherical space to obtain the plurality of reconstruction models, and finally the plurality of reconstruction models obtained by reconstruction are distributed and stored in the corresponding blocks of the spherical space according to the reconstruction model information. And unified management of a plurality of reconstruction models is realized.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular, to a three-dimensional scene reconstruction method, a method for intelligent driving simulation, a computer device, a storage medium, and a program product. Background Art

[0002] During the development of an intelligent driving system, it is necessary to perform three-dimensional reconstruction of the real world and simulate it in a simulator to meet the testing and development requirements of perception, planning, control, etc. in the development of the intelligent driving system. The reconstruction methods in related technologies mainly assume that the constructed scene is planar and reconstruct the scene within a small range. When reconstructing the scene, a scene consistent with the real world is created by scanning. This method has a small acquisition range, loses geographical location information, and lacks an efficient management method for multiple small scenes. Summary of the Invention

[0003] Embodiments of this application provide a three-dimensional scene reconstruction method, a method for intelligent driving simulation, a computer device, a storage medium, and a program product, which are used to solve at least one of the above technical problems.

[0004] In a first aspect, embodiments of this application provide a three-dimensional scene reconstruction method, including: Dividing the earth space into multiple blocks, where the earth space includes a spherical space within a preset range above the earth's surface; Obtaining scene data of the earth space for three-dimensional reconstruction to obtain a three-dimensional reconstruction result, where the three-dimensional reconstruction result includes a reconstruction model and reconstruction model information; According to the reconstruction model information, dividing the reconstruction model into corresponding blocks.

[0005] In some embodiments, the multiple blocks are distributed in multiple layers, each layer includes at least one block, the number of blocks included in the layer farther from the earth's surface in the multiple layers is smaller, and the size of the blocks included in the layer closer to the earth's surface in the multiple layers is smaller.

[0006] In some embodiments, the reconstruction model information includes the longitude and latitude information and the size information of the reconstruction model; the step of dividing the reconstruction model into corresponding blocks according to the reconstruction model information includes: Determining the layer where the target block is located according to the size information of the reconstruction model and the sizes of the multiple blocks; Determining the target block from the layer where the target block is located according to the longitude and latitude information of the reconstruction model; Dividing the reconstruction model into the target block.

[0007] In some embodiments, obtaining scene data of the earth space for three-dimensional reconstruction includes: The scene data is obtained by a collection terminal scanning a scene; the collection terminal includes at least one of a radar device, an image sensor, and an RTK device, and the collection terminal is mounted on at least one of a drone, a collection vehicle, and a handheld device; Three-dimensional reconstruction is performed according to the scene data.

[0008] In some embodiments, it further includes: performing semantic segmentation on the reconstruction model to obtain semantic segmentation information; constructing OpenDrive data according to the semantic segmentation information.

[0009] In some embodiments, it further includes: while obtaining the scene data by the collection terminal scanning the scene, recording at least one of the position, altitude, temperature, humidity, and season of the scene.

[0010] In a second aspect, an embodiment of the present application provides a method for intelligent driving simulation, which is characterized by including: Performing three-dimensional scene reconstruction by the three-dimensional scene reconstruction method according to any embodiment of the present application; Performing intelligent driving simulation based on the reconstruction model obtained by three-dimensional scene reconstruction.

[0011] In some embodiments, performing intelligent driving simulation based on the reconstruction model obtained by three-dimensional scene reconstruction includes: According to the position information of the observation area, streaming and loading the reconstruction models of the corresponding blocks from multiple blocks in the earth space for scene rendering, so as to perform intelligent driving simulation according to the scene rendering result.

[0012] In some embodiments, according to the position information of the observation area, streaming and loading the reconstruction models of the corresponding blocks from multiple blocks in the earth space for scene rendering includes: Determining the visible blocks among the multiple blocks according to the position information of the observation area; Loading the reconstruction models corresponding to the visible blocks, and loading the ego vehicle simulation information; Combining the reconstruction models of the visible blocks and the ego vehicle simulation information for scene rendering.

[0013] In some embodiments, when the number of the visible blocks is at least two, the loading the reconstruction models corresponding to the visible blocks includes: According to the reconstruction model longitude and latitude information of the reconstruction models of at least two of the visible blocks, splicing the reconstruction models of at least two of the visible blocks, and loading the spliced reconstruction model.

[0014] In some embodiments, the position information of the observation area includes the longitude and latitude information and altitude information of the observation area; the determining the visible blocks among the multiple blocks according to the position information of the observation area includes: Determine the visible blocks among the multiple blocks according to the longitude and latitude information and altitude information of the observation area and the reconstructed model longitude and latitude information of the multiple reconstructed models corresponding to the multiple blocks.

[0015] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of any one of the above three-dimensional scene reconstruction methods and / or methods for intelligent driving simulation in the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of any one of the above three-dimensional scene reconstruction methods and / or methods for intelligent driving simulation in the present application are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of any one of the above three-dimensional scene reconstruction methods and / or methods for intelligent driving simulation in the present application are implemented.

[0018] In this embodiment of the present application, a spherical space within a preset range above the earth's surface is divided into multiple blocks, and three-dimensional reconstruction is performed based on the acquired scene data in the spherical space to obtain multiple reconstructed models. Finally, according to the reconstructed model information, the multiple reconstructed models obtained by reconstruction are allocated and stored in the corresponding blocks of the spherical space, realizing the unified management of the multiple reconstructed models. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a flowchart of an embodiment of the three-dimensional reconstruction method of the present application; Figure 2 It is a schematic flowchart of another embodiment of the three-dimensional reconstruction method of the present application; Figure 3 It is a schematic flowchart of an embodiment of the method for intelligent driving simulation in the present application; Figure 4 It is a schematic flowchart of another embodiment of the method for intelligent driving simulation in the present application; Figure 5Schematic flowchart of another embodiment of the method for intelligent driving simulation in this application; Figure 6 Schematic structural diagram of one embodiment of the computer device in this application. Detailed implementation manners

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0022] It should also be noted that in this text, the terms "include" and "comprise" not only include those elements, but also other elements not explicitly listed, or elements inherent to such a process, method, article, or device. Without further limitation, the elements defined by the statement "include..." do not exclude the existence of additional identical elements in the process, method, article, or device including the said elements.

[0023] As Figure 1 shown, the embodiments of this application provide a three-dimensional scene reconstruction method, including: S10. Divide the earth space into multiple blocks, where the earth space includes a spherical space within a preset range above the earth's surface.

[0024] Exemplarily, the multiple blocks are distributed in multiple layers, each layer includes at least one block, the number of blocks in the layer farther from the earth's surface in the multiple layers is less, and the size of the blocks in the layer closer to the earth's surface in the multiple layers is smaller.

[0025] S20. Obtain the scene data of the earth space for three-dimensional reconstruction to obtain a three-dimensional reconstruction result, where the three-dimensional reconstruction result includes a reconstruction model and reconstruction model information.

[0026] Exemplarily, obtain the scene data of multiple scenes in the earth space for three-dimensional reconstruction to obtain multiple reconstruction models and corresponding reconstruction model information. Among them, the multiple scenes may be scenes with a physical range from several meters to dozens of meters (such as a single object, a room, furniture, a statue, a human body, etc.).

[0027] S30. According to the reconstruction model information, divide the reconstruction model into corresponding blocks.

[0028] In this embodiment, a spherical space within a preset range above the earth's surface is divided into multiple blocks. Based on three-dimensional earth rendering, and according to the scene data obtained in the spherical space, multiple reconstruction models are obtained through three-dimensional reconstruction. Finally, according to the reconstruction model information, the multiple reconstructed models are allocated and stored in the corresponding blocks of the spherical space, realizing the unified management of the multiple reconstruction models.

[0029] As Figure 2 shown in the flowchart of another embodiment of the three-dimensional reconstruction method of the present application. In this embodiment, the reconstruction model information includes the longitude and latitude information of the reconstruction model and the size information of the reconstruction model; the step of dividing the reconstruction model into the corresponding blocks according to the reconstruction model information includes: S31. Determine the layer where the target block is located according to the size information of the reconstruction model and the sizes of the multiple blocks.

[0030] Exemplarily, the sizes of the multiple blocks are divided according to the size of the reconstruction model to be stored, so as to store the reconstruction models corresponding to scenes of different sizes. In addition, the multiple blocks of the spherical space are distributed in multiple layers, and the sizes of the blocks included in the layers closer to the earth's surface are smaller, that is, blocks of different sizes are distributed in different layers (for example, each layer may include multiple blocks of the same size, or each layer may include multiple blocks with sizes close to each other). Therefore, the layer where the target block is located can be determined according to the size information of the reconstruction model and the sizes of the multiple blocks.

[0031] S32. Determine the target block from the layer where the target block is located according to the longitude and latitude information of the reconstruction model. Exemplarily, after determining the layer where the target block is located, there may still be multiple blocks in this layer available for storing the reconstruction model. However, since the scene corresponding to the reconstruction model has a physical location, the target block with a matching physical location is further determined from the layer where the target block is located according to the longitude and latitude information of the corresponding reconstruction model.

[0032] S33. Divide the reconstruction model into the target block.

[0033] In this embodiment, a correspondence relationship between the reconstruction model and the blocks in the spherical space is established from two dimensions: the size of the reconstruction model and the position of the reconstruction model, realizing the true modeling of the real world and the unified storage management of the reconstruction models.

[0034] In some embodiments, obtaining scene data of the geospace for 3D reconstruction includes: acquiring scene data by a collection terminal scanning a scene; the collection terminal includes at least one of a radar device, an image sensor, and an RTK device, and the collection terminal is mounted on at least one of a drone, a collection vehicle, and a handheld device; performing 3D reconstruction based on the scene data. Exemplarily, technologies such as Nerf or GaussianSplatting are used to scan the scene.

[0035] In this embodiment, drones, collection vehicles, and handheld devices are comprehensively utilized to collect data for different scenes and road structures. Among them, drones, collection vehicles, and handheld devices can be configured with lidar and / or image sensors and / or RTK devices to collect scene data, and a high-precision reconstruction model can be created to meet the requirements of intelligent driving simulation for scene resolution. The created reconstruction model can be in formats such as Gltf (supporting PBR and relighting) or 2DGS (supporting PBR and relighting), etc.

[0036] Exemplarily, acquiring scene data by a collection terminal scanning a scene includes: Step 1, data preparation: Ensure that both the collected scene and the constructed reconstruction model have accurate geographical location information (such as longitude, latitude, altitude, etc.); Step 2, unify the coordinate system: All data uses the same geographical coordinate system (WGS84) to ensure that stitching can be performed under the same reference framework; Step 3, block and layer storage: The constructed reconstruction model is stored in different layers according to its size (the constructed reconstruction model should be saved to the corresponding data layer according to its size. Larger reconstruction models are at higher levels, and smaller reconstruction models are at lower levels. When the viewing angle is close to the ground, the higher-level models are preferentially loaded).

[0037] In some embodiments, it further includes: performing semantic segmentation on the reconstruction model to obtain semantic segmentation information; constructing OpenDrive data based on the semantic segmentation information.

[0038] Exemplarily, after 3D reconstruction, semantic segmentation is performed on the reconstruction model to extract road edges and lane lines. Then, based on this information, the corresponding OpenDrive data is constructed (OpenDrive is a format standard for high-precision maps, and a high-precision map can be obtained based on the xodr file of OpenDrive), meeting the requirements of intelligent driving tests for high-precision maps.

[0039] In some embodiments, it further includes: while acquiring scene data by a collection terminal scanning a scene, recording at least one of the position, altitude, temperature, humidity, and season of the scene.

[0040] When reconstructing the model for the scene in this embodiment, information such as the position, altitude, temperature, humidity, season, and weather of the reconstructed model is automatically recorded and marked, which can provide basic feature information for the later matching of the reconstructed model (for example, when conducting intelligent driving training or synthetic data generation, there will be different requirements for the scene, such as: night, rainy day, foggy day, etc. Based on this information, the area that meets the requirements can be quickly matched and found on the terrain).

[0041] As Figure 3 shown, it is a schematic flowchart of an embodiment of the method for intelligent driving simulation in the present application. In this embodiment, the following steps are included: S40. Perform three-dimensional scene reconstruction through the three-dimensional scene reconstruction method described in any embodiment of the present application. Exemplarily, the following steps are pre-adopted for three-dimensional scene reconstruction: divide the earth space into multiple blocks, where the earth space includes a spherical space within a preset range above the earth's surface; obtain the scene data of the earth space for three-dimensional reconstruction to obtain a three-dimensional reconstruction result, where the three-dimensional reconstruction result includes a reconstructed model and reconstructed model information; according to the reconstructed model information, divide the reconstructed model into the corresponding blocks.

[0042] S50. Perform intelligent driving simulation based on the reconstructed model obtained from three-dimensional scene reconstruction. Exemplarily, according to the needs of the simulation, obtain the required reconstructed model from the multiple reconstructed models obtained from three-dimensional scene reconstruction for intelligent driving simulation.

[0043] In this embodiment, the three-dimensional reconstruction method described in any of the foregoing embodiments of the present application is pre-adopted to perform three-dimensional reconstruction of the real world, obtaining reconstructed models of multiple scenes in the real world and uniformly managing and storing them in the blocks of the spherical space, so that the required reconstructed model can be quickly queried and obtained for simulation in intelligent driving simulation.

[0044] In some embodiments, the performing intelligent driving simulation based on the reconstructed model obtained from three-dimensional scene reconstruction includes: according to the position information of the observation area, streamingly load the reconstructed models of the corresponding blocks from multiple blocks of the earth space for scene rendering, so as to perform intelligent driving simulation according to the scene rendering result.

[0045] As Figure 4 shown, it is a schematic flowchart of an embodiment of the method for intelligent driving simulation in the present application. In this embodiment, according to the position information of the observation area, streamingly loading the reconstructed models of the corresponding blocks from multiple blocks of the earth space for scene rendering includes: S51. Determine the visible blocks among the multiple blocks according to the position information of the observation area.

[0046] Exemplarily, when displaying the Earth through a simulation software or a simulation engine (such as the Unreal Engine), an observation camera (e.g., a pinhole camera) is used. The visible area (observation area) of the pinhole camera is a cone. The observation area determines which areas of the terrain are visible and need to be loaded for display. The blocks observed by the rectangular view of the observation camera are the current visible blocks.

[0047] Exemplarily, the user can select the observation area by themselves. Additionally, one or more of the location, altitude, temperature, humidity, and season of the scene can be used to search for or filter out eligible blocks, and then the recommended observation areas are displayed based on these blocks for the user to select.

[0048] Exemplarily, the location information of the observation area includes the longitude and latitude information and altitude information of the observation area; determining the visible blocks among the multiple blocks according to the location information of the observation area includes: determining the visible blocks among the multiple blocks according to the longitude and latitude information, altitude information of the observation area, and the longitude and latitude information of the reconstruction models of the multiple reconstruction models corresponding to the multiple blocks. For example, a planet plugin implemented based on Unreal will stream and load the reconstruction models in the corresponding layer from the multiple reconstruction models stored in multiple blocks according to the position (longitude and latitude + altitude) of the observation camera and the rendering settings of the observation angle, etc., to achieve the display of the planet-level scene.

[0049] S52. Load the reconstruction model corresponding to the visible block and load the ego-vehicle simulation information. Exemplarily, the ego-vehicle simulation information includes an ego-vehicle model, sensor data, and a dynamics model; load the reconstruction model and the ego-vehicle simulation information into the simulator.

[0050] S53. Combine the reconstruction model of the visible block and the ego-vehicle simulation information for scene rendering.

[0051] Exemplarily, the simulator performs scene rendering according to the reconstruction model and the ego-vehicle simulation information. In this embodiment, by combining the reconstruction model and the ego-vehicle simulation information simultaneously, not only the real-world scene is simulated, but also the vehicle model, sensor data, and dynamics model are combined, making the simulated scene closer to the real driving scene.

[0052] In some embodiments, when the number of the visible blocks is at least two, loading the reconstruction model corresponding to the visible block includes: splicing the reconstruction models of at least two visible blocks according to the longitude and latitude information of the reconstruction models of at least two visible blocks, and loading the spliced reconstruction model.

[0053] Exemplarily, the multi-win scenario of the reconstruction models of at least two of the visible blocks is a small scenario, such as a scenario with a physical range of several meters to dozens of meters (such as a single object, a room, furniture, a statue, a human body, etc.).

[0054] In this embodiment, by splicing the reconstruction models of at least two adjacent visible blocks according to the longitude and latitude information of the reconstruction models of multiple small scenarios, a reconstruction model of a large scenario or an ultra-large scenario is obtained by splicing the reconstruction models based on small scenarios. Among them, a large scenario can be, for example, a scenario with a physical range above the hundred-meter level (such as an urban block, a mountain range, a forest, a factory park, etc.); an ultra-large scenario can be, for example, a scenario with a physical range from the ten-thousand-meter level (10 kilometers) to the global scale (such as a city, a national park, a mountain range system, a marine area, the surface of a planet).

[0055] This embodiment effectively organizes and manages the collected small scenarios according to the geographic coordinate system, and can splice the small scenarios collected multiple times into a large scenario or an ultra-large scenario through the geographic coordinate system, thus meeting the requirements of long-distance testing of intelligent driving vehicles.

[0056] As Figure 5 shown, it is a schematic flowchart of an embodiment of the method for intelligent driving simulation of the present application. In this embodiment, the following steps are included: S100. Divide the earth space according to the hierarchy and block size.

[0057] Exemplarily, in the pyramid tile (Tile-Pyramid) management scheme adopted by the present application, the earth area is split into multiple layers, similar to a pyramid structure. When the observation camera is far from the earth's surface, the data of layer 0 is displayed. When the camera is closer to the earth, the data of higher levels is used in turn. In the pyramid tiles, each layer is composed of a different number of blocks. According to the Tile-Pyramid rule: there is one block in layer 0, four blocks in layer 1, sixteen blocks in layer 2... and so on. The number of blocks is equal to 4 to the power of the layer index.

[0058] Exemplarily, the earth space refers to the spatial positions and distributions on the earth's surface or its three-dimensional environment. The WGS84 (World Geodetic System 1984) coordinate system is used in the present application. The range of the earth space is as follows: the latitude range is from -90 to +90; the longitude range is from -180 to +180; the earth radius is taken as 6378137 meters; the altitude range is taken from the earth's surface to 3 * 6378137 meters.

[0059] S200. Use Nerf or GaussianSplatting technology to scan the scenario to obtain a reconstruction model.

[0060] Exemplarily, a reconstruction model is obtained by using a drone, a scanning vehicle, a handheld device, etc. to perform scene scanning using Nerf or GaussianSplatting technology.

[0061] S300. According to the longitude, latitude and size of the reconstruction model, divide the reconstruction model into corresponding Earth Tiles (i.e., blocks).

[0062] Exemplarily, an Earth Tile refers to a specific block in a corresponding layer after the Earth is divided into layers and blocks. Its main function is to facilitate the confirmation of which resources need to be loaded and displayed and which resources need to be unloaded when the camera observes the Earth. Because it is unrealistic to load and display all the resources of the Earth at the same time.

[0063] Exemplarily, according to the size of the reconstruction model, determine the size of the required block. According to the size of the block, the layer where the reconstruction model is saved can be determined; according to the longitude and latitude of the reconstruction model, find the index of the block in the corresponding layer, and store the reconstruction model in the storage directory corresponding to the block.

[0064] S400. According to the observation area, load the surface and elevation data of the Earth and automatically triangulate and render the Earth.

[0065] Exemplarily, when displaying the Earth in Unreal, a pinhole camera is used. The visible area (observation area) of the pinhole camera is a cone. The observation area determines which terrain areas are visible and need to be loaded for display. Among them, the elevation data is a height map that determines the shape of the Earth's surface.

[0066] The method of automatically triangulating and rendering the Earth includes, for example: first define the Earth as a regular icosahedron; the triangles that make up the icosahedron are all equilateral triangles; according to the height of the observation camera from the ground, recursively divide the corresponding equilateral triangle into three new equilateral triangles; for the vertices of the divided equilateral triangles, update the height according to the elevation data; fill the seams with newly added triangles. It should be noted that the above is only an example, and this application is not limited thereto. The specific implementation method can refer to related technologies.

[0067] S500. According to the visible Tiles, load the models and display them.

[0068] Exemplarily, according to the visible Tiles, load and display the reconstruction models corresponding to the visible Tiles. Among them, the Tiles observed by the rectangular view of the observation camera, that is, the currently visible Tiles, are convenient for the block management of the Earth resources. The closer to the ground, the more resources need to be displayed. Blocking and only loading the visible blocks can reduce the amount of resources that need to be loaded into the memory and video memory.

[0069] S600. Load the vehicle simulation information and start the simulation.

[0070] Exemplarily, a model of the vehicle itself, sensor data, a dynamics model, etc. can be loaded. The rendering of the Earth and the rendering of the vehicle itself can be carried out in the simulator. In this embodiment, the simulator can be used to load the simulation information of the vehicle itself and the required reconstruction model for intelligent driving simulation.

[0071] In some embodiments, the present application further provides a computer device, including a memory, a processor, and a computer program stored on the memory. It is characterized in that the processor executes the computer program to implement the steps of any one of the above three-dimensional scene reconstruction methods and / or the method for intelligent driving simulation in the present application.

[0072] In some embodiments, the present application further provides a computer-readable storage medium, on which a computer program / instructions are stored. It is characterized in that when the computer program / instructions are executed by the processor, the steps of any one of the above three-dimensional scene reconstruction methods and / or the method for intelligent driving simulation in the present application are implemented.

[0073] In some embodiments, the present application further provides a computer program product, including a computer program / instructions. It is characterized in that when the computer program / instructions are executed by the processor, the steps of any one of the above three-dimensional scene reconstruction methods and / or the method for intelligent driving simulation in the present application are implemented.

[0074] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of actions combined. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application. In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] Figure 6 is a schematic diagram of the hardware structure of a computer device that executes the three-dimensional scene reconstruction method and / or the method for intelligent driving simulation provided by another embodiment of the present application. As Figure 6 shown, the device includes: One or more processors 610 and a memory 620. Figure 6 Here, one processor 610 is taken as an example.

[0076] The device that executes the three-dimensional scene reconstruction method and / or the method for intelligent driving simulation may further include: an input device 630 and an output device 640.

[0077] The processor 610, the memory 620, the input device 630, and the output device 640 can be connected via a bus or other means. Figure 6 Taking the connection via the bus as an example.

[0078] As a non-volatile computer-readable storage medium, the memory 620 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the three-dimensional scene reconstruction method and / or the method for intelligent driving simulation in the embodiments of the present application. By running the non-volatile software programs, instructions, and modules stored in the memory 620, the processor 610 executes various functional applications and data processing of the server, that is, implements the three-dimensional scene reconstruction method and / or the method for intelligent driving simulation in the above method embodiments.

[0079] The memory 620 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the three-dimensional scene reconstruction device and / or the device for intelligent driving simulation (for example, the computer device described in the foregoing embodiments), etc. In addition, the memory 620 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 620 optionally includes a memory remotely set relative to the processor 610, and these remote memories can be connected to the three-dimensional scene reconstruction device and / or the device for intelligent driving simulation through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0080] The input device 630 can receive input digital or character information, and generate signals related to the user settings and function controls of the three-dimensional scene reconstruction device and / or the device for intelligent driving simulation. The output device 640 can include a display device such as a display screen.

[0081] The one or more modules are stored in the memory 620 and, when executed by the one or more processors 610, execute the three-dimensional scene reconstruction method and / or the method for intelligent driving simulation in any of the above method embodiments.

[0082] The above product can execute the method provided by the embodiments of the present application, and has the corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided by the embodiments of the present application.

[0083] The computer device in the embodiments of the present application exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by having mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones, etc.

[0084] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have the feature of mobile Internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as iPad.

[0085] (3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio and video players (such as iPod), handheld game consoles, e-books, and smart toys and portable in-vehicle navigation devices.

[0086] (4) Servers: Devices that provide computing services. The composition of a server includes a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but due to the need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability, etc.

[0087] (5) Other electronic devices with data interaction functions.

[0088] The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solutions or the part that contributes to the related technologies can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A three-dimensional scene reconstruction method, characterized in that, Including: Dividing the earth space into multiple blocks, where the earth space includes a spherical space within a preset range above the earth's surface; Obtaining scene data of the earth space for 3D reconstruction to obtain a 3D reconstruction result, where the 3D reconstruction result includes a reconstruction model and reconstruction model information; Dividing the reconstruction model into corresponding blocks according to the reconstruction model information.

2. The method according to claim 1, characterized in that, The multiple blocks are distributed in multiple layers, each layer includes at least one block, the number of blocks included in the layer farther from the earth's surface in the multiple layers is smaller, and the size of the blocks included in the layer closer to the earth's surface in the multiple layers is smaller.

3. The method according to claim 1, wherein The reconstruction model information includes reconstruction model longitude and latitude information and reconstruction model size information; the dividing the reconstruction model into corresponding blocks according to the reconstruction model information includes: Determining the layer where the target block is located according to the reconstruction model size information and the sizes of the multiple blocks; Determining the target block from the layer where the target block is located according to the reconstruction model longitude and latitude information; Dividing the reconstruction model into the target block.

4. The method according to claim 1, characterized in that, The obtaining scene data of the earth space for 3D reconstruction includes: Collecting scene data by a collection terminal through scanning a scene; the collection terminal includes at least one of a radar device, an image sensor, and an RTK device, and the collection terminal is carried on at least one of a drone, a collection vehicle, and a handheld device; Performing 3D reconstruction according to the scene data.

5. The method according to claim 4, characterized in that, It also includes: Performing semantic segmentation on the reconstruction model to obtain semantic segmentation information; Constructing OpenDrive data according to the semantic segmentation information.

6. The method according to claim 4, wherein It also includes: While collecting scene data by a collection terminal through scanning a scene, recording at least one of the position, altitude, temperature, humidity, and season of the scene.

7. A method for intelligent driving simulation, characterized in that, Including: Performing 3D scene reconstruction by the method according to any one of claims 1-6; Performing intelligent driving simulation based on the reconstruction model obtained from 3D scene reconstruction.

8. The method according to claim 7, characterized in that, The performing intelligent driving simulation based on the reconstruction model obtained from 3D scene reconstruction includes: According to the position information of the observation area, streaming and loading the reconstruction models of the corresponding blocks from multiple blocks in the earth space for scene rendering, so as to perform intelligent driving simulation according to the scene rendering result.

9. The method according to claim 8, characterized in that, The streaming and loading the reconstruction models of the corresponding blocks from multiple blocks in the earth space according to the position information of the observation area for scene rendering includes: Determining the visible blocks among the multiple blocks according to the position information of the observation area; Loading the reconstruction models corresponding to the visible blocks and loading the ego vehicle simulation information; Combining the reconstruction models of the visible blocks and the ego vehicle simulation information for scene rendering.

10. The method according to claim 9, wherein When the number of the visible blocks is at least two, the loading the reconstruction models corresponding to the visible blocks includes: According to the reconstruction model longitude and latitude information of the reconstruction models of at least two visible blocks, splicing the reconstruction models of at least two visible blocks and loading the spliced reconstruction model.

11. The method according to any one of claims 9-10, characterized in that, The position information of the observation area includes the longitude and latitude information and altitude information of the observation area; the determining the visible blocks among the multiple blocks according to the position information of the observation area includes: Determine visible blocks among the multiple blocks based on the longitude and latitude information, altitude information of the observation area, and the reconstructed model longitude and latitude information of the multiple reconstructed models corresponding to the multiple blocks.

12. A computer device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-11.

13. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1-11 are implemented.

14. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1-11 are implemented.

Images