Three-dimensional scene simulation method and system and readable storage medium

Abstract

The invention provides a three-dimensional scene simulation method and system and a readable storage medium, and the method comprises the steps: obtaining input data, and constructing a test basic scene database according to the input data, the input data comprising unmanned ship data and environment observation data; and based on the test basic scene database, modeling an unmanned ship test scene through a neural radiation field technology to obtain a dynamic three-dimensional scene simulation model, and generating multi-modal data. According to the technical scheme, the generated three-dimensional scene has dynamic characteristics and is associated with a real scene, a virtual test environment based on real test field mapping is formed, and the fidelity of the test environment is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of three-dimensional scene simulation, and in particular, relates to a three-dimensional scene simulation method and system, and a readable storage medium. Background Art

[0002] The application of intelligent unmanned ship technology has brought new demands for the generation and evaluation of unmanned ship mission capabilities. On the one hand, new algorithms make unmanned ships (groups) more "intelligent", continuously expanding the capability boundaries and environmental applicability of unmanned ships (groups), and thus giving rise to new mission styles and modes. At the same time, the assessment of the mission capabilities of unmanned ships (groups) in complex environments and extreme conditions has become more urgent. At present, there are gaps in many aspects of the domestic standard system and test technology.

[0003] The rapid generation of mission capabilities of unmanned ships requires the vigorous development of virtual test and verification technology. With the development of unmanned ships, the requirements for the generation cycle and level of mission capabilities are constantly increasing, which brings new opportunities and challenges to the virtual test and verification of unmanned ships. On the one hand, shortening the R&D cycle of unmanned ships requires developers to complete the R&D and testing of unmanned ships, especially intelligent control systems, in a shorter time, which provides a more urgent need for virtual test and verification. On the other hand, shortening the cycle of unmanned ship capability generation also requires developers to discover and solve problems more quickly during the R&D stage of unmanned ships, which puts higher requirements on virtual test and verification. Rapidly, completely and accurately reflecting the real situation of unmanned ship mission capabilities is of great significance for solving the design verification means of unmanned ships and promoting the development process of unmanned ships and the generation of mission capabilities. Summary of the invention

[0004] This application aims to solve or improve the above technical problems.

[0005] To this end, an embodiment of the present application provides a three-dimensional scene simulation method.

[0006] The embodiment of the present application also provides a three-dimensional scene simulation system.

[0007] The embodiment of the present application also provides a three-dimensional scene simulation system.

[0008] The embodiment of the present application also provides a readable storage medium.

[0009] To achieve the above-mentioned purpose, an embodiment of the present application provides a three-dimensional scene simulation method, including: obtaining input data and constructing a test basic scene database based on the input data, the input data including unmanned ship data and environmental observation data; based on the test basic scene database, the unmanned ship test scene is modeled through neural radiation field technology to obtain a dynamic three-dimensional scene simulation model, and generate multimodal data.

[0010] According to the three-dimensional scene simulation method provided by the present application, firstly, input data is obtained, and a test basic scene database is constructed according to the input data, and the input data includes unmanned ship data and environmental observation data. Then, based on the test basic scene database, the unmanned ship test scene is modeled by neural radiation field technology to obtain a dynamic three-dimensional scene simulation model, and multimodal data is generated. It can be understood that by using the unmanned ship itself and the environmental observation data collected in the real test site as one of the input elements of the scene, the test basic scene database is constructed by collecting and recording the data of the real environment test site. By using the neural radiation field technology to construct a dynamic three-dimensional simulation scene and generate multimodal data, the insufficiency of the dynamic scene generation ability based on the generative model is compensated. The neural radiation field scene simulation mainly generates a new scene based on 3D reconstruction. The advantage of the 3D scene synthesis method based on the neural radiation field is that the rendering result has a high degree of restoration, which is closer to the real scene, and more 3D modal information can be obtained. The neural radiation field method can reconstruct a complete scene, obtain a scene rendering view of any perspective at the internal position of the scene, and can also simulate the information of the 3D sensor. In 3D models, dynamic synthesis can be achieved by manipulating the observation position, viewing angle, and objects in the scene. After the three-dimensional scene based on the generative model is constructed, input is injected based on typical tasks to drive the behavior and actions of key elements in the scene, so that the generated dynamic scene has dynamic characteristics and is associated with the real scene, forming a virtual test environment based on the mapping of the real test site, increasing the fidelity of the test environment.

[0011] Specifically, with the help of the multi-view reconstruction capability of the neural radiation field, a high degree of restoration of the real scene can be achieved. Objects in the scene are distinguished by 3D target boxes, and the scene and different objects are modeled independently. Objects in the scene can be manipulated by manipulating object modeling, such as changing their position and posture. Using a similar method, real scenes can be reconstructed using unmanned ship records, and can be replayed on this basis. The dynamic foreground and static background are modeled separately, objects and scenes are decoupled, and operations such as addition, deletion, and replacement are performed on the unmanned ship test environment to obtain multiple different scene lines. While obtaining the rendering results, the point cloud information of the 3D modality can be obtained.

[0012] In addition, the technical solution provided by this application may also have the following additional technical features:

[0013] In some technical solutions, optionally, input data is obtained and a test basic scenario database is constructed based on the input data, including: obtaining real unmanned ship driving data on the sea; preprocessing the real unmanned ship driving data and extracting and identifying typical scenes to obtain a typical test scenario digital twin of the real unmanned ship driving data; and constructing a test basic scenario database based on the typical test scenario digital twin.

[0014] In this technical solution, input data is obtained and a test basic scenario database is constructed based on the input data. Specifically, real unmanned ship driving data on the sea is collected through a data acquisition platform. After data preprocessing and typical scene extraction and identification, a digital twin of a typical test scene in the real unmanned ship driving data can be obtained, based on which a basic test scenario library for unmanned ship tests can be constructed.

[0015] In some technical solutions, optionally, the unmanned ship test scenario is modeled through neural radiation field technology, including: constructing a boundary test sample generation model based on an adversarial generative model; obtaining key element parameters, and embedding the key element parameters into the feature space of the boundary test sample generation model; generating the expected boundary test scenario through the boundary test sample generation model.

[0016] In this technical solution, the unmanned ship test scenario is modeled through neural radiation field technology. Specifically, a boundary test sample generation method is constructed based on the adversarial generative model. With the help of the imagination and generalization ability of the adversarial generative model, combined with the analysis of typical mission scenarios and the description of key elements, the key elements are embedded in the feature space of the generative model in a parameterized form to guide the generative model to generate the desired boundary test scenario. By introducing multiple factors such as weather, sea conditions, viewing angle, load status, etc., an environment that meets diverse test conditions can be generated.

[0017] In some technical solutions, optionally, the key element parameters include one of the following or a combination thereof: weather data, sea state data, viewing angle data and load status data.

[0018] In this technical solution, the key element parameters include one of the following or a combination thereof: weather data, sea condition data, viewing angle data and load status data.

[0019] In some technical solutions, optionally, the unmanned ship test scene is modeled using neural radiation field technology, including: distinguishing objects in the unmanned ship test scene using a 3D target box, and independently modeling the unmanned ship test scene and objects.

[0020] In this technical solution, the unmanned ship test scene is modeled through the neural radiation field technology. Specifically, the objects in the unmanned ship test scene are distinguished by a 3D target frame, and the unmanned ship test scene and objects are independently modeled. In this way, the objects in the scene can be operated by operating the object modeling, such as changing its position and posture.

[0021] In some technical solutions, optionally, modeling the unmanned ship test scene through neural radiation field technology also includes: separately modeling the dynamic foreground and static background through neural radiation field technology.

[0022] In this technical solution, the unmanned ship test scene is modeled through neural radiation field technology, and the dynamic foreground and static background are modeled separately through neural radiation field technology to decouple objects and scenes, so that the unmanned ship test environment can be added, deleted, replaced, and other operations can be performed to obtain multiple different scenario lines.

[0023] In some technical solutions, optionally, the three-dimensional scene simulation method further includes: manipulating the foreground object modeling of the dynamic three-dimensional scene simulation model.

[0024] In this technical solution, the three-dimensional scene simulation method also includes manipulating the modeling of foreground objects in the dynamic three-dimensional scene simulation model. Specifically, for objects in the scene, the use of generative models such as diffusion models and generative adversarial networks is mainly to control the generation through semantics in the feature space. Since the objects and scenes are highly coupled in the feature space, it is difficult to manipulate the objects at a fine granularity. At the same time, since it is a static generation, it is difficult to achieve dynamic interaction between objects. In the 3D method, the display is decoupled by independently modeling the foreground and background, which is more controllable. Dynamic interaction between objects or between objects and scenes is achieved by manipulating the modeling of foreground objects.

[0025] An embodiment of the present application provides a three-dimensional scene simulation system, including: an acquisition module, used to acquire input data and construct a test basic scene database based on the input data, the input data including unmanned ship data and environmental observation data; a simulation module, used to model the unmanned ship test scene based on the test basic scene database through neural radiation field technology, obtain a dynamic three-dimensional scene simulation model, and generate multimodal data.

[0026] According to the three-dimensional scene simulation system provided by the present application, it includes an acquisition module and a simulation module. Among them, the acquisition module is used to acquire input data and construct a test basic scene database according to the input data, and the input data includes unmanned ship data and environmental observation data. The simulation module is used to model the unmanned ship test scene based on the test basic scene database through neural radiation field technology, obtain a dynamic three-dimensional scene simulation model, and generate multimodal data. It can be understood that by using the unmanned ship itself and the environmental observation data collected in the real test site as one of the input elements of the scene, the test basic scene database is constructed by collecting and recording the data of the real environment test site. By using the neural radiation field technology to construct a dynamic three-dimensional simulation scene and generate multimodal data, the insufficiency of the dynamic scene generation ability based on the generative model is compensated. The neural radiation field scene simulation mainly generates a new scene based on 3D reconstruction. The advantage of the 3D scene synthesis method based on the neural radiation field is that the rendering result has a high degree of restoration, which is closer to the real scene, and more 3D modal information can be obtained. The neural radiation field method can reconstruct a complete scene, obtain a scene rendering view of any perspective at the internal position of the scene, and can also simulate the information of the 3D sensor. In 3D models, dynamic synthesis can be achieved by manipulating the observation position, viewing angle, and objects in the scene. After the three-dimensional scene based on the generative model is constructed, input is injected based on typical tasks to drive the behavior and actions of key elements in the scene, so that the generated dynamic scene has dynamic characteristics and is associated with the real scene, forming a virtual test environment based on the mapping of the real test site, increasing the fidelity of the test environment.

[0027] An embodiment of the present application provides a three-dimensional scene simulation system, including: a memory and a processor, wherein the memory stores programs or instructions that can be run on the processor, and when the processor executes the program or instruction, it implements the three-dimensional scene simulation method of any one of the technical solutions of the first aspect, and thus has the technical effect of any of the technical solutions of the first aspect above, which will not be repeated here.

[0028] An embodiment of the present application provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the three-dimensional scene simulation method of any one of the technical solutions of the first aspect are implemented, and thus the technical effects of any of the technical solutions of the first aspect are obtained, which will not be repeated here.

[0029] Additional aspects and advantages of the present application will become apparent in the following description or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0031] Figure 1 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0032] Figure 2 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0033] Figure 3 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0034] Figure 4 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0035] Figure 5 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0036] Figure 6 A schematic diagram of the steps of a three-dimensional scene simulation method according to an embodiment of the present application;

[0037] Figure 7 A schematic block diagram of the structure of a three-dimensional scene simulation system according to an embodiment of the present application;

[0038] Figure 8 A schematic block diagram of the structure of a three-dimensional scene simulation system according to an embodiment of the present application;

[0039] Fig. 9 A schematic diagram of a NeRF-based dynamic three-dimensional scene modeling and generation technology of a three-dimensional scene simulation method according to an embodiment of the present application;

[0040] Fig.10 A schematic diagram of three-dimensional scene modeling and multimodal data generation of a three-dimensional scene simulation method according to an embodiment of the present application;

[0041] in, Figure 7 and Figure 8 The corresponding relationship between the reference numerals and component names in the figure is:

[0042] 10: 3D scene simulation system; 110: acquisition module; 120: simulation module; 20: 3D scene simulation system; 300: memory; 400: processor. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] Refer to the following Figures 1 to 10 Describe the three-dimensional scene simulation method and system, and readable storage medium of some embodiments of the present application.

[0045] like Figure 1 As shown, the embodiment of the first aspect of the present application provides a three-dimensional scene simulation method, comprising the following steps:

[0046] Step S102: obtaining input data, and constructing a test basic scenario database according to the input data, wherein the input data includes unmanned ship data and environmental observation data;

[0047] Step S104: Based on the test basic scene database, the unmanned ship test scene is modeled through the neural radiation field technology to obtain a dynamic three-dimensional scene simulation model and generate multimodal data.

[0048] According to the three-dimensional scene simulation method provided by this embodiment, firstly, input data is obtained, and a test basic scene database is constructed according to the input data, and the input data includes unmanned ship data and environmental observation data. Then, based on the test basic scene database, the unmanned ship test scene is modeled by neural radiation field technology to obtain a dynamic three-dimensional scene simulation model, and multimodal data is generated. It can be understood that by using the unmanned ship itself and the environmental observation data collected in the real test site as one of the input elements of the scene, the test basic scene database is constructed by collecting and recording the data of the real environment test site. By using the neural radiation field technology to construct a dynamic three-dimensional simulation scene and generate multimodal data, the insufficiency of the dynamic scene generation ability based on the generative model is compensated. The neural radiation field scene simulation mainly generates a new scene based on 3D reconstruction. The advantage of the 3D scene synthesis method based on the neural radiation field is that the rendering result has a high degree of restoration, which is closer to the real scene, and more 3D modal information can be obtained. The neural radiation field method can reconstruct a complete scene, obtain a scene rendering view of any perspective at the internal position of the scene, and can also simulate the information of the 3D sensor. In 3D models, dynamic synthesis can be achieved by manipulating the observation position, viewing angle, and objects in the scene. After the three-dimensional scene based on the generative model is constructed, input is injected based on typical tasks to drive the behavior and actions of key elements in the scene, so that the generated dynamic scene has dynamic characteristics and is associated with the real scene, forming a virtual test environment based on the mapping of the real test site, increasing the fidelity of the test environment.

[0049] Specifically, with the help of the multi-view reconstruction capability of the neural radiation field, a high degree of restoration of the real scene can be achieved. Objects in the scene are distinguished by 3D target boxes, and the scene and different objects are modeled independently. Objects in the scene can be manipulated by manipulating object modeling, such as changing their position and posture. Using a similar method, real scenes can be reconstructed using unmanned ship records, and can be replayed on this basis. The dynamic foreground and static background are modeled separately, objects and scenes are decoupled, and operations such as addition, deletion, and replacement are performed on the unmanned ship test environment to obtain multiple different scene lines. While obtaining the rendering results, the point cloud information of the 3D modality can be obtained.

[0050] like Figure 2 As shown, according to a three-dimensional scene simulation method of an embodiment proposed in the present application, input data is obtained, and a test basic scene database is constructed according to the input data, including the following steps:

[0051] Step S202: Acquire real unmanned ship driving data on the sea surface;

[0052] Step S204: preprocessing the real unmanned ship driving data and extracting and identifying typical scenes to obtain a typical test scene digital twin of the real unmanned ship driving data;

[0053] Step S206: Construct a test basic scenario database based on the digital twin of the typical test scenario.

[0054] In this embodiment, input data is obtained, and a test basic scenario database is constructed based on the input data. Specifically, real unmanned ship driving data on the sea surface is collected through a data acquisition platform. After data preprocessing and typical scene extraction and identification, a digital twin of a typical test scene in the real unmanned ship driving data can be obtained, and a basic test scenario library for unmanned ship tests can be constructed based on it.

[0055] like Figure 3 As shown, according to a three-dimensional scene simulation method of an embodiment proposed in the present application, the unmanned ship test scene is modeled by neural radiation field technology, including the following steps:

[0056] Step S302: constructing a boundary test sample generation model based on the adversarial generative model;

[0057] Step S304: Acquire key element parameters, and embed the key element parameters into the feature space of the boundary test sample generation model;

[0058] Step S306: Generate a desired boundary test scenario through a boundary test sample generation model.

[0059] In this embodiment, the unmanned ship test scenario is modeled by neural radiation field technology. Specifically, a boundary test sample generation method is constructed based on the adversarial generative model. With the help of the imagination and generalization ability of the adversarial generative model, combined with the analysis of typical mission scenarios and the description of key elements, the key elements are embedded in the feature space of the generative model in a parameterized form to guide the generative model to generate the desired boundary test scenario. By introducing multiple factors such as weather, sea conditions, viewing angle, load status, etc., an environment that meets diverse test conditions can be generated.

[0060] In some embodiments, optionally, the key element parameters include one or a combination of the following: weather data, sea state data, viewing angle data, and load status data.

[0061] like Figure 4 As shown, according to a three-dimensional scene simulation method of an embodiment proposed in the present application, the unmanned ship test scene is modeled by neural radiation field technology, including the following steps:

[0062] Step S402: objects in the unmanned ship test scene are distinguished by a 3D target frame, and the unmanned ship test scene and objects are independently modeled.

[0063] In this embodiment, the unmanned ship test scene is modeled by neural radiation field technology. Specifically, objects in the unmanned ship test scene are distinguished by a 3D target frame, and the unmanned ship test scene and objects are independently modeled. This allows the operation of objects in the scene to be achieved by operating the object modeling, such as changing its position, posture, etc.

[0064] like Figure 5 As shown, according to a three-dimensional scene simulation method of an embodiment proposed in the present application, the unmanned ship test scene is modeled by neural radiation field technology, and the following steps are also included:

[0065] Step S502: Separately model the dynamic foreground and the static background using neural radiance field technology.

[0066] In this embodiment, the unmanned ship test scene is modeled through neural radiation field technology, and the dynamic foreground and static background are separately modeled through neural radiation field technology to decouple objects and scenes, so that the unmanned ship test environment can be added, deleted, replaced, and the like to obtain multiple different scenario lines.

[0067] like Figure 6 As shown, the three-dimensional scene simulation method according to an embodiment of the present application further includes the following steps:

[0068] Step S602: manipulating the foreground object modeling of the dynamic three-dimensional scene simulation model.

[0069] In this embodiment, the three-dimensional scene simulation method also includes manipulating the modeling of foreground objects of the dynamic three-dimensional scene simulation model. Specifically, for objects in the scene, the use of generative models such as diffusion models and generative adversarial networks is mainly to control the generation through semantics in the feature space. Since the objects and scenes are highly coupled in the feature space, it is difficult to manipulate the objects at a fine granularity. At the same time, since it is a static generation, it is difficult to achieve dynamic interaction between objects. In the 3D method, the display is decoupled by independently modeling the foreground and background, which is more controllable. Dynamic interaction between objects or between objects and scenes is achieved by manipulating the modeling of foreground objects.

[0070] like Figure 7 As shown, an embodiment of the second aspect of the present application provides a three-dimensional scene simulation system 10, including: an acquisition module 110, used to acquire input data and build a test basic scene database based on the input data, the input data including unmanned ship data and environmental observation data; a simulation module 120, used to model the unmanned ship test scene based on the test basic scene database through neural radiation field technology, obtain a dynamic three-dimensional scene simulation model, and generate multimodal data.

[0071] According to the three-dimensional scene simulation system 10 provided in this embodiment, it includes an acquisition module 110 and a simulation module 120. Among them, the acquisition module 110 is used to acquire input data and construct a test basic scene database according to the input data, and the input data includes unmanned ship data and environmental observation data. The simulation module 120 is used to model the unmanned ship test scene based on the test basic scene database through the neural radiation field technology, obtain a dynamic three-dimensional scene simulation model, and generate multimodal data. It can be understood that by using the unmanned ship itself and the environmental observation data collected in the real test site as one of the input elements of the scene, the test basic scene database is constructed by collecting and recording the data of the real environment test site. By using the neural radiation field technology to construct a dynamic three-dimensional simulation scene and generate multimodal data, the lack of dynamic scene generation capability based on the generative model is compensated. The neural radiation field scene simulation mainly generates a new scene based on 3D reconstruction. The advantage of the 3D scene synthesis method based on the neural radiation field is that the rendering result has a high degree of restoration, which is closer to the real scene, and more 3D modal information can be obtained. The neural radiation field method can reconstruct a complete scene, obtain a scene rendering view of any perspective within the scene, and simulate the information of the 3D sensor. In the 3D model, dynamic synthesis can be achieved by manipulating the observation position, perspective, and objects in the scene. After the three-dimensional scene based on the generative model is constructed, input injection is performed based on typical tasks to drive the behavior and actions of key elements in the scene, so that the generated dynamic scene has dynamic characteristics and is associated with the real scene, forming a virtual test environment based on the mapping of the real test site, increasing the realism of the test environment.

[0072] like Figure 8 As shown, an embodiment of the third aspect of the present application provides a three-dimensional scene simulation system 20, including: a memory 300 and a processor 400, wherein the memory 300 stores a program or instruction that can be run on the processor 400, and when the processor 400 executes the program or instruction, the steps of the three-dimensional scene simulation method of any one of the embodiments of the first aspect are implemented, and thus the technical effect of any one of the embodiments of the first aspect mentioned above is not repeated here.

[0073] The embodiment of the fourth aspect of the present application provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the three-dimensional scene simulation method of any one of the embodiments of the first aspect are implemented, and thus it has the technical effect of any one of the embodiments of the first aspect mentioned above, which will not be repeated here.

[0074] like Fig. 9 , Fig.10As shown, according to a three-dimensional scene simulation method of a specific embodiment provided in the present application, a dynamic high-fidelity three-dimensional scene simulation based on neural radiation fields is carried out to address the problems of multiple scenes, complex tasks and low efficiency in the unmanned ship test environment construction.

[0075] The main body of similarity theory is the three similarity theorems. With its development, it has been widely used in the theoretical guidance of model experiments. The core of virtual scene production is the modeling and experiment of the physical world. How to build a highly realistic virtual model is the core key. Although the generative model has the potential to quickly fit the characteristics of nonlinear real-world data, it is still prone to problems such as model failure to converge and unstable generation accuracy. In response to this key technology, this embodiment intends to construct a boundary test sample generation method based on an adversarial generative model. With the help of the imagination and generalization ability of the adversarial generative model, combined with the analysis of typical task scenarios and the description of key elements, the key elements are embedded in the feature space of the generative model in a parameterized form to guide the generative model to generate the desired boundary test scenario. By introducing multiple factors such as weather, sea conditions, viewing angles, and load states, an environment that meets diverse test conditions can be generated.

[0076] Specifically, this embodiment intends to use Neural Radiation Field (NeRF) technology to construct a dynamic three-dimensional simulation scene and generate multimodal data to make up for the lack of dynamic scene generation capabilities based on generative models. NeRF scene simulation mainly generates new scenes based on 3D reconstruction. The advantage of the NeRF-based 3D scene synthesis method is that the rendering results obtained have a high degree of restoration, are closer to the real scene, and can obtain more 3D modal information. The NeRF method can reconstruct a complete scene, obtain a scene rendering view of any perspective at the internal position of the scene, and can also simulate the information of the 3D sensor. The existing BEV scene generation content is mostly static scenes. In the 3D model, dynamic synthesis can be achieved by manipulating the observation position, perspective, and objects in the scene.

[0077] For objects in the scene, the use of generative models such as diffusion models and generative adversarial networks (GANs) is mainly to control the generation through semantics in the feature space. Since objects and scenes are highly coupled in the feature space, it is difficult to manipulate objects at a fine granularity; at the same time, since the generation is static, it is difficult to achieve dynamic interaction between objects. In the 3D method, the display is decoupled by independently modeling the foreground and background, which is more controllable. Dynamic interaction between objects or between objects and scenes is achieved by manipulating the modeling of foreground objects.

[0078] In order to obtain a complete and continuous unmanned ship test scene, it can be achieved by building a virtual scene and manipulating the environmental factors and unmanned ship behavior. This effect can be achieved by using an unmanned ship test simulation platform to obtain rich simulation data for the expansion of perception, prediction and planning tasks. However, there is still a difference between the rendering that is better than the virtual engine and reality, and it cannot be directly migrated. In order to obtain scene simulation data close to reality, one current approach is to collect data once in a real scene, use NeRF to model the scene and objects, and operate the elements in the scene on this basis.

[0079] With the help of NeRF's multi-view reconstruction capability, a high degree of restoration of the real scene can be achieved. Objects in the scene are distinguished by 3D target boxes, and the scene and different objects are modeled independently. Objects in the scene can be manipulated by manipulating object modeling, such as changing their position and posture. Using a similar method, real scenes can be reconstructed using unmanned ship records, and can be replayed on this basis. Dynamic foreground and static background are modeled separately, objects and scenes are decoupled, and operations such as addition, deletion, and replacement are performed on the unmanned ship test environment to obtain multiple different scene lines. While obtaining the rendering result, point cloud information of the 3D modality can be obtained.

[0080] The data acquisition platform collects real unmanned ship driving data on the sea. After data preprocessing and typical scene extraction and identification, the digital twin of the typical test scene in the real unmanned ship driving data can be obtained, based on which a basic scene library for unmanned ship testing can be constructed.

[0081] In summary, the beneficial effects of the embodiments of the present application are: using the observation data of the unmanned ship itself and the environment collected in the real test site as one of the input elements of the scene, and building a test basic scene database by collecting and recording the data of the real environment test site. After the three-dimensional scene based on the generative model is built, input injection is performed based on typical tasks to drive the behavior and actions of the key elements in the scene, so that the generated dynamic scene has dynamic characteristics, and is associated with the real scene, forming a virtual test environment based on the mapping of the real test site, and increasing the fidelity of the test environment.

[0082] In this application, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance; the term "plurality" refers to two or more, unless otherwise expressly defined. Terms such as "installed", "connected", "connected", and "fixed" should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0083] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "rear", etc., indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the system or module referred to must have a specific direction, be constructed and operated in a specific orientation, and therefore, should not be understood as a limitation on the present application.

[0084] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0085] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional scene simulation method, characterized in that: include: Acquire input data, and construct a test basic scenario database according to the input data, wherein the input data includes unmanned ship data and environmental observation data; Based on the test basic scenario database, the unmanned ship test scenario is modeled using neural radiation field technology to obtain a dynamic three-dimensional scene simulation model and generate multimodal data.

2. The three-dimensional scene simulation method according to claim 1, characterized in that: The step of obtaining input data and constructing a test basic scenario database according to the input data includes: Obtain real unmanned ship driving data on the sea; Preprocessing the real unmanned ship driving data and extracting and identifying typical scenes to obtain a typical test scene digital twin of the real unmanned ship driving data; A test basic scenario database is constructed based on the typical test scenario digital twin.

3. The three-dimensional scene simulation method according to claim 1, characterized in that: The unmanned ship test scenario is modeled by neural radiation field technology, including: Construct a boundary test sample generation model based on the adversarial generative model; Acquire key element parameters, and embed the key element parameters into the feature space of the boundary test sample generation model; The desired boundary test scenario is generated by the boundary test sample generation model.

4. The three-dimensional scene simulation method according to claim 3, characterized in that: The key element parameters include one of the following or a combination thereof: weather data, sea state data, viewing angle data and load status data.

5. The three-dimensional scene simulation method according to claim 1, characterized in that: The unmanned ship test scenario is modeled by neural radiation field technology, including: Objects in the unmanned ship test scene are distinguished by a 3D target frame, and the unmanned ship test scene and the objects are independently modeled.

6. The three-dimensional scene simulation method according to claim 5, characterized in that: The method of modeling the unmanned ship test scenario by using the neural radiation field technology also includes: Dynamic foreground and static background are modeled separately through neural radiance field technology.

7. The three-dimensional scene simulation method according to any one of claims 1 to 6, characterized in that: Also includes: The foreground object modeling of the dynamic three-dimensional scene simulation model is manipulated.

8. A three-dimensional scene simulation system, characterized in that: include: An acquisition module (110) is used to acquire input data and construct a test basic scenario database according to the input data, wherein the input data includes unmanned ship data and environmental observation data; The simulation module (120) is used to model the unmanned ship test scene based on the test basic scene database through neural radiation field technology, obtain a dynamic three-dimensional scene simulation model, and generate multimodal data.

9. A three-dimensional scene simulation system, characterized in that: include: A memory (300) and a processor (400), wherein the memory (300) stores a program or instruction that can be run on the processor (400), and when the processor (400) executes the program or the instruction, the steps of the three-dimensional scene simulation method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or the instruction is executed by a processor, the steps of the three-dimensional scene simulation method according to any one of claims 1 to 7 are implemented.

Images