Campus three-dimensional reconstruction method and device, terminal and storage medium

Abstract

The invention provides a campus three-dimensional reconstruction method and device, a terminal and a storage medium. The method comprises the steps of planning a flight path of an unmanned aerial vehicle based on an aerial photography range of a current campus; controlling the unmanned aerial vehicle to fly according to the flight path and collecting campus image data; performing neural radiation field modeling on the campus image data by using a pre-trained NeRF model, and judging whether the campus three-dimensional reconstruction scene meets a preset reconstruction requirement or not; and if not, identifying a to-be-repaired area in the campus three-dimensional reconstruction scene to plan a new flight path, controlling the unmanned aerial vehicle to collect new data under the new flight path, and performing scene repair by using the new data until the campus three-dimensional reconstruction scene meets a preset reconstruction requirement, thereby obtaining a target campus three-dimensional reconstruction scene. According to the invention, through feedback of the reconstruction effect, the unmanned aerial vehicle can repeatedly carry out data acquisition in an area with a poor reconstruction effect, the campus three-dimensional reconstruction scene is perfected, and the precision and efficiency of scene reconstruction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction technology, and in particular to a campus three-dimensional reconstruction method, device, terminal and storage medium. Background Art

[0002] At present, traditional 3D reconstruction of outdoor campuses has some shortcomings. The data collection efficiency is low, and traditional aerial photography requires manual path planning. It cannot automatically optimize the shooting angle for densely built areas such as teaching buildings, resulting in the omission of blind spots and reduced scene reconstruction accuracy.

[0003] Therefore, the existing technology has defects and needs to be improved and developed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a campus three-dimensional reconstruction method, device, terminal and storage medium in response to the above-mentioned defects of the existing technology, which can improve the accuracy and efficiency of the reconstructed model.

[0005] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0006] A three-dimensional reconstruction method for a campus, wherein the method comprises:

[0007] Determine the aerial photography range of the current campus and plan the flight path of the drone based on the aerial photography range;

[0008] Controlling the drone equipped with the camera to fly along the flight path and collect corresponding campus image data;

[0009] Using a pre-trained NeRF model to perform neural radiation field modeling on the campus image data, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets the preset reconstruction requirements;

[0010] If the three-dimensional reconstruction scene of the campus does not meet the preset reconstruction requirements, identifying an area to be repaired in the three-dimensional reconstruction scene of the campus to plan a new flight path for the drone;

[0011] The drone is controlled to fly along the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be repaired in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining a target campus 3D reconstruction scene.

[0012] In one implementation, the pre-trained NeRF model is a three-dimensional reconstruction model based on a preset neural network structure, and during the training process, the illumination invariance loss function is used to constrain the pixel or feature consistency of the three-dimensional reconstruction model for the same scene under different illumination.

[0013] In one implementation, the process of performing neural radiation field modeling on the campus image data using a pre-trained NeRF model to generate a three-dimensional reconstruction scene of the campus further includes:

[0014] Dynamic objects in the campus image data are filtered using a preset temporal consistency detection algorithm.

[0015] In one implementation, the campus 3D reconstruction method further includes:

[0016] Based on the pre-set communication protocol and communication method between the drone and the ground station or cloud server, the campus image data is transmitted to the ground station or the cloud server.

[0017] In one implementation, after the 3D reconstruction scene of the campus meets the preset reconstruction requirements and the target 3D reconstruction scene of the campus is obtained, the method further includes:

[0018] The target campus 3D reconstruction scene is converted into a Mesh model using a preset progressive Mesh simplification algorithm, so as to construct a virtual 3D campus model of the campus landmark buildings using the Mesh model.

[0019] In one implementation, after constructing the virtual model of the campus landmark building using the Mesh model, the method further includes:

[0020] Corresponding POI information is added to the campus landmark buildings in the virtual three-dimensional campus model.

[0021] In one implementation, identifying the area to be repaired in the three-dimensional reconstructed campus scene to plan a new flight path for the drone includes:

[0022] A pre-trained semantic segmentation model is used to identify the area to be repaired in the three-dimensional reconstruction scene of the campus, and a preset data collection weight is assigned to the area to be repaired to plan a new flight path for the drone.

[0023] The present invention also discloses a campus three-dimensional reconstruction device, wherein the device comprises:

[0024] Aerial photography range determination module, used to determine the aerial photography range of the current campus;

[0025] A first path planning module is used to plan a flight path of the UAV based on the aerial photography range;

[0026] A data acquisition module is used to control the drone equipped with a camera to fly along the flight path and collect corresponding campus image data;

[0027] A scene reconstruction module is used to perform neural radiation field modeling on the campus image data using a pre-trained NeRF model, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets preset reconstruction requirements;

[0028] an area recognition module, configured to identify an area to be repaired in the 3D reconstructed campus scene if the 3D reconstructed campus scene does not meet the preset reconstruction requirements, so as to plan a new flight path for the UAV;

[0029] The scene restoration module is used to control the UAV to fly according to the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be restored in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining the target campus 3D reconstruction scene.

[0030] The present invention also discloses a terminal, which includes: a memory, a processor, and a campus three-dimensional reconstruction program stored in the memory and runnable on the processor. When the campus three-dimensional reconstruction program is executed by the processor, the steps of the campus three-dimensional reconstruction method described above are implemented.

[0031] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the campus three-dimensional reconstruction method as described above.

[0032] The present invention provides a campus three-dimensional reconstruction method, device, terminal and storage medium. The campus three-dimensional reconstruction method includes: determining the aerial photography range of the current campus, and planning the flight path of an unmanned aerial vehicle (UAV) based on the aerial photography range; controlling the UAV equipped with a camera to fly along the flight path and collect corresponding campus image data; using a pre-trained NeRF model to perform neural radiation field modeling on the campus image data to generate a campus three-dimensional reconstruction scene, and judging whether the campus three-dimensional reconstruction scene meets preset reconstruction requirements; if the campus three-dimensional reconstruction scene does not meet the preset reconstruction requirements, identifying the area to be repaired in the campus three-dimensional reconstruction scene to plan a new flight path for the UAV; controlling the UAV to fly along the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be repaired in the campus three-dimensional reconstruction scene until the campus three-dimensional reconstruction scene meets the preset reconstruction requirements, thereby obtaining a target campus three-dimensional reconstruction scene. It can be seen from this that the present invention determines whether the campus 3D reconstruction scene needs to be repaired by judging whether it meets the preset reconstruction requirements. If necessary, the area to be repaired with poor reconstruction effect is identified, and a new flight path of the drone is planned based on the area to be repaired, so that the drone can repeat data collection in the area to be repaired with poor reconstruction effect, improve the campus 3D reconstruction scene, and thus improve the accuracy and efficiency of the reconstruction scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of a preferred embodiment of the campus 3D reconstruction method of the present invention;

[0034] Figure 2 This is a schematic diagram of a four-propeller UAV disclosed in the present invention;

[0035] Figure 3 This is a schematic diagram of the architecture of an Internet of Things control system based on a local area network and socket communication disclosed in the present invention;

[0036] Figure 4 This is a schematic diagram of a specific campus three-dimensional reconstruction method disclosed in the present invention;

[0037] Figure 5 This is a functional principle block diagram of a preferred embodiment of the campus 3D reconstruction device of the present invention;

[0038] Figure 6 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] See Figure 1 , Figure 1 This is a flow chart of the campus 3D reconstruction method in the present invention. Figure 1 As shown, the campus 3D reconstruction method according to the embodiment of the present invention includes:

[0041] Step S11: determine the aerial photography range of the current campus, and plan the flight path of the drone based on the aerial photography range.

[0042] In this embodiment, during the stage of acquiring data through fixed-point cruising by a drone, the aerial photography range of the current campus must be determined first, that is, the area that needs to be covered must be determined to achieve efficient scene coverage and data collection. The drone flight path is then planned based on the aerial photography range. For example, during the path planning process, a heat map of building density within the aerial photography range can be constructed. High-density areas can then be determined based on the building density heat map, and the waypoint density can be automatically increased for high-density areas, such as for places such as libraries.

[0043] Step S12: Control the drone equipped with the camera to fly along the flight path and collect corresponding campus image data.

[0044] In this embodiment, after completing the drone flight path, the drone equipped with a camera device is controlled to fly according to the drone flight path and collect corresponding campus image data. For example, a preset flight control algorithm can be used to control the drone equipped with a camera device to fly according to the drone flight path to ensure the stability and safety of the drone during the cruising process. That is, the drone is controlled to take off and enable the drone to cruise at a fixed point along the flight path. During the fixed-point cruising along the flight path, the drone stays at each observation point for a period of time according to the flight plan and camera settings to collect images.

[0045] It should be noted that see Figure 2As shown, the drone can be a four-propeller drone. Its flight control system primarily includes a flight control system, a main control chip, a gyroscope, sensors, an accelerometer, a geomagnetic sensor, and a GPS (Global Positioning System). Furthermore, when equipped with an RGBD (Red Green Blue–Depth) camera or LiDAR (Light Detection and Ranging) device, it can perform a panoramic scan of the campus, acquiring high-precision point cloud data and texture information. This high-precision point cloud data and texture information can be combined to further optimize subsequent steps and obtain a 3D reconstruction of the target campus scene.

[0046] In the process of data collection, the influence of parameters such as the flight altitude and field of view of the drone on the data collection effect can be considered, and the parameter settings can be further optimized to maximize the efficiency and quality of data collection. The parameters of the camera equipment carried by the drone can be pre-set. The parameters of the camera equipment can include parameters such as exposure time, aperture size and ISO (International Standards Organization, sensitivity) to adapt to different lighting conditions and scene requirements.

[0047] In this embodiment, after completing data collection using a drone equipped with a camera, the campus image data can be transmitted to the ground station or cloud server based on a pre-set communication protocol and communication method between the drone and the ground station or cloud server. The communication protocol and communication method between the drone and the ground station or cloud server can include wireless networks, Bluetooth, satellite communications, etc. Specifically, factors such as communication distance, bandwidth, and stability can be considered to determine the most appropriate communication protocol and communication method to implement the data transmission function, transmit the collected image data or other sensor data to the ground station or cloud server, ensure the security and integrity of the data transmission, avoid data loss or damage, and ensure that the data can be transmitted in a timely and effective manner.

[0048] For example, the IoT control system architecture based on LAN and Socket communication can be found in Figure 3 As shown in the figure, the main control end includes a Java client APP developed with Android Studio and Eclipse, which sends control commands to the server computer and receives data through the local area network. The server communicates with the slave controller (STIR2 microcontroller) through the SPI-WIFI module, realizing the bridge function of command issuance and data transmission, thus forming a two-way control and data interaction link of "client → server → WIFI module → microcontroller".

[0049] In this embodiment, the flight parameter setting of the UAV, as well as the display of flight mission planning and data monitoring can also be achieved by setting a visual interactive interface.

[0050] Step S13: Use the pre-trained NeRF model to perform neural radiation field modeling on the campus image data, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets the preset reconstruction requirements.

[0051] In this embodiment, after obtaining campus image data by achieving full-area cruising flight of the drone and collecting data with full regional coverage, the pre-trained NeRF (Neural Radiance Fields) model can be directly used to perform neural radiation field modeling on the campus image data to generate a three-dimensional reconstruction scene of the campus, and then further determine whether the three-dimensional reconstruction scene of the campus meets the preset reconstruction requirements.

[0052] Among them, the pre-trained NeRF model is a 3D reconstruction model based on a preset neural network structure. During the training process, the illumination invariance loss function is used to constrain the 3D reconstruction model to ensure pixel or feature consistency for the same scene under different illumination conditions. That is, embedding the illumination invariance loss function in NeRF can reduce artifacts in shaded or reflective areas, allowing it to better handle fine details, overcome the effects of luminosity changes and transient object changes, and solve the problem of illumination sensitivity. It overcomes the problem of distortion of reconstructed textures caused by complex illumination such as shaded trees and glass curtain wall reflections on campus, and achieves accurate reconstruction of the 3D scene from the input image sequence. After training is completed, the performance of the model can be evaluated using an independent test set or through cross-validation. If its performance does not meet the requirements, the model can continue to be trained according to the preset training strategy, so that it continues to learn the depth and color information of the scene from images from different perspectives, and can infer relatively accurate scene reconstruction results until its performance meets the requirements.

[0053] In addition, when using the pre-trained NeRF model to model the neural radiation field of campus image data and generate a three-dimensional reconstruction scene of the campus, a preset temporal consistency detection algorithm is used to filter dynamic objects in the campus image data. For example, transient objects such as students and vehicles are automatically removed, thereby solving the problem of poor dynamic scene processing and avoiding the problem of "ghosting" in the NeRF model caused by student activities and vehicle movement, without the need for manual labeling and removal.

[0054] Step S14: If the three-dimensional reconstructed campus scene does not meet the preset reconstruction requirements, an area to be repaired in the three-dimensional reconstructed campus scene is identified to plan a new flight path for the UAV.

[0055] In this embodiment, if the campus 3D reconstruction scene does not meet the preset reconstruction requirements, the area to be repaired in the campus 3D reconstruction scene is identified to plan the new flight path of the drone. Specifically, the pre-trained semantic segmentation model is used to identify the area to be repaired in the campus 3D reconstruction scene, and preset data collection weights are assigned to the area to be repaired to plan the new flight path of the drone, wherein the area to be repaired represents the area where the reconstruction effect does not meet the requirements and needs to be optimized and improved. It is understandable that the pre-trained semantic segmentation model is used to identify key areas, such as some roads or building entrances, and then higher data collection weights are assigned to the key areas in path planning. For example, the data collection weights corresponding to the key areas are pre-configured. After using the pre-trained semantic segmentation model to identify some key areas in the aerial photography range, the preset data collection weights corresponding to the key areas can be assigned to the key areas. In the process of using the preset drone path planning algorithm to plan the drone flight path, the weight of each key area can be combined to achieve the planning of the drone flight path. In addition, the pre-trained semantic segmentation model can be Mask R-CNN (Mask Region-based Convolutional Neural Network).

[0056] Step S15: Control the UAV to fly along the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be repaired in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining the target campus 3D reconstruction scene.

[0057] It is understandable that the NeRF model outputs a depth uncertainty map, which is then combined with a semantic segmentation mask to identify low-quality areas, thereby triggering the drone to retake photos of the low-quality areas. For example, areas such as sculpture details are prone to being low-quality areas with poor reconstruction effects. These areas need to trigger the drone to retake photos of the low-quality areas, and use the retaken image data to repair the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, and the target campus 3D reconstruction scene is obtained.

[0058] In this embodiment, after the campus 3D reconstruction scene meets the preset reconstruction requirements and the target campus 3D reconstruction scene is obtained, the target campus 3D reconstruction scene can be converted into a Mesh model using a preset progressive Mesh simplification algorithm to construct a virtual 3D campus model of the campus landmark building using the Mesh model. It is understandable that implicit models such as NeRF are converted into display representations to support various application scenarios. Among them, after 3D reconstruction, the model is converted into a Mesh model, which can be used to produce campus landmark buildings, and the converted display model can be directly used for interaction, navigation and experience in the virtual environment.

[0059] In this embodiment, a mesh model is used to construct a virtual 3D campus model of landmark buildings. Within this virtual environment, interactive navigation allows users to freely explore the campus and understand the characteristics and locations of campus buildings. Points of Interest (POIs) can also be added to these landmark buildings within the virtual 3D campus model. For example, POI information can be embedded in different buildings, such as classroom buildings, libraries, dormitories, and cafeterias.

[0060] For example, the constructed virtual 3D campus model can be used in recruitment promotional materials, such as freshman admission letters and campus websites, to showcase the campus's scenery and appeal. That is, when freshmen register, they usually need to use paper maps or campus brochures to understand the campus environment, but this method is inefficient and does not provide an immersive experience. However, through SLAM (Simultaneous Localization and Mapping) and NeRF technology, a 3D campus model can be constructed, a link or mini-program generated, and embedded into a QR code in the freshman admission letter. Users only need to scan the QR code to browse the virtual campus on their mobile phone or computer and familiarize themselves with the campus layout in advance. Alternatively, a campus business card can be created with a QR code on it, allowing interested people to scan the QR code to enter the virtual campus environment, display the 3D campus scenery, and learn more about the school.

[0061] It should be pointed out that, in combination with SLAM technology, automated map construction is achieved to ensure the accuracy of the model. NeRF technology is used to perform neural radiation field modeling on images of the campus from different angles to generate high-quality 3D scenes. Finally, through lighting rendering optimization, a realistic virtual campus roaming experience can be achieved. In addition, in the embodiment of the present application, deep reinforcement learning can also be used to optimize the data collection strategy, so that the drone can dynamically adjust the flight path to improve data coverage. When the modeling effect of a single drone is not good, other drones can be dispatched to supplement the data, and multi-drone collaborative optimization can be achieved to achieve more efficient scene reconstruction. In addition, in combination with semantic segmentation technology, high-precision reconstruction is prioritized in key target areas such as roads and building entrances to improve practicality.

[0062] It can be seen that in the embodiment of the present invention, whether the campus three-dimensional reconstruction scene meets the preset reconstruction requirements is judged to determine whether the campus three-dimensional reconstruction scene needs to be repaired. If necessary, the area to be repaired with poor reconstruction effect is identified, and a new flight path of the drone is planned based on the area to be repaired, so that the drone can repeat data collection in the area to be repaired with poor reconstruction effect, improve the campus three-dimensional reconstruction scene, and thus improve the accuracy and efficiency of the reconstruction scene.

[0063] For example, see Figure 4 As shown in the figure, in the data collection stage, the aerial photography range of the current campus is determined, the influence of parameters such as the flight altitude and field of view of the drone on the data collection effect is considered, the drone parameters and the parameters of the camera equipment on the drone are set, and then the flight path of the drone is planned based on the aerial photography range. The drone equipped with the camera equipment is controlled to fly along the flight path and collect data. Data transmission and data feedback are realized through a pre-established communication system. According to the communication protocol and communication method of the communication system, the data is transmitted to the personal computer or cloud server and other devices selected by the user. The data is pre-processed by the personal computer or cloud server and other devices, such as image correction, and then a data set is created to obtain the pose. In the process of algorithm training, the model training of the determined appropriate algorithm is carried out to realize model rendering, and implicit models such as NeRF are converted into display representations to support various practical application scenarios.

[0064] In one embodiment, if Figure 5 As shown, based on the above-mentioned campus 3D reconstruction method, the present invention also provides a campus 3D reconstruction device, including:

[0065] Aerial photography range determination module 11, used to determine the aerial photography range of the current campus;

[0066] A first path planning module 12 is used to plan a flight path of the UAV based on the aerial photography range;

[0067] A data acquisition module 13 is used to control the drone equipped with the camera to fly along the flight path and collect corresponding campus image data;

[0068] A scene reconstruction module 14 is configured to perform neural radiation field modeling on the campus image data using a pre-trained NeRF model, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets preset reconstruction requirements;

[0069] an area recognition module 15 for identifying an area to be repaired in the 3D reconstructed campus scene if the 3D reconstructed campus scene does not meet the preset reconstruction requirements, so as to plan a new flight path for the UAV;

[0070] The scene restoration module 16 is used to control the UAV to fly according to the new flight path and collect new campus image data, so as to use the new campus image data to restore the area to be restored in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining the target campus 3D reconstruction scene.

[0071] Figure 6This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0072] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0073] When the processor 502 executes the program, the campus three-dimensional reconstruction method provided in the above embodiment is implemented.

[0074] Furthermore, the terminal further includes:

[0075] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0076] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0077] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0078] If the memory 501, processor 502, and communication interface 503 are implemented independently, the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to enable communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified as address buses, data buses, control buses, etc. For ease of illustration, the figure shows only one line, but this does not mean that there is only one bus or only one type of bus.

[0079] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0080] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0081] This embodiment also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the above-mentioned campus three-dimensional reconstruction method is implemented.

[0082] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

[0083] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0084] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as a sequenced list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can read and execute instructions from an instruction execution system, apparatus, or device).

[0085] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0086] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A campus 3D reconstruction method, characterized in that: The method comprises: Determine the aerial photography range of the current campus and plan the flight path of the drone based on the aerial photography range; Controlling the drone equipped with the camera to fly along the flight path and collect corresponding campus image data; Using a pre-trained NeRF model to perform neural radiation field modeling on the campus image data, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets the preset reconstruction requirements; If the three-dimensional reconstruction scene of the campus does not meet the preset reconstruction requirements, identifying an area to be repaired in the three-dimensional reconstruction scene of the campus to plan a new flight path for the drone; The drone is controlled to fly along the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be repaired in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining a target campus 3D reconstruction scene.

2. The campus 3D reconstruction method according to claim 1, characterized in that: The pre-trained NeRF model is a 3D reconstruction model based on a preset neural network structure, and during the training process, the illumination invariance loss function is used to constrain the pixel or feature consistency of the 3D reconstruction model for the same scene under different illumination.

3. The campus 3D reconstruction method according to claim 1, characterized in that: In the process of using the pre-trained NeRF model to perform neural radiation field modeling on the campus image data to generate a three-dimensional reconstruction scene of the campus, the following steps are also included: Dynamic objects in the campus image data are filtered using a preset temporal consistency detection algorithm.

4. The campus 3D reconstruction method according to claim 1, characterized in that: Also includes: Based on the pre-set communication protocol and communication method between the drone and the ground station or cloud server, the campus image data is transmitted to the ground station or the cloud server.

5. The campus 3D reconstruction method according to claim 1, characterized in that: After the 3D reconstruction scene of the campus meets the preset reconstruction requirements and the target 3D reconstruction scene of the campus is obtained, the method further includes: The target campus 3D reconstruction scene is converted into a Mesh model using a preset progressive Mesh simplification algorithm, so as to construct a virtual 3D campus model of the campus landmark buildings using the Mesh model.

6. The campus 3D reconstruction method according to claim 5, characterized in that: After constructing the virtual model of the campus landmark building using the Mesh model, the method further includes: Corresponding POI information is added to the campus landmark buildings in the virtual three-dimensional campus model.

7. The campus three-dimensional reconstruction method according to any one of claims 1 to 6, characterized in that: The identifying the area to be repaired in the three-dimensional reconstruction scene of the campus to plan a new flight path for the drone includes: A pre-trained semantic segmentation model is used to identify the area to be repaired in the three-dimensional reconstruction scene of the campus, and a preset data collection weight is assigned to the area to be repaired to plan a new flight path for the drone.

8. A campus 3D reconstruction device, characterized in that: The device comprises: Aerial photography range determination module, used to determine the aerial photography range of the current campus; A first path planning module is used to plan a flight path of the UAV based on the aerial photography range; A data acquisition module is used to control the drone equipped with a camera to fly along the flight path and collect corresponding campus image data; A scene reconstruction module is used to perform neural radiation field modeling on the campus image data using a pre-trained NeRF model, generate a three-dimensional reconstruction scene of the campus, and determine whether the three-dimensional reconstruction scene of the campus meets preset reconstruction requirements; an area recognition module, configured to identify an area to be repaired in the 3D reconstructed campus scene if the 3D reconstructed campus scene does not meet the preset reconstruction requirements, so as to plan a new flight path for the UAV; The scene restoration module is used to control the UAV to fly according to the new flight path and collect new campus image data, so as to use the new campus image data to repair the area to be restored in the campus 3D reconstruction scene until the campus 3D reconstruction scene meets the preset reconstruction requirements, thereby obtaining the target campus 3D reconstruction scene.

9. A terminal, characterized in that: include: A memory, a processor, and a campus three-dimensional reconstruction program stored in the memory and executable on the processor, wherein the campus three-dimensional reconstruction program, when executed by the processor, implements the steps of the campus three-dimensional reconstruction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which can be executed to implement the steps of the campus three-dimensional reconstruction method according to any one of claims 1 to 7.

Images