Unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition method and system based on reinforcement learning

Abstract

The application relates to a kind of unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition method and system based on reinforcement learning, wherein the method comprises the following steps: constructing virtual indoor simulation space and virtual unmanned aerial vehicle model, and laying target acquisition point;Reinforcement learning model is constructed and trained, in the training process, at each time step, unmanned aerial vehicle obtains target acquisition point information input reinforcement learning model, reinforcement learning model generates action instruction and is transmitted to virtual unmanned aerial vehicle model, virtual unmanned aerial vehicle executes geometric transformation, updates position and orientation, and reconstructs visual field information;The trained reinforcement learning model is deployed in the untrained virtual simulation environment, unmanned aerial vehicle autonomous flight experiment and data acquisition are carried out, and the acquisition performance is quantitatively evaluated through multiple performance indicators.Compared with the prior art, the application can realize efficient and stable data acquisition in complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of building informatization and unmanned aerial vehicle autonomous control technology, in particular to an indoor three-dimensional reconstruction autonomous acquisition method and system of unmanned aerial vehicle based on reinforcement learning. BACKGROUND

[0002] In recent years, the emergence of many image-based intelligent reconstruction algorithms (such as NeRF, 3DGS) has lowered the threshold of the acquisition device for the reconstruction task to sensors such as monocular cameras, greatly reducing the weight of the sensor device and the data transmission requirement, and promoting the universality of lightweight unmanned aerial vehicles carrying sensors as a data acquisition method. The characteristics of this type of intelligent reconstruction algorithm include using image data instead of laser point cloud, image data covering the entire field, etc. However, since the unmanned aerial vehicle only has a single position and angle for each shot, an efficient path needs to be planned to deal with resource waste. Currently, there are studies at home and abroad that apply PPO, DDPG and other algorithms in deep reinforcement learning to robot obstacle avoidance, path optimization and other tasks, mainly focusing on outdoor navigation, target reaching, path following and other scenarios. For example, Chinese patent CN115826621A discloses a kind of unmanned aerial vehicle motion planning method and system based on deep reinforcement learning, but this kind of method focuses on "safe arrival" and "obstacle avoidance efficiency", and there is still a research gap in the design of vision modeling, space coverage and other strategies.

[0003] Under this background, lightweight unmanned aerial vehicles have shown advantages over traditional surveying and mapping techniques in narrow and complex indoor environments. For example, a study found that unmanned aerial vehicles can pass through small windows into attics and collect surfaces that ground laser scanning cannot cover, while maintaining an accuracy of 1 cm and reducing equipment costs. Another study showed that unmanned aerial vehicles can complete autonomous mapping through ventilation windows or collapsed openings in environments without GPS, demonstrating higher flexibility and safety. A study also pointed out that unmanned aerial vehicles can save about 60% of time in indoor construction monitoring while maintaining mapping accuracy.

[0004] Therefore, there is currently a lack of a lightweight unmanned aerial vehicle autonomous acquisition method with three-dimensional space reconstruction as the ultimate goal. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art and provide an indoor three-dimensional reconstruction autonomous acquisition method and system of unmanned aerial vehicle based on reinforcement learning, which takes into account the acquisition efficiency and completeness, and provides effective data support for subsequent indoor three-dimensional reconstruction.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] An indoor three-dimensional reconstruction autonomous acquisition method of unmanned aerial vehicle based on reinforcement learning, the method comprising the following steps:

[0008] S1, scenario simulation modeling and spatial target setting: constructing a virtual indoor simulation space and a virtual UAV model, and arranging target acquisition points;

[0009] S2, strategy learning and capability training: constructing and training a reinforcement learning model, wherein the reward function of the reinforcement learning model includes coverage reward, new discovery reward, exploration reward, out-of-boundary punishment, collision punishment and efficiency punishment; during the training process, at each time step, the UAV obtains target acquisition point information inputting into the reinforcement learning model, the reinforcement learning model generates action instructions and delivers them to the virtual UAV model, the virtual UAV performs geometric transformation, updates the position and orientation, and reconstructs the visual field information;

[0010] S3, autonomous flight evaluation and data acquisition implementation: deploying the trained reinforcement learning model in an untrained virtual simulation environment, carrying out UAV autonomous flight experiments and data acquisition, and quantitatively evaluating the acquisition performance through multiple performance indicators.

[0011] The virtual indoor simulation space is constructed, specifically:

[0012] The virtual indoor simulation space is constructed by using the Rhino modeling platform to simulate typical rooms, corridors and corner node structures in buildings; during the modeling process, first, basic space units are established, and then openings, penetrations and blockages are realized through Boolean operations, and then the units are combined in a geometric splicing manner to form the overall environment; thereafter, furniture and obstacles are introduced, and simplification is realized by abstracting complex components into equivalent geometric bodies.

[0013] During the modeling process of the virtual indoor simulation space, the difficulty is adjusted according to three dimensions of room quantity, functional combination and boundary obstacle, wherein for the room quantity, the difficulty is increased from a single room to a multi-room combination; for the functional combination, a diversified space is formed by connecting corridors and corners; for the boundary obstacle, furniture or obstacles are added at the wall or node to increase the complexity of shielding and obstacle avoidance.

[0014] The method for arranging target acquisition points is to arrange them based on the Grasshopper script, select the exposed surfaces of each component in the simulation space as acquisition targets, and arrange orthogonal point clouds at a preset interval, wherein the preset interval is dynamically adjusted according to the complexity.

[0015] In the reinforcement learning model, the environment state at each time step is represented as a multi-dimensional vector, including the current position of the UAV, the current orientation angle, the number of visible target acquisition points in the simulated visual field, the coverage rate, and the distance from the UAV to the nearest obstacle in the front, back, left and right directions, wherein the coverage rate is defined as the proportion of the number of visible target acquisition points in the simulated visual field to the total number of target acquisition points.

[0016] The action output by the reinforcement learning model comprises:

[0017] A translation action: performing a translation of a preset fixed step length in any one of the front, back, left and right directions on the UAV;

[0018] A rotation action: performing a rotation of a multiple of 45 degrees in the horizontal direction on the UAV;

[0019] In-place holding: staying for a period of time, observing or waiting for a new strategy;

[0020] Wherein, only one action is allowed to be performed at each time step.

[0021] The reward function of the reinforcement learning model is represented as:

[0022] ,

[0023] Wherein, is the immediate value of the i-th reward or penalty at time t, is the weight of the i-th reward or penalty at time t, is the number of items of the reward or penalty, and the weight is dynamically adjusted in the early and late stages of the training phase; Wherein, the coverage reward is defined as a preset value multiplied by the proportion of the number of newly visible target collection points in the total number of target collection points in the simulated field of view; the new discovery reward is defined as a preset value multiplied by the number of newly discovered target collection points; the exploration reward is defined as a step reward, which assigns a first score to the first visit and a second score to the second visit, and assigns no score to subsequent repeated visits, wherein the first score is greater than the second score; the out-of-bound penalty is defined as a preset penalty value when the UAV is out of bounds; the collision penalty is defined as a gradient penalty preset according to the distance between the UAV and the obstacle, and the closer the distance, the greater the penalty value; and the efficiency penalty is defined as the number of repeated visits. n The weights of the coverage reward and the exploration reward are positive values, and the weight in the early stage is greater than the weight in the late stage; the weight of the new discovery reward is positive, and the weight in the early stage is less than the weight in the late stage; the weights of the out-of-bound penalty and the collision penalty are fixed at 1; and the weight of the efficiency penalty is negative, and the weight in the early stage is greater than the weight in the late stage.

[0024] The performance indicators include coverage, path step number and collection efficiency.

[0025] A reinforcement learning-based indoor three-dimensional reconstruction autonomous collection system for a UAV, for implementing the method, the system comprising:

[0026] The performance indicators include coverage, path step number and collection efficiency.

[0027] A reinforcement learning-based indoor three-dimensional reconstruction autonomous collection system for a UAV, for implementing the method, the system comprising:

[0028] ​​The scene simulation modeling and space target setting module is used for constructing a virtual indoor simulation space in a Rhino environment and a virtual unmanned aerial vehicle model through a Grasshopper platform, and combining a target collection point arranged by a Grasshopper script;

[0029] The strategy learning and ability training module is used for constructing and training a reinforcement learning model, and designing a bidirectional interface of the reinforcement learning model and the Grasshopper platform for data transmission, wherein a reward function of the reinforcement learning model comprises a coverage reward, a new discovery reward, an exploration reward, an out-of-bound punishment, a collision punishment and an efficiency punishment; in the training process, at each time step, the unmanned aerial vehicle obtains target collection point information input into the reinforcement learning model, the reinforcement learning model generates an action instruction and transmits the action instruction to the virtual unmanned aerial vehicle model, the virtual unmanned aerial vehicle performs geometric transformation, updates a position and an orientation, and reconstructs visual field information;

[0030] The autonomous flight evaluation and data collection implementation module is used for deploying the trained reinforcement learning model in an untrained virtual simulation environment, carrying out unmanned aerial vehicle autonomous flight experiments and data collection, and quantitatively evaluating collected performance through multiple indexes.

[0031] Compared with the prior art, the present application has the following beneficial effects:

[0032] The application provides an unmanned aerial vehicle (UAV) building indoor three-dimensional reconstruction autonomous acquisition method based on reinforcement learning, which can realize efficient and stable data acquisition in a complex environment. The application constructs multiple types of virtual indoor scenes and virtual UAV models, designs dynamically adjustable acquisition points, provides a structured spatial environment and evaluation benchmark, and selects a Proximal Policy Optimization (PPO) algorithm as a core reinforcement learning framework, designs a composite reward function, combines coverage, efficiency and collision penalty factors, and dynamically adjusts the weight to realize dual-objective optimization of "acquisition integrity-path efficiency". This mechanism effectively avoids the problems of low acquisition efficiency and high cost of traditional methods. Through continuous learning and adaptation of the reinforcement learning model, good performance can be maintained when the environment changes. This adaptive capability enables the UAV to effectively perform three-dimensional reconstruction in various indoor scenes. The application innovatively constructs a bidirectional communication interface based on GH-Python, realizes real-time closed-loop of perception input, strategy generation, behavior evaluation and action output, achieves efficient linkage of the model and the algorithm, breaks through the limitation that the Grasshopper environment usually only supports static modeling, enables the model to efficiently link between geometric modeling and strategy training, and greatly improves the experimental iteration efficiency. In the test experiment, the time consumption of the UAV autonomous acquisition of each group of rooms is 10-20 minutes, and the acquisition efficiency and integrity are significantly improved compared with traditional path planning. The application has outstanding innovation in algorithm design, interface mechanism and path optimization, and provides an efficient and reliable solution for three-dimensional reconstruction in complex indoor environments such as old and new changes. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A method flowchart of the application;

[0034] Figure 2 An indoor simulation model diagram in an embodiment;

[0035] Figure 3 A target acquisition point layout diagram in an embodiment;

[0036] Figure 4 A control process diagram of the application;

[0037] Figure 5 A state information diagram in the reinforcement learning model of the application;

[0038] Figure 6 A reinforcement learning model training process diagram of the application;

[0039] Figure 7 A sample strategy optimization curve and fitting slope diagram of the application in an embodiment. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of the present application.

[0041] Unless otherwise defined, technical terms or scientific terms used in the present application should be understood as their common meanings in the technical field to which the present application pertains. The terms "one", "a", "an", "the", and similar terms in the present application do not denote a singular number but can denote one or more. The terms "comprise", "include", "have", and any variations thereof in the present application are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a list of steps or modules (units) is not limited to the listed steps or units, but can further include other steps or units not listed or can further include other steps or units inherent to such processes, methods, products, or devices. The terms "connect", "connected", "couple", and similar terms in the present application are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" in the present application refers to two or more. The term "and / or" describes the association relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", and the like in the present application are only to distinguish similar objects, and do not represent a specific order for the objects.

[0042] The embodiment provides an unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition method based on reinforcement learning, as shown in the figure, the method comprises the following steps: Figure 1

[0043] S1, scenario simulation modeling and space target point setting: constructing a virtual indoor simulation space and a virtual unmanned aerial vehicle model, and arranging target acquisition points.

[0044] S11, constructing a virtual indoor simulation space.

[0045] ​Rhino modeling platform is used to build a virtual indoor simulation space to simulate typical rooms, corridors and corner nodes in buildings. In the modeling process, first, the basic space unit is established, and then the opening, penetration and blocking are realized through Boolean operation. Then, the unit is combined in a geometric splicing way to form a whole environment with controllability and complexity. After that, furniture and obstacles are introduced to improve the authenticity. The complex components are simplified by abstracting them into equivalent geometric bodies such as cuboids and cylinders, so as to balance the computational efficiency and environmental feature expression.

[0046] In the modeling process of virtual indoor simulation space, the model construction is adjusted in three dimensions, forming a multi-path planning and obstacle avoidance difficulty, as shown in Figure 2 : According to the number of rooms, functional combination and boundary obstacles, the difficulty is adjusted in three dimensions. For the number of rooms, the difficulty increases from single room to multi-room combination. For functional combination, diversified space is formed by connecting corridors and corners. For boundary obstacles, furniture or obstacles are added to walls or nodes to increase the complexity of shielding and obstacle avoidance.

[0047] S12, layout target acquisition points.

[0048] Based on the Grasshopper script, indoor acquisition points are laid out. The exposed surfaces of each component in the room (wall, ceiling, floor, furniture surface) are selected as the acquisition target, and orthogonal point clouds are laid out at an interval of 600 mm, as shown in Figure 3 . To control the training complexity, the point spacing and total number of points can be dynamically adjusted.

[0049] S13, build a virtual UAV model.

[0050] A virtual UAV model is built through the Grasshopper platform, which has the basic characteristics of typical UAVs and can simulate flight, environmental perception and data acquisition tasks.

[0051] S2, strategy learning and ability training: build a reinforcement learning model and train it. In the training process, at each time step, the UAV obtains the target acquisition point information input into the reinforcement learning model, which generates action instructions and transmits them to the bidirectional communication interface in the Grasshopper environment. After analysis, it is transmitted to the virtual UAV model, which performs geometric transformation, updates the position and orientation, and rebuilds the visual information. These information returns to the Python control end to promote training iteration. As shown in Figure 4 .

[0052] In the reinforcement learning model, the environment state at each time step is represented as a multi-dimensional vector, as shown in Figure 5 , including:

[0053] (1) current position of the UAV; (2) current orientation angle (one of 8 directions); (3) number of visible target collection points in the simulated field of view: (4) coverage (the proportion of the number of visible target collection points in the simulated field of view in the total number of target collection points); (5) distance from the UAV to the nearest obstacle in the front, rear, left and right directions.

[0054] The action output by the reinforcement learning model includes:

[0055] (1) translation action: performing translation of a fixed step length of 600mm in any one of the front, rear, left and right directions on the UAV;

[0056] (2) rotation action: performing rotation of a multiple of 45 degrees in the horizontal direction on the UAV;

[0057] (3) stay in place: staying for a period of time to observe or wait for a new strategy;

[0058] Among them, only one action is allowed to be executed at each time step, ensuring the explainability and computational efficiency of trajectory control.

[0059] In the reinforcement learning training process, based on the multi-objective optimization theory and dynamic weight adjustment mechanism, a composite reward function is designed, including coverage reward, new discovery reward, exploration reward, out-of-bound penalty, collision penalty and efficiency penalty, and the overall reward is obtained by weighted summation to realize comprehensive guidance of the UAV path planning, expressed as:

[0060] ,

[0061] Among them, is the instantaneous value of the th reward or penalty at time , is the weight of the th reward or penalty at time , n is the number of rewards or penalties, and the weights are dynamically adjusted in the early and late stages of the training phase, so that the strategy focuses on different aspects in different stages, and can achieve dynamic balance between coverage, efficiency and safety, thereby significantly improving the overall completion and training effect of the UAV path planning task.

[0062] The coverage reward is defined as a preset value multiplied by the proportion of the number of new visible target collection points in the simulation field of view to the total number of target collection points; the new discovery reward is defined as a preset value multiplied by the number of newly discovered target collection points; the exploration reward is defined as a step reward, a first score is given for the first visit, a second score is given for the second visit, and subsequent repeated visits are given scores, wherein the first score is greater than the second score; the out-of-bound penalty is defined as: when out of bounds, a preset penalty value is given; the collision penalty is defined as a gradient penalty preset according to the distance between the unmanned aerial vehicle and the obstacle, and the closer the distance, the greater the penalty value; and the efficiency penalty is defined as the number of repeated visits. The weights of the coverage reward and the exploration reward are positive values, and the weight in the early stage is greater than the weight in the later stage, the weight of the new discovery reward is positive, and the weight in the early stage is less than the weight in the later stage, the weights of the out-of-bound penalty and the collision penalty are fixed as 1, and the weight of the efficiency penalty is negative, and the weight in the early stage is greater than the weight in the later stage.

[0063] In a preferred embodiment, the specific settings are as shown in Table 1.

[0064] Table 1

[0065]

[0066] The Proximal Policy Optimization (PPO) algorithm is used as the core framework for reinforcement learning model training. The training process design is as shown in Figure 6 : (1) Initial environment generation: randomly set the starting position and orientation of the unmanned aerial vehicle, and load the indoor target point layout information; (2) state input extraction: the current state is extracted by the Grasshopper control end and transmitted to the PPO module in Python; (3) policy implementation and action output: the PPO network outputs the action, indicating the next step movement or rotation of the unmanned aerial vehicle; (4) action execution and feedback acquisition: the Grasshopper receives the action output by the execution algorithm, the environment is updated, and the changes in the collection points and the trajectory are recorded; (5) reward calculation and network update: the immediate reward is calculated according to the collection effect, which is used for PPO policy update; (6) policy optimization loop: multiple rounds of training are performed until the policy converges. Through the integration of data obtained by training the same room multiple times, it can be found that the collection efficiency increases with the increase of the number of training, as shown in Figure 7 .

[0067] S3, autonomous flight evaluation and data collection implementation: deploy the trained reinforcement learning model in an untrained virtual simulation environment, carry out unmanned aerial vehicle autonomous flight experiments and data collection, and quantitatively evaluate the collection performance through multiple performance indicators.

[0068] S31, simulation flight experiment setting.

[0069] The experimental environment is based on a Rhino constructed indoor space model, and the process is carried out according to the trained process. The starting point and direction are initialized, and the target point cloud is read; according to the current state, the action instruction is output by calling the strategy network; the action (movement or rotation) is executed by Grasshopper, and the space position is updated; it is judged whether there is a new collection point in the current field of view, and the collection situation is recorded; repeat the execution until the coverage target or the step limit is reached.

[0070] S32, performance evaluation.

[0071] Independent experiments are carried out in a set scene, and key performance indicators including coverage, path step number, collection efficiency, etc. are recorded.

[0072] S33, according to the performance indicators, a new round of adjustment and training is carried out.

[0073] The embodiment also provides an unmanned aerial vehicle indoor three-dimensional reconstruction autonomous collection system based on reinforcement learning, which is used to realize the method, and the system comprises:

[0074] Scenario simulation modeling and space target point setting module: used for constructing a virtual indoor simulation space in a Rhino environment and a virtual unmanned aerial vehicle model through a Grasshopper platform, and combining a Grasshopper script to arrange target collection points;

[0075] Strategy learning and ability training module: used for constructing a reinforcement learning model and training, and designing a bidirectional interface of the reinforcement learning model based on GH-Python and the Grasshopper platform, supporting state transmission, action execution and environment feedback. The reward function of the reinforcement learning model comprises a coverage reward, a new discovery reward, an exploration reward, an out-of-bound penalty, a collision penalty and an efficiency penalty; in the training process, at each time step, the unmanned aerial vehicle acquires target collection point information and inputs the reinforcement learning model, the reinforcement learning model generates an action instruction and transmits it to the virtual unmanned aerial vehicle model, the virtual unmanned aerial vehicle executes geometric transformation, updates the position and orientation, and reconstructs the field of view information;

[0076] Autonomous flight evaluation and data collection implementation module: used for deploying the trained reinforcement learning model in an untrained virtual simulation environment, carrying out unmanned aerial vehicle autonomous flight experiments and data collection, and quantitatively evaluating the collection performance through multiple indicators.

[0077] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0078] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition system based on reinforcement learning, characterized in that, The system comprises: A scene simulation modeling and space target point setting module: used for constructing a virtual indoor simulation space in a Rhino environment and a virtual unmanned aerial vehicle model through a Grasshopper platform, and setting target collection points by combining a Grasshopper script; A strategy learning and ability training module: used for constructing and training a reinforcement learning model, and designing a two-way interface based on GH-Python of the reinforcement learning model and the Grasshopper platform for data transmission, wherein a reward function of the reinforcement learning model comprises a coverage reward, a new discovery reward, an exploration reward, an out-of-bound penalty, a collision penalty and an efficiency penalty, the coverage reward is defined as a preset value multiplied by a proportion of a number of newly visible target collection points in a simulation visual field to a total number of target collection points, the new discovery reward is defined as a preset value multiplied by a number of newly discovered target collection points, and the exploration reward is defined as a step reward, a first score is given for a first visit, a second score is given for a second visit, and subsequent repeated visits are not scored, wherein the first score is greater than the second score; the out-of-bound penalty is defined as a preset penalty value when out of bound; the collision penalty is defined as a gradient penalty preset according to a distance between the unmanned aerial vehicle and an obstacle, and the closer the distance, the greater the penalty value; and the efficiency penalty is defined as a number of repeated visits; in the training process, at each time step, the unmanned aerial vehicle obtains target collection point information and inputs the information into the reinforcement learning model, the reinforcement learning model generates an action instruction and transmits the instruction to the virtual unmanned aerial vehicle model, the virtual unmanned aerial vehicle performs geometric transformation, updates a position and an orientation, and reconstructs visual field information; An autonomous flight evaluation and data collection implementation module: used for deploying the trained reinforcement learning model in an untrained virtual simulation environment, carrying out unmanned aerial vehicle autonomous flight experiments and data collection, and quantitatively evaluating collection performance through multiple indexes. 2.The indoor 3D reconstruction autonomous acquisition system based on reinforcement learning of claim 1, wherein, The virtual indoor simulation space is constructed by using a Rhino modeling platform to simulate typical room, corridor and corner node structures in a building; in the modeling process, a basic space unit is first established, openings, through-penetrations and blockages are realized through Boolean operations, and the unit is combined in a geometric splicing manner to form an overall environment; thereafter, furniture and obstacles are introduced, and simplification is realized by abstracting complex components into equivalent geometric bodies. In the modeling process of the virtual indoor simulation space, difficulty is adjusted according to three dimensions of room quantity, functional combination and boundary obstacles, wherein for the room quantity, difficulty is increased from a single room to a multi-room combination; for the functional combination, a diversified space is formed through corridors and corner connections; and for the boundary obstacles, furniture or obstacles are added at walls or nodes to increase the complexity of shielding and obstacle avoidance. 3.The system of claim 2, wherein, The method for setting the target collection points is to set them based on a Grasshopper script, select exposed surfaces of components in the simulation space as collection targets, and set orthogonal point clouds at a preset interval, which is dynamically adjusted according to complexity. 4.The system of claim 1, wherein, ​ 5.The unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition system based on reinforcement learning of claim 1, wherein, In the reinforcement learning model, an environment state of each time step is represented as a multi-dimensional vector, including a current position of the unmanned aerial vehicle, a current orientation angle, a number of visible target collection points in a simulated field of view, a coverage rate, and distances from the unmanned aerial vehicle to the nearest obstacles in four directions of front, back, left and right. 6.The system of claim 1, wherein, The action output by the reinforcement learning model includes: a translation action: performing a translation of a preset fixed step length in any one direction of front, back, left and right on the unmanned aerial vehicle; a rotation action: performing a rotation of a multiple of 45 degrees in a horizontal direction on the unmanned aerial vehicle; a stay-in-place action: staying for a period of time to observe or wait for a new strategy; wherein only one action is allowed to be performed at each time step.

7. The unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition system based on reinforcement learning according to claim 1, characterized in that, A reward function of the reinforcement learning model is represented as: ， wherein, is the first reward or penalty at time is the immediate value of the reward or penalty at time is the first weight of the reward or penalty at time is the immediate value of the reward or penalty at time n is the number of terms of the reward or penalty, the weights being dynamically adjusted over time as the training session progresses from an early to a later stage.

8. The unmanned aerial vehicle indoor three-dimensional reconstruction autonomous acquisition system based on reinforcement learning according to claim 7, characterized in that, weights of the coverage rate reward and the exploration reward are positive values, and a weight in an early stage is greater than a weight in a later stage, a weight of the new discovery reward is positive, and a weight in the early stage is less than a weight in the later stage, weights of the out-of-bound penalty and the collision penalty are fixed as 1, and a weight of the efficiency penalty is negative, and a weight in the early stage is greater than a weight in the later stage. 9.The system of claim 1, wherein, The performance indicators include the coverage rate, the path step number and the collection efficiency.

Images