Image acquisition method, device and 3D Gaussian splash modeling method
By obtaining point cloud data of the region of interest and generating grid data and labeling attribute labels, route planning is carried out to collect image data, the shortcomings of the existing 3D Gaussian splash modeling algorithm in the training set acquisition method are solved, and the construction is improved The model effect and scene coverage achieve more accurate motion structure recovery and all-round coverage.
Patent Information
- Application Number
- CN202410931763.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-12
AI Technical Summary
The existing 3D Gaussian splatter modeling algorithm has flaws in the training set acquisition method, resulting in poor modeling effect, especially the accuracy of motion structure recovery and insufficient scene coverage.
By obtaining point cloud data of the region of interest, generating grid data and labeling attribute tags for each cell, route planning is performed based on these tags, target routes are generated to collect image data, and ensuring that the image data better meets the scene modeling model Training requirements.
The modeling effect of scene modeling models is improved, the accuracy of motion structure recovery and the uniformity of scene coverage is ensured, the objects in the region of interest can be fully covered, and new high-quality views are generated at different distances.
Smart Images

Figure CN118470579B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of route planning, and in particular to an image acquisition method, device and 3D Gaussian splash modeling method. Background Art
[0002] Realistic 3D technology is a technology that presents real-world objects and scenes in three dimensions. It captures objects and scenes in the real world, and then uses computer graphics, image processing, computer vision and other technologies to model, render and display them, so that users can feel the spatial and three-dimensional sense of the real world in a virtual environment. At present, related technologies provide modeling algorithms such as Mesh, NeRF, and 3D Gaussian Splatting (3DGS). Compared with the two modeling algorithms of Mesh and NeRF, the 3D Gaussian Splatting modeling algorithm has faster training and rendering speeds.
[0003] However, the modeling effect of the modeling algorithm depends on the training set (that is, a set of images of objects or regions of interest), such as the accuracy of the results of motion structure recovery of the training set, the coverage of the training set for the scene, the near and far distribution of the training set in space, etc. Existing training set collection methods usually have defects in these aspects, resulting in suboptimal modeling results. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide an image acquisition method, device and 3D Gaussian splash modeling method, which can make the acquired image data better meet the training requirements of the scene modeling model, thereby improving the modeling effect of the scene modeling model.
[0005] In a first aspect, an embodiment of the present invention provides an image acquisition method, including:
[0006] Get the point cloud data corresponding to the area of interest;
[0007] Generate grid data corresponding to the region of interest, and determine an attribute label corresponding to each cell in the grid data based on the point cloud data; wherein the attribute label is used to characterize the point cloud occupancy of the cell and the distance between cells;
[0008] Performing route planning according to the attribute tags to generate a target route corresponding to the area of interest;
[0009] Image data corresponding to the region of interest is collected based on the target route, and the image data is used to train a scene modeling model.
[0010] In one embodiment, the step of determining the attribute label corresponding to each cell in the grid data based on the point cloud data includes:
[0011] Based on the relative position relationship between the point cloud data and each cell in the grid data, labeling the cell with an occupancy status label to divide the cell into occupied cells and unoccupied cells;
[0012] Based on the relative position relationship between the occupied cells and the unoccupied cells, distance labels are marked for the unoccupied cells.
[0013] In one embodiment, based on the relative position relationship between the point cloud data and each cell in the grid data, the step of labeling the cell with a point cloud occupancy label to divide the cell into occupied cells and unoccupied cells includes:
[0014] Determine whether there is a point located in the cell in the point cloud data;
[0015] If yes, an occupied label is marked for the cell to classify the cell as an occupied cell; if no, an unoccupied label is marked for the cell to classify the cell as an unoccupied cell.
[0016] In one embodiment, based on the relative position relationship between the occupied cells and the unoccupied cells, the step of marking the unoccupied cells with distance labels includes:
[0017] Based on the relative positional relationship between the occupied cell and the unoccupied cell, a shortest distance between the occupied cell and the unoccupied cell is determined, and the shortest distance is marked as a distance label to the unoccupied cell.
[0018] In one embodiment, the step of performing route planning according to the attribute tag to generate a target route corresponding to the area of interest includes:
[0019] Filtering candidate cells that meet preset conditions from the cells according to the attribute tags;
[0020] A specified path planning algorithm is used to perform route planning for the candidate cells based on preset route planning constraints to generate a target route corresponding to the area of interest.
[0021] In one embodiment, the step of selecting candidate cells that meet preset conditions from the cells according to the attribute tags includes:
[0022] If the cell is divided into an unoccupied cell and the distance tag marked with the cell satisfies the distance range, it is determined that the cell meets the preset condition and the cell is determined as a candidate cell.
[0023] In one embodiment, the step of collecting image data corresponding to the region of interest based on the target route includes:
[0024] Extracting a target waypoint set from the target route, and determining a shooting direction corresponding to each waypoint in the target waypoint set;
[0025] Controlling the drone to navigate along the target route, and performing a shooting action on the region of interest according to the shooting direction corresponding to each waypoint during the navigation process, so as to obtain image data corresponding to the region of interest;
[0026] Alternatively, the drone is controlled to navigate along the target route, and a video recording action is performed on the region of interest during the navigation process to obtain image data corresponding to the region of interest.
[0027] In a second aspect, an embodiment of the present invention further provides an image acquisition device, comprising:
[0028] A data acquisition module, used to acquire point cloud data corresponding to the area of interest;
[0029] A label determination module, used to generate grid data corresponding to the region of interest, and determine an attribute label corresponding to each cell in the grid data based on the point cloud data; wherein the attribute label is used to characterize the point cloud occupancy of the cell and the distance between grids;
[0030] A route planning module, used to perform route planning according to the attribute tags and generate a target route corresponding to the area of interest;
[0031] An image acquisition module is used to acquire image data corresponding to the area of interest based on the target route, and the image data is used to train the scene modeling model.
[0032] In a third aspect, an embodiment of the present invention further provides a 3D Gaussian splash modeling method, comprising:
[0033] Acquire image data corresponding to the region of interest; wherein the image data is acquired by using any one of the image acquisition methods provided in the first aspect;
[0034] A training data set is constructed based on the image data, and a 3D Gaussian splash model is trained using the training data set. The trained 3D Gaussian splash model is used to model the region of interest.
[0035] In a fourth aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.
[0036] An image acquisition method, device and 3D Gaussian splash modeling method provided by an embodiment of the present invention first obtain point cloud data corresponding to an area of interest; then generate grid data corresponding to the area of interest, and determine the attribute label corresponding to each cell in the grid data based on the point cloud data, the attribute label is used to characterize the point cloud occupancy of the cell and the distance between cells; then perform route planning according to the attribute label to generate a target route corresponding to the area of interest; finally, based on the target route, collect image data corresponding to the area of interest, and the image data is used to train a scene modeling model. The above method generates grid data corresponding to the area of interest and labels attribute labels for each cell in the grid data, so as to reflect the cell where the object is located and the cell that the drone can pass through, and the distance information between the cell where the object is located and the cell that the drone can pass through, and on this basis, perform route planning. If the drone can reach it, the target route planned by the embodiment of the present invention can completely cover the objects in the area of interest, so that the collected image data can better meet the training requirements of the scene modeling model, thereby improving the modeling effect of the scene modeling model.
[0037] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1 A schematic diagram of a flow chart of an image acquisition method provided by an embodiment of the present invention;
[0041] Figure 2A schematic diagram of a target waypoint set provided by an embodiment of the present invention;
[0042] Figure 3 A schematic diagram of a 3D Gaussian splash modeling method provided by an embodiment of the present invention;
[0043] Figure 4 A schematic diagram of the structure of an image acquisition device provided by an embodiment of the present invention;
[0044] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] At present, real-life 3D technology has been widely used in many fields, such as games, movies, virtual reality, geographic information systems, urban planning, cultural relics protection, and drone smart inspections.
[0047] With the continuous development of computer graphics, image processing, computer vision and other technologies, real-life 3D technology will achieve more important application value in the future. Currently commonly used modeling methods include Mesh, which directly describes the spatial volume and color of objects, through polygon mesh (Polygonmesh), which is a collection of vertices and polygons that represent the shape of a polyhedron in 3D computer graphics. It is a kind of irregular structure data. These meshes are usually composed of triangles, quadrilaterals or other simple convex polygons. Among them, the most commonly used is triangular mesh, which usually needs to store three types of information: vertices, edges and faces, but it requires a large number of modeling images, and the established model has poor restoration of texture-free parts and simulation of real environments;
[0048] Another newer modeling algorithm, NeRF, uses a scalar field to describe the ability of spatial absorption and emission of light. A neural network is used to approximate this scalar field function. All weight parameters of the neural network are optimized through training. Only 2D posed images are used as supervision to represent complex 3D scenes and achieve large-scale spatial reconstruction.
[0049] The newly proposed 3D Gaussian splash uses a set of 3D Gaussian functions distributed in space to describe the changes in the intensity and color of light as it passes through space. When light passes through these Gaussian functions, their colors and transparency are accumulated to eventually form the color of the corresponding pixel. Both NeRF and 3D Gaussian Splash use a certain method to approximate the behavior of light as it passes through space, and use neural networks to train and optimize a large number of complex parameters to handle light and shadow effects at different angles, reflective effects on glass and water surfaces, and small volumes. Compared with traditional methods, fewer images are required for reconstruction, and the model requires less storage space. And its biggest advantage is that the training and rendering speeds are much faster than NeRF.
[0050] Although 3D Gaussian splashing has the above advantages, it also has its limitations. One important point is that the final modeling effect of 3D Gaussian splashing depends on the training set (a set of photos of objects or areas of interest), such as the accuracy of the results of motion structure recovery of the training set, the coverage of the training set photos for the scene, the distribution of the training set photos in space, etc. The existing training set collection methods usually have defects in these aspects, resulting in suboptimal final modeling results.
[0051] Based on this, the present invention provides an image acquisition method, device and 3D Gaussian splash modeling method, which can make the acquired image data better meet the training requirements of the scene modeling model, thereby improving the modeling effect of the scene modeling model.
[0052] To facilitate understanding of this embodiment, firstly, an image acquisition method disclosed in an embodiment of the present invention is described in detail. Figure 1 The flowchart of an image acquisition method shown in FIG. 1 mainly includes the following steps S102 to S108:
[0053] Step S102, obtaining point cloud data corresponding to the area of interest.
[0054] In one example, point cloud data corresponding to the area of interest can be obtained through lidar scanning or autonomous exploration by a depth camera.
[0055] Step S104: generate grid data corresponding to the region of interest, and determine the attribute label corresponding to each cell in the grid data based on the point cloud data.
[0056] Among them, the attribute label is used to characterize the point cloud occupancy of the cell and the distance between cells. For example, the attribute label includes an occupancy label and a distance label; among them, the occupancy label is divided into an occupied label and an unoccupied label. The cell marked with the occupied label can be understood as the cell where the object in the area of interest is located, and the cell marked with the unoccupied label can be understood as the cell that the drone can pass through (that is, there is no object in the cell); the distance label can be only marked on the cell marked with the unoccupied label. The distance label will represent the shortest distance between the cell marked with the unoccupied label and the cell marked with the occupied label, which is equivalent to the distance when the drone is located in the cell to shoot the object.
[0057] In one example, the region of interest may be voxelized or gridded to generate grid data, and then a corresponding attribute label may be labeled for each cell based on the relative position relationship between the point cloud data and each cell in the grid data, and the relative position relationship between each cell.
[0058] Step S106, performing route planning according to the attribute tags to generate a target route corresponding to the area of interest.
[0059] In one example, based on the point cloud occupancy and the distance between cells, candidate cells that meet preset conditions can be screened out from the cells contained in the grid data. The preset condition may be that the cell is not occupied and the distance between the cell and the occupied cell meets the distance range. Then, path planning is performed for the screened candidate cells to obtain a target route that can be navigated by the drone and meets the shooting requirements.
[0060] Step S108, collecting image data corresponding to the area of interest based on the target route, and the image data is used to train the scene modeling model.
[0061] In one example, waypoints can be extracted from the target route and the shooting directions corresponding to each waypoint can be set, so that when the UAV is navigating along the target route, based on the waypoints and their corresponding shooting directions, the image acquisition device carried by the UAV can be controlled to collect image data in the form of photographs; in another example, when the UAV is navigating along the target route, the image acquisition device carried by the UAV can be controlled to collect image data in the form of video recording.
[0062] The above-mentioned image acquisition method provided by the embodiment of the present invention generates grid data corresponding to the area of interest and marks attribute labels for each cell in the grid data, so as to reflect the cells where objects are located in the area of interest and the cells that the drone can pass through, as well as the distance information between the cells where the objects are located and the cells that the drone can pass through through the point cloud occupancy represented by the attribute labels. On this basis, route planning is performed. If the drone can reach the area, the target route planned by the embodiment of the present invention can completely cover the objects in the area of interest, so that the collected image data can better meet the training requirements of the scene modeling model, thereby improving the modeling effect of the scene modeling model.
[0063] For ease of understanding, the embodiment of the present invention provides a specific implementation of an image acquisition method for a 3D Gaussian splash reconstruction algorithm. The method is an optimized training set acquisition method for a 3D Gaussian splash reconstruction method based on a point cloud of an area of interest. The method can achieve the following effects:
[0064] First, 3D Gaussian splashing relies on the training set to generate a sparse point cloud through motion structure recovery to initialize the position of the 3D Gaussian function. Therefore, a more accurate motion structure recovery result can obtain a better 3D Gaussian splashing model. A more accurate motion structure recovery result requires that the training set has uniform coverage of the area of interest and meets sufficient overlap. The embodiment of the present invention will ensure that the planned route can maintain a uniform and stable overlap rate.
[0065] Second, the 3D model generated by 3D Gaussian splashing cannot cover areas that are not covered by the training set. Existing methods for automatically planning routes usually have difficulty in fully covering objects and scenes in the area of interest. The route planned by the embodiment of the present invention can capture photos of the area of interest without blind spots (assuming that the drone can reach it).
[0066] Third, the existing data set collection method usually only collects photos near a certain fixed distance from the object of interest. When the 3D Gaussian splash model trained with this data set generates new views, it can only obtain new views with good effects near the above distance. When collecting data sets, the embodiments of the present invention collect photos at different distances if space permits. Although the training set is increased to a certain extent, the trained model can obtain new views with good effects in generating new views at different distances.
[0067] In specific implementation, the image acquisition method mainly includes the following steps 1 to 5:
[0068] Step 1: Obtain the point cloud data corresponding to the area of interest.
[0069] In one example, point cloud data corresponding to the area of interest can be obtained through lidar scanning or autonomous exploration by a depth camera.
[0070] Step 2: Generate grid data corresponding to the area of interest.
[0071] In one example, the three-dimensional space of the region of interest may be voxelized to generate voxel grid data. Optionally, the range and voxel size of the region of interest are adjustable parameters, and the specific values of the parameters may be set based on actual needs, which is not limited in the embodiment of the present invention.
[0072] Step 3, determining the attribute label corresponding to each cell in the grid data based on the point cloud data, including the following steps 3.1 to 3.2:
[0073] Step 3.1, based on the relative position relationship between each cell in the point cloud data and the grid data, annotate the cell with an occupancy label to divide the cell into occupied cells and unoccupied cells.
[0074] In specific implementation, it is possible to determine whether there are points in the cell in the point cloud data; if so, the cell is labeled with an occupied label to classify the cell as an occupied cell; if not, the cell is labeled with an unoccupied label to classify the cell as an unoccupied cell. For example, for a cell, if there is a point cloud in the cell, the cell is assigned an "occupied" label; conversely, if there is no point cloud in the cell, the cell is assigned an "unoccupied" label.
[0075] Step 3.2: Based on the relative position relationship between the occupied cells and the unoccupied cells, the distance labels are annotated for the unoccupied cells.
[0076] In a specific implementation, the shortest distance between the occupied cell and the unoccupied cell can be determined based on the relative position relationship between the occupied cell and the unoccupied cell, and the shortest distance can be used as a distance label to mark the unoccupied cell. In one example, each unoccupied cell is traversed, the distance between the unoccupied cell and each occupied cell is calculated, the shortest distance is used as a distance label, and the distance label is then assigned to the unoccupied cell.
[0077] Step 4: Plan the route according to the attribute labels and generate the target route corresponding to the area of interest, including the following steps 4.1 to 4.2:
[0078] Step 4.1, filter out candidate cells that meet preset conditions from cells based on attribute labels.
[0079] In a specific implementation, if a cell is classified as an unoccupied cell and the distance tag annotated with the cell satisfies the distance range, it is determined that the cell satisfies the preset condition and the cell is determined as a candidate cell. In one example, based on the distance tag annotated with the unoccupied cell, candidate cells within the distance range are screened out from the unoccupied cells.
[0080] Step 4.2, using the specified path planning algorithm, perform route planning for the candidate cells based on the preset route planning constraints to generate a target route corresponding to the area of interest.
[0081] Among them, the path planning algorithm can adopt A* algorithm, Dijkstra algorithm, BFS (Breadth-First Searching) algorithm, DFS (Depth-First Searching) algorithm, etc. For example, given the starting point and the end point, the A* path planning algorithm is used to plan a route with the shortest distance. There are many options for the implementation of this step. First, the choice of path planning algorithm is relatively flexible, and secondly, the construction of the cost function during planning is also relatively flexible, such as adding constraints such as increasing height as little as possible. The specific algorithm selection and the construction of the cost function depend on business needs. Sometimes it is necessary to maximize the overlap rate between photos as much as possible, and sometimes it is necessary to take efficiency into account. A very important point is that the algorithm will take pictures of objects of interest from different distances according to the input parameter settings to meet the diversity of observation distances in the 3D Gaussian splash training set.
[0082] Step 5: Collect image data corresponding to the area of interest based on the target route, mainly including the following methods 1 and 2:
[0083] Method 1: First, extract the target waypoint set from the target route, and determine the shooting direction corresponding to each waypoint in the target waypoint set; then control the drone to navigate along the target route, and perform shooting actions on the area of interest according to the shooting direction corresponding to each waypoint during the navigation process to obtain image data corresponding to the area of interest.
[0084] In one example, when extracting the target waypoint set, you can use fixed-point extraction and shooting (referred to as fixed-point shooting) or isochronous extraction and shooting (referred to as isochronous shooting). Fixed-point shooting is as follows: waypoints and shooting directions are evenly planned according to the overlap rate and the distance to the object being photographed; isochronous shooting is as follows: some key frames (i.e. waypoints) are automatically selected from the target route, and the shooting directions of these key frames are set, and the flight speed and shooting interval when the drone smoothly executes this target route are set.
[0085] Method 2: Control the drone to navigate along the target route, and perform video recording for the area of interest during the navigation process to obtain image data corresponding to the area of interest.
[0086] In one example, when a drone smoothly executes a target route, image data corresponding to the target area is collected in the form of video recording.
[0087] According to the above method 1 or method 2, the following can be obtained: Figure 2 A schematic diagram of a target waypoint set is shown, in which the cells (dots) marked with arrows on the outside are unoccupied cells at a certain distance from the surface of the object, the cells (dots) not marked with arrows in the central area are cells occupied by the object, and the arrows are used to represent the shooting direction.
[0088] In one example, the image data collected in the above steps 1 to 5 are used to construct a training set, and the 3D Gaussian splash model is further trained using the training set to obtain a 3D Gaussian splash model with better modeling effect.
[0089] In summary, the embodiments of the present invention have at least the following characteristics: (1) Uniform waypoints that meet the overlap ratio make the motion structure recovery result more accurate, and improve the accuracy of 3D Gaussian splash initialization. (2) All-round coverage of the scene or object of interest without blind spots that cannot be reconstructed. (3) The image data is distributed at all distances, which improves the effect of generating new views at different distances when using the 3D Gaussian splash model to generate new views.
[0090] Based on the above embodiments, the present invention also provides a 3D Gaussian splash modeling method, see Figure 3 The flowchart of a 3D Gaussian splash modeling method shown in FIG. 1 includes the following steps S302 to S304:
[0091] Step S302, obtaining image data corresponding to the region of interest; wherein the image data is acquired by using the image acquisition method provided in the aforementioned embodiment, and the embodiment of the present invention will not be described in detail here.
[0092] Step S304: construct a training data set based on the image data, and use the training data set to train a 3D Gaussian splash model. The trained 3D Gaussian splash model is used to model the region of interest.
[0093] The new training set acquisition strategy proposed by the aforementioned image acquisition method will rely on the UAV platform to optimize the technical problems existing in the existing technology, so that after training the 3D Gaussian splash model using the training data set constructed with image data, the potential of the 3D Gaussian splash model can be exploited as much as possible.
[0094] Based on the above embodiments, the present invention provides an image acquisition device. Figure 4The structure diagram of an image acquisition device shown in FIG. 1 mainly includes the following parts:
[0095] The data acquisition module 402 is used to acquire point cloud data corresponding to the area of interest;
[0096] The label determination module 404 is used to generate grid data corresponding to the region of interest, and determine the attribute label corresponding to each cell in the grid data based on the point cloud data; wherein the attribute label is used to characterize the point cloud occupancy of the cell and the distance between grids;
[0097] A route planning module 406 is used to plan a route according to the attribute tags and generate a target route corresponding to the area of interest;
[0098] The image acquisition module 408 is used to collect image data corresponding to the area of interest based on the target route, and the image data is used to train the scene modeling model.
[0099] The above-mentioned image acquisition device provided by the embodiment of the present invention generates grid data corresponding to the area of interest and marks attribute labels for each cell in the grid data, so as to reflect the cells where objects are located in the area of interest and the cells that the drone can pass through, as well as the distance information between the cells where the objects are located and the cells that the drone can pass through through the point cloud occupancy represented by the attribute labels. On this basis, route planning is performed. If the drone can reach the area, the target route planned by the embodiment of the present invention can completely cover the objects in the area of interest, so that the collected image data can better meet the training requirements of the scene modeling model, thereby improving the modeling effect of the scene modeling model.
[0100] In one implementation, the tag determination module 404 is further configured to:
[0101] Based on the relative position relationship between each cell in the point cloud data and the grid data, the occupancy labels are marked for the cells to divide the cells into occupied cells and unoccupied cells;
[0102] Based on the relative position relationship between occupied cells and unoccupied cells, distance labels are annotated for unoccupied cells.
[0103] In one implementation, the tag determination module 404 is further configured to:
[0104] Determine whether there is a point located in a cell in the point cloud data;
[0105] If yes, an occupied label is given to the cell to classify the cell as an occupied cell; if no, an unoccupied label is given to the cell to classify the cell as an unoccupied cell.
[0106] In one implementation, the tag determination module 404 is further configured to:
[0107] Based on the relative positional relationship between the occupied cells and the unoccupied cells, the shortest distance between the occupied cells and the unoccupied cells is determined, and the shortest distance is annotated to the unoccupied cells as a distance label.
[0108] In one embodiment, the route planning module 406 is further configured to:
[0109] Filter out candidate cells that meet preset conditions from cells according to attribute labels;
[0110] Using the specified path planning algorithm, route planning is performed for the candidate cells based on the preset route planning constraints to generate the target route corresponding to the area of interest.
[0111] In one embodiment, the route planning module 406 is further configured to:
[0112] If the cell is divided into an unoccupied cell and the distance label annotated with the cell satisfies the distance range, it is determined that the cell satisfies the preset condition and the cell is determined as a candidate cell.
[0113] In one implementation, the image acquisition module 408 is further configured to:
[0114] Extracting a target waypoint set from the target route, and determining a shooting direction corresponding to each waypoint in the target waypoint set;
[0115] Control the UAV to navigate along the target route, and perform shooting actions on the region of interest according to the shooting direction corresponding to each waypoint during the navigation process to obtain image data corresponding to the region of interest;
[0116] Alternatively, the drone is controlled to navigate along a target route, and a video recording action is performed on the region of interest during the navigation process to obtain image data corresponding to the region of interest.
[0117] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0118] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.
[0119] Figure 5A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected via the bus 52; the processor 50 is used to execute an executable module stored in the memory 51, such as a computer program.
[0120] The memory 51 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0121] The bus 52 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0122] Among them, the memory 51 is used to store programs, and the processor 50 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 50 or implemented by the processor 50.
[0123] The processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 50. The above processor 50 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51 and completes the steps of the above method in combination with its hardware.
[0124] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.
[0125] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0126] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. An image acquisition method, characterized in that: include: Get the point cloud data corresponding to the area of interest; Generate grid data corresponding to the region of interest, and determine the attribute label corresponding to each cell in the grid data based on the point cloud data; wherein the attribute label is used to characterize the point cloud occupancy of the cell and the distance between cells, and the attribute label includes an occupancy label and a distance label, and the occupancy label is divided into an occupied label and an unoccupied label. The cell marked with the occupied label is the cell where the object in the region of interest is located, and the cell marked with the unoccupied label is the cell that the drone can pass through. The cell marked with the unoccupied label is also marked with a distance label, and the distance label will characterize the shortest distance between the cell marked with the unoccupied label and the cell marked with the occupied label; Performing route planning according to the attribute tags to generate a target route corresponding to the area of interest; Collecting image data corresponding to the region of interest based on the target route, wherein the image data is used to train a scene modeling model; The step of determining the attribute label corresponding to each cell in the grid data based on the point cloud data includes: based on the relative position relationship between the point cloud data and each cell in the grid data, marking the cell with an occupancy label to divide the cell into occupied cells and unoccupied cells; based on the relative position relationship between the occupied cells and the unoccupied cells, marking the unoccupied cells with a distance label.
2. The image acquisition method according to claim 1, characterized in that: Based on the relative position relationship between the point cloud data and each cell in the grid data, the step of labeling the cell with a point cloud occupancy label to divide the cell into occupied cells and unoccupied cells includes: Determine whether there is a point located in the cell in the point cloud data; If yes, an occupied label is marked for the cell to classify the cell as an occupied cell; if no, an unoccupied label is marked for the cell to classify the cell as an unoccupied cell.
3. The image acquisition method according to claim 1, characterized in that: The step of marking distance labels for the unoccupied cells based on the relative positional relationship between the occupied cells and the unoccupied cells comprises: Based on the relative positional relationship between the occupied cell and the unoccupied cell, a shortest distance between the occupied cell and the unoccupied cell is determined, and the shortest distance is marked as a distance label to the unoccupied cell.
4. The image acquisition method according to claim 1, characterized in that: The step of performing route planning according to the attribute tags to generate a target route corresponding to the area of interest includes: Filtering candidate cells that meet preset conditions from the cells according to the attribute tags; A specified path planning algorithm is used to perform route planning for the candidate cells based on preset route planning constraints to generate a target route corresponding to the area of interest.
5. The image acquisition method according to claim 4, characterized in that: The step of selecting candidate cells that meet preset conditions from the cells according to the attribute tags includes: If the cell is divided into an unoccupied cell and the distance tag marked with the cell satisfies the distance range, it is determined that the cell meets the preset condition and the cell is determined as a candidate cell.
6. The image acquisition method according to claim 1, characterized in that: The step of collecting image data corresponding to the area of interest based on the target route includes: Extracting a target waypoint set from the target route, and determining a shooting direction corresponding to each waypoint in the target waypoint set; Controlling the drone to navigate along the target route, and performing a shooting action on the region of interest according to the shooting direction corresponding to each waypoint during the navigation process, so as to obtain image data corresponding to the region of interest; Alternatively, the drone is controlled to navigate along the target route, and a video recording action is performed on the region of interest during the navigation process to obtain image data corresponding to the region of interest.
7. An image acquisition device, characterized in that: include: A data acquisition module, used to acquire point cloud data corresponding to the area of interest; a label determination module, used to generate grid data corresponding to the region of interest, and determine the attribute label corresponding to each cell in the grid data based on the point cloud data; wherein the attribute label is used to characterize the point cloud occupancy of the cell and the distance between grids, the attribute label includes an occupancy label and a distance label, the occupancy label is divided into an occupied label and an unoccupied label, the cell marked with the occupied label is the cell where the object in the region of interest is located, the cell marked with the unoccupied label is the cell that the drone can pass through, and the cell marked with the unoccupied label is also marked with a distance label, and the distance label will characterize the shortest distance between the cell marked with the unoccupied label and the cell marked with the occupied label; A route planning module, used to perform route planning according to the attribute tags and generate a target route corresponding to the area of interest; An image acquisition module, used for acquiring image data corresponding to the region of interest based on the target route, wherein the image data is used for training a scene modeling model; The label determination module is specifically used to: based on the relative position relationship between the point cloud data and each cell in the grid data, label the cell with an occupancy label to divide the cell into occupied cells and unoccupied cells; based on the relative position relationship between the occupied cells and the unoccupied cells, label the unoccupied cells with a distance label.
8. A 3D Gaussian splash modeling method, characterized in that: include: Acquire image data corresponding to the region of interest; wherein the image data is acquired by using the image acquisition method according to any one of claims 1 to 6; A training data set is constructed based on the image data, and a 3D Gaussian splash model is trained using the training data set. The trained 3D Gaussian splash model is used to model the region of interest.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 6.
