Incremental point cloud-based viewpoint generation method and device, equipment and medium

Abstract

The invention discloses a viewpoint generation method and device based on incremental point cloud, equipment and a medium, and the method comprises the steps: obtaining point cloud data in a target region, and converting the point cloud data into raster data; performing down-sampling processing on detection points in the raster data to obtain second raster data; generating a triangular patch graph according to the second raster data; and generating a viewpoint according to the triangular patch graph. Compared with the prior art, the method has the following effects that the effective sparse point cloud is obtained by downsampling the detection points in the raster data, then the triangular patch diagram is generated according to the sparse point cloud, and finally, the viewpoint pose is directly calculated and generated according to the triangular patch diagram.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional environment reconstruction, and in particular to a viewpoint generation method, device, equipment and medium based on incremental point cloud. Background Art

[0002] In the application fields of mobile robots, such as object detection, environmental exploration, 3D reconstruction, etc., higher requirements are placed on the work efficiency of mobile robots. For the above task scenarios, what kind of posture to observe the target object (i.e., viewpoint posture) is a key issue that needs to be solved.

[0003] At present, the existing coping strategy is to use a sampling scheme to generate a large number of viewpoints and establish relevant viewpoint judgment indicators to evaluate the viewpoints. However, the above scheme requires preprocessing of a large amount of point cloud data. In the task of generating viewpoints online, this will lead to a decrease in the efficiency of viewpoint generation and affect the progress of the task. Summary of the invention

[0004] The purpose of this application is to provide a viewpoint generation method, device, equipment and medium based on incremental point cloud to solve the problem of reduced viewpoint generation efficiency in the task of scene requirements for online viewpoint generation in related technologies.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a viewpoint generation method based on incremental point cloud, comprising:

[0007] Acquire point cloud data within the target area, and convert the point cloud data into raster data;

[0008] Downsampling the detection points in the raster data to obtain second raster data;

[0009] Generate a triangular patch image according to the second raster data;

[0010] A viewpoint is generated according to the triangular patch graph.

[0011] Furthermore, the converting the point cloud data into raster data includes: converting the point cloud data into raster data based on projection or interpolation.

[0012] Furthermore, the downsampling of the detection points in the raster data to obtain the second raster data includes:

[0013] Determine whether there is a detection point in each grid;

[0014] If there are detection points, one detection point is selected to replace all detection points in the grid, and the remaining detection points are deleted;

[0015] After all grids have been judged, the second grid data is obtained.

[0016] Furthermore, the downsampling process is specifically calculated as follows:

[0017]

[0018] in, is the result of downsampling, is the detection point data in different dimensions, r μ is the resolution.

[0019] Further, generating a triangular patch image according to the second raster data includes:

[0020] The second raster data is converted into a triangular facet map by using geographic information software or programming.

[0021] Further, generating a viewpoint according to the triangular patch graph includes:

[0022] Each triangular face in the triangular face graph is screened for viewpoints in turn. If the viewpoint meets a preset condition, the viewpoint is a reasonable viewpoint.

[0023] Furthermore, the preset condition is specifically adopted in the following formula:

[0024] |V i -P A |=|V i -P B |=|V i -P C |

[0025]

[0026] IsFree(V i )=1

[0027] Among them, A, B, C are the three vertices of the triangle, V i , P A , P B , P C are the viewpoint and the positions of the three vertices in the triangle patch, IsFree(V i ) indicates that the area where the viewpoint is located is not occupied.

[0028] In a second aspect, the present application also provides a viewpoint generation device based on incremental point cloud, comprising:

[0029] An acquisition module, used for acquiring point cloud data in a target area and converting the point cloud data into raster data;

[0030] A downsampling processing module, used for performing downsampling processing on the detection points in the raster data to obtain second raster data;

[0031] A triangular patch generation module, used for generating a triangular patch image according to the second raster data;

[0032] A viewpoint generation module is used to generate viewpoints according to the triangular patch graph.

[0033] In a third aspect, the present application also provides a computer electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above-mentioned viewpoint generation methods based on incremental point clouds are implemented.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned viewpoint generation methods based on incremental point clouds are implemented.

[0035] The present application provides a method, device, equipment and medium for generating a viewpoint based on an incremental point cloud. The method obtains point cloud data in a target area and converts the point cloud data into raster data; performs downsampling processing on the detection points in the raster data to obtain second raster data; generates a triangular patch image based on the second raster data; and generates a viewpoint based on the triangular patch image. Compared with the prior art, the present application has the following effects: the present application first obtains an effective sparse point cloud by downsampling the detection points in the raster data, then generates a triangular patch image based on the sparse point cloud, and finally directly calculates and generates the viewpoint pose based on the triangular patch image. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a method for generating viewpoints based on incremental point clouds according to an embodiment of the present application;

[0037] Figure 2 is a comparative schematic diagram of point cloud data before and after downsampling in an embodiment of the present application;

[0038] Figure 3 is a schematic diagram of a global triangle facet in an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of a local triangular facet in an embodiment of the present application;

[0040] Figure 5 is a schematic diagram of viewpoint generation restriction conditions in an embodiment of the present application;

[0041] Figure 6 This is a schematic diagram of generating a single viewpoint in an embodiment of the present application;

[0042] Figure 7 is a schematic diagram of global viewpoint generation in an embodiment of the present application;

[0043] Figure 8 is a schematic diagram of local viewpoint generation in an embodiment of the present application;

[0044] Fig. 9 It is a structural schematic diagram of a viewpoint generation device based on incremental point cloud according to an embodiment of the present application;

[0045] Fig.10 It is a structural schematic diagram of a computer electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may also be a centered element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a centered element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0048] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0050] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms "a", "said" and "the" used in one or more embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0052] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when the time of".

[0053] At present, the existing mainstream viewpoint generation algorithms usually use sampling schemes to generate a large number of viewpoints and establish relevant viewpoint judgment indicators for evaluation. However, this process consumes a huge amount of computation in the preprocessing of point cloud information. In addition, it is necessary to establish a viewpoint evaluation function to solve the optimization problem. Therefore, this scheme is not applicable to online viewpoint generation scenes.

[0054] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes are not repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0055] Please refer to Figure 1 The present application provides a method for generating viewpoints based on incremental point clouds, which comprises at least the following steps:

[0056] S10, acquiring point cloud data in the target area, and converting the point cloud data into raster data.

[0057] Specifically, point cloud data can be obtained through various sensors (laser scanners and lidar, etc.). For example, a robot equipped with a lidar can obtain point cloud data along the way by continuously scanning the surrounding environment during movement.

[0058] It can be understood that point cloud data is a collection of vectors in a three-dimensional coordinate system. These vectors are usually represented in the form of points. Each point has a three-dimensional space coordinate (X, Y, Z), and sometimes contains other attribute information, such as color, normal vector, reflection intensity, etc.

[0059] It should be noted that after obtaining the point cloud data, it is necessary to convert the point cloud data into raster data to facilitate subsequent processing.

[0060] In a certain embodiment of the present application, the point cloud data is converted into raster data by projection or interpolation.

[0061] S20, performing downsampling processing on the detection points in the raster data to obtain second raster data.

[0062] It is understandable that after the point cloud data is converted into raster data, there are generally multiple detection points (point clouds) in each raster. In this embodiment, in order to reduce the point cloud data to be processed without losing valid data, a raster downsampling method is adopted, that is, first check whether there is a detection point in each raster. If there is a detection point, a detection point is selected to replace all the detection points in the raster. Usually, this sampling point can be the average value (center of mass) of the coordinates of all points in the voxel, or it can be the center point or the point closest to the center point.

[0063] In one embodiment of the present application, step S20 includes: determining whether there are detection points in each grid; if there are detection points, selecting a detection point to replace all detection points in the grid, and deleting the remaining detection points; after all grids are determined, obtaining second grid data.

[0064] Specifically, assume that the point cloud data in the downsampled grid is Depend on Calculated. Where μ represents the dimension, assuming the map resolution is r μ , define the minimum value in each dimension as mapmin μ , then we have the following projection formula:

[0065]

[0066] in, is the result of downsampling, is the detection point data in different dimensions, r μ is the resolution.

[0067] For example, see Figure 2 , Figure 2 The left area in the middle is the point cloud data before downsampling. Figure 2 The right side shows the point cloud data after downsampling.

[0068] S30: Generate a triangular patch map according to the second raster data.

[0069] See also Figure 3 and Figure 4 In one embodiment of the present invention, geographic information software or programming is used to generate a triangular facet map from the second raster data.

[0070] For example, the geographic information software ArcGIS or the open source GIS software QGIS can be used to generate a triangular facet map from the second raster data.

[0071] It should be noted that ArcGIS has powerful spatial analysis and data processing capabilities. Through the "Create TIN" tool in the "3DAnalyst" toolset, raster data can be used as input, and relevant parameters such as elevation fields can be set to generate irregular triangulated networks (TINs), which are essentially composed of triangular facets. The generated TIN can also be further edited, visualized, and analyzed, such as adding textures and performing lighting rendering.

[0072] QGIS is an open source GIS software. Use the "GRASS" algorithm or "SAGA" algorithm in the "Processing Toolbox" to convert raster data into triangular patch maps. For example, through the "v.surf.rst" module of GRASS, a triangulated network can be generated based on the elevation values ​​of raster data, and the generated triangulated network can be simplified, smoothed, and other processing can be performed.

[0073] S40: Generate a viewpoint according to the triangular patch graph.

[0074] Specifically, see Figure 5 In this embodiment, a viewpoint generation method based on triangular facets is proposed. Here, the following triangle is assumed to be one of the triangular facets ( Figure 5 (a)); let the circumcenter of the triangle be point D ( Figure 5(b) part), the purpose is to make the distance between the observation point and each vertex equal; draw a perpendicular line OD through D ( Figure 5 (c) Part), at this time, sampling is performed on the vertical line at a set safety distance, and the direction of the observation point is determined by judging the occupancy of the Sanger map ( Figure 5 Part (d)), through the above conditions, the viewpoints are comprehensively generated.

[0075] In one embodiment of the present application, the step S40 includes: screening the viewpoint of each triangular face in the triangular face graph in turn, and if the viewpoint meets the preset conditions, the viewpoint is a reasonable viewpoint.

[0076] In a specific embodiment of the present application, the preset condition is specifically adopted by the following formula:

[0077] |V i -P A |=|V i -P B |=|V i -P C |

[0078]

[0079] IsFree(V i )=1

[0080] Among them, A, B, C are the three vertices of the triangle, V i , P A , P B , P C are the viewpoint and the positions of the three vertices in the triangle patch, IsFree(V i ) indicates that the area where the viewpoint is located is not occupied.

[0081] See also Figure 6-Figure 8 , Figure 6 A schematic diagram generated for a single viewpoint, Figure 7 Schematic diagram generated for the global viewpoint, Figure 8 Schematic diagram of generating local viewpoints.

[0082] The present application provides a method for generating a viewpoint based on an incremental point cloud, which obtains point cloud data in a target area and converts the point cloud data into raster data; performs downsampling processing on the detection points in the raster data to obtain second raster data; generates a triangular patch image based on the second raster data; and generates a viewpoint based on the triangular patch image. Compared with the prior art, the present application has the following effects: the present application first obtains an effective sparse point cloud by downsampling the detection points in the raster data, then generates a triangular patch image based on the sparse point cloud, and finally directly calculates based on the triangular patch image to generate the viewpoint pose.

[0083] See also Fig. 9 , a viewpoint generation device 200 based on incremental point cloud, comprising:

[0084] An acquisition module 201 is used to acquire point cloud data in a target area and convert the point cloud data into raster data;

[0085] A downsampling processing module 202 is used to perform downsampling processing on the detection points in the raster data to obtain second raster data;

[0086] A triangular patch generation module 203, used to generate a triangular patch image according to the second raster data;

[0087] The viewpoint generation module 204 is used to generate a viewpoint according to the triangular patch image.

[0088] See also Fig.10 The embodiment of the present application also provides a computer electronic device 300, including a memory 303 and a processor 302, wherein the memory 303 stores a computer program, and when the processor executes the computer program, the steps of a viewpoint generation method based on incremental point cloud described in any one of the above items are implemented.

[0089] Specifically, the electronic device 300 includes: a transceiver 301, a bus interface and a processor 302, wherein the processor 302 is used to obtain point cloud data in a target area and convert the point cloud data into raster data; downsample the detection points in the raster data to obtain second raster data; generate a triangular facet map based on the second raster data; and generate a viewpoint based on the triangular facet map.

[0090] In the embodiment of the present application, the electronic device 300 further includes: a memory 303. Fig.10In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 302 and memory represented by memory 303. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 301 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 302 is responsible for managing the bus architecture and general processing, and the memory 303 may store data used by the processor 302 when performing operations.

[0091] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any one of the above-mentioned viewpoint generation methods based on incremental point clouds are implemented.

[0092] In this embodiment, the computer readable storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0093] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0094] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0095] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0096] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0097] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0098] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A viewpoint generation method based on incremental point cloud, characterized in that: include: Acquire point cloud data within the target area, and convert the point cloud data into raster data; Downsampling the detection points in the raster data to obtain second raster data; Generate a triangular patch image according to the second raster data; A viewpoint is generated according to the triangular patch graph.

2. The viewpoint generation method according to claim 1, characterized in that: The step of converting the point cloud data into raster data comprises: The point cloud data is converted into raster data based on projection or interpolation.

3. The viewpoint generation method according to claim 1, characterized in that: The downsampling of the detection points in the raster data to obtain second raster data includes: Determine whether there is a detection point in each grid; If there are detection points, one detection point is selected to replace all detection points in the grid, and the remaining detection points are deleted; After all grids have been judged, the second grid data is obtained.

4. The viewpoint generation method according to claim 3, characterized in that: The specific calculation formula for the downsampling process is as follows: in, is the result of downsampling, is the detection point data in different dimensions, r μ is the resolution.

5. The viewpoint generation method according to claim 1, characterized in that: Generating a triangular patch image according to the second raster data includes: The second raster data is converted into a triangular facet map by using geographic information software or programming.

6. The viewpoint generation method according to claim 1, characterized in that: Generating a viewpoint according to the triangular patch graph includes: Each triangular face in the triangular face graph is screened for viewpoints in turn. If the viewpoint meets a preset condition, the viewpoint is a reasonable viewpoint.

7. The viewpoint generation method according to claim 6, characterized in that: The preset condition is specifically formulated as follows: |In i -P A |=|V i -P B |=|V i -P C | IsFree(V i )=1 Among them, A, B, C are the three vertices of the triangle, V i , P A , P B , P C are the viewpoint and the positions of the three vertices in the triangle patch, IsFree(V i ) indicates that the area where the viewpoint is located is not occupied.

8. A viewpoint generation device based on incremental point cloud, characterized in that: include: An acquisition module, used for acquiring point cloud data in a target area and converting the point cloud data into raster data; A downsampling processing module, used for performing downsampling processing on the detection points in the raster data to obtain second raster data; A triangular patch generation module, used for generating a triangular patch image according to the second raster data; A viewpoint generation module is used to generate viewpoints according to the triangular patch graph.

9. A computer electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the viewpoint generation method based on incremental point cloud described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the viewpoint generation method based on incremental point cloud described in any one of claims 1 to 7 are implemented.

Images