Generation method for surface mesh, estimation method for normal vector, generation method for three-dimensional point cloud, measuring device, measuring system, and program

Abstract

To provide a generation method for a surface mesh that obtains a surface mesh reflecting a surface shape of an object more accurately, even when the number of acquired data by a laser sensor is small.SOLUTION: A generation method of a surface mesh includes: emitting laser while sequentially changing at least one parameter selected from the group consisting of a position and a direction of a laser sensor with respect to a space including a target object, and receiving reflected light if present; assuming that the laser is emitted from the laser sensor facing a reference direction at a reference position, and converting changes in the parameter into a scatter diagram of laser emission angles to generate a first scatter diagram; dividing the first scatter diagram into polygons to generate a first mesh; extracting points associated with received reflected light from among the points constituting the first scatter diagram; and extracting portions corresponding to the extracted points from the first mesh to be set as a surface mesh of the target object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a surface mesh generation method, a normal vector estimation method, a three-dimensional point cloud generation method, a measurement device, a measurement system, and a program. [Background technology]

[0002] As an application of LiDAR (Laser Imaging Detection and Ranging) technology, a measuring device is known in which a laser sensor is mounted on a vehicle, and while moving through a space containing an object, a laser is emitted toward the space, and the reflected light from the object is received, thereby generating a three-dimensional model including the surface shape of the object. An example of such a device is disclosed in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-7743 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, 3D models of objects have been generated primarily through the following procedure. First, a 3D point cloud, which is a collection of points (vertices) originating from the surface of the object and representing specific positions in space (X, Y, Z coordinates), is obtained from reflected light obtained by a laser sensor, etc. Next, lines and surfaces connecting the vertices are generated based on the 3D point cloud, and a surface mesh that approximately represents the surface shape of the object is obtained. The surface mesh is then applied to the 3D point cloud (and textures, etc., are further applied as necessary) to generate a 3D model. The above technology can be applied in various ways by mounting a laser sensor on a moving object, for example, by mounting a laser sensor on a car to collect data for autonomous driving, or by mounting a laser sensor on an air vehicle such as a drone to measure topography.

[0005] However, conventional methods for creating surface meshes based on 3D point clouds have had several problems. The premise is that, simply put, a 3D point cloud is a collection of vertices where pulsed emitted laser light is scattered (reflected) on the surface of an object and the reflected light is detected. If no reflected light is detected, the pulse information (sensor information) is deleted and is not reflected in the 3D point cloud. However, in the coordinate system of a 3D point cloud, locations where no vertices exist can include not only the aforementioned cases where "there is no object in the direction of laser emission," but also locations where "the laser is not emitted at that position in the first place (and therefore it is unclear whether an object exists)."

[0006] Next, a surface mesh is generally generated by connecting the vertices of the 3D point cloud with lines and surfaces based on the distances between the vertices. Note that the generation of lines and surfaces is often performed using a known algorithm. In this case, the locations where there is no sensor information, i.e., the locations where there are gaps in the point cloud, are locations where reflected light was not detected for some reason. If the laser was emitted and the target object (reflected light) was not detected, this location does not need to be connected with a line or surface (first case).

[0007] However, as mentioned above, the areas where no point cloud exists may also include areas where the laser is not emitted in the first place. Such areas tend to occur when the density of the 3D point cloud is low, i.e., when the measurement is simpler or performed faster (second case).

[0008] When a 3D point cloud is sparse, the distance between vertices inevitably tends to be large. Suppose that in a three-dimensional point cloud, the distance D1 between vertices due to the first case (for example, the object has a hole and the vertices are separated by a certain distance) may be greater than the distance D2 between vertices due to the second case (the vertices are separated due to the measurement interval). In this case, the first case takes priority, and adjacent vertices that should not be connected by lines or faces may end up being connected by lines or faces, which may result in an inaccurate surface mesh that does not reflect the surface shape of the object.

[0009] This problem is particularly pronounced when analysis is forced to be based on relatively "sparse" 3D point clouds, for example, on hardware with limited capabilities, such as on-site or real-time measurements, or in limited time. One known solution to this problem is to adjust the parameters of the surface mesh generation algorithm according to the density of the 3D point cloud, etc. However, parameter adjustment requires skilled techniques and a lot of trial and error, and is also problematic in that it is cumbersome.

[0010] The present disclosure solves at least one of the problems of the above-mentioned conventional techniques. Specifically, one of the problems to be solved is to provide a surface mesh generation method that obtains a surface mesh that more accurately reflects the surface shape of an object even when the number of data points acquired by a laser sensor is small. [Means for solving the problem]

[0011] One of the surface mesh generation methods disclosed herein is a surface mesh generation method that generates a surface mesh that reflects the surface shape of an object by emitting a laser from a laser sensor into a space that includes an object and receiving reflected light from the object with the laser sensor, the surface mesh generation method including: emitting the laser into the space while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor, and receiving the reflected light, if any; assuming that the laser is emitted from the laser sensor facing a reference direction at a reference position, converting the change in the parameter into a scatter plot of the laser emission angle to generate a first scatter plot; dividing the first scatter plot into polygons to generate a first mesh; extracting the points associated with the reception of the reflected light from the points that constitute the first scatter plot; and extracting portions of the first mesh that correspond to the extracted points to form the surface mesh of the object. [Effects of the Invention]

[0012] According to the present disclosure, at least one of the problems of the above-described conventional techniques can be solved. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram of a measurement system according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram of a situation in which the shape of an object is measured by a moving body equipped with a laser sensor. [Figure 3] FIG. 10 is an explanatory diagram of a first scatter plot. [Figure 4] FIG. 10 is an explanatory diagram of a first mesh. [Figure 5] FIG. 10 is an explanatory diagram of a state in which a mesh corresponding to a light receiving point is extracted from a first mesh. [Figure 6] FIG. 10 is a diagram showing an extracted mesh. [Figure 7] FIG. 1 is a flow diagram of an embodiment of a method for generating a surface mesh using a measurement system. [Figure 8] FIG. 10 is a block diagram of a measurement system according to a second embodiment. [Figure 9] FIG. 10 is an explanatory diagram of a generated three-dimensional model. [Figure 10] FIG. 1 is a flow diagram of a method for generating a three-dimensional model using a measurement system. [Figure 11] FIG. 10 is a block diagram of a measurement system according to a third embodiment. [Figure 12] FIG. 1 is a flow diagram of a method for estimating a normal vector using a measurement system. [Figure 13] FIG. 10 is a block diagram of a measurement system according to a fourth embodiment. [Figure 14] FIG. 10 is a diagram illustrating a specific example of a second scatter diagram. [Figure 15] FIG. 10 is a diagram showing the state in which the first mesh is superimposed on the second scatter plot. [Figure 16] FIG. 1 is a flow diagram of a method for obtaining a colored 3D point cloud using a measurement system. [Figure 17] An example of a colorized 3D point cloud. DETAILED DESCRIPTION OF THE INVENTION

[0014] A first surface mesh generation method of the present disclosure is a surface mesh generation method that generates a surface mesh that reflects the surface shape of the object by emitting a laser from a laser sensor into a space including an object and receiving reflected light from the object with the laser sensor, the surface mesh generation method including: emitting the laser into the space while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor, and receiving the reflected light, if any; assuming that the laser is emitted from the laser sensor facing a reference direction at a reference position, converting the change in the parameter into a scatter plot of the laser emission angle to generate a first scatter plot; dividing the first scatter plot into polygons to generate a first mesh; extracting the points associated with the reception of the reflected light from the points that constitute the first scatter plot; and extracting portions of the first mesh that correspond to the extracted points to form the surface mesh of the object.

[0015] One of the features of the above surface mesh generation method is that the surface mesh can be generated without going through a three-dimensional point cloud (although it may be generated through a three-dimensional point cloud). In this method, a surface mesh is generated based on a first scatter plot of the emission angle of a laser emitted from a reference position toward a reference direction.

[0016] A laser is emitted from a laser sensor, and light reflected by an object is received by the laser sensor. In this method, a parameter selected from the position and / or direction of the laser sensor is changed sequentially (e.g., changed for each pulse) to emit the laser. The first scatter plot is obtained by converting the change in this parameter into the emission angle of this laser, assuming that the laser is emitted from a laser sensor facing a reference direction at a reference position. In other words, all measurement data is converted by assuming that the data was obtained by emitting the laser while constantly moving the laser sensor, as if it was obtained by emitting the laser from a fixed single location.

[0017] At this time, the first scatter diagram includes all points where the laser is emitted. Regardless of whether or not reflected light is received, the first scatter diagram reflects the points. For example, if the parameter changes with each pulse of the laser pulse, the scatter plot will reflect both pulses that produce reflected light and pulses that do not produce reflected light as a single point (a point on the scatter plot).

[0018] Then, points associated with the reception of reflected light are extracted from the points constituting the first scatter plot. A point associated with the reception of reflected light means that reflected light was received from the object at that measurement timing (e.g., pulse), i.e., that point is a vertex that should be reflected in the 3D point cloud. Note that points not extracted in the first scatter plot typically refer to points where the laser was emitted but no reflected light was detected. Furthermore, points not extracted in the first scatter plot may also include points where reflected light was detected but it was determined that no reflected light was detected based on information such as the distance to the reflection point.

[0019] Cases in which it is determined that a reflection point has not been detected include, for example, the following. First, in cases where the position of an object in space is known in advance, reflection from a position significantly different from the predicted position of the object may be regarded as reflection from an object other than the object (for example, an object constituting the background), and reflected light may be regarded as not being detected. Second, there may be cases where reflection from the surface on which the object is placed (the ground) is unnecessary, in which case reflected light from a position at or below the height of the aircraft 10 may be regarded as not being detected.

[0020] This is clearly different from conventional 3D point clouds, in which no distinction is made between areas where no point cloud exists, where the laser was not emitted in the first place, and areas where the laser was emitted but no reflected light was present. Conventional surface mesh generation based on 3D point clouds does not distinguish whether gaps where no points exist are gaps caused by the absence of the object (surface) or gaps caused by sparse acquired data. As a result, even if gaps between vertices actually reflect the "hole" shape of the object, they may end up being filled with lines or faces when generating the surface mesh.

[0021] On the other hand, in this method, all of the points that make up the first scatter plot represent "locations where the laser was emitted." A first mesh is generated from these points in advance, and then a mesh within a predetermined range is extracted based on the points associated with the reflected light, resulting in a surface mesh that accurately reflects the shape of the target object. This is true even if the sensor information is sparse.

[0022] This method allows for the generation of accurate surface meshes without the need for special parameter settings for the mesh generation algorithm. Furthermore, because accurate surface meshes can be generated even when sensor information is sparse, the computational load is reduced and the required hardware configuration can be made lower spec. More accurate surface meshes can be generated than before, even using more compact and lightweight hardware. This makes on-site analysis easier, making it useful for infrastructure inspections, etc. It can also be used for autonomous driving of vehicles such as automobiles and drones.

[0023] The normal vector estimation method of the present disclosure is a normal vector estimation method that includes estimating a normal vector of a surface mesh based on the surface mesh obtained by the first surface mesh generation method.

[0024] In this specification, a normal vector refers to a surface normal and / or a vertex normal. Conventional methods for estimating normal vectors, for example, in the case of vertex normals, involve searching for a point to be calculated and a point group used to calculate that point using parameters such as distance information, and treating the surface normal of an approximate plane of the detected point group as the vertex normal. Such normal vector estimation methods require skilled techniques such as adjusting search parameters and a great deal of trial and error, making them cumbersome. Furthermore, if the positions of the searched point group are biased, the accuracy of normal vector estimation deteriorates.

[0025] On the other hand, the normal vector estimation method uses the surface mesh obtained by the surface mesh generation method (of the first embodiment) described above, calculates the surface normals of all faces that include the point to be calculated, and normalizes the sum of these surface normals to obtain the vertex normal. This method can avoid problems such as parameter adjustment and bias in the position of the point cloud, and can obtain accurate normal vectors.

[0026] The first three-dimensional point cloud generation method of the present disclosure is a three-dimensional point cloud generation method that emits a laser from a laser sensor into a space including an object, receives reflected light from the object, and combines it with a captured image of the space to generate a colored three-dimensional point cloud of the object, and the method includes: emitting the laser while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor with respect to the space, receiving the reflected light if any; assuming that the laser is emitted from the laser sensor at a reference position in a reference direction, and The method for generating a three-dimensional point cloud includes converting a change in a meter into a scatter plot of the laser emission angle to generate a first scatter plot, dividing the first scatter plot into polygons to generate a first mesh, converting a color image obtained by capturing an image of the space from the reference position toward the reference direction into a scatter plot of the light-receiving angle for each pixel to generate a second scatter plot, setting a color for each point constituting the first scatter plot based on the first mesh and the second scatter plot, and generating the colored three-dimensional point cloud by associating the points with the colors.

[0027] The above-mentioned method for generating a 3D point cloud is characterized in that, similar to the method for generating a surface mesh in the first embodiment, a first mesh is generated, and a second scatter plot is generated based on a color image, and a colored 3D point cloud is obtained by associating these.

[0028] A first measuring device of the present disclosure is a measuring device that emits a laser from a laser sensor into a space including an object and receives reflected light from the object, thereby obtaining a surface mesh that reflects the surface shape of the object, and includes: a memory; a processor; and a program stored in the memory and configured to be executable by the processor, the program including: instructions to emit the laser and receive reflected light, if any, while sequentially changing at least one parameter selected from the group consisting of the position and direction of the laser sensor; instructions to assume that the laser is emitted from the laser sensor in a reference position toward a reference direction, convert the change in the parameter into a scatter plot of the laser emission angle, and generate a first scatter plot; instructions to polygonally divide the first scatter plot to generate a first mesh, and extract the points associated with the reception of the reflected light from the points that make up the first scatter plot; and instructions to extract portions corresponding to the extracted points from the first mesh, thereby forming the surface mesh of the object.

[0029] A second measurement device of the present disclosure is the first measurement device, wherein the program includes an instruction to use the surface mesh and estimate a normal vector of the surface mesh.

[0030] A first measurement system of the present disclosure is a measurement system further comprising a first or second measurement device and a mobile body on which the laser sensor is mounted.

[0031] A first program of the present disclosure is a measurement device that emits a laser from a laser sensor into a space including an object, receives reflected light from the object, and combines the received light with a captured image of the space to obtain a colored three-dimensional point cloud of the object, the measurement device including a memory, a processor, and a program stored in the memory and configured to be executable by the processor, the program including instructions to emit the laser and receive reflected light, if any, while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor with respect to the space; The measurement device includes: an instruction to generate a first scatter plot by converting a change in the parameter into a scatter plot of the laser emission angle, assuming that the laser is emitted in a reference direction; an instruction to generate a first mesh by dividing the first scatter plot into polygons; an instruction to convert a color image obtained by capturing an image of the space from the reference position in the reference direction into a scatter plot of the light-receiving angle for each pixel, and generate a second scatter plot; an instruction to set a color for each point constituting the first scatter plot based on the first mesh and the second scatter plot; and an instruction to generate the colored three-dimensional point cloud by associating the points with the colors.

[0032] A second program of the present disclosure is a program that causes a computer to execute the following instructions: assume that the laser is emitted from the laser sensor in a reference position in a reference direction while sequentially changing at least one parameter selected from the group consisting of the position and direction of the laser sensor with respect to a space including an object, and based on data obtained by receiving any reflected light, convert the changes in the parameter into a scatter plot of the laser emission angle and generate a first scatter plot; divide the first scatter plot into polygons to generate a first mesh; extract the points associated with the reception of the reflected light from the points that make up the first scatter plot; and extract portions of the first mesh that correspond to the extracted points and use them as a surface mesh of the object.

[0033] A third program of the present disclosure is a program that causes a computer to execute the following instructions: assume that the laser is emitted from the laser sensor in a reference position in a reference direction while sequentially changing at least one parameter selected from the group consisting of the position and direction of the laser sensor with respect to a space including an object, and, based on data obtained by receiving any reflected light, convert the changes in the parameter into a scatter plot of the laser emission angle and generate a first scatter plot; divide the first scatter plot into polygons to generate a first mesh; convert a color image obtained by capturing an image of the space from the reference position in the reference direction into a scatter plot of the light reception angle for each pixel and generate a second scatter plot; and set a color for each point constituting the first scatter plot based on the first mesh and the second scatter plot.; and generate a colored three-dimensional point cloud by associating the points with the colors.

[0034] An embodiment of the measurement system and an embodiment of a method for generating a surface mesh using the measurement system will be described in detail below. The following description may be based on representative embodiments, but the present disclosure is not limited to such embodiments. In this specification, a numerical range expressed using "to" means a range that includes the numerical values ​​before and after "to" as the lower and upper limits.

[0035] The examples shown below are examples that embody the technical ideas of the present disclosure, and the technical ideas of the present disclosure do not limit the materials, shapes, structures, and arrangements of the components to the following embodiments. The drawings are schematic. Therefore, the relationship and ratio between thickness and planar dimensions may differ from the actual ones, and the relationship and ratio between dimensions may also differ between the drawings.

[0036] Example 1 Fig. 1 is a block diagram of a measurement system according to a first embodiment, and Fig. 2 is an explanatory diagram of a situation in which the shape of an object is measured by a moving body equipped with a laser sensor. The measurement system 100 is composed of a controller 40 and an aircraft 10, which is a moving body. As shown in FIG. 2, the aircraft 10 emits a laser toward the space while moving through a space including an object 200. The aircraft 10 then receives the light reflected from the object 200. Based on this data, the controller 40 generates a surface mesh that represents the surface shape of the object 200 (for example, reflecting the surface shape of the object 200 having a through-hole 200A).

[0037] The flying vehicle 10 includes a controller 12 , a position sensor 14 , an attitude sensor 16 , a camera 18 , an RF circuit 20 , a motor 22 , a propeller 24 , and a laser sensor 30 .

[0038] Each part of the aircraft 10 is controlled by a controller 12. The controller 12 is a computer equipped with a processor and memory (not shown). Specifically, a microcomputer board or the like can be used as the controller 12. The controller 12 controls each part of the aircraft 10 based on various commands received from the controller 40 via the RF circuit 20, or based on programs stored in advance in memory, etc. At this time, the motor 22 may be feedback-controlled based on data obtained from the position sensor 14 and the attitude sensor 16, which will be described later.

[0039] The position sensor 14 is a sensor for measuring the position coordinates of the aircraft 10 or a device including this sensor. The position sensor 14 is, for example, a position detection means such as a GNSS (Global Navigation Satellite System) sensor attached at an arbitrary position. The position coordinates of the aircraft 10 can be acquired by the position sensor 14. The measurement values ​​of the position sensor 14 are used for navigation control of the aircraft 10 and also for recording the position of the laser sensor 30.

[0040] There are no particular limitations on the method for converting the position coordinates of the aircraft 10 into the position of the laser sensor 30. For example, a method can be employed in which the relative positional relationship between the position coordinates acquired by the position sensor 14 and the laser sensor 30 is stored in advance in the memory of the aircraft 10, and the position coordinates of the aircraft 10 are converted into the position coordinates of the laser sensor 30 based on this relative positional relationship.

[0041] The attitude sensor 16 is a sensor for measuring the attitude of the aircraft 10 or a device including this sensor. The attitude of the aircraft 10 includes rotational movement in three orthogonal axes (yaw, pitch, and roll).

[0042] The attitude sensor 16 may be, for example, an inertial measurement unit mounted at any position on the air vehicle 10. The measurements of the attitude sensor 16 are used for navigation control of the air vehicle 10 and also for recording the direction in which the laser sensor 30 is pointing. There are no particular limitations on the method for converting the attitude of the aircraft 10 into the direction in which the laser sensor 30 is pointing. For example, the relative relationship between the attitude acquired by the attitude sensor 16 and the attitude of the laser sensor 30 may be stored in the memory of the aircraft 10. This allows the measurement value of the attitude sensor 16 to be converted into the direction in which the laser sensor 30 is pointing.

[0043] The camera 18 is disposed near the laser sensor 30 and captures an image of a space including an object to obtain a color image. The camera 18 is preferably disposed at approximately the same position as the laser sensor 30 and facing in approximately the same direction. In other words, the camera 18 is preferably disposed so as to be able to obtain an image of the same field of view as the laser sensor 30. In this case, the coordinates of the imaging position of the camera 18 can be converted into the coordinates of the laser sensor 30 based on the measurement values ​​of the position sensor 14.

[0044] Electromagnetic waves are transmitted and received by the RF circuitry 20. The RF circuitry 20 converts between electrical signals and electromagnetic waves and can communicate with the controller 40 via these electromagnetic waves. Specific examples of the RF circuitry 20 include an antenna system, an RF transceiver, an amplifier, a tuner, an oscillator, a digital signal processor, a CODEC chipset, and memory, and can further include known circuits for performing these functions. The data acquired by the position sensor 14 , the attitude sensor 16 , and the camera 18 is transmitted to the controller 40 via the RF circuit 20 .

[0045] The aircraft 10 is equipped with four propellers 24. The propellers 24 are independently controlled by motors 22 connected to them. The rotation speed and other parameters of each propeller 24 are controlled by a navigation control program pre-stored in a memory (not shown) provided in the controller 12, based on commands and other information received from the controller 40 via the RF circuit 20.

[0046] The laser sensor 30 is a sensor that can measure the distance to an object by irradiating the object with a laser and receiving the reflected light. Specifically, a LiDAR sensor can be used. The laser sensor 30 includes a laser emitter 32 that emits a laser in a predetermined direction, a reflected light receiver 34 that receives the reflected light, and a rotation mechanism 35 that changes the direction of the laser emitter 32 and the reflected light receiver 34 within a predetermined range.

[0047] The emission surface of the laser emitter 32 and the light receiving surface of the reflected light receiving unit 34 are parallel and aligned toward a predetermined direction. The laser emitter 32 emits, for example, infrared laser light. More specifically, a pulsed laser of approximately 532 nm, 905 nm, or 1550 nm may be emitted. When the laser emitted from the laser emitter 32 in a predetermined direction hits an object present in that direction, the reflected light is received by the reflected light receiving unit 34. The distance between the reflection point (the surface or vertex of the object) and the reflected light receiving unit 34 can be calculated based on the time it takes for the light to be emitted from the laser emitter 32 and received by the reflected light receiving unit 34.

[0048] The rotation mechanism 35 sequentially changes the laser emission angle by changing the angle between the laser emission unit 32 and the reflected light receiving unit 34. As a non-limiting example, the rotation mechanism 35 changes the laser emission angle in the range of -30° to +30° in the horizontal direction and -30° to +30° in the vertical direction based on a predetermined sequence.

[0049] According to the laser sensor 30 configured as described above, when the housing is at a specific position (reference position) and facing a specific direction (reference direction), the rotation mechanism 35 operates to emit a laser while changing the emission angle within a predetermined range, and the reflected light can be received.

[0050] The flying object 10 performs measurements using the laser sensor 30 while moving within a space including the target object 200. In Fig. 2, the flying object 10 is depicted as if it were orbiting the target object 200, but the laser does not have to be irradiated onto the target object, and may be irradiated (randomly) onto the space including the target object.

[0051] The aircraft 10 moves while laser measuring the distance between the reflected light receiving unit 34 and the target object 200. The position and attitude (direction) of the aircraft 10 (laser sensor 30) are acquired by a position sensor 14 and an attitude sensor 16 housed in the main body 36 of the aircraft 10. Distance measurement data at a certain measurement timing is associated with the position coordinates and attitude (direction) of the reflected light receiving unit 34 at that measurement timing. This makes it possible to determine the position coordinates of the reflection point in space from the distance measurement data. This provides three-dimensional information of the reflection point (vertex) in space, i.e., a three-dimensional point cloud.

[0052] 1, the controller 40 includes a processor 44 (e.g., a CPU (Central Processing Unit)), a touch panel 42, a memory controller 46, a memory 60, an I / O interface 48 (I / O: Input / Output), an I / O controller 52, an I / O subsystem 50, and an RF circuit (RF: Radio Frequency) 54. Each component communicates with each other via a communication bus or a signal line.

[0053] Memory 60 may include random access memory (RAM), magnetic disk storage devices, and non-volatile memory devices such as flash memory devices.

[0054] Memory 60 may also include remotely located storage devices. Such storage devices may be accessed via RF circuitry 54, an external port (not shown), a communications network, or a combination thereof. Communications networks may include the Internet, an intranet, a local area network (LAN), a wide local area network (WLAN), a storage area network (SAN), or the like. Access to memory 60 by components of controller 40, including processor 44, is controlled by memory controller 46. The processor 44, memory controller 46, etc. may be implemented on one chip, or may be implemented separately on multiple chips.

[0055] RF circuitry 54 transmits and receives electromagnetic waves. RF circuitry 54 converts between electrical signals and electromagnetic waves and may communicate with air vehicle 10 and, if desired, with communications networks and other communications devices via these electromagnetic waves. Examples of RF circuitry 54 include an antenna system, an RF transceiver, an amplifier, a tuner, an oscillator, a digital signal processor, a CODEC chipset, and memory, and may include other known circuits for performing these functions.

[0056] The I / O subsystem 50 includes an I / O interface 48 and an I / O controller 52. The I / O subsystem 50 transmits and receives electrical signals for controlling the touch panel 42.

[0057] The touch panel 42 provides both an output interface and an input interface between the controller 40 and the user 1. The touch panel 42 displays visual output to the user. This visual output may include text, images (still images), video (moving images), and combinations thereof. The input interface, output interface, and visual information output by the touch panel 42 provide a GUI.

[0058] The touch panel 42 accepts input from a user based on contact. The touch panel 42 forms a touch-sensitive surface that accepts user input. The touch panel 42 (and the I / O subsystem 50) (together with associated modules and / or instruction sets in memory 60) detects the contact (and any movement or interruption resulting from the contact) and translates the detected contact into information that interacts with graphical objects displayed on the touch panel 42. The unit of contact detected corresponds to a stylus pen, one or more fingers of a user, and the like.

[0059] The touch panel 42 may be of any type, such as a resistive type, a capacitive type, an infrared type, an optical type, or an electromagnetic induction type, with a resistive type or a capacitive type being preferred. Furthermore, the display technology may be a liquid crystal display technology, a light-emitting diode display technology, a plasma display technology, an electroluminescence display technology, a quantum dot display technology, or the like, without any particular restrictions. The user 1 can touch the touch panel 42 with a stylus pen and / or a finger, etc.

[0060] The memory 60 stores software components including instructions or collections thereof (instruction sets, hereinafter also referred to as "modules") that can be executed by the processor 44. The software components may include an operating system 60A, a communications module 60B, a navigation control module 60C, a sensor control module 60D, a UI module 60E, a data processing module 60F, and other applications 60L.

[0061] Operating system 60A includes various software components and / or drivers for controlling general system tasks such as memory control and power management, and is responsible for communication between the hardware and software components included in controller 40.

[0062] The communications module 60B includes various software components for controlling communications with the air vehicle 10 and processing received data via an external port (not shown) and / or the RF circuitry 54.

[0063] The navigation control module 60C includes software components for controlling the navigation of the aircraft 10. Specifically, it includes an instruction set for acquiring measurements from the position sensor 14 and the attitude sensor 16, and an instruction set for controlling the rotation speed of the motor 22 based on the acquired measurements. The navigation control module 60C may also include an instruction set for navigating the aircraft 10 so as to sequentially change at least one parameter selected from the group consisting of the position and direction of the laser sensor 30 within a space including an object. Note that, for example, "sequentially changing" may mean changing the parameter at regular intervals of pulses when a laser is irradiated in pulses. Note that the parameter may also be changed by direct manipulation by the user 1. In this case, the navigation control module 60C does not need to include an instruction for changing the parameter.

[0064] The sensor control module 60D includes software components for controlling the position sensor 14, attitude sensor 16, camera 18, and laser sensor 30 equipped on the aircraft 10. Specifically, it includes a set of instructions for causing the laser sensor 30 to emit a laser and receive the reflected light. The emitted laser may be a continuous wave or a pulsed wave. The sensor control module 60D obtains data from the laser sensor 30 regarding the timing of laser emission and the presence or absence of reflected light at that time.

[0065] The sensor control module 60D also obtains the position of the laser sensor 30 (converted from the position of the flying object 10) from the position sensor 14. The sensor control module 60D also obtains the attitude of the laser sensor from the attitude sensor 16.

[0066] The position (coordinates) and attitude (direction) of the laser sensor 30 at the timing of laser emission, the laser emission angle adjusted by the rotation mechanism 35, and the presence or absence of reflected light are each associated with each other and stored as a set of entries. These are stored at regular intervals (for example, for each laser pulse) and can be organized as, for example, table-like data including a plurality of entries.

[0067] The UI module 60E (UI: User Interface) includes various software components for controlling the overall user interface including the GUI of the controller 40.

[0068] The data processing module 60F includes software components for processing data acquired by each sensor of the air vehicle 10.

[0069] The data processing module 60F includes an instruction set for generating a surface mesh based on the data acquired by the sensor and applying the generated surface mesh to a 3D point cloud to generate a 3D model of the object. Specifically, the data processing module 60F includes an instruction set for generating a 3D point cloud reflecting the vertices of the object from the entries where reflected light is obtained. The data processing module 60F may also include an instruction set for cleaning, classifying, filtering, etc. the point cloud data, as needed.

[0070] The data processing module 60F also includes various subcomponents for generating a surface mesh, such as a first scatter plot generation module 60G, a first mesh generation module 60H, a point extraction module 60J, and a surface mesh generation module 60K.

[0071] The first scatter plot generation module 60G includes a software component for converting changes in at least one parameter selected from the group consisting of position and direction during laser emission into a scatter plot of emission angles, which may further include the laser emission angle.

[0072] When a laser is emitted at regular intervals from a moving aircraft 10, the position and direction of the laser sensor 30 and the laser emission angle due to the operation of the rotation mechanism 35 change successively at each laser emission timing (for example, each pulse).

[0073] The first scatter plot generation module 60G includes an instruction set for converting changes in these parameters (position, direction, and emission angle) into a scatter plot (first scatter plot) of the laser emission angle when emitted from the laser sensor 30 in a certain reference state. Note that the reference state refers to when the laser sensor 30 (its housing) is at a certain reference position (coordinates x, y, z) and facing a certain direction. This reference state is preferably selected to be the same position and direction as the position and direction at which the image was captured by the camera 18 (although it is not necessary to select the same position and direction). This makes it easier to combine the surface mesh with the image captured by the camera 18, which captures the same field of view as the laser sensor 30.

[0074] 3 is an explanatory diagram of the first scatter plot. The first scatter plot 210 is a two-dimensional scatter plot, with a horizontal axis 218 and a vertical axis 216 representing the emission angle (unit: rad) in mutually orthogonal directions. Typically, the horizontal axis 218 represents the emission angle in the horizontal direction (X-axis direction), and the vertical axis 216 represents the emission angle in the vertical direction (Z-axis direction).

[0075] Here, the center point 214 of the first scatter diagram 210 represents the reference state. Each of the multiple points 212 corresponds to a laser emission at a certain point in time. For example, each corresponds to one pulse of laser emission. Note that in the example of FIG. 3, the outline of the first scatter diagram is approximately circular, but the shape of the first scatter diagram is not limited to this. It may also be a polygonal shape such as a rectangle.

[0076] Point 212 on first scatter plot 210 corresponds to the emission of a laser beam, but is not related to the presence or absence of reflected light. That is, whether or not reflected light is detected as a result of the laser emission, it is reflected as point 212 on first scatter plot 210. All of the laser beams emitted for measurement are reflected on the first scatter plot.

[0077] The first scatter plot generation module 60G may further include instructions for various correction processes to be executed when converting the parameter changes into laser emission angles, such as correction performed using a 3D point cloud obtained by reflected light.

[0078] For example, when generating the first scatter plot in the first scatter plot generation module 60G, if a laser emission from which no reflected light is detected is considered to have a reflection point at an infinite distance, differences in the distance from the laser sensor 30 to the object 200 may cause an error in the resulting first scatter plot 210. This error is a systematic error, and may have a certain effect on the results obtained, mainly when measurements are taken of different objects under different conditions.

[0079] When such an error occurs, it can be appropriately corrected using the three-dimensional point cloud information (three-dimensional coordinates of the reflection points) obtained from the reflected light, i.e., the distance from the laser sensor 30 to the object 200. However, in many cases, the error can be ignored in practice. Furthermore, from the viewpoint of speeding up processing, it is preferable that the first scatter plot generation module 60G does not include a command for correction processing using the three-dimensional point cloud information. In this respect, it can be said that three-dimensional point cloud information is not essential for generating a first scatter plot by the first scatter plot generation module 60G.

[0080] The first mesh generation module 60H includes a software component for dividing the first scatter plot into polygons to generate a first mesh. A known algorithm such as Delaunay triangulation can be appropriately employed to generate the first mesh. The first mesh generation module 60H may include such a known algorithm.

[0081] 4 is an explanatory diagram of the first mesh. Note that the drawing is for explanatory purposes and is not intended to limit the mesh generation command set included in the first mesh generation module 60H to one derived from a specific algorithm.

[0082] The first mesh 219 is obtained by triangulating the first scatter plot 210, and the coordinate system is the same as that of the first scatter plot 210. That is, the vertical axis 216 represents the laser emission angle in the vertical direction, and the horizontal axis 218 represents the laser emission angle in the horizontal direction. In the first scatter plot 210, the points 212 are connected by lines or surfaces using a polygon division algorithm included in the first mesh generation module 60H, and a mesh is generated. As with the first scatter plot 210, all points 212 are used in generating the first mesh 219, regardless of whether or not reflected light is received.

[0083] The point extraction module 60J includes a software component for extracting points associated with the reception of reflected light from the points 212 that make up the first scatter plot 210.

[0084] The entries acquired by the sensor control module 60D include information on whether or not reflected light from the emitted laser beam has been received. This information on whether or not reflected light has been received may include information on whether or not reflected light has been received using information such as the distance to the reflection point, as described above (reflected light was received, but it was deemed not to have been received based on the distance to the reflection point, etc.). The point extraction module 60J refers to this information and extracts points 212 associated with the reception of reflected light (hereinafter simply referred to as "light-receiving points") from the points 212 constituting the first scatter plot 210.

[0085] Receiving reflected light means that the entries (ranging information, position information, direction information) are used to generate a 3D point cloud. For the point extraction module 60J to extract the point 212, at least the fact that "reflected light has been received" is essential. On the other hand, the point cloud information (e.g., coordinates, etc.) obtained by the reflected light is not essential. In other words, for the process of extracting the point 212 by the point extraction module 60J, a 3D point cloud (coordinates, etc.) is not essential.

[0086] The surface mesh generation module 60K includes a software component for extracting a portion corresponding to the extracted point (light-receiving point 222) from the first mesh 219 and forming it into a surface mesh of the object (obtaining a surface mesh).

[0087] 5 is an explanatory diagram showing a state in which a mesh 220 corresponding to a light receiving point 222 is extracted from a first mesh 219. The first mesh 219 is shown by a dashed line, and the extracted mesh 220 is shown by a solid line.

[0088] The extracted mesh 220 is a mesh formed by light-receiving points 222, and its lines and faces are common to the first mesh 219. Fig. 6 is a diagram showing the extracted mesh 220. This mesh becomes the surface mesh of the object.

[0089] Next, a detailed description will be given of an embodiment of a method for generating a surface mesh using the measurement system 100. Figure 7 is a flow diagram of an embodiment of a method for generating a surface mesh using the measurement system. First, in step S10, a laser is emitted from the laser sensor and reflected light, if any, is received while at least one parameter selected from the group consisting of the position and direction of the laser sensor is successively changed with respect to the space including the target object. At this time, the laser emission direction may also be changed as one of the parameters by the function of the rotation mechanism 35.

[0090] As already explained with reference to Fig. 2, the measurement system 100 emits a laser beam into a space including the target object 200 using the flying object 10 equipped with the laser sensor 30, etc. At this time, the parameters can be changed by moving the flying object 10.

[0091] Note that, as long as the laser can be emitted while changing the above parameters, the flying object 10 does not need to be used. In other words, the flying object 10 is not an essential component of the measurement system 100. For example, the laser sensor 30 may be mounted on a moving object other than the flying object 10. An example of such a moving object is a vehicle. Furthermore, the laser sensor 30 does not need to be mounted on a moving object and moved. The laser sensor 30 may be fixed at a certain point, and the laser may be emitted from that point while changing the emission direction using the function of the rotation mechanism 35. In this case, the measurement system does not need to include a moving object. Note that, when the laser sensor 30 is fixed, that position becomes the reference position described below, and the direction in which the laser sensor 30 (its housing) faces becomes the reference direction.

[0092] The controller 40 can also function independently as a measurement device. In the measurement system 100, the controller 40 controls the aircraft 10 and analyzes data obtained from each sensor of the aircraft 10. On the other hand, the controller 40 can also only analyze data input from an external device via the I / O interface 48 and the RF circuit 54. For example, if the aircraft 10 is independently controlled and only necessary data is input to the controller 40, the controller 40 functions independently as a measurement device capable of generating a surface mesh, etc. In this case, the controller 40 does not need to have functions related to control of the aircraft 10, such as the navigation control module 60C.

[0093] Next, in step S11, assuming that a laser is emitted from a laser sensor facing a reference direction at a reference position, the changes in the parameters are converted into a scatter plot of the laser emission angle to generate a first scatter plot. The first scatter plot is generated by the processor 44 executing an instruction set included in the first scatter plot generation module 60G. The generation method has already been described.

[0094] Next, in step S12, the first scatter diagram is divided into polygons to generate a first mesh. The first mesh is generated by executing an instruction set included in the first mesh generation module 60H by the processor 44. The generation method has already been described.

[0095] Next, in step S13, points associated with reception of reflected light are extracted from the points constituting the first scatter diagram. The points are extracted by the processor 44 executing an instruction set included in the point extraction module 60J. The extraction method has already been described.

[0096] Next, in step S14, portions corresponding to the extracted points (light-receiving points) are extracted from the first mesh to form a surface mesh of the object. The surface mesh is generated by the processor 44 executing an instruction set contained in the surface mesh generation module 60K. The generation method has already been described.

[0097] The above surface mesh generation method generates a surface mesh of an object without going through a 3D point cloud, which avoids the problem with the conventional technology, namely, "there is no distinction between a location where a point cloud (vertex) does not exist and a location where the laser was emitted but no reflected light (first case) and a location where the laser was not emitted at all (second case)."

[0098] The above problem has a significant impact on the quality (accuracy) of the generated surface mesh when the surface mesh is generated based on a 3D point cloud.

[0099] In a 3D point cloud, locations with no vertices are mainly locations where reflected light was not detected due to Case 1 or Case 2. However, when a surface mesh is generated based on a 3D point cloud using a known algorithm, the reasons for the absence of vertices in the locations are not generally taken into consideration, and the vertices are connected by lines or surfaces based on the distance between the vertices, etc.

[0100] In this case, although a location where no points exist based on the first case should not be connected, if the distance between the vertices in the second case is greater, the location may be mistakenly connected by a line or a surface as a combination of closer vertices. Specifically, the through-hole 200A of the object 200 may be connected (blocked) by a line or a surface, and the generated surface mesh may not accurately reflect the shape of the object 200. This tendency is more pronounced when the density of the 3D point cloud is low (when there are many gaps due to the second case).

[0101] In contrast, the above-described surface mesh generation method does not require the use of a three-dimensional point cloud to generate the surface mesh, and further, it involves the steps of generating a first mesh from a first scatter plot and then extracting a mesh based on the points where reflected light is detected, so the distinction between the first case and the second case is naturally reflected in the generated mesh.

[0102] In summary, compared to the conventional method in which a mesh is generated based on the vertices where reflected light is detected, this method differs in that a mesh is extracted from a mesh created based on the laser emission points based on the points (= vertices) where reflected light is detected. This achieves the desired effect.

[0103] Example 2 FIG. 8 is a block diagram of a measurement system according to a second embodiment. The measurement system 201 is a measurement system comprising an aircraft 10 and a controller 40, and differs from the measurement system 100 of Example 1 in the software components stored in memory 60. Other aspects are the same, including the functions of the aircraft 10. Therefore, a description of the same parts will be omitted, and the description will focus on the differences.

[0104] The memory 60 includes a 3D point cloud generation module 60M and a 3D model generation module 60N as subcomponents of the data processing module 60F.

[0105] The 3D point cloud generation module 60M includes a software component for obtaining a 3D point cloud of an object from information obtained by sensors mounted on the aircraft 10. Specifically, the 3D point cloud generation module 60M includes a command set for determining the position coordinates of the laser reflection point based on the ranging information obtained by the laser sensor 30, the position information of the laser sensor 30 obtained by the position sensor 14, the direction information of the laser sensor 30 obtained by the attitude sensor 16, and, if necessary, the laser emission angle, and generating 3D information of the reflection point in space, i.e., a 3D point cloud.

[0106] The 3D model generation module 60N includes software components for generating a 3D model of the object 200 based on the surface mesh generated by the surface mesh generation module 60K and the 3D point cloud generated by the 3D point cloud generation module 60M.

[0107] 9 is an explanatory diagram of the generated three-dimensional model 230. The three-dimensional model 230 is obtained by applying the mesh 220 (surface mesh) to the three-dimensional point group 232. When generating the three-dimensional model, texture data and the like may also be applied in addition to the above.

[0108] 10 is a flow diagram of a method for generating a three-dimensional model using measurement system 201. In this flow, steps S10 to S14 are similar to the flow of the surface mesh generation method in FIG. 7, and therefore description thereof will be omitted.

[0109] This method includes, in step S21, obtaining a three-dimensional point cloud based on data from the laser sensor 30, the position sensor 14, the attitude sensor 16, and, if necessary, the laser emission angle. Note that the order in which step S21 is performed within the overall flow is not limited to that of the flow in Fig. 10. That is, it does not have to be performed after step S14 or before step S22. Since the three-dimensional point cloud can be obtained after or in parallel with data acquisition by the laser sensor 30 or the like, it may be performed after or in parallel with step S10.

[0110] Next, in step S22, the surface mesh is applied to the 3D point cloud to obtain a 3D model of the object. In the above 3D model generation method, the method of Example 1 is used to generate the surface mesh, so the obtained 3D model is likely to be more accurate.

[0111] Example 3 FIG. 11 is a block diagram of a measurement system according to a third embodiment. The measurement system 300 is a measurement system comprising an aircraft 10 and a controller 40, and differs from the measurement system 100 of Example 1 in the software components stored in memory 60. Other aspects are the same, including the functions of the aircraft 10. Therefore, a description of the same parts will be omitted, and the description will focus on the differences.

[0112] The memory 60 includes a three-dimensional point cloud generation module 60M and a normal vector estimation module 60P as subcomponents of the data processing module 60F.

[0113] The normal vector estimation module 60P includes a software component for estimating normal vectors based on the surface mesh and the 3D point cloud. In this specification, normal vectors include vertex normals and surface normals. There are no particular limitations on the method for estimating surface normals and vertex normals, and any known method can be applied as appropriate.

[0114] Fig. 12 is a flow diagram of a method for estimating a normal vector using measurement system 300. In this flow, steps S10 to S14 are common to the flow of the surface mesh generation method in Fig. 7, and step S21 is common to the flow in Fig. 10, so a description thereof will be omitted.

[0115] This method includes, in step S31, estimating a normal vector based on a surface mesh and a 3D point cloud. In the above normal vector estimation method, the method of Example 1 is used to generate the surface mesh, so the obtained normal vector is likely to be more accurate.

[0116] Example 4 FIG. 13 is a block diagram of a measurement system according to a fourth embodiment. The measurement system 400 is a measurement system comprising an aircraft 10 and a controller 40, and differs from the measurement system 100 of Example 1 in the software components stored in memory 60. All other aspects are common, including the functions of the aircraft 10. Therefore, a description of the commonalities will be omitted, and the description will focus on the differences.

[0117] The memory 60 includes, as subcomponents of the data processing module 60F, a three-dimensional point cloud generation module 60M, a three-dimensional model generation module 60N, a second scatter plot generation module 70A, and a color setting module 70B.

[0118] The three-dimensional point cloud generation module 60M includes a software component for obtaining a three-dimensional point cloud of an object from information obtained by sensors mounted on the aircraft 10. Specifically, the three-dimensional point cloud generation module 60M includes an instruction set for determining the position coordinates of the laser reflection point based on the distance measurement information obtained by the laser sensor 30, the position information of the laser sensor 30 obtained by the position sensor 14, and the direction information of the laser sensor 30 obtained by the attitude sensor 16 (laser emission angle, if necessary), and for generating three-dimensional information of the reflection point in space, i.e., a three-dimensional point cloud.

[0119] One of the features of the measurement system 400 is that a colored three-dimensional point cloud is generated by the functions of the second scatter plot generation module 70A and the color setting module 70B.

[0120] The 3D model generation module 60N includes software components for generating a 3D model of the object 200 based on the surface mesh generated by the surface mesh generation module 60K and the 3D point cloud generated by the 3D point cloud generation module 60M.

[0121] The second scatter plot generation module 70A includes a software component for converting a color image acquired by a camera 18 that captures images of the same field of view as the laser sensor 30 into a scatter plot of the light receiving angle for each pixel, thereby obtaining the second scatter plot. The image used to generate the second scatter plot is the image captured from the center point 214 when the first scatter plot was generated. The center point 214 when the first scatter plot was generated indicates that the laser sensor 30 was at a reference position and facing a reference direction. At this time, the camera 18 captures images of the same field of view. In other words, the center of the second scatter plot is the same as the center point 214.

[0122] However, even if the same field of view is captured, there may be a discrepancy between the positions of the camera 18 and the laser sensor 30, the origins of their respective coordinate systems, etc. Therefore, when generating the second scatter plot, a correction process may be performed to match the coordinate systems of the camera 18 and the laser sensor 30. Specifically, a process may be performed to align the coordinate system of the laser sensor 30 or the camera 18 with the coordinate system of the other.

[0123] The second scatter diagram is obtained by unfolding the image around the center point 214 and converting each pixel into a scatter diagram of the light-receiving angle. Note that, in order to make it easier to convert the captured image into the second scatter diagram, a calibration process may be performed to correct distortion of the lens of the camera 18, etc.

[0124] The second scatter diagram shares a central point 214 with the first scatter diagram, with the horizontal axis representing the horizontal light-receiving angle (rad) and the vertical axis representing the vertical light-receiving angle (rad). FIG. 14 is a diagram showing a specific example of the second scatter diagram. As shown in the diagram, the horizontal axis represents the horizontal light-receiving angle (rad), and the vertical axis represents the vertical light-receiving angle (rad). The inset is a partially enlarged view. In the enlarged view, each pixel is represented as a colored dot. Note that although the image in FIG. 14 is monochrome, it is actually a color image.

[0125] The color setting module 70B includes a software component for setting a color for each point constituting the first scatter plot based on the first mesh and the second scatter plot.

[0126] The color setting module 70B includes a set of instructions for determining the intersections between polygons (typically triangles) included in the first mesh obtained by dividing the first scatter plot and the light receiving angles of each pixel in the second scatter plot, and for calculating the coordinates of the pixels, thereby setting the colors of the points that make up the polygons based on the colors of the pixels.

[0127] 15 is a diagram showing the first mesh superimposed on the second scatter plot. The inset is a partial enlargement. As can be seen in the enlarged view, the pixels of the second scatter plot are included within the surface that makes up the first mesh. If pixels having different colors are included in the plane, correction such as averaging may be performed according to the distance from the point.

[0128] Next, the procedure will be described. Fig. 16 is a flow diagram of a method for obtaining a colored 3D point cloud using measurement system 400. In this flow, steps S10 to S14 are the same as the flow of the surface mesh generation method in Fig. 7, and therefore their description will be omitted. This method includes, in step S41, converting a color image obtained by capturing an image of a space from a reference position toward a reference direction into a scatter plot of the light-receiving angle for each pixel to obtain a second scatter plot. The method for generating the second scatter plot has already been described.

[0129] Next, in step S42, a color is set for each point constituting the first scatter diagram based on the first mesh and the second scatter diagram. The color setting is performed by color setting module 70B, and the method for doing so has already been described.

[0130] Next, in step S43, the colorized 3D point cloud is obtained by associating the points with colors. The method for obtaining the 3D point cloud has already been described. In this method, a color is set for each point constituting the first scatter plot, and when the 3D point cloud generation module 60M generates a 3D point cloud based on this, a colored 3D point cloud is automatically obtained. Figure 17 is an example of a colored 3D point cloud. Note that although it is shown in monochrome in the drawing, it is actually colored.

[0131] The method may further include a step of obtaining a colored three-dimensional model by a three-dimensional model generation module 60N using the colored three-dimensional point cloud and the first mesh.

[0132] This method not only generates an accurate 3D point cloud based on a more accurate surface mesh, but also allows colorization with simple calculations. Because both the surface mesh generation and colorization require simpler calculations, they can be performed on computers with limited performance, making them suitable for on-site measurements, etc. [Explanation of symbols]

[0133] 10 Flying Objects 12 Controllers 14 Position Sensor 16 Attitude Sensor 18 Camera 20 RF circuit 22 Motor 24 propellers 30 Laser Sensor 32 Laser emission unit 34 Reflected light receiver 35 Rotation mechanism 36 Main Unit 40 Controller 42 Touch Panel 44 processors 46 Memory Controller 48 I / O interfaces 50 I / O Subsystem 52 I / O controller 54 RF circuit 60 memory 60A Operating System 60B Communication Module 60C Navigation Control Module 60D Sensor Control Module 60E UI Module 60F Data Processing Module 60G 1st Scatterplot Generation Module 60H First mesh generation module 60J Point Extraction Module 60K Surface Mesh Generation Module 60L Application 60M 3D point cloud generation module 60N 3D model generation module 60P Normal Vector Estimation Module 70A Second scatter plot generation module 70B Color Setting Module 100 Measurement System 200 objects 200A through hole 201 Measurement System 210 First Scatter Plot 212 points 214 Center point 216 Vertical Axis 218 Horizontal axis 219 1st mesh 220 mesh 222 light receiving points 230 3D models 232 3D point cloud 300 Measurement System 400 Measurement System

Claims

1. A method for generating a surface mesh, comprising: emitting a laser from a laser sensor into a space including an object; and receiving reflected light from the object by the laser sensor, thereby generating a surface mesh that reflects a surface shape of the object, emitting the laser while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor with respect to the space, and receiving the reflected light if any; Assuming that the laser is emitted from the laser sensor facing a reference direction at a reference position, converting the change in the parameter into a scatter diagram of the laser emission angle to generate a first scatter diagram; dividing the first scatter plot into polygons to generate a first mesh; extracting the points associated with reception of the reflected light from the points constituting the first scatter diagram; extracting a portion corresponding to the extracted points from the first mesh to form a surface mesh of the object.

2. A method for estimating a normal vector, comprising estimating a normal vector of the surface mesh based on the surface mesh obtained by the surface mesh generation method according to claim 1.

3. A method for generating a three-dimensional point cloud, comprising: emitting a laser from a laser sensor into a space including an object, receiving reflected light from the object, and combining the received light with a captured image of the space to generate a colored three-dimensional point cloud of the object; emitting the laser while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor with respect to the space, and receiving the reflected light if any; Assuming that the laser is emitted from the laser sensor at a reference position in a reference direction, converting the change in the parameter into a scatter diagram of the emission angle of the laser to generate a first scatter diagram; dividing the first scatter plot into polygons to generate a first mesh; converting a color image obtained by capturing an image of the space from the reference position toward the reference direction into a scatter diagram of light-receiving angles for each pixel to generate a second scatter diagram; setting a color for each point constituting the first scatter diagram based on the first mesh and the second scatter diagram; and generating a colored three-dimensional point cloud by associating the points with the colors.

4. A measuring device that emits a laser from a laser sensor into a space including an object and receives reflected light from the object, thereby obtaining a surface mesh that reflects the surface shape of the object, A memory, a processor, and a program stored in the memory and configured to be executable by the processor, The program a command to emit the laser and receive the reflected light, if any, while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor; an instruction to convert the change in the parameter into a scatter diagram of laser emission angles and generate a first scatter diagram, assuming that the laser is emitted from the laser sensor at a reference position in a reference direction; dividing the first scatter plot into polygons to generate a first mesh; an instruction to extract the points associated with reception of the reflected light from the points constituting the first scatter diagram; and an instruction to extract a portion corresponding to the extracted points from the first mesh to form a surface mesh of the object.

5. 5. The measurement device of claim 4, wherein the program includes instructions for using the surface mesh to estimate a normal vector of the surface mesh.

6. A measurement system further comprising: the measurement device according to claim 4 or 5; and a mobile object on which the laser sensor is mounted.

7. A measurement device that emits a laser from a laser sensor into a space including an object, receives reflected light from the object, and combines the received light with a captured image of the space to obtain a colored three-dimensional point cloud of the object, A memory, a processor, and a program stored in the memory and configured to be executable by the processor, The program a command to emit the laser and receive the reflected light, if any, while sequentially changing at least one parameter selected from the group consisting of a position and a direction of the laser sensor with respect to the space; an instruction to convert the change in the parameter into a scatter diagram of the laser emission angle, assuming that the laser is emitted from the laser sensor at a reference position in a reference direction, and to generate a first scatter diagram; A command to divide the first scatter plot into polygons to generate a first mesh; a command to convert a color image obtained by capturing an image of the space from the reference position toward the reference direction into a scatter diagram of light-receiving angles for each pixel, and to generate a second scatter diagram; a command to set a color for each point constituting the first scatter diagram based on the first mesh and the second scatter diagram; and instructions for generating a colorized three-dimensional point cloud by associating the points with the colors.

8. While sequentially changing at least one parameter selected from the group consisting of the position and direction of the laser sensor with respect to a space including the object, a laser is emitted from the laser sensor, and reflected light, if any, is received, and based on the obtained data, On the computer, an instruction to convert the change in the parameter into a scatter diagram of laser emission angles and generate a first scatter diagram, assuming that the laser is emitted from the laser sensor at a reference position in a reference direction; dividing the first scatter plot into polygons to generate a first mesh; an instruction to extract the points associated with reception of the reflected light from the points constituting the first scatter diagram; A command to extract a portion corresponding to the extracted points from the first mesh and set it as a surface mesh of the object; A program that executes the following.

9. While sequentially changing at least one parameter selected from the group consisting of the position and direction of the laser sensor with respect to a space including the object, a laser is emitted from the laser sensor, and reflected light, if any, is received, and based on the obtained data, On the computer, an instruction to convert the change in the parameter into a scatter diagram of the laser emission angle, assuming that the laser is emitted from the laser sensor at a reference position in a reference direction, and to generate a first scatter diagram; dividing the first scatter plot into polygons to generate a first mesh; a command to convert a color image obtained by capturing an image of the space from the reference position toward the reference direction into a scatter diagram of light-receiving angles for each pixel, and to generate a second scatter diagram; a command to set a color for each point constituting the first scatter diagram based on the first mesh and the second scatter diagram; and instructions for generating a colored three-dimensional point cloud by associating the points with the colors.

Images