Processing unit

Abstract

This provides technology to assist in adjusting the attitude and position of measuring devices. [Solution] The processing device comprises an acquisition unit that acquires point cloud data representing a point cloud of multiple objects located within a predetermined range from a measuring device, and a display control unit that causes a display unit to display a screen showing the point cloud represented by the point cloud data, wherein the screen includes a first image showing the point cloud as viewed from a first viewpoint, and a second image showing the point cloud as viewed from a second viewpoint different from the first viewpoint.

Description

[Technical Field]

[0001] The technology disclosed in the present specification relates to a processing apparatus that processes point cloud data. [Background Art]

[0002] Patent Document 1 discloses an object detection apparatus that detects an image representing an object from a distance image generated by a distance measuring apparatus. The object detection apparatus displays the detected image. [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Publication No. 2017-219385 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] There are cases where the posture and position of a measuring device are adjusted while viewing an image showing measurement results obtained by the measuring device. The present specification provides a technology for assisting adjustment of the posture and position of a measuring device. [Means for Solving the Problem]

[0005] The present specification discloses one aspect of a processing apparatus that processes point cloud data. The processing apparatus includes: an acquisition unit that acquires point cloud data representing a point cloud of a plurality of objects existing within a predetermined range from a measuring device; and a display control unit that causes a display unit to display a screen that displays the point cloud represented by the point cloud data, wherein the screen includes a first image showing the point cloud viewed from a first viewpoint, and a second image showing the point cloud viewed from a second viewpoint different from the first viewpoint.

[0006] With the above configuration, the measurement results from the measuring device can be viewed from multiple perspectives. Compared to a configuration where the measurement results from the measuring device are viewed from only one perspective, the situation in which the measuring device is installed can be quickly grasped. This helps to adjust the attitude and position of the measuring device.

[0007] The first viewpoint is a viewpoint that provides an overview of the multiple objects, and the second viewpoint may be a viewpoint that views the multiple objects from a direction perpendicular to the vertical line extending from the first viewpoint.

[0008] Each point represented by the point cloud data is represented in a Cartesian coordinate system, and the first image may be generated by convolving each point of the point cloud data along a first axis in the Cartesian coordinate system, and the second image may be generated by convolving each point of the point cloud data along a second axis in the Cartesian coordinate system which is orthogonal to the first axis.

[0009] This specification discloses other embodiments of a processing apparatus for processing point cloud data. The processing apparatus comprises an acquisition unit that acquires point cloud data representing a plurality of objects located within a predetermined range from a measuring device, and a display control unit that causes a display unit to display a screen showing the point cloud represented by the point cloud data, wherein the screen includes a point cloud image representing the point cloud and an information image showing additional information, the information image including at least one of the edges of the point cloud, a grid line defined along the height direction of the predetermined range and a grid line defined along the horizontal direction of the predetermined range.

[0010] With the above configuration, users can grasp not only the measurement results of the measuring device but also additional information. Compared to a configuration where additional information is not displayed, users can quickly understand the situation in which the measuring device is installed. This helps in adjusting the attitude and position of the measuring device.

[0011] This specification discloses other embodiments of a processing apparatus for processing point cloud data. The processing apparatus comprises an acquisition unit that acquires point cloud data representing a plurality of objects within a predetermined range from a measuring device, and a display control unit that causes a display unit to display a screen showing a specific point cloud obtained by excluding points that satisfy predetermined conditions from the point cloud represented by the point cloud data.

[0012] The measurement results from the measuring device may include points that hinder understanding the conditions in which the measuring device is installed. The specific conditions described above are set as conditions for identifying these hindering points. Compared to a configuration in which points that satisfy the predetermined conditions are not excluded, the conditions in which the measuring device is installed can be quickly understood. This helps in adjusting the attitude and position of the measuring device.

[0013] The point that satisfies the aforementioned predetermined conditions may be a point measured within at least one of the ranges of the predetermined range that is less than or equal to the first height, and the range that is greater than or equal to the second height.

[0014] Points that could cause problems include, for example, points where the ground was measured or points where the ceiling was measured. Here, the range below the first height represents the ground, and the range above the second height represents the ceiling. With the above configuration, it is possible to display a specific point cloud excluding points where the ground was measured or points where the ceiling was measured.

[0015] Furthermore, the computer program for the above-mentioned processing device, the storage medium for storing the computer program, and the control method for controlling the above-mentioned processing device are also novel and useful. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram of the measurement system. [Figure 2] This is a flowchart illustrating the processing steps of a terminal device. [Figure 3] This is a schematic diagram of the screen displayed on the terminal device. [Figure 4]This is a schematic diagram of the screen displayed on the terminal device. [Modes for carrying out the invention]

[0017] (Configuration of measurement system 2; Figure 1) The measurement system 2 measures the position of objects within a predetermined space. The objects consist of moving objects 4 and stationary objects 6. Moving objects 4 are, for example, people. Stationary objects 6 are, for example, walls, shelves, desks, ceilings, floors 8, etc. The predetermined space is, for example, an indoor space, an outdoor space, etc. Indoor spaces are, for example, offices, factories, shopping malls, exhibition halls, etc. Outdoor spaces are, for example, the area outside a train station, a building entrance, etc. Note that XYZ coordinates are defined in Figure 1. The following explanation will use XYZ coordinates as appropriate.

[0018] The measurement system 2 comprises a terminal device 10 and a LiDAR 100. The terminal device 10 is a desktop PC, laptop PC, smartphone, etc. The LiDAR 100 is a 3D LiDAR that measures the distance to surrounding objects in three dimensions, a technique called light detection and ranging. The terminal device 10 and the LiDAR 100 are connected to each other via wired or wireless connections so that they can communicate with each other. In a modified example, the measurement system 2 may be a single device, and the functions of the terminal device 10 and the LiDAR 100 may be integrated into that single device.

[0019] The LiDAR 100 is configured to repeatedly scan a laser over a range R around the LiDAR 100, and output the distance to the reflection point of the laser on an object within the range R as point cloud data. The laser is emitted by a laser light source built into the LiDAR 100. The laser light source rotates, for example, using a rotation mechanism such as a motor. Scanning over the range R is achieved by rotating the laser light source by a predetermined angle in the XY plane (for example, 180 degrees) and by a predetermined upper and lower angle in the Z direction (for example, 120 degrees). For example, the laser light source rotates at a rotation speed of 5 rotations per second. Note that in a modification, the LiDAR 100 may scan the laser by a method that does not use a rotation mechanism, which is called a solid-state type.

[0020] Point cloud data is a collection of coordinate points of orthogonal coordinates (X, Y, Z) defined within the range R. Note that in a modification, the point cloud data may be a collection of coordinate points of polar coordinates defined within the range R.

[0021] In addition, the point cloud data includes the intensity (for example, reflectance) of reflected light from the reflection point corresponding to each of the plurality of coordinate points.

[0022] (Configuration of Terminal Device 10; Figure 1) The terminal device 10 includes a display unit 12, an operation unit 14, a communication interface 16, and a control unit 30. Hereinafter, "interface" is referred to as "I / F".

[0023] The display unit 12 is a display for displaying various information. The operation unit 14 includes a plurality of keys for receiving instructions from a user. The operation unit 14 is, for example, a mouse, a keyboard, or the like. Note that the display unit 12 may also function as a touch screen (that is, an operation unit). The communication I / F 16 is an I / F for performing communication with the LiDAR 100.

[0024] The control unit 30 comprises a CPU 32 and a memory 34. The memory 34 consists of volatile memory and non-volatile memory. The CPU 32 executes various processes according to programs 40 and 42 stored in the memory 34. The OS program 40 is a program that controls the basic operation of the terminal device 10.

[0025] Application program 42 (hereinafter referred to as "App 42") is a program for processing point cloud data received from LiDAR 100. App 42 is provided, for example, by the provider of the measurement system 2.

[0026] (Processing by terminal device 10; Figures 2, 3, and 4) Figure 2 is a flowchart of the process that the CPU 32 executes according to the application 42. The process in Figure 2 is executed in response to user instructions input into the operation unit 14.

[0027] In S10, CPU32 begins repeatedly acquiring point cloud data from LiDAR100.

[0028] In S12, the CPU 32 displays the main screen SC1, which shows the point cloud data acquired from the LiDAR 100, on the display unit 12. As shown in Figure 3, the main screen SC1 includes a point cloud image F1, which is an image of the point cloud data. The point cloud image F1 includes a point cloud image representing a stationary object 6 and a point cloud image representing a moving object 4. Note that although a point cloud image is a collection of multiple points, in this application, the point cloud image is schematically depicted with solid lines.

[0029] In S14, the CPU 32 determines whether or not a command to add a viewpoint has been input from the user to the operation unit 14. The command to add a viewpoint is a command to display a multi-screen SC2 that displays multiple images corresponding to multiple viewpoints. As shown in Figure 4, the multi-screen SC2 includes a point cloud image F1, an overhead view image F2, and a front view image F3. The overhead view image F2 and the front view image F3 are images generated from point cloud data. The overhead view image F2 is a viewpoint that surveys objects within range R, i.e., a point cloud image viewed from the Z-axis direction. The front view image F3 is a viewpoint that looks at objects within range R facing the front of the LiDAR 100, i.e., a point cloud image viewed from the X-axis direction or the Y-axis direction.

[0030] If the CPU 32 determines that the user has input a command to add a viewpoint to the control unit 14 (YES in S14), it proceeds to S20. In S20, the CPU 32 executes the process of generating an overhead view image F2 and a front view image F3 from the point cloud data.

[0031] For example, the overhead image F2 is generated by the following procedure. First, the CPU32 sorts all points indicated by the point cloud data in descending order of the distance calculated from the XY coordinate points other than the Z coordinate point. The CPU32 prepares an image with a predetermined number of pixels. The position of each pixel corresponds to the XY coordinates. Each pixel in the prepared image is initially assigned the value "0". The CPU32 inputs all sorted points as pixel values ​​into the image. If a pixel has multiple pixel values ​​assigned to it, the CPU32 adopts the smallest value, i.e., the value with the smallest distance. Here, the pixel value is, for example, a fixed value "1". When the pixel value is a fixed value "1", the overhead image F2 is a binary image. In other examples, the pixel value is a normalized value of the distance calculated from the XY coordinate points. The normalized value is, for example, a value within the range of 0 to 255. In other examples, the pixel value is a normalized value of the intensity of the reflected light of the corresponding point. If the pixel values ​​are normalized, the overhead image F2 is an image represented in grayscale or as a color gradient. Following the procedure described above, the CPU32 convolves the point cloud data along the Z-axis to generate the overhead image F2, which is an image viewed from the Z-axis direction.

[0032] Furthermore, the front view image F3, viewed from the X-axis direction, is generated by convolving the point cloud data along the X-axis direction using the procedure described for the overhead view image F2. Similarly, the front view image F3, viewed from the Y-axis direction, is generated by convolving the point cloud data along the Y-axis direction using the procedure described for the overhead view image F2.

[0033] In S22, the CPU 32 displays the multiple screen SC2 on the display unit 12 instead of the main screen SC1. This configuration allows the measurement results from the LiDAR 100 to be viewed from multiple viewpoints. Compared to the main screen SC1, the situation in which the LiDAR 100 is installed can be quickly grasped. This helps in adjusting the attitude and position of the LiDAR 100. When S22 ends, the CPU 32 returns to S14.

[0034] Furthermore, if the CPU 32 determines that no instruction to add a viewpoint has been input from the user to the operation unit 14 (NO in S14), it proceeds to S16. In S16, the CPU 32 determines whether or not an instruction to add information has been input from the user to the operation unit 14. The instruction to add information is an instruction to add multiple information images that show additional information to the main screen SC1. The multiple information images include an edge image M1, a center image M2, depth grid lines M3, and height grid lines M4.

[0035] Edge image M1 is an image showing the edge of point cloud image F1. Edge image M1 shows a closed boundary, and all points represented by the point cloud data are located within that boundary. In other words, no points represented by the point cloud data exist outside of edge image M1. Edge image M1 allows the user to understand the status of objects that can be measured at the current position and orientation of LiDAR100.

[0036] The central image M2 shows the center line of the LiDAR100's field of view. The depth scale M3 shows the contour lines of distance in the depth direction of the LiDAR100's field of view, i.e., in the XY plane. The height scale M4 shows the contour lines of distance in the height direction of the LiDAR100's field of view, i.e., in the Z direction. The depth scale M3 and height scale M4 are composed of lines that are spaced at equal intervals. The central image M2, depth scale M3, and height scale M4 allow the user to understand the size of objects that can be measured at the current position and orientation of the LiDAR100, as well as the distance to those objects.

[0037] If the CPU 32 determines that the user has input an instruction to add information to the operation unit 14 (YES in S16), it proceeds to S30. In S30, the CPU 32 performs edge identification processing to identify the edges of the point cloud image F1. For example, in edge identification processing, the CPU 32 scans all pixels of the point cloud image F1 and identifies the boundary between pixels with pixel values ​​and pixels without pixel values ​​as edges.

[0038] In S32, the CPU 32 adds the edge image M1, the center image M2, the depth grid lines M3, and the height grid lines M4 to the main screen SC1. Compared to the main screen SC1 where multiple information images are not displayed, the situation in which the LiDAR 100 is installed can be quickly grasped. This helps to adjust the attitude and position of the LiDAR 100. In the modified example, at least one of the edge image M1, the center image M2, the depth grid lines M3, and the height grid lines M4 may be added to the main screen SC1. When S32 is completed, the CPU 32 proceeds to S50.

[0039] Furthermore, if the CPU 32 determines that no instruction to add information has been entered by the user into the operation unit 14 (NO in S16), it proceeds to S18. In S18, the CPU 32 determines whether or not an instruction to exclude information has been entered by the user into the operation unit 14. The measurement results of the LiDAR 100 may include points that are obstacles when understanding the situation in which the LiDAR 100 is installed. For example, obstacle points are points measured on the floor surface 8 and the ceiling. An instruction to exclude information is an instruction to exclude obstacle points from the point cloud data shown by the point cloud data.

[0040] If the CPU 32 determines that an instruction to exclude information has been entered by the user into the operation unit 14 (YES in S18), it proceeds to S40. In S40, the CPU 32 sets the exclusion range. The exclusion range includes the range representing the floor 8 and the range representing the ceiling. The range representing the floor 8 is the range within range R that is less than or equal to the first height. The range representing the ceiling is the range that is greater than or equal to the second height. Here, the first height and the second height may be values ​​predetermined by the application 42. Alternatively, the first height and the second height may be values ​​entered by the user.

[0041] In S42, the CPU 32 identifies a specific point cloud by excluding points included in the exclusion range from the point cloud data. For example, points included in the exclusion range are those whose Z-axis value is less than or equal to the first height, or those whose Z-axis value is greater than or equal to the second height. The CPU 32 then displays an image showing the specific point cloud on the main screen SC1. This configuration allows for a quicker understanding of the LiDAR 100's installation status compared to a configuration in which points included in the exclusion range are not excluded. This helps in adjusting the attitude and position of the LiDAR 100.

[0042] Points excluded by the information exclusion instruction are not limited to points included in the exclusion range. For example, points in the point cloud data that have a reflected light intensity above the first threshold, or points that have a reflected light intensity below the second threshold, are excluded. This is because these points may be noise that does not accurately measure the object.

[0043] Furthermore, points excluded by the information exclusion instruction may also be points indicating the moving object 4. Points indicating the moving object 4 are identified, for example, by clustering. Clustering is performed on point cloud data accumulated as time-series data. Clustering is a known technique, and a detailed explanation is omitted in this specification. By excluding points indicating the moving object 4, a point cloud image showing only the stationary object 6 is displayed on the main screen SC1. The status of the stationary object 6 in the space where the LiDAR 100 is installed can be quickly grasped.

[0044] Furthermore, if the CPU 32 determines that no instruction to exclude information has been entered by the user into the operation unit 14 (NO in S18), it proceeds to S50. In S50, the CPU 32 determines whether or not an instruction to terminate has been entered by the user into the operation unit 14. If the CPU 32 determines that an instruction to terminate has been entered by the user into the operation unit 14 (YES in S50), it terminates the process shown in Figure 2. On the other hand, if the CPU 32 determines that no instruction to terminate has been entered by the user into the operation unit 14 (NO in S50), it returns to S14.

[0045] (Correspondence) Terminal device 10 and range R are examples of "processing device" and "predetermined range," respectively. Moving object 4 and stationary object 6 are examples of "multiple objects." Main screen SC1 and multiple screen SC2 are examples of "screens." Overhead view image F2 and front view image F3 are examples of "first image" and "second image," respectively. The Z axis is an example of "first axis," and the X and Y axes are examples of "second axes." Point cloud image F1 is an example of a "point cloud image." Edge image M1, center image M2, depth scale line M3, and height scale line M4 are examples of "information images."

[0046] S10 in Figure 2 is an example of the process implemented by the "acquisition unit". S22, S32, and S42 are examples of the processes implemented by the "display control unit".

[0047] The above describes specific examples of the technology disclosed herein, but these are merely illustrative and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples described above. For example, the following modifications may be adopted.

[0048] (Modification 1) Processes S14, S20, and S22 in Figure 2 do not need to be executed. In this modification, the "first image" and the "second image" are omitted.

[0049] (Modification 2) Multiple screen SC2 may include one of the overhead view image F2 and the front view image F3, but may not include the other of the overhead view image F2 and the front view image F3. In this modification, the point cloud image F1 is an example of the "first image," and either the overhead view image F2 or the front view image F3 is an example of the "second image."

[0050] (Modification 3) Processes S16, S30, and S32 in Figure 2 do not need to be executed. In this modification, the "information image" is omitted.

[0051] (Modification 4) The processes S18, S40, and S42 in Figure 2 do not need to be performed. In this modification, the "specific point cloud" is omitted.

[0052] (Modification 5) The "processing device" may be a server on the Internet or an intranet.

[0053] The technical elements described herein or in the drawings demonstrate technical usefulness individually or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technologies illustrated herein or in the drawings achieve multiple objectives simultaneously, and achieving even one of these objectives constitutes technical usefulness in itself. [Explanation of Symbols]

[0054] 2: Measurement System 4: Mobile 6: Stationary object 8: Floor surface 10: Terminal device 12:Display section 14:Operation section 16: Communication I / F 30: Control Unit 32:CPU 34: Memory 40: OS Program 42: App F1: Point cloud image F2: Overhead view F3: Front view image M1: Edge image M2: Center image M3: Depth scale line M4: Height scale line R: Range SC1: Main screen SC2: Multiple screens

Claims

1. A processing device for processing point cloud data, An acquisition unit that acquires point cloud data representing a point cloud of multiple objects within a predetermined range from a measuring device, A display control unit that causes a screen displaying the point cloud shown by the point cloud data to be displayed on the display unit, Equipped with, The aforementioned screen includes a first image showing the point cloud as viewed from a first viewpoint, and a second image showing the point cloud as viewed from a second viewpoint different from the first viewpoint, wherein the processing apparatus.

2. The first viewpoint described above is a viewpoint that provides an overview of the multiple objects, The apparatus according to claim 1, wherein the second viewpoint is a viewpoint from which the plurality of objects are viewed from a direction perpendicular to the vertical line extending from the first viewpoint.

3. Each point represented by the aforementioned point cloud data is shown in a Cartesian coordinate system. The first image is generated by convolving each point of the point cloud data along a first axis in the Cartesian coordinate system. The processing apparatus according to claim 1, wherein the second image is generated by convolving each point of the point cloud data along a second axis in the Cartesian coordinate system that is orthogonal to the first axis.

4. A processing device for processing point cloud data, An acquisition unit that acquires point cloud data representing a point cloud of multiple objects within a predetermined range from a measuring device, A display control unit that causes a screen displaying the point cloud shown by the point cloud data to be displayed on the display unit, Equipped with, The aforementioned screen includes a point cloud image showing the point cloud and an information image showing additional information. The information image includes at least one of the following: the edges of the point cloud, a grid line defined along the height direction of the predetermined range, and a grid line defined along the horizontal direction of the predetermined range.

5. A processing device for processing point cloud data, An acquisition unit that acquires point cloud data representing a point cloud of multiple objects within a predetermined range from a measuring device, A display control unit causes a display unit to display a screen showing a specific point cloud obtained by excluding points that satisfy predetermined conditions from the point cloud data shown above, A processing apparatus equipped with the following features.

6. The apparatus according to claim 5, wherein the point that satisfies the predetermined conditions is a point measured in at least one of the ranges of the predetermined range that is less than or equal to a first height and the range that is greater than or equal to a second height.

Images