Computer program for the processing unit, and the processing unit.

Abstract

It provides technology for accurately superimposing moving objects onto a map. [Solution] The processing device comprises: a first acquisition unit that acquires point cloud data representing objects as point clouds from a measuring device; a cropping unit that crops partial point cloud data indicating objects within a predetermined area from the point cloud data; a generation unit that generates map data indicating a map of the predetermined area from the partial point cloud data; a calculation unit that calculates moving body data indicating a moving body moving within the predetermined area from the point cloud data; and a first display control unit that overlays the moving body indicated by the moving body data onto the map indicated by the map data and displays the map with the moving body overlaid.

Description

Technical Field

[0001] The technology disclosed in this specification relates to a processing device for processing point cloud data.

Background Art

[0002] Patent Document 1 discloses a technology for constructing an indoor map of a target facility based on scan information generated by scanning the interior of the target facility.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Technologies for measuring the movement of a moving object, such as a person, moving within an area such as a facility, are becoming widespread. The measured movement of the moving object is displayed overlaid on a map. This specification provides a technology for accurately overlaying a moving object on a map.

Means for Solving the Problems

[0005] This specification discloses a processing device for processing point cloud data. The processing device includes a first acquisition unit that acquires point cloud data representing an object as a point cloud from a measurement device, a cutting unit that cuts out partial point cloud data indicating an object within a predetermined area from the point cloud data, a generation unit that generates map data indicating a map of the predetermined area from the partial point cloud data, a calculation unit that calculates moving object data indicating a moving object moving within the predetermined area from the point cloud data, and a first display control unit that overlays the moving object indicated by the moving object data on the map indicated by the map data and displays the map with the overlaid moving object.

[0006] According to the above configuration, map data and moving object data are generated from point cloud data obtained from the measurement device. In other words, the reference point for map data and the reference point for moving object data are the same. This allows the moving object to be accurately overlaid on the map.

[0007] The processing device further includes a memory for storing location information indicating the location where the measuring device is installed, and the location information may include the latitude and longitude of the measuring device and the direction in which the measuring device is facing.

[0008] With the above configuration, a map generated from point cloud data can be reflected on other maps based on latitude and longitude.

[0009] The position information may further include at least one of the elevation angle of the measuring device and the installation height of the measuring device.

[0010] With the above configuration, a moving object can be superimposed on a map corrected using at least one of the elevation / depression angle and installation height of the measuring device.

[0011] The predetermined region has a predetermined horizontal range and a predetermined height range, and the cropping section may crop the partial point cloud data from the point cloud data by cropping the point data belonging to both the predetermined horizontal range and the predetermined height range from the point cloud data.

[0012] According to the above configuration, it is possible to generate a map of a region limited by a predetermined horizontal range and a predetermined height range.

[0013] The cropping portion may be formed by removing point data that are greater than or equal to a first height from the point cloud data, thereby cropping the partial point cloud data from the point cloud data.

[0014] Point data indicating heights greater than or equal to the first height represent indoor ceilings. Ceilings are located above the areas where moving objects such as people move. With the above configuration, map data can be generated by removing point data representing ceilings where moving objects do not move.

[0015] The aforementioned cropping portion may be obtained by cropping the partial point cloud data from the point cloud data by removing point data that indicates a height of a second height or less from the point cloud data.

[0016] Point data indicating heights below the second height represent the floor or ground. The floor or ground is not an obstacle for moving objects such as people. According to the above configuration, map data can be generated by removing point data indicating the floor or ground that is not an obstacle for moving objects.

[0017] The first display control unit may display the map with the measuring device superimposed on it, in addition to the moving object.

[0018] With the above configuration, you can view the map and confirm the location of not only the moving object but also the measuring device.

[0019] The processing device further includes a second display control unit that displays a designation screen for specifying the predetermined area, and the cropping unit may crop the partial point cloud data showing the object within the predetermined area specified by the designation screen.

[0020] According to the above configuration, the user can specify a predetermined area for generating the map.

[0021] The aforementioned designation screen may have a graphical user interface that allows the user to specify a predetermined area, which is part of the entire area, using a frame.

[0022] With the above configuration, a predetermined area can be easily specified using a graphical user interface.

[0023] The map may be a two-dimensional bitmap.

[0024] According to the above configuration, the moving object can be confirmed on a two-dimensional bitmap.

[0025] In addition, a computer program for the above processing device, a storage medium storing the computer program, and a control method for controlling the above processing device are also novel and useful.

Brief Description of the Drawings

[0026] [Figure 1] It is a block diagram of a measurement system. [Figure 2] It is a flowchart of map generation processing. [Figure 3] It is a flowchart of display processing.

Embodiments for Carrying Out the Invention

[0027] (First Embodiment) (Configuration of Measurement System 2; FIG. 1) The measurement system 2 measures the positions of objects in a predetermined space 4. The objects are a moving object and a stationary object 8. The moving object is, for example, a person 6a, 6b, etc. The stationary object 8 is, for example, a wall, a shelf, a desk, etc. The predetermined space 4 is, for example, an indoor space, an outdoor space, etc. The indoor space is, for example, an office, a factory, a shopping mall, an exhibition hall, etc. The outdoor space is, for example, the outside of a station, the entrance of a building, etc. In addition, XYZ coordinates are defined in FIG. 1. Hereinafter, the XYZ coordinates will be used for explanation as appropriate.

[0028] The measurement system 2 comprises a terminal device 10 and a LiDAR 100. The terminal device 10 is a desktop PC, laptop PC, smartphone, etc. The LiDAR 100 is a 3D LiDAR that measures the distance to surrounding objects in three dimensions, a technique called light detection and ranging. The terminal device 10 and the LiDAR 100 are connected to each other via wired or wireless connections so that they can communicate with each other. In a modified example, the measurement system 2 may be a single device, and the functions of the terminal device 10 and the LiDAR 100 may be integrated into that single device.

[0029] The LiDAR100 is configured to output point cloud data obtained by repeatedly scanning a laser over a region R surrounding the LiDAR100. The laser is emitted by a laser light source built into the LiDAR100. The laser light source rotates using a rotation mechanism such as a motor. Scanning over region R is achieved by rotating the laser light source by a predetermined angle in the XY plane (e.g., 180 degrees) and a predetermined angle up and down in the Z direction (e.g., 120 degrees). For example, the laser light source rotates at a rotation speed of 5 times / second. In a modified example, the LiDAR100 may scan the laser using a method that does not utilize a rotation mechanism, known as a solid-state type.

[0030] Point cloud data is a collection of coordinate points in Cartesian coordinates (X, Y, Z) defined within space 4. In a modified example, point cloud data may also be a collection of coordinate points in polar coordinates defined within space 4.

[0031] (Configuration of terminal device 10; Figure 1) The terminal device 10 comprises a display unit 12, an operation unit 14, a communication interface 16, and a control unit 30. Hereafter, "interface" will be abbreviated as "I / F".

[0032] The display unit 12 is a display for showing various information. The operation unit 14 has multiple keys for receiving instructions from the user. The display unit 12 may also function as a touchscreen (i.e., operation unit). The communication interface 16 is an interface for performing communication with the LiDAR 100.

[0033] The control unit 30 comprises a CPU 32 and a memory 34. The memory 34 consists of volatile memory and non-volatile memory. The CPU 32 performs various processes according to programs 40 and 42 stored in the memory 34. The OS program 40 is a program that controls the basic operation of the terminal device 10. The application program 42 (hereinafter referred to as "app 42") is a program for processing point cloud data received from the LiDAR 100. App 42 is provided, for example, by a company that provides the measurement system 2.

[0034] The application 42 repeatedly receives point cloud data from the LiDAR 100 in a time series and stores the time series point cloud data as measurement data 50 in the memory 34. The memory 34 can also store installation information 52, map point cloud information 54, and mobile object data 56.

[0035] The installation information 52 includes the latitude and longitude of the installation location of the Lider 100, the orientation of the Lider 100, the elevation angle of the Lider 100, and the installation height of the Lider 100. The installation information 52 is input by the user, for example. In the modified version, at least a part of the installation information 52, for example, the elevation angle, may be a value measured by a sensor.

[0036] Map point cloud information 54 is data that shows a map of a predetermined area in space 4. Map point cloud information 54 is generated from measurement data 50 by application 42. Map point cloud information 54 includes map point cloud data 60, corner information 62, and Lider installation information 64.

[0037] The map point cloud data 60 is point cloud data that shows a map of a predetermined area in space 4. The predetermined area is defined by a predetermined horizontal range A1 and a predetermined height range H1 to H2. The height range H1 to H2 is the range from the lower limit height H1 to the upper limit height H2. The predetermined area is, for example, an area specified by the user.

[0038] The map point cloud data 60 is a collection of point data representing stationary objects 8 located within a predetermined area. The corner information 62 includes the coordinate points of each of the four corners of the horizontal range A1. Furthermore, the corner information 62 also includes the latitude and longitude of each of the four corners of the horizontal range A1. The map point cloud data 60 and the corner information 62 are generated by the map generation process described later.

[0039] Lider installation information 64 consists of the latitude and longitude of the installation location of Lider 100 and the orientation of Lider 100, as specified in installation information 52.

[0040] The moving object data 56 represents a moving object moving within a predetermined area in space 4. App 42 generates the moving object data 56 by analyzing the measurement data 50. For example, App 42 extracts a group of point data having a specific distance relationship from the measurement data 50. App 42 identifies the extracted group as person 6a. The specific distance relationship is a so-called feature point. App 42 analyzes the measurement data 50 and calculates time-series data of the coordinate points of person 6a having the specific distance relationship. App 42 also extracts a group of point data having a different distance relationship from person 6a from the measurement data 50 and identifies this group as another person 6b. App 42 calculates time-series data of the coordinate points of the other person 6b. The moving object data 56 is the time-series data of the coordinate points calculated for each of the multiple people.

[0041] (Map generation process; Figure 2) Figure 2 is a flowchart of the map generation process executed by the CPU 32 according to the application 42. The map generation process is the process for generating map point cloud information 54. The process in Figure 2 is started when the operation unit 14 receives a generation instruction from the user.

[0042] In S10, the CPU 32 retrieves measurement data 50 from memory 34. In the process shown in Figure 2, for example, measurement data 50 measured by the Lider 100 during a time when there is little foot traffic in space 4 is used. This is because people, being moving objects, become noise when generating the map. The measurement time for the measurement data 50 retrieved in S10 can be relatively short, for example, a few minutes.

[0043] In S12, the CPU 32 performs plane estimation processing on the measurement data 50 retrieved in S10. For example, it is assumed that the Lider 100 is installed at a high altitude and facing downwards. In this case, the Lider 100 is not installed parallel to the actual floor or ground, and as a result, the height of the object in space 4 is measured at a value different from the actual height. In the plane estimation processing described above, the elevation angle and installation height in the installation information 52 are used to correct the measurement data 50 from the Lider 100 to data that represents the actual height. The algorithm for the plane estimation processing is known, so its explanation is omitted in this specification.

[0044] In S14, the CPU 32 analyzes the measurement data 50 corrected in S12 and extracts point data representing the moving object from the measurement data 50. The method for extracting point data representing the moving object is the same as the method for identifying the moving object in the moving object data 56. Then, the CPU 32 removes the point data representing the moving object from the measurement data 50.

[0045] In S16, the CPU 32 displays a point cloud screen SC1 on the display unit 12, which displays the point cloud represented by the measurement data 50 with the moving object removed. The point cloud screen SC1 displays the point cloud P1 representing the stationary object 8 in space 4 in three dimensions. The point cloud screen SC1 also displays a frame W1 for specifying the horizontal range A1. The user specifies the horizontal range A1 by changing the size of the frame W1 and moving the frame W1 using the operation unit 14. The frame W1 is a graphical user interface. The horizontal range A1 can be easily specified using the frame W1. In the modified example, instead of the frame W1, an input screen may be displayed for inputting the horizontal and vertical lengths of the horizontal range A1.

[0046] In S18, the CPU 32 monitors whether the operation unit 14 receives an instruction from the user to confirm the horizontal range A1 specified on the point cloud screen SC1. If the CPU 32 receives an instruction from the user to confirm the horizontal range A1 on the operation unit 14 (YES in S16), the process proceeds to S20.

[0047] In S20, the CPU32 identifies the coordinates of the four corners of the horizontal range A1 defined in S16 from the measurement data.

[0048] In S22, the CPU 32 calculates the latitude and longitude of the four corners of the horizontal range A1 based on the latitude and longitude of the installation location of the Lider 100 and the coordinate points of the four corners identified in S20. Here, the latitude and longitude of the installation location of the Lider 100 are pre-entered into the application 42 by the user.

[0049] In S24, the CPU32 removes point data from the measurement data50 that indicates points located outside the horizontal range A1.

[0050] In S26, the CPU 32 displays a height specification screen SC2 on the display unit 12 for specifying the height range H1 to H2. The height specification screen SC2 includes a field for entering the lower height limit H1 and a field for entering the upper height limit H2.

[0051] In S28, the CPU 32 monitors whether the operation unit 14 receives instructions from the user to confirm the lower limit height H1 and the upper limit height H2 entered in the height specification screen SC2. If the CPU 32 receives instructions from the user to confirm the lower limit height H1 and the upper limit height H2 (YES in S28), the process proceeds to S30.

[0052] In S30, the CPU 32 removes point data from the measurement data 50 that indicates points located at a height greater than or equal to the upper limit height H2. An object with a height greater than or equal to the upper limit height H2 is, for example, the ceiling of an indoor building. Normally, moving objects such as people do not pass over the ceiling. Point data indicating ceilings where moving objects do not move can be removed from the measurement data 50 used to generate the map.

[0053] In S32, the CPU 32 removes point data from the measurement data 50 that indicates points located at a height below the lower limit height H1. Objects with a height below the lower limit height H1 are, for example, the floor or the ground. Normally, the floor or the ground is not an obstacle for moving objects such as people. Point data indicating the floor or the ground that does not pose an obstacle to moving objects can be removed from the measurement data 50 used to generate the map.

[0054] In S34, the CPU 32 updates the point cloud in the point cloud screen SC1 with the point cloud represented by the measurement data 50 processed in S24, S30, and S32 described above.

[0055] In S36, the CPU 32 monitors whether the operation unit 14 receives a storage instruction from the user to store the map point cloud information 54 in the memory 34. If the CPU 32 receives a storage instruction from the user (YES in S36), it proceeds to S40.

[0056] In S40, the CPU 32 stores map point cloud information 54 in memory 34, which includes the measurement data 50 processed in S24, S30, and S32 as map point cloud data 60. Here, the coordinate points identified in S20 and the latitude and longitude calculated in S22 are stored in memory 34 as corner information 62. When S40 is completed, the process shown in Figure 2 is finished.

[0057] (Display processing; Figure 3) Figure 3 is a flowchart of the display processing performed by the CPU 32 according to the application 42. The display processing is the process of overlaying a moving object onto the map shown by the map point cloud information 54. The processing in Figure 2 is started when the operation unit 14 receives a display instruction from the user.

[0058] In S50, the CPU 32 retrieves map point cloud information 54 from memory 34. In S52, the CPU 32 converts the map point cloud data 60 in the map point cloud information 54 into bitmap data. Bitmap data is data that represents a two-dimensional bitmap. The bitmap is a map of a predetermined area viewed from the vertical direction. The size of each dot in the bitmap is predetermined by the user. In the modified example, the size of each dot may be determined by the application 42 based on the measurement accuracy of the Lider 100. For each point data in the map point cloud data 60, the CPU 32 sets the data in the dot corresponding to the position indicated by that point data. This generates a bitmap based on the map point cloud data 60. The resolution of the bitmap is set lower than the resolution of the map point cloud data 60. Therefore, multiple point data may correspond to a single dot.

[0059] Furthermore, the CPU 32 determines the latitude and longitude of the four corners of the bitmap based on the corner information 62. The CPU 32 also sets data for the dots corresponding to the positions indicated by the placement information 52 in the map point cloud information 54. As a result, the Lider 100 is displayed on the bitmap.

[0060] In S54, the CPU 32 retrieves the mobile data 56 from the memory 34. The mobile data 56 is generated, for example, based on the measurement data 50 received from the Lider 100 after executing the process shown in Figure 2. In a modified example, the mobile data 56 may be generated based on the measurement data 50 received from the Lider 100 before executing the process shown in Figure 2.

[0061] In S56, the CPU 32 overlays the moving object represented by the moving object data 56 onto the bitmap data represented by the bitmap data converted in S52. Specifically, the CPU 32 sets data in the dots corresponding to the positions represented by the moving object data 56. Here, the moving object data 56 is time-series data. The position of the moving object overlaid on the bitmap changes over time.

[0062] In S58, the CPU 32 displays the map screen SC3, which includes a bitmap with the moving objects superimposed in S52, on the display unit 12. Figure 3 shows an example of the map screen SC3. The map screen SC3 displays stationary objects 8, as well as people 6a and 6b, which are displayed as moving objects moving between the stationary objects 8. Here, the stationary objects 8 are displayed in a different color from the moving objects such as people 6a and 6b. The moving objects can be seen on a two-dimensional bitmap.

[0063] Furthermore, the Lider 100 is also displayed on the map screen SC3. By displaying the Lider 100, the position of the Lider 100 can be confirmed not only by looking at the map screen SC3. Note that the people 6a, 6b, and the Lider 100 only need to be displayed as schematic objects. When S58 is completed, the CPU 32 terminates the process shown in Figure 3.

[0064] (Effects of this embodiment) In this embodiment, map point cloud data 60 and mobile object data 56 are generated from measurement data 50 obtained from the Lider 100. That is, the reference for map point cloud data 60 and the reference for mobile object data 56 are the same. This allows the mobile object to be accurately superimposed on the map.

[0065] In this embodiment, the map point cloud information 54 includes installation information 52. Based on the latitude and longitude in the installation information 52, the map shown by the map point cloud information 54 can be overlaid with other maps, such as maps provided on a website.

[0066] (Correspondence) Terminal device 10 and Lider 100 are examples of "processing device" and "measuring device," respectively. Memory 34 is an example of "memory." Measurement data 50 and map point cloud data 60 are examples of "point cloud data" and "partial point cloud data," respectively. Mobile object data 56 is an example of "mobile object data." Bitmap data and map screen SC3 are examples of "map data" and "map," respectively. Installation information 52 is an example of "location information." Horizontal range A1 and height range H1~H2 are examples of "horizontal range" and "height range," respectively. Upper limit height H2 and lower limit height H1 are examples of "first height" and "second height," respectively. Point cloud screen SC1 and height specification screen SC2 in Figure 2 are examples of "specification screens." Frame W1 is an example of "frame."

[0067] S10 in Figure 2 is an example of the process implemented by the "first acquisition unit". S24, S30, and S32 in Figure 2 are examples of the processes implemented by the "cutting unit". S52 and S58 in Figure 3 are examples of the processes implemented by the "generation unit" and the "first display unit", respectively.

[0068] The above describes specific examples of the technology disclosed herein, but these are merely illustrative and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples described above. For example, the following modifications may be adopted.

[0069] (Modification 1) The installation information 52 does not need to be stored in memory 34. In this modification, "location information" can be omitted.

[0070] (Modification 2) The installation information 52 does not have to include the elevation angle and installation height. Furthermore, the process S12 in Figure 2 does not have to be executed.

[0071] (Modification 3) At least one of the following may be omitted: the horizontal range A1 in S24 of Figure 2, and the height range H1 to H2 in S30 and S32.

[0072] (Modification 4) The process in S30 of Figure 2 does not need to be executed. In this modification, the "first height" can be omitted.

[0073] (Modification 5) The process in S32 of Figure 2 does not need to be executed. In this modification, the "second height" can be omitted.

[0074] (Modification 6) In the map screen SC3, the Lider 100 does not need to be displayed. In this modification, the phrase "the map on which the measuring device is superimposed" can be omitted.

[0075] (Modification 7) The processes S18 and S26 in Figure 2 do not need to be executed. In this modification, the predetermined area may be automatically set by the application 42. In this modification, the "specified screen" can be omitted.

[0076] (Modification 8) The process in S52 of Figure 3 does not need to be executed. In this modification, the point cloud may be displayed as a map on the map screen SC3. In this modification, "bitmap" can be omitted.

[0077] (Modification 9) The "processing device" is not limited to the terminal device 10, but may be, for example, a server connected to a LAN or the Internet.

[0078] The technical elements described herein or in the drawings demonstrate technical usefulness individually or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technologies illustrated herein or in the drawings achieve multiple objectives simultaneously, and achieving even one of these objectives constitutes technical usefulness in itself. [Explanation of Symbols]

[0079] 2: Measurement System 4: Space 6a: person 6b: Person 8: Stationary object 10: Terminal device 12: Display section 14:Operation section 16: Communication I / F 30: Control Unit 32:CPU 34: Memory 40: OS Program 42: App 50: Measurement data 52: Installation information 54: Map point cloud information 56: Mobile Data 60: Map point cloud data 62: Corner Information 64: Lider Installation Information A1: Horizontal range H1: Height H2: Height P1: Point cloud R: area SC1: Point cloud screen SC2: Height specification screen SC3: Map screen W1: Frame

Claims

1. A processing device for processing point cloud data, A first acquisition unit that acquires point cloud data representing an object as a point cloud from a measuring device, A cropping unit that extracts partial point cloud data representing an object within a predetermined region from the aforementioned point cloud data, A generation unit that generates map data showing a map of a predetermined region from the partial point cloud data, A calculation unit calculates moving object data indicating a moving object moving within a predetermined area from the point cloud data, A first display control unit overlays the moving object, represented by the moving object data, onto the map represented by the map data, and displays the map with the moving object superimposed. A processing device equipped with the following features.

2. The aforementioned processing apparatus further, The device includes a memory that stores location information indicating the location where the measuring device is installed. The processing apparatus according to claim 1, wherein the position information includes the latitude and longitude of the measuring device and the direction in which the measuring device is facing.

3. The processing apparatus according to claim 2, wherein the position information further includes at least one of the elevation angle of the measuring device and the installation height of the measuring device.

4. The aforementioned predetermined region has a predetermined horizontal range and a predetermined height range. The processing apparatus according to claim 1, wherein the cutting unit cuts out point data belonging to both the predetermined horizontal range and the predetermined height range from the point cloud data, thereby cutting out the partial point cloud data from the point cloud data.

5. The processing apparatus according to claim 1, wherein the cutting portion cuts out the partial point cloud data from the point cloud data by removing point data that have a height greater than or equal to a first height from the point cloud data.

6. The processing apparatus according to claim 1, wherein the cutting portion cuts out the partial point cloud data from the point cloud data by removing point data indicating a height of a second height or less from the point cloud data.

7. The processing apparatus according to claim 1, wherein the first display control unit displays the map with the measuring device superimposed on the moving body.

8. The aforementioned processing apparatus further, The system includes a second display control unit that displays a designation screen for specifying the predetermined area, The processing apparatus according to claim 1, wherein the cropping section crops the partial point cloud data showing the object within the predetermined area specified by the designated screen.

9. The processing apparatus according to claim 8, wherein the designated screen has a graphical user interface that allows the user to specify the predetermined area, which is part of the entire area, with a frame.

10. The processing apparatus according to claim 1, wherein the map is a two-dimensional bitmap.

11. A computer program for a processing unit that processes point cloud data, The computer program controls the computer of the processing unit, A first acquisition unit that acquires point cloud data representing an object as a point cloud from a measuring device, A cropping unit that extracts partial point cloud data representing an object within a predetermined region from the aforementioned point cloud data, A generation unit that generates map data showing a map of a predetermined region from the partial point cloud data, A calculation unit calculates moving object data indicating a moving object moving within a predetermined area from the point cloud data, A first display control unit overlays the moving object, represented by the moving object data, onto the map represented by the map data, and displays the map with the moving object superimposed. A computer program that functions as such.

Images