Information processing device, crane monitoring system, distance calculation method, and distance calculation program

Abstract

To suppress an operation amount in calculation of distance between objects detected from point group data.SOLUTION: An information processing device (1) includes: an object detection unit (104) configured to detect a plurality of objects from point group data; and a distance calculation unit (107) for specifying a contour section in which both objects face each other with an arbitrary position in each of the two objects to be a distance calculation target among the plurality of detected objects being a reference to calculate distance between the contour sections as distance between the objects.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device or the like that calculates the distance between objects detected from point cloud data. [Background technology]

[0002] Techniques for detecting objects from point cloud data have been known for some time. For example, Patent Document 1 below discloses a crane equipped with a guide information display device. This guide information display device has a function for estimating the ground surface, the top surface of a suspended load, and the top surfaces of features around the suspended load from point cloud data acquired by a laser scanner placed at the tip of the crane's telescopic boom. The guide information display device also has a function for determining that there is a risk of contact if the horizontal distance between the suspended load and the feature when projected onto a horizontal plane is equal to or less than a predetermined threshold, and the vertical distance is equal to or less than a predetermined threshold. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-23567 Summary of the Invention [Problem to be solved by the invention]

[0004] Although Patent Document 1 does not disclose a specific method for calculating the horizontal distance, in general, when calculating the distance between objects represented by point cloud data, the distance between each point on the contour of one object and each point on the contour of the other object is calculated. The minimum value of the calculated distances is then set as the distance between the two objects. For example, if there are n points on the contour of each object, the number of combinations of points for which distances should be calculated is n squared.

[0005] As described above, the amount of calculation required to calculate the distance between objects detected from point cloud data is large, and there is room for improvement in this respect. One aspect of the present invention aims to realize an information processing device or the like that can reduce the amount of calculation required to calculate the distance between objects detected from point cloud data. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one embodiment of the present invention includes an object detection unit that detects multiple objects from point cloud data that includes at least points indicating the surfaces of the multiple objects, and a distance calculation unit that identifies opposing contour portions of two objects to be subjected to distance calculation based on an arbitrary position on each of the multiple objects detected by the object detection unit, and calculates the distance between the contour portions as the distance between the objects.

[0007] In addition, in order to solve the above-mentioned problems, a distance calculation method according to one embodiment of the present invention is a distance calculation method executed by one or more information processing devices, and includes an object detection step of detecting multiple objects from point cloud data including at least points indicating the surfaces of the multiple objects, and a distance calculation step of identifying opposing contour portions of two objects to be subjected to distance calculation among the multiple objects detected in the object detection step, using any position on each of the objects as a reference, and calculating the distance between the contour portions as the distance between the objects. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to reduce the amount of calculation required to calculate the distance between objects detected from point cloud data. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of a configuration of a main part of an information processing device according to a first embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of the configuration of a monitoring system including the information processing device. [Figure 3]FIG. 10 is a diagram illustrating an example of a scan pattern. [Figure 4] FIG. 10 is a diagram showing an example in which each point included in one cluster is projected onto a horizontal plane on which a grid is set. [Figure 5] FIG. 10 is a diagram illustrating an example of a procedure for detecting a contour. [Figure 6] 10A and 10B are diagrams illustrating conditions related to the height of an object and the depth of a surface of the object, among the conditions for integrating objects. [Figure 7] FIG. 10 is a diagram showing an example in which points included in each cluster to be determined as to whether or not to merge are projected onto a horizontal plane on which a grid is set. [Figure 8] FIG. 10 is a diagram illustrating a method for calculating the distance between objects. [Figure 9] FIG. 10 is a diagram showing an example of a display screen of a detected object. [Figure 10] 10 is a flowchart illustrating an example of processing executed by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] [System Configuration] The configuration of a crane monitoring system 100 according to one embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the monitoring system 100. The monitoring system 100 is a system for safety management of a crane CR, and as shown in the figure, the monitoring system 100 includes an information processing device 1, an alarm device 2, and a measuring device 3.

[0011] The crane CR is a machine that lifts and moves heavy objects (loads). The crane CR includes a carriage CR1, a base CR2, a boom CR3, a cable CR4, a hook CR5, and a crane room CR6.

[0012] The carriage section CR1 carries a base section CR2. The base section CR2 is a structure that supports the boom CR3 and other components. The base section CR2 and the carriage section CR1 are connected so that the base section CR2 can rotate on the carriage section CR1. This allows the crane CR to transport loads located within a 360-degree radius. Note that the rotation range may be less than 360 degrees. The crane CR may also be unable to rotate within a horizontal plane.

[0013] One end of the boom CR3 is rotatably connected to the base CR2. A pulley (not shown) is provided at the other end of the boom CR3, and a cable CR4 hangs down from the pulley.

[0014] The cable CR4 is a rope for lifting a load, and has a hook CR5 attached to the end for hanging the load. When a load is to be moved by the crane CR, the load is sling-hooked onto the hook CR5 and lifted up.

[0015] The crane room CR6 is a space that houses various devices for operating the crane CR, and the operator of the crane CR (hereinafter simply referred to as the operator) operates the crane CR from the crane room CR6. For example, the operator can adjust the length of the boom CR3, raise and lower the hook CR5 (load) by winding up / lowering the cable CR4, and rotate the structure of the base CR2.

[0016] Additionally, a measuring device 3 is attached near the tip of the boom CR3 of the crane CR. The measuring device 3 generates point cloud data that includes at least points indicating the surfaces of the load suspended by the crane CR and the surrounding objects. For example, when point cloud data is generated by the measuring device 3 in the state shown in FIG. 2, point cloud data is generated that indicates the surface positions of each object within the conical measurement area AR1, specifically the load A1, objects B1 and B2, etc. Note that the size of the measurement area AR1 can change depending on the height of the measuring device 3 (the length and angle of the boom CR3).

[0017] The measuring device 3 may be any device capable of generating the above-described point cloud data. For example, various laser scanners and the like can be used as the measuring device 3. For example, a LiDAR (Light Detection and Ranging) device may also be used as the measuring device 3. The data format of the point cloud data is not particularly limited. In this embodiment, an example will be described in which each point included in the point cloud data is represented by coordinate values ​​(x, y, z) in a Cartesian coordinate system defined by three mutually orthogonal axes, x, y, and z. For example, as shown in FIG. 2, a coordinate system may be applied in which the vertical downward direction is defined as the positive direction of the x-axis, and mutually perpendicular y- and z-axes are defined on a horizontal plane orthogonal to the x-axis. In this case, the yz plane is the surface to be measured. Furthermore, the origin of the coordinate system, i.e., the point with coordinate values ​​(0, 0, 0), may be defined as the position of the measuring device 3. The coordinate values ​​may be in meters, for example.

[0018] The information processing device 1 detects a suspended load and objects around the suspended load and calculates the distance between them using the point cloud data generated by the measurement device 3. For example, in the state shown in Fig. 2, the information processing device 1 calculates the distance between the suspended load A1 and object B1, and the distance between the suspended load A1 and object B2.

[0019] The alarm device 2 issues an alarm based on the distance calculated by the information processing device 1, i.e., the distance between the suspended load and an object surrounding the suspended load. For example, the alarm device 2 may issue an alarm when the distance between the suspended load A1 and the object B1 or B2 falls below a predetermined threshold (e.g., 2 m). This prevents the operator from accidentally bringing the suspended load A1 into contact with the object B1 or B2 without realizing that the suspended load A1 and the object B1 or B2 are approaching. The alarm may be issued in a manner that can be recognized by the operator. For example, the alarm device 2 may issue an alarm using any one or a combination of sound, light, vibration, and / or image display.

[0020] As described above, the monitoring system 100 includes a measurement device 3 that generates point cloud data including at least points representing the surfaces of the load A1 suspended from the crane CR and the surrounding objects B1 and B2; an information processing device 1 that calculates the distance between the load A1 and the surrounding object B1 or B2 using the point cloud data generated by the measurement device 3; and an alarm device 2 that issues an alarm based on the distance calculated by the information processing device 1. As will be described in detail later, the information processing device 1 can calculate the distance between the load A1 and the surrounding object B1 or B2 with a small amount of calculation. Furthermore, by issuing an alarm based on the calculated distance, it contributes to safety management of the crane CR. Furthermore, if a worker or the like enters the measurement area AR1, the information processing device 1 calculates the distance between the load A1 and the person who entered, thereby contributing to ensuring the safety of the worker or the like.

[0021] The device configuration of the monitoring system 100 can be changed as appropriate. For example, the information processing device 1 and the measuring device 3 can be configured as a single device having a single housing. The same applies to the information processing device 1 and the alarm device 2. Furthermore, the monitoring system 100 may include, for example, in addition to the alarm device 2 that issues an audio warning, a monitoring device that displays an image showing the suspended load and objects around it.

[0022] Furthermore, each device included in the monitoring system 100 may be battery-powered. Furthermore, communication between each device included in the monitoring system 100 may be performed by wireless communication or by wired communication. Furthermore, in the monitoring system 100, the location where the information processing device 1 is placed is arbitrary and is not particularly limited. For example, the information processing device 1 may be placed in a control center or the like located away from the crane CR. Furthermore, for example, the information processing device 1 may be placed in the crane room CR6. In this case, the information processing device 1 may be given the function of the alarm device 2, and the alarm device 2 may be omitted.

[0023] [About the scan pattern] A variety of scan patterns can be applied when scanning the measurement area using the measurement device 3. This will be explained with reference to FIG. 3. FIG. 3 is a diagram showing examples of scan patterns. More specifically, FIG. 3 shows two scan patterns, 31 and 32. In the figure, dashed arrows indicate the scanning paths.

[0024] Scan pattern 31 is a pattern for scanning at equal intervals parallel to the y direction (horizontal direction). Scan pattern 31 is a general pattern also described in Patent Document 1. When scan pattern 31 is applied, a series of contour lines can be identified by sequentially detecting the contours of an object from point cloud data obtained by scanning along the path indicated by the dashed arrow at the farthest back, for example.

[0025] Scan pattern 31 has the drawback that it cannot obtain information about the area between adjacent dashed arrows, which may result in missed detection if an object is present in that area. Scan pattern 32 overcomes this drawback. Scan pattern 32 is a flower-shaped pattern in which multiple spindle-shaped paths intersect at the center. In scan pattern 32, the scanning paths are curved and include paths diagonal to the z-axis and y-axis directions, reducing the possibility of missed detection of an object compared to scan pattern 31. As will be described in detail later, the information processing device 1 can appropriately perform processing such as object detection from the point cloud data obtained by the scan, regardless of whether scan pattern 31 or 32 is applied, or whether a scan pattern other than these is applied.

[0026] [Device configuration] The configuration of an information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the main parts of the information processing device 1. As shown in the figure, the information processing device 1 includes a control unit 10 that controls all parts of the information processing device 1, and a storage unit 11 that stores various data used by the information processing device 1. The information processing device 1 may be, for example, a personal computer, a server, or a workstation.

[0027] The information processing device 1 also includes a communication unit 12 that enables the information processing device 1 to communicate with other devices. The communication unit 12 functions as an interface for controlling the alarm device 2, and also functions as an interface for acquiring point cloud data from the measurement device 3. Note that communication with the alarm device 2 and communication with the measurement device 3 may be performed via separate communication units. The communication unit 12 may perform wired communication or wireless communication. For example, the communication unit 12 may be one that performs communication via a USB (Universal Serial Bus), or one that performs communication via a wireless or wired LAN (Local Area Network).

[0028] The information processing device 1 also includes an input unit 13 that accepts input of various data to the information processing device 1, and an output unit 14 that outputs various data from the information processing device 1. The output unit 14 may be, for example, a display unit that displays images. In this case, the information processing device 1 can display, on the output unit 14, point cloud data, objects detected from the point cloud data, calculation results of distances between objects, and the like.

[0029] The display destination of such information is not particularly limited. For example, it may be displayed on a display device connected to the information processing device 1, or if the alarm device 2 shown in FIG. 2 has a display unit, it may be displayed on that display unit, or it may be displayed on a display device connected to the alarm device 2. When displaying on an external display device, for example, a remote desktop function, a VNC (Virtual Network Computing) function, or an SSH (Secure Shell) X11 forward function may be used. Also, the information may be displayed on an external display device using a dedicated app. Also, a normal imaging device may be placed near the measurement device 3, and images captured by the imaging device may be displayed together with the above-mentioned information.

[0030] The control unit 10 also includes a data acquisition unit 101, a clustering unit 102, a projection unit 103, an object detection unit 104, a contour detection unit 105, an integration unit 106, a distance calculation unit 107, a notification control unit 108, an identity determination unit 109, and a display control unit 110.

[0031] The data acquisition unit 101 acquires point cloud data including at least points representing the surfaces of a plurality of objects. For example, the data acquisition unit 101 may acquire point cloud data generated by the measurement device 3 shown in Fig. 2 through communication via the communication unit 12. The point cloud data may be coordinate values ​​representing the position of each point in three-dimensional space.

[0032] The clustering unit 102 clusters each point included in the point cloud data acquired by the data acquisition unit 101. Clustering of point cloud data can also be described as dividing or classifying the point cloud data. Clustering of point cloud data can be performed based on the positional relationship of each point. In other words, the clustering unit 102 can classify points that are close to each other into one cluster, or points that are located on the same plane into one cluster. Specific clustering methods will be explained in the "Clustering" section below.

[0033] The projection unit 103 projects the point cloud onto a horizontal plane on which a grid is set. Details of the grid and the projection method will be explained later in the section "Projection of point cloud data."

[0034] The object detection unit 104 detects a plurality of objects from point cloud data including at least points indicating the surfaces of the plurality of objects, acquired by the data acquisition unit 101. A specific method for object detection will be described later in the section "Object Detection."

[0035] The contour detection unit 105 detects the contour of the object detected by the object detection unit 104. A specific method for contour detection will be described later in the section "Contour Detection."

[0036] The integrating unit 106 integrates, into a single object, those objects that satisfy a predetermined condition among the multiple objects detected by the object detection unit 104. A specific integration method will be described later in the section "Object Integration."

[0037] Distance calculation unit 107 identifies the opposing contour portions of two objects to be the distance calculation targets among the multiple objects detected by object detection unit 104, using any position on each of the two objects as a reference, and calculates the distance between the contour portions as the distance between the objects. A specific method for calculating the distance will be described later in the section "Calculating the distance."

[0038] The notification control unit 108 issues a notification to the operator of the crane CR or the like according to the distance calculated by the distance calculation unit 107. For example, the notification control unit 108 may cause the alarm device 2 to output an alarm when the distance calculated by the distance calculation unit 107, i.e., the distance between the suspended load and an object around it, becomes equal to or less than a predetermined threshold. Of course, the manner and content of the notification are arbitrary and are not limited to this example. For example, the notification control unit 108 may issue a notification via the output unit 14, or may issue a notification of the distance calculated by the distance calculation unit 107. Alternatively, the notification may be issued by displaying an image. In this case, the notification control unit 108 may be omitted and the display control unit 110 may issue the notification.

[0039] The identity determination unit 109 determines whether the objects detected from the point cloud data acquired at different times are the same object. A specific method for determining identity will be described later in the section "Display Screen Example."

[0040] The display control unit 110 causes the display device to display the object detected by the object detection unit 104. The display device may be provided in the information processing device 1 or in another device. The display of the detected object will be described later in the section "Display Screen Example."

[0041] As described above, the information processing device 1 includes an object detection unit 104 that detects multiple objects from point cloud data that includes at least points indicating the surfaces of the multiple objects, and a distance calculation unit 107 that identifies opposing contour portions of two objects to be the subject of distance calculation among the multiple objects detected by the object detection unit 104, using any position on each of the two objects as a reference, and calculates the distance between the contour portions as the distance between the objects.

[0042] According to the above configuration, the opposing contour portions of two objects to be distance-calculated are identified using an arbitrary position on each of the objects as a reference, and the distance between the contour portions is calculated as the distance between the objects. Therefore, when calculating the distance between the objects, it is not necessary to use all of the points on the contour lines of the detected objects; it is sufficient to use only a portion of them.

[0043] Therefore, the above configuration has the advantage of being able to reduce the amount of calculation required to calculate the distance compared to conventional technology that calculates the distance between objects using all points on the contour lines of each object detected from point cloud data.

[0044] [Clustering] The clustering unit 102 may perform clustering using, for example, RANSAC (Random Sample Consensus) using a surface function. In this case, the clustering unit 102 randomly samples points from the three-dimensional point cloud data and classifies a plurality of sampled points that lie on a predetermined function (here, a surface function) into one cluster. That is, the clustering unit 102 can acquire points on one surface (not all points on the surface) by executing RANSAC once. Next, the clustering unit 102 executes RANSAC again, excluding the points for which the clusters were determined from the original point cloud data, to acquire another cluster. The clustering unit 102 repeats this process multiple times (for example, until the number of remaining points becomes 10% or less of the number of points included in the original point cloud data), thereby classifying the point cloud data into clusters that respectively correspond to a plurality of surfaces.

[0045] Alternatively, for example, the clustering unit 102 may perform clustering of the point cloud data by Euclidean cluster extraction. In this case, the clustering unit 102 first samples one point from the point cloud data. Then, the clustering unit 102 searches for points within a predetermined distance from the sampled point, and then searches for points within a further predetermined distance from the detected point, repeating this process until no points within the predetermined distance are detected, and the multiple points detected in this process are grouped into one cluster. As with the use of RANSAC, the point cloud data can be classified into multiple clusters by repeating Euclidean clustering multiple times.

[0046] [Projection of point cloud data] In order to process the point clouds of each cluster classified by the clustering unit 102 in a regular manner, the projection unit 103 projects the point clouds onto a horizontal plane on which a grid is set. Here, "regular" means that each element is arranged in order. For example, in a two-dimensional photographic image, each pixel is arranged in order, so the two-dimensional image is "regular." On the other hand, points included in each cluster classified by the point cloud data using the above-mentioned RANSAC or Euclidean clustering are not arranged in order and are "irregular." Projection by the projection unit 103 makes it possible to process such irregular points in a regular manner.

[0047] Projection of point cloud data will be described below with reference to Fig. 4. Fig. 4 is a diagram showing an example in which each point included in one cluster is projected onto a horizontal plane (gy-gz plane) on which a grid is set.

[0048] Projecting a point cloud onto a horizontal plane means using only the y and z values ​​of the original point coordinates (x, y, z). The grid coordinates (gy, gz) are integer values ​​calculated based on the (y, z) values. For example, the projection unit 103 may convert the (y, z) values ​​into grid coordinates (gy, gz) using the following formula: gy = y ÷ coefficient gz=z÷coefficient As mentioned above, the grid coordinates (gy, gz) are integer values. The coefficients are adjusted so that one grid element (an element corresponding to one grid coordinate) contains as many points as possible. In the example of FIG. 4, a grid is set with gy ranging from 0 to 13 and gz ranging from 0 to 11. By setting such a grid and projecting point cloud data onto the grid, it becomes possible to process point cloud data measured by an irregular scan, such as the scan pattern 32 shown in FIG. 3, in a regular manner.

[0049] By performing the above-described projection, the range of points classified into a certain cluster can be expressed using grid coordinates. For example, in the column where gy=9 in FIG. 4, points are included in each grid in the range from gz=1 to gz=8. Therefore, the range occupied by the cluster (i.e., the object) in this column extends from grid element 41 at grid coordinate (9,1) to grid element 42 at grid coordinate (9,8). Hereinafter, the minimum gz value of the grids containing points in a certain column will be referred to as gMin_z, and the maximum gz value will be referred to as gMax_z. For example, in the column where gy=2, gMin_z is 4 and gMax_z is 6. The contour detection unit 105 identifies gMin_z and gMax_z for each column in which a point exists. Note that in FIG. 4, grid elements whose gz value is gMin_z or gMax_z are indicated by gray shading.

[0050] 4 also shows an enlarged view of grid element 41. As shown in the figure, grid element 41 includes a total of four points, points 411 to 414. Grid element 41 has a gz value of gMax_z and is located at the top end of the cluster (object). Therefore, point 412, which is located at the top (closest to gz=0) of points 411 to 414, can be considered to be the outermost part of the cluster (object), i.e., a point located on the contour line.

[0051] [Contour detection] As described above, the contour detection unit 105 identifies gMin_z and gMax_z for each column of grid elements onto which points are projected by the projection unit 103. The contour detection unit 105 also detects contours (more precisely, points that make up the contours). The points that make up the contours are the outermost points among the points included in the grid elements that make up the contours, and these points are indicated by black stars in FIG. 4. For example, the above-mentioned point 412 is also indicated by a black star.

[0052] Specifically, in a column of grid elements onto which points are projected, contour detection unit 105 detects the point with the smallest z value included in the grid element gMin_z and the point with the largest z value included in the grid element gMax_z as points constituting the contour in that column. The points constituting the contour may be detected, for example, by a procedure such as that shown in Fig. 5. Fig. 5 is a diagram showing an example of the contour detection procedure.

[0053] In the example of Fig. 5, the contour detection unit 105 executes steps (1) to (4) in the order of the numbers, thereby identifying gMin_z and gMax_z and detecting points that constitute a contour. Specifically, in step (1), the contour detection unit 105 searches for grid elements included in the column where gy = 0 in order from top to bottom (starting with the grid element where gz = 0) and attempts to detect a grid element that includes a point. In the example of Fig. 5, since there is no grid element that includes a point in the column where gy = 0, the contour detection unit 105 proceeds to search the next column (the column where gy = 1). Since there is also no grid element that includes a point in this column, the contour detection unit 105 proceeds to search the next column (the column where gy = 2).

[0054] Here, the column where gy=2 contains a grid element that includes a point. The contour detection unit 105 searches for the grid elements in that column in order from top to bottom (starting from gz=0) and identifies the gz value of grid element 51 that is the first grid element detected that contains the point, which is 4, as the gMin_z of that column. The contour detection unit 105 also detects point 511, which is located at the top (closest to gz=0) of the points included in grid element 51, as a point that constitutes the contour.

[0055] Next, the contour detection unit 105 identifies gMin_z in the column where gy=2 and detects the points that make up the contour, and then proceeds to search the next column (the column where gy=3). In the column where gy=3, as in the column where gy=2, the topmost grid element that contains a point is identified, the value of gz of that grid element is identified as the value of gMin_z, and the topmost point included in that grid element is detected as a point that makes up the contour. The contour detection unit 105 then repeats the same process until it has finished searching all columns (until it has finished searching the column where gy=13 in the example of FIG. 5).

[0056] In the next step (2), the contour detection unit 105 searches sequentially from the top row to the bottom row. This search is performed from the right end to the left end of each row. In the search of step (2), the contour detection unit 105 detects the rightmost point of the grid element containing the first detected point as a point constituting the contour. For example, in the search of the row where gz=0, the grid element where (gy, gz)=(0, 8) is identified, and the rightmost point (the point with the largest y coordinate value) contained in this grid element is detected as a point constituting the contour.

[0057] Note that for grid elements for which contour points have already been detected in step (1), there is no need to detect contour points again. One method for preventing duplicate contour point detection is to set a Done variable for each grid element. Specifically, when contour detection begins, the Done variable for each grid element is set to "Not Corresponded," and the Done variable for grid elements for which contour points have been detected (or gMin_z or gMax_z has been determined) is updated to "Coped." This prevents duplicate contour detection (or gMin_z or gMax_z determination) for "Coped" grid elements. Note that grid elements that are "Coped" are also excluded from detection in steps (3) and (4) described below.

[0058] For grid elements identified in both steps (1) and (2), the contour detection unit 105 may detect the point closest to the upper right corner of the grid elements as a point constituting the contour. Similarly, for grid elements identified in both steps (2) and (3), the contour detection unit 105 may detect the point closest to the lower right corner as a point constituting the contour, and for grid elements identified in both steps (3) and (4), the contour detection unit 105 may detect the point closest to the lower left corner as a point constituting the contour.

[0059] After step (2) is completed, the contour detection unit 105 proceeds to step (3). In step (3), the contour detection unit 105 searches sequentially from the rightmost column to the leftmost column. The search in step (3) is performed in each column from the bottom to the top. In the search in step (3), the contour detection unit 105 identifies the gz value of the grid element containing the first detected point as the gMax_z of that column. Furthermore, the contour detection unit 105 detects the point located at the bottom of the grid elements as a point constituting the contour.

[0060] After step (3) is completed, the contour detection unit 105 proceeds to step (4). In step (4), the contour detection unit 105 searches sequentially from the bottom row to the top row. The search in step (4) is performed from the left end to the right end of each row. In the search in step (4), the contour detection unit 105 detects the leftmost point among the points included in the grid element containing the first detected point as a point constituting the contour.

[0061] Note that points may be detected at positions far from the actual object positions due to noise or other factors contained in the point cloud data measured by the measurement device 3. For this reason, the contour detection unit 105 may search for grid elements that contain points and that are surrounded by a predetermined number of grid elements containing points. For example, for grid element 51 in FIG. 5, the contour detection unit 105 may count how many of the eight grid elements adjacent to grid element 51 contain points, and if the count is equal to or greater than a predetermined threshold, identify the value of gz of grid element 51 as gMin_z. For example, if the threshold is set to 2, there are three grid elements adjacent to grid element 51 that contain points, and therefore the value of gz of grid element 51 is identified as gMin_z.

[0062] When the points constituting the contour of an object are detected using the above procedure, it is possible to draw the contour of the object by connecting the detected points in the order in which they were detected. If it is not necessary to draw the contour, the points constituting the contour may be detected using any procedure. Furthermore, contour detection is not essential, and if contour detection is not performed, the contour detection unit 105 only needs to identify gMin_z and gMax_z using the above steps (1) and (3), and steps (2) and (4) are not necessary.

[0063] As described above, the information processing device 1 may include a contour detection unit 105 that detects points constituting the contours of multiple objects detected by the object detection unit 104 from the point cloud data. As will be described in detail later, when detecting points constituting the contours of an object, the distance calculation unit 107 can identify points constituting the contour portion to be used in calculating the distance from among the points detected by the contour detection unit 105. This makes it possible to reduce the amount of calculation required to calculate the distance between objects compared to when the contour portion to be used in calculating the distance is identified from among all points on the object.

[0064] [Object detection] Here, a specific method of object detection by the object detection unit 104 will be described with reference again to Figure 2. When point cloud data is acquired near the load A1 suspended from the crane CR shown in Figure 2, the object detection unit 104 can detect the load A1, objects B1 and B2, and the ground from the point cloud data. When these are the detection targets, the object detection unit 104 first detects the ground, then detects the suspended load A1, and finally detects objects B1 and B2.

[0065] In detecting the ground, the object detection unit 104 may detect the deepest cluster among the clusters into which the clustering unit 102 has classified the point cloud as the ground. Note that here, a deep position refers to a position that is far from the measurement device 3 (a position with a large x-coordinate value). For example, the object detection unit 104 may calculate the average value of the x-coordinate values ​​of all points included in each cluster into which the clustering unit 102 has classified the point cloud, and detect the cluster with the largest calculated average value as the ground. Alternatively, the object detection unit 104 may detect the cluster with the largest average value of the x-coordinate values ​​of points included in each grid of each cluster as the ground.

[0066] Furthermore, the object detection unit 104 may also detect clusters at a depth close to the cluster detected as the ground as the ground. For example, the object detection unit 104 may detect the deepest cluster and clusters whose distance from the deepest cluster is equal to or less than a predetermined threshold as the ground.

[0067] In detecting the load A1 after detecting the ground, the object detection unit 104 detects the cluster corresponding to the load A1 from among the clusters into which the clustering unit 102 classified the point cloud, excluding the cluster corresponding to the ground. For example, if the measurement device 3 is attached directly above the hook CR5, the point corresponding to the load A1 will be located near the center when the point cloud data is projected onto the yz plane. In this way, the distribution range of the points corresponding to the load A1 can be specified in advance, so the object detection unit 104 simply searches for the cluster corresponding to the load A1 within that distribution range.

[0068] For example, when the points of each cluster are projected onto the yz plane, the object detection unit 104 may detect, as the cluster corresponding to the load A1, the cluster whose center is within the above distribution range, whose area on the yz plane is larger than the upper surface of the hook CR5, and whose depth is shallowest. Note that, taking into consideration the swinging of the hook CR5 and the load A1, it is preferable to set the above distribution range broadly.

[0069] Then, the object detection unit 104 classifies the point cloud into clusters by the clustering unit 102, excluding the cluster corresponding to the ground and the cluster corresponding to the suspended load A1, and classifies the remaining clusters as clusters of the objects B1 and B2. Through the above processing, the ground, the suspended load A1, and the objects B1 and B2 are detected from the point cloud data.

[0070] [Object Integration] When point cloud data is clustered, a single object may be divided into multiple clusters. Furthermore, an object consisting of multiple surfaces at different depths, such as the suspended load A1 in FIG. 2, may have each surface classified into a different cluster. The integrating unit 106 integrates, into a single object, objects that are actually one object but whose individual parts have been classified into different clusters and thus have been detected as objects with different parts.

[0071] The integration may be performed on clusters (objects) that satisfy a predetermined condition. For example, if the heights of a set of objects to be determined as to whether to integrate them and the depths of the surfaces of the objects satisfy a predetermined condition and the grids of the set of objects overlap, the integration unit 106 may integrate the objects into a single object.

[0072] The above conditions regarding the height of an object and the depth of its surface will now be explained with reference to FIG. 6. FIG. 6 is a diagram explaining the conditions regarding the height of an object and the depth of its surface, which are among the conditions for integrating objects. FIG. 6 shows examples where the suspended load is A2 and where the suspended load is A3 with an object B3 located below it. Note that the cables and hooks suspended from the suspended loads A2 and A3 are not shown.

[0073] The height of the load A2 is H1, and the difference in depth between the surface A21 and the surface A22 of the load A2 is d1 (d1

[0074] On the other hand, load A3 has the same height H1 as load A2, but the difference in depth between surface A31, which is the top surface of load A3, and surface B31, which is the top surface of object B3, is d2 (d2>H1). Thus, a surface whose depth difference with surface A31 is greater than the height of load A3 is not a surface on load A3.

[0075] Therefore, when a surface of an object whose height is known is detected, the integrating unit 106 may determine that a surface whose difference in depth from the surface is less than the above height satisfies the integrating condition. On the other hand, the integrating unit 106 may determine that a surface whose difference in depth from the surface of an object whose height is known is greater than the height does not satisfy the integrating condition.

[0076] Furthermore, the integrating unit 106 may make the above determination by adding a predetermined margin to the height of the object. That is, the integrating unit 106 may determine that a surface that satisfies the condition "difference in surface depth<height of object+(predetermined margin)" satisfies the integration condition. The height of the object may be input via the input unit 13, for example, and stored in advance in the information processing device 1.

[0077] 6, the integrating unit 106 calculates the difference d1 in depth between the suspended load A2 detected by the object detecting unit 104 (more precisely, the surface A21 at the shallowest position among the upper surfaces of the suspended load A2) and the surface A22 detected as an object separate from the suspended load A2. The integrating unit 106 then compares d1 with a value obtained by adding a predetermined margin to a pre-stored height H1, and determines that the integrating condition is met.

[0078] ​On the other hand, in the case of the suspended load A3, the integrating unit 106 calculates the difference d2 in depth between the suspended load A3 detected by the object detecting unit 104 (more precisely, the surface A31 at the shallowest position among the upper surfaces of the suspended load A3) and the surface B32 detected as an object separate from the suspended load A3. The integrating unit 106 then compares d2 with a value obtained by adding a predetermined margin to the pre-stored height H1, and determines that the condition for integrating is not met.

[0079] Next, the conditions for grid superimposition will be explained with reference to Fig. 7. Fig. 7 is a diagram showing an example in which points included in each cluster to be determined as to whether or not to merge are projected onto a horizontal plane (gy-gz plane) on which a grid is set. Fig. 7 shows two examples, Example 71 and Example 72.

[0080] In both Example 71 and Example 72, the point cloud classified into Cluster 1 and the point cloud classified into Cluster 2 are projected onto the gy-gz plane. At this stage, the point cloud classified into Cluster 1 and the point cloud classified into Cluster 2 are detected as separate objects.

[0081] Here, the grid enclosed by a solid line includes both points classified into cluster 1 and points classified into cluster 2. When the object corresponding to the point cloud classified into cluster 1 and the object corresponding to the point cloud classified into cluster 2 are one object, such grid overlap occurs. Note that grid overlap means that there is a grid that includes points on both clusters (objects).

[0082] Therefore, when the number of grids including points on two clusters (objects) is equal to or greater than a threshold, the integrating unit 106 may integrate the two objects into one object. For example, the integrating unit 106 can calculate the number of grids by the following calculation.

[0083] First, the integrating unit 106 compares the gMax_z and gMin_z of the grids for two clusters for which integration is being determined. For example, if clusters 1 and 2 are being determined for which integration is to be performed, the integrating unit 106 compares the gMax_z and gMin_z of cluster 1 with the gMax_z and gMin_z of cluster 2. In this case, the size relationship between gMax_z of cluster 1 and gMax_z of cluster 2, the size relationship between gMax_z of cluster 1 and gMin_z of cluster 2, the size relationship between gMin_z of cluster 1 and gMax_z of cluster 2, and the size relationship between gMin_z of cluster 1 and gMin_z of cluster 2 are compared. Note that the size relationships are compared when the values ​​of gMax_z and gMin_z of cluster 1 and gMax_z and gMin_z of cluster 2 are both valid.

[0084] Next, the integrating unit 106 calculates the number of grids to be superimposed according to the result of the comparison. Specifically, the integrating unit 106 calculates the number of grids to be superimposed in one column by any of the following methods 1 to 4. The integrating unit 106 performs this process for the grids in each column to calculate the total number of grids to be superimposed. 1: (gMin_z of cluster 1) ≦ (gMin_z of cluster 2) and If (gMin_z of cluster 2) ≤ (gMax_z of cluster 1), then The number of grids to be superimposed is (gMax_z of cluster 1)-(gMin_z of cluster 2)+1. 2: (gMin_z of cluster 2) ≦ (gMin_z of cluster 1) and If (gMin_z of cluster 1) ≤ (gMax_z of cluster 2), then The number of grids to be superimposed is (gMax_z of cluster 2)-(gMin_z of cluster 1)+1. 3: (gMin_z of cluster 1) ≦ (gMin_z of cluster 2) and If (gMax_z of cluster 2) ≤ (gMax_z of cluster 1), The number of grids to be superimposed is (gMax_z of cluster 2)-(gMin_z of cluster 2)+1. 4: (gMin_z of cluster 2) ≦ (gMin_z of cluster 1) and If (gMax_z of cluster 1) ≤ (gMax_z of cluster 2), The number of grids to be superimposed is (gMax_z of cluster 1)-(gMin_z of cluster 1)+1.

[0085] For example, in the column gy=6 in example 71 in Figure 7, gMin_z=2 in cluster 1, gMax_z=8 in cluster 1, gMin_z=8 in cluster 2, and gMax_z=9 in cluster 2. These values ​​satisfy condition 1 above, so the number of grids to be superimposed in this column is calculated as (gMax_z=8 in cluster 1) - (gMin_z=8 in cluster 2) + 1 = 1. The same applies to the columns gy=7 to gy=10. In example 71, the total number of grids to be superimposed is 5.

[0086] On the other hand, in the column of gy=8 in example 72 in Figure 7, gMin_z=1 in cluster 1, gMax_z=8 in cluster 1, gMin_z=3 in cluster 2, and gMax_z=8 in cluster 2. Since these values ​​satisfy the above condition 3, the number of grids to be superimposed in this column is calculated as (gMax_z=8 in cluster 2) - (gMin_z=3 in cluster 2) + 1 = 6. In example 72, the total number of grids to be superimposed is 6.

[0087] If the total number of grids calculated as described above, i.e., the number of grids including points on each of the two objects, is equal to or greater than a predetermined threshold, the integrating unit 106 may integrate the two objects into a single object. For example, if the threshold is set to 3, the objects in cluster 1 and cluster 2 are integrated in both Examples 71 and 72. This process is repeated until only clusters (objects) that do not satisfy the integration conditions remain.

[0088] Note that if the size of the grid elements is set small, even if one object is detected as multiple separate objects, it is difficult for points on each object to be included in a single grid. In this case, the gMin_z and gMax_z of at least one of the two objects to be integrated may be corrected to larger values ​​before the above-mentioned determination is made. The correction method is not particularly limited, and may be, for example, correction by adding a predetermined value or correction by multiplying by a predetermined value greater than 1.

[0089] As described above, when an object is detected as multiple separate objects, the detected objects will be located close to each other on their contours, and the grid on the contours will contain points on each object.

[0090] Therefore, when the points on the multiple objects detected by the object detection unit 104 are projected onto a horizontal plane on which grids are set, if the number of grids including points on the two objects is equal to or greater than a threshold, the integration unit 106 integrates the two objects into a single object. This makes it possible to correctly handle multiple objects as a single object even if one object is detected as being separated into multiple objects.

[0091] Furthermore, as described above, in addition to the above conditions, the integrating unit 106 may integrate two objects into a single object when the difference in depth between the surfaces of the two objects is smaller than the height of one of the objects (or the height of the object plus a predetermined margin). This makes it possible to prevent objects that are vertically separated from each other from being integrated into a single object.

[0092] Note that some conventional cluster integration determination techniques determine whether or not integration is possible based on the width of each cluster being evaluated. For example, in example 72 in Figure 7, cluster 1 exists in the range from gy = 2 to gy = 8, so its width in the gy direction is 7. On the other hand, cluster 2 exists in the range from gy = 8 to gy = 10, so its width in the gy direction is 2. Therefore, in example 72 in Figure 7, even when the conventional method is applied, clusters 1 and 2 are determined to overlap, and can be correctly integrated.

[0093] In contrast, when the conventional method is applied to Example 71, the width of cluster 1 in the gy direction is 10, from gy=2 to gy=11, which is larger than the actual width of cluster 1. As such, depending on the distribution range of the clusters, when the conventional method is applied, there is a risk that what are actually two objects may be erroneously integrated into one object. In this regard, the integration method by the integration unit 106 described above makes it possible to accurately determine whether or not to integrate, regardless of the distribution range of the clusters.

[0094] [Distance calculation] The method for calculating the distance between objects by distance calculation unit 107 will be described with reference to FIG. 8. FIG. 8 is a diagram illustrating the method for calculating the distance between objects. In FIG. 8, the distribution range of points on each of two objects for which distance calculation is to be performed is indicated by dashed lines 81 and 82. Dashed lines 81 and 82 can also be considered to be the range of a cluster corresponding to each object. Hereinafter, the object corresponding to dashed line 81 will be referred to as the first object, and the object corresponding to dashed line 82 will be referred to as the second object. The first object is, for example, a suspended load, and the second object is, for example, an object located near the suspended load. Although not shown in the figure, the dashed lines 81 and 82 contain point clouds classified into each cluster. These point clouds are projected onto a horizontal plane (yz plane or gy-gz plane).

[0095] When calculating the distance between the first object and the second object, distance calculation unit 107 identifies center position 811 of the first object and center position 821 of the second object. The method of identifying center positions 811 and 821 is not particularly limited. For example, distance calculation unit 107 may determine the average value of all points within dashed line 81 as center position 811 of the first object. Alternatively, distance calculation unit 107 may determine the y coordinate of center position 811 of the first object by subtracting the minimum value from the maximum value of the y coordinates of all points within dashed line 81 and dividing the result by 2, and may determine the z coordinate calculated in a similar manner as the z coordinate of center position 811 of the first object. The same applies to the second object.

[0096] Next, distance calculation unit 107 identifies, from the point cloud data projected onto the horizontal plane, point P1 on the first object that is closest to center position 821 of the second object, and point P2 on the second object that is closest to center position 811 of the first object. The distance between point P1 and point P2 is longer than the distance between the first object and the second object.

[0097] Therefore, the distance calculation unit 107 identifies a point P2' on the second object that is closest to the point P1 and a point P1' on the first object that is closest to the point P2 from the point cloud data projected onto the horizontal plane.

[0098] The line segment P1-P1' connecting point P1 and point P1' thus identified, and the line segment P2-P2' connecting point P2 and point P2' correspond to the opposing contour portions of the first object and the second object, as shown in Fig. 8. Therefore, distance calculation unit 107 can obtain the distance between the first object and the second object by calculating the distance between line segment P1-P1' and line segment P2-P2'.

[0099] Here, when the contour detection unit 105 has detected a contour, the distance calculation unit 107 simply identifies points P1, P1', P2, and P2' from among the points on the detected contour in the point cloud data projected onto the horizontal plane. This reduces the amount of calculation required to identify these points compared to identifying points P1, P1', P2, and P2' from all point cloud data projected onto the horizontal plane.

[0100] It should be noted that it is not essential to detect contours by contour detection unit 105. Even when points P1, P1', P2, and P2' are identified from the entire point group projected onto a horizontal plane, the amount of calculation is reduced compared to the conventional method of calculating the distance between each point on the contour of one object and each point on the contour of the other object.

[0101] Furthermore, the reference positions for identifying the contour portions can be any positions on each of the two objects for which distances are to be calculated, and are not limited to the center positions 811 and 821.

[0102] In other words, when the two objects to be calculated as the distance are a first object and a second object, the distance calculation unit 107 may identify, from the point cloud data projected onto a horizontal plane, a point P1 on the first object that is closest to an arbitrary position on the second object, and a point P2 on the second object that is closest to an arbitrary position on the first object, as well as a point P2' on the second object that is closest to point P1, and a point P1' on the first object that is closest to point P2, and calculate the distance between the line segments P1-P1' and P2-P2', which are opposing contour portions of the first object and the second object.

[0103] According to the above configuration, the opposing contour portions of the first object and the second object can be identified with a small amount of calculation by finding four points P1, P2, P1', and P2', thereby reducing the amount of calculation required to calculate the distance between the objects.

[0104] Further, distance calculation unit 107 may set search range 83 based on center positions 811 and 821, and identify contour portions to be used for distance calculation, i.e., points P1, P1', P2, and P2', within set search range 83. In the example of Fig. 8, search range 83 is set by enlarging a rectangle with center positions 811 and 821 as diagonal vertices.

[0105] In this way, the distance calculation unit 107 may specify the contour portion to be used for calculating the distance within a search range set between the center positions of the two objects to be distance calculated. This configuration can reduce the amount of calculation required for calculating the distance between the objects, compared to specifying the contour portion to be used for calculating the distance from all points on the objects (or all points on the contour detected by the contour detection unit 105).

[0106] [Display screen example] Fig. 9 is a diagram showing an example of a display screen of an object detected by the object detection unit 104. Objects 911 and 912 detected by the object detection unit 104 are depicted in an image 91 shown in Fig. 9. The display control unit 110 generates such an image based on the detection results of the object detection unit 104 and displays it on the display device.

[0107] Specifically, image 91 includes the contour line of object 911 on the yz plane, points on the contour line, and a point cloud on object 911 (projected onto the yz plane), as well as the character string "N2 0.3 7.12 -0.4 0.21." Similarly, image 91 includes the contour line of object 912, points on the contour line, and a point cloud on object 912, as well as the character string "7.69 0.13 0.09." Here, object 912 is a suspended load, and object 911 is an object located around the suspended load. Furthermore, the points that make up the contour portion used in calculating the distance (points corresponding to points P1, P1', P2, and P2' in FIG. 8) are shown in black so that they can be distinguished from the other points.

[0108] The contour line can be drawn based on the detection result of the contour detection unit 105. In other words, the display control unit 110 can draw the contour line of an object by connecting the points that make up the contour detected by the contour detection unit 105 in the order in which they were detected.

[0109] The character string "N2 0.3 7.12 -0.4 0.21" indicates the identification information of the object 911, the distance from the object 911 to the object 912 (hanging load), and the coordinates of the center position of the object 911. Specifically, "N2" is the identification information of the object 911, "0.3" is the distance (unit: meters, for example) from the object 911 to the object 912, and "7.12 -0.4 0.21" is the coordinates of the center position of the object 911 in the xyz space. On the other hand, the character string "7.69 0.13 0.09" indicates the coordinates of the center position of the object 912 in the xyz space. In this way, the display control unit 110 may display various information related to the detected objects.

[0110] Furthermore, in addition to the above-mentioned information, the display control unit 110 may also display information such as the distance from the measuring device 3 to an object. Furthermore, when the distance between the suspended load and the surrounding object calculated by the distance calculation unit 107 is equal to or less than a predetermined threshold, the display control unit 110 may display warning information that notifies that there is a risk of the suspended load coming into contact with the surrounding object. Furthermore, when a normal photographing device is placed near the measuring device 3, the display control unit 110 may display an image photographed by the photographing device along with the above-mentioned various information. Note that the user may be able to select which of the above-mentioned various information to display by, for example, an input operation via the input unit 13.

[0111] Furthermore, the display control unit 110 may display the suspended load and the surrounding objects in different colors, and when multiple surrounding objects are detected, may display each of these objects in a different color.

[0112] Here, the position of the suspended load and the positions of objects around the suspended load may change over time. For this reason, it is desirable that the measurement device 3 generates time-series point cloud data, and the information processing device 1 detects objects at each point in time from the point cloud data at that time and calculates the distance between the objects. Then, it is desirable that the display control unit 110, after drawing an object detected from point cloud data measured at a certain point in time, updates the previous drawing content based on the latest detection results when an object is detected from point cloud data measured at the next point in time.

[0113] For example, when the measurement device 3 generates point cloud data in a time series, the data acquisition unit 101 may acquire point cloud data generated at a certain point in time, write the data to a frame buffer, and delete the old frame buffer each time new point cloud data is acquired. This allows object detection and rendering to be performed using the latest point cloud data at all times.

[0114] Furthermore, when an object is detected from point cloud data measured at one time point and also from point cloud data measured at the next time point, the identity determination unit 109 may determine whether the objects detected from the point cloud data measured at each time point are the same object. Then, the display control unit 110 may display the objects determined to be the same object by the identity determination unit 109 in a common display mode.

[0115] This allows the user to easily recognize that objects detected from point cloud data acquired at different times are the same object. For example, the display control unit 110 may display the entire object or the outline of the object that the identity determination unit 109 has determined to be the same object in the same color. This allows the user to easily recognize the correspondence between objects before and after updating the drawing content.

[0116] To determine identity, the number of superimposed grids can be used, as in the case of integrating objects. Specifically, the identity determination unit 109 determines identity using gMax_z and gMin_z calculated by projecting points on each object detected from point cloud data measured at a certain point in time onto a horizontal plane (gy-gz plane) on which grids are set, and gMax_z and gMin_z calculated by projecting points on each object detected from point cloud data measured next onto the horizontal plane (gy-gz plane) on which grids are set.

[0117] For example, the identity determination unit 109 may calculate the number of overlaps between the grid of a certain object at a certain time point and the grid of a certain object at the next time point, and if the calculated value is equal to or greater than a predetermined threshold, determine that the objects are the same.

[0118] Furthermore, for example, the identity determination unit 109 may determine that two objects (objects whose difference in the number of grids occupied by each object is within a threshold) are identical when the value obtained by dividing the number of grids overlapping between the two objects whose identity is to be determined by the number of grids occupied by the object on the grid is equal to or greater than a predetermined threshold. Note that the above area may be the area of ​​either of the two objects whose identity is to be determined.

[0119] As described above, the information processing device 1 may be provided with an identity determination unit 109 that determines whether the objects detected at each time point are the same object based on the number of grids that include both points on the object detected at each time point by the object detection unit 104 from point cloud data acquired at a certain time point and points on the object detected by the object detection unit 104 from point cloud data acquired at another time point when the points are projected onto a horizontal plane on which grids are set.

[0120] If objects detected from point cloud data acquired at different times occupy similar positions and areas, they are considered to be the same object. Therefore, the above configuration makes it possible to appropriately determine whether the detected objects are the same object. This configuration also makes it possible to track the same object from point cloud data over time.

[0121] [Processing flow] The flow of processing (distance calculation method) executed by the information processing device 1 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of processing executed by the information processing device 1. Fig. 10 shows processing immediately after the measurement of point cloud data by the measuring device 3 is completed.

[0122] In S11, the data acquisition unit 101 acquires point cloud data measured by the measuring device 3. For example, the data acquisition unit 101 may acquire the point cloud data from the measuring device 3 by communication via the communication unit 12.

[0123] In S12, the clustering unit 102 divides the point cloud data acquired in S11 into clusters. As described above, dividing the point cloud data into clusters means classifying the point cloud data into multiple clusters. Next, in S13, the projection unit 103 projects the point cloud data of each cluster onto a horizontal plane on which a grid is set.

[0124] In S14, the object detection unit 104 detects the ground. More specifically, the object detection unit 104 detects a cluster corresponding to the ground from among the multiple clusters used to divide the point cloud data in S12.

[0125] In S15 (object detection step), the object detection unit 104 detects multiple objects. More specifically, the object detection unit 104 excludes clusters detected in S14 as corresponding to the ground from the multiple clusters into which the point cloud data was classified in S12, and then detects a cluster corresponding to the suspended load from among those clusters. The object detection unit 104 then detects each of the clusters that do not correspond to either the suspended load or the ground as an object around the suspended load.

[0126] In S16, an integration process is performed to integrate a single object from among the multiple objects detected in S15. In the integration process, first, the contour detection unit 105 detects gMax_z, gMin_z, and points constituting the contour for each cluster detected as an object in S15. Then, the integration unit 106 determines whether or not to integrate the objects using gMax_z and gMin_z of each object. As described above, the determination of whether or not to integrate may also take into account the difference between the height of the object and the depth of the object's surface. This process is performed first for the suspended load, and then for the objects surrounding the suspended load. The integration process for the objects surrounding the suspended load is repeated until only clusters (objects) that do not satisfy the integration conditions remain.

[0127] In S17 (distance calculation step), the distance calculation unit 107 identifies opposing contour portions of two objects to be subjected to distance calculation among the multiple objects detected in S15, using any position on each of the two objects as a reference, and calculates the distance between the contour portions as the distance between the objects. The method for identifying opposing contour portions is as explained with reference to FIG. 8, and therefore the explanation will not be repeated here. Specifically, in S17, the distance calculation unit 107 calculates the distance between the suspended load and the objects surrounding it among the multiple objects detected in S15. Note that, if multiple objects surrounding the suspended load are detected, the distance calculation unit 107 calculates the distance between the suspended load and each of the multiple detected objects.

[0128] In S18, the notification control unit 108 determines whether an alarm is necessary based on the distance calculated in S17. For example, the notification control unit 108 may determine that an alarm is necessary (YES in S18) when the distance calculated in S17 is equal to or less than a predetermined threshold. If the determination in S18 is YES, the process proceeds to S19, and if the determination in S18 is NO, the process proceeds to S20. In S19, the notification control unit 108 issues an alarm that the distance between the suspended load and the surrounding objects is short. For example, the notification control unit 108 may instruct the alarm device 2 to issue an alarm.

[0129] In S20, the display control unit 110 causes the display device to display the objects detected in S15. Note that, among the objects detected in S15, the display control unit 110 causes the objects that were integrated in S16 to be displayed in an integrated state. Furthermore, the display control unit 110 may also display various information such as the distance calculated in S17 in addition to information indicating the detected objects.

[0130] Furthermore, when executing the process of S20, if there are results of object detection at a previous time point, the identity determination unit 109 may determine the identity of the objects detected at each time point prior to the process of S20. In this case, in S20, the display control unit 110 displays the objects determined to be the same object by the identity determination unit 109 in a common display mode.

[0131] In S21, the data acquisition unit 101 determines whether or not to terminate the process. If the determination in S21 is YES, the process of FIG. 10 terminates. On the other hand, if the determination in S21 is NO, the process returns to S11, and the data acquisition unit 101 acquires new point cloud data (measured at a later time than the previously acquired data). The termination condition in S21 may be determined in advance. For example, the termination condition may be that a user's termination operation has been detected or that a predetermined time has elapsed since the start of the process.

[0132] [Modification] Some of the above-described processing performed by the information processing device 1 may be configured to be performed by another computer system. For example, the processing of S14 and S15 in FIG. 10 may be configured to be performed by another computer system. In this case, the information processing device 1 transmits information required for the calculation to the other computer system via communication over a network. Then, the information processing device 1 receives the calculation results transmitted from the other computer system. In this way, the distance calculation method according to this embodiment can be executed by one or more information processing devices.

[0133] [Software implementation example] The functions of the information processing device 1 can be realized by a program (distance calculation program) that causes a computer to function as the information processing device 1, and that causes a computer to function as each control block of the information processing device 1 (particularly each part included in the control unit 10).

[0134] In this case, the information processing device 1 includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0135] The program may be stored non-transitory on one or more computer-readable storage media. The storage media may or may not be included in the information processing device 1. In the latter case, the program may be supplied to the information processing device 1 via any wired or wireless transmission medium.

[0136] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0137] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0138] 1. Information processing equipment 104 Object detection unit 105 Contour detection unit 107 Distance calculation unit 109 Identity determination section 110 Display control unit 2 Alarm device 3. Measuring equipment 100 Surveillance System

Claims

1. an object detection unit that detects a plurality of objects from point cloud data that includes at least points indicating the surfaces of the plurality of objects; and a distance calculation unit that identifies opposing contour portions of two objects to be subjected to distance calculation among the plurality of objects detected by the object detection unit using an arbitrary position on each of the two objects as a reference, and calculates the distance between the contour portions as the distance between the objects.

2. When the two objects to be distance calculation targets are a first object and a second object, The distance calculation unit Identifying a point P1 on the first object that is closest to an arbitrary position on the second object and a point P2 on the second object that is closest to an arbitrary position on the first object from the point cloud data projected onto a horizontal plane, and Identifying a point P2' on the second object that is closest to the point P1 and a point P1' on the first object that is closest to the point P2; 2. The information processing apparatus according to claim 1, wherein the apparatus calculates a distance between line segments P1-P1' and P2-P2' that are opposing contour portions of the first object and the second object.

3. a contour detection unit that detects points constituting contours of the plurality of objects detected by the object detection unit from the point cloud data; The information processing apparatus according to claim 1 , wherein the distance calculation section identifies points that form the contour portion from among the points detected by the contour detection section.

4. The information processing apparatus according to claim 1 , wherein the distance calculation unit specifies the contour portion within a search range set between center positions of the two objects to be distance-calculated.

5. 2. The information processing device according to claim 1, further comprising an integration unit that, when each point on the plurality of objects detected by the object detection unit is projected onto a horizontal plane on which grids are set, integrates the two objects into a single object if the number of grids that each include points on two objects is equal to or greater than a threshold value.

6. 2. The information processing device according to claim 1, further comprising an identity determination unit that determines whether the objects detected at each time point are the same object based on the number of grids that include both points on the object detected at each time point by the object detection unit from the point cloud data acquired at a certain time point and points on the object detected by the object detection unit from the point cloud data acquired at another time point when the grids are set on a horizontal plane.

7. a display control unit that causes a display device to display an image showing the object detected by the object detection unit; The information processing device according to claim 6 , wherein the display control unit displays the objects that the identity determination unit determines to be the same object in a common display mode.

8. a measurement device that generates point cloud data including at least points that represent the surface of the load suspended by the crane and objects around it; The information processing device according to claim 1 , wherein the information processing device calculates a distance between the suspended load and the object around the suspended load using the point cloud data generated by the measurement device; and an alarm device that issues an alarm according to the distance calculated by the information processing device.

9. A distance calculation method executed by one or more information processing devices, an object detection step of detecting a plurality of objects from point cloud data including at least points indicating the surfaces of the plurality of objects; a distance calculation step of identifying opposing contour portions of two objects to be subjected to distance calculation among the plurality of objects detected in the object detection step, using any position on each of the two objects as a reference, and calculating the distance between the contour portions as the distance between the objects.

10. 2. A distance calculation program for causing a computer to function as the information processing device according to claim 1, the distance calculation program causing the computer to function as the object detection unit and the distance calculation unit.

Images