Background detection device, background detection method, and object detection device

Abstract

To provide a background detection device, a background detection method and an object detection device that enable handling a complicated structure, and can detect a background area automatically.SOLUTION: In an object detection device including: a main body part provided in a lower part of a strut as a control box and the like; and a LiDAR installed in an upper part of the strut, a background detection unit of the main body part comprises: an input part 110 to which point group information is inputted from a LiDAR scanning a scan range while intermittently irradiating the scan range with light, receiving reflection light of the light and acquiring the point group information based on the reflection light; an image generation unit 111 that detects a background area serving as a stationary area within a given space on the basis of the point group information inputted to the input part 110 at a given time interval; a synthesis unit 112; and a circle area generation unit 113. Then, the image generation unit 111, the synthesis unit 112 and the circle area generation unit 113 are configured to make the background area approximate to a shape composed of a plurality of columns on the basis of the point group information.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a background detection device and a background detection method for detecting a background area for use in, for example, an object detection device that detects an object on a road.

Background Art

[0002] For example, as an object detection device that detects an object on a road, a method of detecting an object using a cluster generated by clustering point clouds detected by LiDAR (Light Detection And Ranging) is known.

[0003] For example, in Patent Document 1, when it is determined that a cluster is over-connected based on an image target detected from an ambient light image and a cluster obtained by clustering point clouds, the over-connected cluster is divided, and when it is determined that a cluster is over-segmented, two or more clusters existing in a portion of the point cloud corresponding to the image target are combined, so that an object can be detected with high accuracy in correct units.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As in the method described in Patent Document 1, by detecting over-connection, it is possible to remove the influence of stationary fixed objects (background areas) such as houses and walls, but there is a problem that the processing becomes complicated.

[0006] Therefore, one method is to detect only moving objects by detecting the background region in advance and excluding the point cloud from that background region. However, if, for example, an area with trees is defined as the background region in a planar manner, pedestrians walking under the branches of the trees may not be detected.

[0007] Furthermore, the fixed objects that constitute the background area are not limited to relatively simple shapes such as walls. For example, vegetation is inherently complex in shape, and its form changes due to growth and the presence or absence of leaves depending on the season. There are also environmental changes such as road construction and snowdrifts. Traditionally, such environmental changes were checked manually or the background area was remeasured, but this is very time-consuming, so automation is desirable.

[0008] Therefore, the present invention aims to provide a background detection device, a background detection method, and an object detection device that can handle complex structures and automatically detect background areas. [Means for solving the problem]

[0009] The invention described in claim 1, made to solve the above problems, is a background detection device comprising: an input unit to which point cloud information is input from a sensor installed on the ground that intermittently irradiates a predetermined space with light and receives reflected light to acquire point cloud information based on the reflected light; and a detection unit that detects a background region which is the area of ​​a fixed object in the predetermined space based on the point cloud information input to the input unit at predetermined time intervals, wherein the detection unit approximates the background region to a shape consisting of a plurality of cylinders based on the point cloud information. [Effects of the Invention]

[0010] According to the present invention, since the background region is approximated as a shape consisting of multiple cylinders, objects with complex shapes can be constructed using simpler shapes, enabling the representation of complex structures. Furthermore, by inputting point cloud information, the background region can be automatically detected. [Brief explanation of the drawing]

[0011] [Figure 1] This is an installation example of an object detection device according to an embodiment of the present invention. [Figure 2] This is a block diagram of the object detection device shown in FIG. 1. [Figure 3] This is a block diagram of the background detection unit shown in FIG. 2. [Figure 4] This is a flowchart of the operation of the background detection unit shown in FIG. 3. [Figure 5] This is a schematic explanatory diagram of the operation of creating a time-series multi-stage projection image list shown in FIG. 4. [Figure 6] This is a flowchart of the operation of creating a time-series multi-stage projection image list shown in FIG. 4. [Figure 7] This is an explanatory diagram of spatial division at the time of generating a multi-stage projection image. [Figure 8] This is an explanatory diagram of the operation of setting a binary value. [Figure 9] This is a schematic explanatory diagram of the synthesis operation of the time-series multi-stage projection image list shown in FIG. 4. [Figure 10] [[ID=API]]This is a flowchart of the operation of creating a time-series multi-stage projection image list shown in FIG. 4. [Figure 11] This is an explanatory diagram of the operation of generating a synthesized multi-stage projection image. [Figure 12] This is a schematic explanatory diagram of the operation of creating a circular region shown in FIG. 4. [Figure 13] This is a flowchart of the operation of creating a circular region shown in FIG. 4. [Figure 14] This is an explanatory diagram of a label region and a bounding rectangle region. [Figure 15] This is an explanatory diagram when dividing a label region. [Figure 16] This is an explanatory diagram when extending a bounding rectangle region. [Figure 17] This is an explanatory diagram of a circular region. [Figure 18] This is an explanatory diagram of a label region not included in the circular region. [Figure 19] This is a flowchart of the operation of the object detection device shown in FIG. 1.

Mode for Carrying Out the Invention

[0012] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 to 19. FIG. 1 is an installation example of an object detection device according to an embodiment of the present invention.

[0013] The object detection device 10 shown in FIG. 1 is installed on a support column P fixed, for example, near an intersection. The object detection device 1 detects moving objects such as pedestrians and vehicles passing through the intersection. That is, the object detection device 10 is installed on the ground. Being installed on this ground means being fixed to the ground, whether directly or indirectly, that is, being installed on an object fixedly installed on the ground such as a building rather than a moving object such as a vehicle.

[0014] The object detection device 10 includes a main body part 1 and a LiDAR 2. The main body part 1 is provided as a control box or the like at the lower part of the support column P. As shown in the block diagram of FIG. 2, the main body part 1 includes a control part 11 and a communication part 12.

[0015] The control part 11 is composed of, for example, a computer or the like. The control part 11 is in charge of the overall control of the object detection device 10. In FIG. 1, the control part 11 functionally includes a removal part 11a, a detection part 11b, and a background detection part 11c.

[0016] The removal part 11a removes the point cloud information outside the detection area based on the detection result of the background detection part 11c described later for the point cloud information acquired by the LiDAR 2. The detection part 11b detects an object based on the point cloud information remaining after the removal by the removal part 11a. The background detection part 11c detects a range that becomes a background area, which is an area excluded from the object detection target in the detection part 11b. That is, the background detection part 11c functions as a background detection device according to this embodiment.

[0017] The communication unit 12 outputs information about the object detected by the control unit 11 to the outside. The communication unit 12 can be configured as, for example, a wireless communication device (it may also be a wired communication device). Information about the object includes the type of object, direction of movement, and speed of movement. Specific examples of output destinations include moving objects such as vehicles passing near the support column P, and traffic control centers.

[0018] LiDAR2 is a sensor that scans a predetermined space by intermittently irradiating it with laser light, receives the reflected light, and acquires point cloud information (3D point cloud information) based on the reflected light. LiDAR2 discretely measures the distance to objects in the space within the irradiation range based on the reflected light of the irradiated laser light, and recognizes the position and shape of the objects as a 3D point cloud. In this embodiment, as shown in Figure 1, LiDAR2 is installed on the top of the support column P and scans the road surface R (road surface R and the space above it) as the predetermined space.

[0019] In this embodiment, the object detection device and the background detection device are configured as an integrated unit, but the background detection device may be a separate unit. In other words, the background detection device may be any other server device, etc., as long as it can acquire point cloud information from LiDAR2. The object detection device 10 should acquire information about the background region from the background detection device and execute the object detection process described later.

[0020] Next, the background region detection method in the background detection unit 11c with the above configuration will be explained with reference to Figures 3 to 18. Figure 3 is a functional configuration diagram of the background detection unit 11c. The background detection unit 11c comprises an input unit 110, an image generation unit 111, a synthesis unit 112, and a circular region generation unit 113.

[0021] The input unit 110 receives point cloud information acquired by LiDAR2. The input point cloud information undergoes a road coordinate system transformation process to convert its coordinate system to the coordinate system of the scanning area, such as a road.

[0022] The image generation unit 111 compresses the point cloud information acquired by the LiDAR in the height direction to generate multiple layers of 2D images for each point cloud. The synthesis unit 112 synthesizes the 2D images generated by the image generation unit at predetermined time intervals. The circular region generation unit 113 generates a cylinder from the circular region based on the result of the synthesis by the synthesis unit 112. Details of the processing in each part of the image generation unit 111, synthesis unit 112, and circular region generation unit 113 will be described later.

[0023] Next, Figure 4 shows a flowchart of the operation of the background detection unit 11c. The flowchart shown in Figure 4 is executed periodically, for example, once every hour. In Figure 4, first, the image generation unit 111 creates a time-series multi-stage projected image list (step S11). The outline of step S11 will be explained with reference to Figure 5.

[0024] As shown in Figure 5, if a point cloud of trees is acquired as the background object (left side of Figure 5), it is divided into multiple sections in the height direction (vertical direction), and the point cloud of each section is compressed in the height direction, that is, a 2D image is generated by projecting it with the height coordinates of the points in the same section set to a constant value (right side of Figure 5). Then, such images are generated for each time series and listed. Listing here means saving the images generated for each time series in a way that allows the chronological order to be identified. The storage location can be the memory of the control unit 11 or the like.

[0025] Figure 6 shows a flowchart of the more detailed operation of step S11. First, the image generation unit 111 prepares a multi-stage projection image in which all pixels have a value of 0 (step S21). The spatial division that is the premise for the multi-stage projection image is explained with reference to Figure 7. In Figure 7, the installation position of LiDAR2 is taken as the origin, the irradiation direction of LiDAR2 is taken as the y-axis, the direction perpendicular to the y-axis in the horizontal plane is taken as the x-axis, and the height direction is taken as the z-axis.

[0026] In Figure 7, if the space is divided equally in the height (z) direction into H sections at intervals of, for example, 25 cm, and the xy plane is divided equally into, for example, M × N squares of 10 cm each, then H 2D images with M × N pixels are generated. In this embodiment, these images are called multi-stage projection images. These H 2D images are binary images in which the pixel values ​​are either "0" or "1". Here, a multi-stage projection image in which all pixels are 0 means an image in which all pixel values ​​of the H 2D images are 0.

[0027] Next, reflected point cloud data (point cloud information) is sent from the sensor (LiDAR2) and input to the input unit 110 (step S22).

[0028] Next, the image generation unit 111 determines which pixel in the multi-stage projection image the point from the point cloud information input in step S22 is contained in (step S23). For example, in the example in Figure 7, it determines which stage out of H and which pixel out of M×N pixels the point belongs to. Since the coordinate system of each point included in the point cloud information has been converted to the road coordinate system, the position can be determined by comparing it with the coordinate range of each pixel in the space divided into M×N pixels in H stages.

[0029] Next, the image generation unit 111 sets the pixels determined in step S23 to a value of 1 (step S24). As shown in Figure 8, pixels that contain points are set to a value of "1". In other words, a specific value is set for pixels that contain points among multiple pixels. Pixels have a range in terms of coordinates in space, but it is sufficient that a point is contained within that range. Also, since pixels are defined horizontally, the z-axis coordinate of each point is fixed to a constant value for each stage. For example, points whose z-axis coordinate is within the range of 0 to 25 cm above the ground are set to "0", points within the range of 25 to 50 cm are set to "25", and so on. In this way, the point cloud of each stage is compressed in the height direction, that is, a 2D image is obtained by projection with the height coordinates set to a constant value.

[0030] Furthermore, as mentioned above, since a binary image is generated, the value will be 1 even if multiple points are present. In other words, the value of pixels with one or more acquired points is set to "1".

[0031] Steps S23 and S24 described above are repeated for each reflection point. For example, if there are 32,000 reflection points (points obtained in step S22), the process is repeated 32,000 times.

[0032] Next, the image generation unit 111 performs a closing process (step S25). The closing process is a well-known process in which the binary image is subjected to several dilation processes, followed by the same number of condensation processes. This closing process is repeated for H stages (the number of 2D images).

[0033] Next, the image generation unit 111 adds the multi-stage projection image after the closing process has been completed to the end of the list of time-series multi-stage projection images (step S26).

[0034] Steps S21 to S26 described above are repeated K times, for example, at predetermined time intervals of 5 minutes. If K=10, a list containing 10 multi-stage projection images generated every 5 minutes is generated.

[0035] Next, the synthesis unit 112 synthesizes the time-series multi-stage projection image list (step S12). The outline of step S12 will be explained with reference to Figure 9.

[0036] Figure 9 is an illustrative diagram of a synthesis method for a list consisting of, for example, multi-stage projected images i1, i2, and i3. Multi-stage projected image i1 is generated from point cloud information acquired at time t1 and consists of 2D images i1a, i1b, i1c, and i1d. Multi-stage projected image i1 is generated from point cloud information acquired at time t2, after time t1, and consists of 2D images i2a, i2b, i2c, and i2d. Multi-stage projected image i1 is generated from point cloud information acquired at time t3, after time t2, and consists of 2D images i3a, i3b, i3c, and i3d. In each 2D image, pixels set to "1" are filled in black.

[0037] The synthesis unit 112 counts the number of pixels in each multi-stage projection image that have a value of "1". If, for example, the number is more than half, the value of that pixel is set to "1". For example, in the 2D image i2a of the multi-stage projection image i2, there is a region X generated from a point cloud that does not exist at other time points. The pixels in this region X are black and therefore have a value of "1", but at other time points they are not black and therefore have a value of "0". Consequently, if the number is less than half, it is removed from the synthesized multi-stage projection image ci.

[0038] Figure 10 shows a flowchart of the more detailed operation of step S12. First, the synthesis unit 112 prepares a composite multi-stage projection image where the entire screen has a value of 0 (step S31). In other words, it generates the data for the multi-stage projection image that will serve as the basis for synthesis.

[0039] Next, in the multi-stage projection image list generated in step S11, the values ​​of the pixels of interest are added in time series (step S32). In other words, if the number of multi-stage projection images in the multi-stage projection image list is K, the addition of the values ​​of the pixels of interest is repeated K times.

[0040] Next, the synthesis unit 112 sets the value of the corresponding pixel in the synthesized multi-stage projection image to "1" if the sum is greater than or equal to a threshold (step S33). For example, a majority can be used as the threshold. Step S33 will be explained with reference to Figure 11. Figure 11 is a modified version of Figure 9 with some additions. The basic configuration of Figure 11 is the same as that of Figure 9. In Figure 11, the pixels at the coordinates marked with an "X" are a minority in the time series, so even after summing, they do not amount to a majority, and the pixel value in the synthesized multi-stage projection image ci is not set to "1". Therefore, the pixels at the coordinates marked with an "X" are removed in the synthesized multi-stage projection image ci.

[0041] In other words, the synthesis unit 112 counts the number of single values ​​(specific values) for each pixel of interest (a specific pixel), and removes pixels where the number of single values ​​(specific values) is less than half (below a predetermined threshold) during K cycles (the counting period). In the explanation above, a single value is set when there is a majority or more, but in other words, if there is less than half, there is no single value, and as a result, the pixels are removed.

[0042] The K loops in step S32 and step S33 described above are repeated for M × N pixels. This generates one stage of composite projection image (2D image).

[0043] Next, the synthesis unit 112 performs a closing process on the synthesized projection image generated up to step S33 (step S34), followed by a labeling process (step S35). Labeling is a well-known process in which labels are attached to connected regions (predetermined groups) in a binary image.

[0044] The closing and labeling processes described above are repeated for H stages. In this way, a composite multi-stage projected image is generated.

[0045] Returning to the flowchart in Figure 4, the next step is for the circular region generation unit 113 to set the circular region from the composite multi-stage projection image (step S13). The outline of step S13 will be explained with reference to Figure 12.

[0046] Figure 12 is an illustrative diagram of a method for creating circular regions from a composite multi-stage projection image. The left side of Figure 12 is the composite multi-stage projection image. By creating circular regions for each stage that cover the black areas of this composite multi-stage projection image, and then expanding them in the height direction, cylindrical regions are formed for each stage (right side of Figure 12). In Figure 12, cylinders Cy1, Cy2, Cy3, and Cy4 are formed. Specifically, the circular region generation unit 113 sets a bounding rectangle region for the region labeled by the synthesis unit 112 (described later), sets a circular region based on the bounding rectangle region, and generates cylinders based on that circular region.

[0047] The object detection device 10 uses this generated cylindrical region as the background region. That is, by performing the processing up to the circular region generation unit 113, the background region is approximated to a shape consisting of multiple cylinders based on the point cloud information.

[0048] Figure 13 shows a flowchart of the more detailed operation of step S11. First, the circular region generation unit 113 determines whether the bounding rectangle of a specific label region in the stage of interest satisfies the condition that the longer side > shorter side × 2 (step S41). The label region refers to the region that has been labeled by the labeling process performed in step S35.

[0049] The circumscribed rectangular region will be explained with reference to Figure 14. As shown in Figure 14, if there is a label region L, then the rectangular region B that is circumscribed around that label region L is the circumscribed rectangular region.

[0050] Next, the circular region generation unit 113, if the result of the determination in step S41 satisfies the condition that the longer side > the shorter side × 2 (step S41; yes), divides the longer side of the rectangle into two equal parts (step S42). Step S42 will be explained with reference to Figure 15. The condition that the longer side > the shorter side × 2 means that the rectangular region has an elongated shape, as shown in Figure 15. In this case, if a circle is set as is, the circular region will be too large for the label region, and if the background region is set based on that circular region, it will include a lot of areas other than the area that should be the background region. Therefore, in this embodiment, it is divided into two. In Figure 14, it is divided by a dashed line d that passes through the center of the longer side.

[0051] In this embodiment, step S113 sets the ratio of the long side to the short side to 2:1, i.e., the long side > short side × 2, but it is not limited to this. However, if the ratio of the long side to the short side is too large, the rectangle will be too elongated and will include a large area other than the original background area, as described above. On the other hand, if the ratio of the long side to the short side is too small, divisions will occur frequently and the processing load will increase. Therefore, it is preferable to set it appropriately, taking into account the processing capacity of the equipment that will serve as the background detection device. Also, while dividing into two is preferable in terms of processing load, it may be divided into three or more.

[0052] Next, the circular region generation unit 113 performs labeling again on the pixels within each divided rectangle and returns to step S41 (step S43). That is, it sets the circumscribing rectangular region again for the divided region.

[0053] On the other hand, if the result of the determination in step S41 does not satisfy the condition that the longer side > shorter side × 2 (step S41; no), the circular region generation unit 113 determines whether the longer side of the label rectangular region is parallel to the x-axis (step S44). Here, as described above, the x-axis and y-axis are defined as the direction of illumination of the LiDAR2 as the y-axis and the direction perpendicular to the y-axis in the horizontal plane as the x-axis.

[0054] Next, the circular region generation unit 113, if the result of the determination in step S44 is that the longer side of the label rectangular region is parallel to the x-axis (step S44; yes), extends the rectangle in the positive y-axis direction to make it a square (step S45). That is, it extends the shorter side of the circumscribed rectangular region in the direction away from the sensor.

[0055] Step S45 will be explained with reference to Figure 16. In Figure 16, the installation position of LiDAR2 is taken as the origin, and the x and y axes are defined as described above, similar to Figure 7. Here, let's assume that the label rectangular area B1 is in the positional relationship shown in the figure. The longer side of the label rectangular area B1 is parallel to the x axis. Therefore, the rectangle is extended in the positive y-axis direction (arrow direction) to make it a square.

[0056] This extended portion is the area that falls under the shadow of the labeled object from the perspective of LiDAR2. Whether an object is present in this extended portion or not, the laser beam does not hit it, so it has little impact on object detection. Conversely, extending the rectangle in the negative y-axis direction within the labeled rectangular area B1 is undesirable because it turns the area hit by the laser beam into a background region.

[0057] On the other hand, if the result of the determination in step S44 is that the longer side of the label rectangular region is not parallel to the x-axis (step S44; no), the circular region generation unit 113 determines whether the x-coordinate of the center point of the label rectangular region is > 0 (step S46).

[0058] Next, if the result of the determination in step S46 satisfies the x-coordinate of the center point of the label rectangular region > 0 (step S46; yes), the circular region generation unit 113 extends the rectangle in the positive x-axis direction to form a square (step S47).

[0059] Step S47 will be explained with reference to Figure 16. In Figure 16, assume that the label rectangular area B2 is in the positional relationship shown. The x-coordinate of the center point of the label rectangular area B2 is greater than 0. Therefore, extend the rectangle in the positive x-axis direction to make it a square.

[0060] This extended portion is the area that falls under the shadow of the labeled object from the perspective of LiDAR2. Whether or not an object is present in this extended portion, the laser beam does not hit it, so it has little impact on object detection. Conversely, extending the rectangle in the negative x-axis direction is undesirable because it turns the area hit by the laser beam into a background region.

[0061] On the other hand, if the result of the determination in step S46 does not satisfy the condition that the x-coordinate of the center point of the label rectangular region is > 0 (step S46; no), the circular region generation unit 113 extends the rectangle in the negative x-axis direction to make it a square (step S48). Step S48 extends the rectangle in the opposite direction to step S47 along the x-axis.

[0062] Next, the circular region generation unit 113 sets a circular region where the center point of the rectangle is the circle center point and the length of the rectangle's side is the circle diameter (step S49). As shown in Figure 17, the rectangular region is square-shaped by steps S45, S47, and S48, so the length of the rectangle's side becomes the circle diameter. In step S49, it is sufficient to save only the data for the circle center point and the circle diameter. The circular region itself is determined to be the region where the circle center point and the circle diameter are used as the background region. In addition to the circle center point and the circle diameter, information on the height of each step may also be saved. That is, the circular region is set by finding the inscribed circle of the square.

[0063] Next, the circular region generation unit 113 determines whether there are any label regions that are not included in the circle set in step S49 (step S4A). If the determination in step S4A finds that there are label regions that are not included in the circle (extending from the circle) (step S4A; yes), the unit relabels only the pixels outside the circle and returns to step S41 (step S4B).

[0064] Step S4B will be explained with reference to Figure 18. The left side of Figure 18 shows that a bounding rectangle B is set around the label area L, extended into a square shape, and a circle C is set with the center point of the square as the circle's center point and the length of the rectangle's side as the circle's diameter.

[0065] However, circle C does not encompass the entire label region L (center of Figure 18). Therefore, for the region La (pixels outside the circle) not included in circle C, each step from step S41 is repeated to set the circular region Ca (right side of Figure 18). As shown on the right side of Figure 18, in this embodiment, the circles may overlap.

[0066] Steps S41 to S4B are repeated for each label area present in the stage of interest. Once the stage of interest is finished, the next stage of interest is performed, and this process is repeated for H stages.

[0067] In this way, information indicating a circular region is generated for each label area of ​​each stage. As shown in Figure 12, the circular region expands in the z-axis direction within each stage, resulting in a cylindrical background area. Since the point cloud information is divided into multiple stages for processing, at least one cylinder is created in each stage, and when these are combined, they form a shape where multiple cylinders are connected in the height direction.

[0068] Returning to the flowchart in Figure 4, the next step is for the circular region generation unit 113 to update the existing background data (information indicating the background region) (step S14). The circular region generation unit 113 generates a cylindrical background region based on the set circular region information (circle center point, circle diameter) and step height information, and updates the existing background region. The updated information indicating the background region is used by the removal unit 11a, as will be described later.

[0069] As is clear from the above description, the image generation unit 111, the synthesis unit 112, and the circular region generation unit 113 function as detection units that detect background regions, which are areas of fixed objects, in a predetermined space based on point cloud information input to the input unit 110 at predetermined time intervals. Therefore, step S11 functions as an input process, and steps S11 to S13 function as detection processes.

[0070] In this embodiment, "fixed object" refers to a stationary object, but this includes vegetation that grows or sheds leaves over time. In other words, "stationary" in this embodiment is a relative concept used to distinguish it from moving vehicles, pedestrians, etc. For example, a person standing still for a long time or a parked vehicle may be detected as a stationary object, i.e., a fixed object, and become part of the background area. However, since the flowchart in Figure 4 is executed periodically, the information indicating the background area is also updated periodically, so it is unlikely that people or other objects will remain in the background area for an extended period.

[0071] Next, an object detection method (object detection method) that detects objects such as moving objects using information of the background region detected by the background detection device (background detection unit 11c) described above will be explained with reference to the flowchart in Figure 19. The flowchart shown in Figure 19 is executed by the control unit 11. Note that the process shown in Figure 19 is executed independently of the process shown in Figure 4.

[0072] First, the control unit 11 performs point cloud data reading processing (step S101). Point cloud data reading processing is the process by which the control unit 11 reads the point cloud information acquired by LiDAR2.

[0073] Next, the control unit 11 performs a point cloud data calibration process to adjust the obtained point cloud so that it is parallel to the ground (step S102). Next, the control unit 11 performs a road coordinate system transformation process to convert the coordinate system of the scanning area, such as a road, from the coordinate system of the point cloud information acquired by LiDAR2 (step S103). Since the point cloud information acquired by LiDAR2 is in a coordinate system based on LiDAR2, it is converted to the coordinate system of the space in which LiDAR2 is installed.

[0074] Next, the removal unit 11a performs point cloud removal processing outside the detection area (step S104). The point cloud removal processing outside the detection area is a process that identifies the background area based on the information of the circular area generated by the background detection unit 11c and removes the point clouds included in the background area. In other words, the removal unit 11a removes the point cloud information within the background area from the point cloud information acquired by the LiDAR2 (sensor).

[0075] Next, the detection unit 11b performs point cloud clustering on the point cloud remaining after the exclusion process in step S104 (step S105). The specific method for point cloud clustering is not particularly limited and can be any well-known method. Clusters are generated by the point cloud clustering process.

[0076] Next, the detection unit 11b performs a cluster center derivation process to derive the center of the cluster generated in step S105 (step S106). The cluster center may be the center of the coordinate system that defines the cluster's range, or it may be the centroid of the cluster.

[0077] Next, the detection unit 11b performs detection target discrimination processing on the clusters generated in step S105 (step S107). Detection target discrimination processing is the process of determining the type of moving object, such as a pedestrian or a vehicle, from the shape and size of the clusters. In other words, the detection unit 11b detects objects based on the point cloud information remaining after removal by the removal unit 11a.

[0078] Next, the detection unit 11b performs a detection target tracking process to track the target identified in step S107 (step S108). The detection target tracking process is performed by detecting the movement of the cluster center position between frames for the target identified in step S107.

[0079] Next, the detection unit 11b performs a velocity derivation process to derive the velocity of the target identified in step S107 (step S109). In the velocity derivation process, the velocity of the target identified in step S107 is derived from the distance traveled by the cluster center position between frames and the frame rate.

[0080] Next, the detection unit 11b performs new registration, update, and deletion processing of detected pedestrian information (step S110). New registration, update, and deletion processing of detected pedestrian information involves registering, updating, or deleting the position, speed, and size of the pedestrian identified in steps S107 to S109 in the memory of the control unit 11. For example, if a pedestrian is newly detected, the pedestrian is newly registered and subsequently updated each time their position changes while being tracked. Also, if the tracked pedestrian moves out of the scanning range of the LiDAR2 and can no longer be detected, the information is deleted. Note that although step S110 is described as being performed on pedestrians, it may also be performed on moving objects other than pedestrians, such as vehicles.

[0081] Next, the control unit 11 outputs target information about the pedestrian (target) processed in step S110 to the outside via the communication unit 12 (step S111).

[0082] According to this embodiment, the background detection unit 11c includes an input unit 110 that receives point cloud information from a LiDAR2 installed on the ground, which intermittently irradiates light to scan a scanning range S, receives the reflected light, and acquires point cloud information based on the reflected light; and an image generation unit 111, a synthesis unit 112, and a circular region generation unit 113 that detect a background region, which is a fixed area in a predetermined space, based on the point cloud information input to the input unit 110 at predetermined time intervals. The image generation unit 111, the synthesis unit 112, and the circular region generation unit 113 then approximate the background region to a shape consisting of multiple cylinders based on the point cloud information.

[0083] With the background detection unit 11c configured as described above, the background region is approximated as a shape consisting of multiple cylinders, allowing complex-shaped objects to be represented by simpler shapes, thus enabling the system to handle complex structures. Furthermore, by inputting point cloud information, the background region can be automatically detected.

[0084] Furthermore, the system includes an image generation unit 111 that compresses point cloud information in the height direction to generate multiple layers of 2D images for each point cloud, a synthesis unit 112 that synthesizes the 2D images generated by the image generation unit 111, and a circular region generation unit 113 that generates a cylinder based on the results of the synthesis by the synthesis unit 112. In this way, complex processing such as machine learning is unnecessary, and a background region with a shape close to that of an actual fixed object can be generated from the point cloud.

[0085] Furthermore, the image generation unit 111 divides the vertical direction of the target space into H sections and sets the height coordinates of the points in the point cloud contained in the same section to a constant value. It then divides the target space horizontally into M × N pixels and sets a value of 1 for the pixels in which points exist. Based on the pixels to which a value of 1 is set, a 2D image is generated. In this way, a 3D point cloud can be converted into a 2D image.

[0086] Furthermore, the synthesis unit 112 aggregates the number of single values ​​for specific pixels in the two-dimensional images generated at predetermined time intervals, and removes pixels where the number of single values ​​is below a predetermined threshold, such as a majority, during the aggregation period. In this way, point clouds relating to objects that appeared only for a short time, such as merely passing through the space, can be removed.

[0087] Furthermore, the synthesis unit 112 labels the image after the removal described above, in predetermined groups. By doing so, regions considered to be fixed objects can be identified by a well-known labeling process, making it easier to generate the circular regions described later.

[0088] Furthermore, the circular region generation unit 113 sets a bounding rectangle for the labeled region, sets a circular region based on that bounding rectangle, and forms a cylinder based on the circular region. In this way, it is possible to treat a region considered to be a fixed object as a circular region and form a cylinder based on that circle. Therefore, the formed cylinder can be used as a background region.

[0089] Furthermore, if the ratio of the long side to the short side of the circumscribed rectangular region is 2:1 or greater, the circular region generation unit 113 divides the circumscribed rectangular region and sets a new circumscribed rectangular region for the divided region. By doing so, it is possible to prevent the inclusion of areas other than the original background region.

[0090] Furthermore, the circular region generation unit 113 sets the circular region by extending the shorter side of the circumscribed rectangular region into a square shape and finding the inscribed circle of the square. In this way, the circular region can be easily set.

[0091] Furthermore, the circular region generation unit 113 extends the shorter side of the circumscribed rectangular region in a direction away from the LiDAR2. By doing so, the region can be extended in the direction that casts a shadow when viewed from the LiDAR2, thereby reducing the influence of the resulting circular region on the object detection process.

[0092] Furthermore, if there are labeled areas not included in the generated circular region, the circular region generation unit 113 sets a new bounding rectangle region for those unincluded labeled areas. In this way, it is possible to generate a circular region that covers all labeled areas while excluding areas other than the labeled areas.

[0093] Furthermore, the object detection device 10 includes a LiDAR 2, a removal unit 11a that acquires information on the background region detected by the background detection unit 11c and removes point cloud information within the background region from the point cloud information acquired by the LiDAR 2, and a detection unit 11b that detects objects based on the point cloud information remaining after removal by the removal unit 11a.

[0094] With the object detection device 10 configured as described above, it becomes possible to set a background area that corresponds as closely as possible to complex structures, thereby enabling more accurate object detection.

[0095] It should be noted that the present invention is not limited to the embodiments described above. That is, those skilled in the art can implement the invention in various ways, without departing from the core principles, in accordance with prior art knowledge. Such modifications, as long as they still possess the configuration of the background detection device, background detection method, and object detection device of the present invention, are of course included within the scope of the present invention. [Explanation of symbols]

[0096] 1. Main body 2 LiDAR (sensor) 10. Object detection device 11 Control Unit 11a Removal section 11b Detection unit 11c Background detection unit 110 Input Section 111 Image generation unit (detection unit) 112 Synthesis Unit (Detection Unit) 113 Circle Region Generation Unit (Detection Unit) L Label area (labeled area) B circumscribed rectangular area C circular area

Claims

1. An input unit receives point cloud information from a sensor installed on the ground that intermittently irradiates light to scan a predetermined space, receives the reflected light, and acquires point cloud information based on the reflected light. The system includes a detection unit that detects a background region, which is the region of a fixed object, within a predetermined space based on the point cloud information input to the input unit at predetermined time intervals, The detection unit approximates the background region to a shape consisting of multiple cylinders based on the point cloud information. A background detection device characterized by the following features.

2. The detection unit is An image generation unit that compresses the point cloud information in the height direction to generate multiple layers of two-dimensional images, The image generation unit combines the multiple stages of two-dimensional images generated at predetermined time intervals, A circular region generating unit generates the cylinder based on the results of synthesis by the aforementioned synthesis unit, The background detection device according to claim 1, characterized by comprising the following:

3. The background detection device according to claim 2, characterized in that the image generation unit divides the vertical direction of the target space into multiple stages and sets the height coordinates of the points in the point cloud included in the same stage to a constant value, further divides the horizontal plane of the target space into multiple pixels, sets a specific value for the pixels in which points exist among the multiple pixels, and generates the two-dimensional image based on the pixels to which the specific value has been set.

4. The background detection device according to claim 3, characterized in that the synthesis unit aggregates the number of specific values ​​for specific pixels of the two-dimensional image generated at predetermined time intervals, and removes pixels in which the number of specific values ​​is less than or equal to a predetermined threshold during the aggregation period.

5. The background detection device according to claim 4, characterized in that the synthesis unit labels the image after the removal has been performed in predetermined groups.

6. The background detection device according to claim 5, characterized in that the circular region generation unit sets a bounding rectangular region for the labeled region, sets a circular region based on the bounding rectangular region, and generates the cylinder based on the circular region.

7. The background detection device according to claim 6, characterized in that the circular region generation unit divides the circumscribed rectangular region and sets the circumscribed rectangular region again for the divided region if the ratio of the long side to the short side of the circumscribed rectangular region is greater than or equal to a predetermined value.

8. The background detection device according to claim 6, characterized in that the circular region generation unit sets the circular region by extending the shorter side of the circumscribed rectangular region into a square shape and determining the inscribed circle of the square.

9. The background detection device according to claim 8, characterized in that the circular region generation unit extends the shorter side in a direction away from the sensor.

10. The background detection device according to claim 6, characterized in that, if there is a labeled area that is not included in the circular area, the circular area generation unit sets the circumscribed rectangular area for the labeled area that is not included.

11. The aforementioned sensor and, A background detection device according to any one of claims 1 to 10 acquires information about the background region, and a removal unit removes the point cloud information within the background region from the point cloud information acquired by the sensor, A detection unit detects an object based on the point cloud information remaining after removal by the removal unit, An object detection device characterized by comprising the following features.

12. An input step in which point cloud information is input from a sensor installed on the ground that intermittently irradiates light to scan a predetermined space, receives the reflected light, and acquires point cloud information based on the reflected light; The process includes a detection step that detects a background region which is the region of a fixed object within a predetermined space based on the point cloud information input to the input step at predetermined time intervals, The detection step approximates the background region to a shape consisting of multiple cylinders based on the point cloud information. A background detection method characterized by the following features.

Images