Object detection device, object detection method, and program

Abstract

To accurately detect an object regardless of an arrangement state of the object.SOLUTION: An object detection apparatus acquires point cloud data of an object, and extracts a plurality of candidate positions of the object from the point cloud data. The apparatus specifies a plurality of combinations of the candidate positions where a degree of inference between objects assumed to be present in the candidate positions is less than a reference value, and calculates, for each of the combinations, for each of the candidate positions included in the combinations, a total value of first scores for evaluating a matching degree between the point cloud data and templates assumed to be present in the candidate positions. The apparatus specifies that the object exists in the candidate position included in the combinations, on the basis of the total value of the first scores.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an object detection device, an object detection method, and a program. [Background technology]

[0002] There are known remote control technologies for forklift vehicles for transporting pallets. For example, in Patent Document 1, an image of a pallet to be transported is captured by an imaging device such as a camera, and the forklift vehicle is moved toward a pallet selected by a user from at least one pallet recognized on the image.

[0003] Here, the detection of the pallet, which is the detection target, is performed based on point cloud data acquired by a distance measuring device using a laser beam, such as 2D-LiDAR, in addition to images acquired by an imaging device such as that disclosed in Patent Document 1. For example, Patent Document 2 discloses an example of a pallet detection technology using a laser beam, in which a light reflecting surface for reflecting a laser beam is provided on the pallet in advance, thereby enabling reliable and easy detection of the pallet. Furthermore, Non-Patent Document 1 discloses a technology for detecting a pallet included in an image by matching a template corresponding to the pallet, which is the target object, with point cloud data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 225390 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-298000 [Non-patent literature]

[0005] [Non-Patent Document 1] Baglivo, L., etal. "Autonomous pallet localization and picking for industrial forklifts: a robust range and look method." Measurement Science and Technology 22.8(2011): 085502. Summary of the Invention [Problem to be solved by the invention]

[0006] In Non-Patent Document 1, the object is detected by applying a template corresponding to a pallet, which is the detection object, to point cloud data. However, a pallet has an opening into which the claws of a forklift vehicle can be inserted, and if multiple such pallets are placed in the area to be detected at intervals corresponding to the openings, there is a risk that each individual pallet cannot be detected accurately.

[0007] At least one embodiment of the present disclosure has been made in consideration of the above-mentioned circumstances, and aims to provide an object detection device, an object detection method, and a program that can accurately detect an object regardless of the object's placement state. [Means for solving the problem]

[0008] In order to solve the above problem, an object detection device according to at least one embodiment of the present disclosure includes: a point cloud data acquisition unit for acquiring point cloud data of at least one object; a candidate position extraction unit for extracting a plurality of candidate positions of the object from the point cloud data; a candidate position combination specifying unit for specifying a plurality of combinations of the candidate positions in which the degree of interference between the objects assumed to be at the candidate positions is less than a reference value; a score calculation unit for calculating, for each of the candidate positions included in the combination, a total value of first scores for evaluating a degree of match between a template of the object hypothesized to be at the candidate position and the point cloud data; an object identification unit for identifying the object as being present at the candidate position included in the combination based on the total value of the first scores; Equipped with.

[0009] In order to solve the above problem, an object detection method according to at least one embodiment of the present disclosure includes: acquiring point cloud data of at least one object; extracting a plurality of candidate positions of the object from the point cloud data; identifying a plurality of combinations of the candidate positions that make the degree of interference between the objects assumed to be at the candidate positions less than a reference value; calculating a sum of first scores for each of the candidate positions included in the combination to evaluate a degree of match between a template of the object hypothesized to be at the candidate position and the point cloud data; determining that the target object is located at the candidate position included in the combination based on the sum of the first scores; Equipped with.

[0010] In order to solve the above problem, a program according to at least one embodiment of the present disclosure includes: In a computer device, acquiring point cloud data of at least one object; extracting a plurality of candidate positions of the object from the point cloud data; identifying a plurality of combinations of the candidate positions that make the degree of interference between the objects assumed to be at the candidate positions less than a reference value; calculating a sum of first scores for each of the candidate positions included in the combination to evaluate a degree of match between a template of the object hypothesized to be at the candidate position and the point cloud data; determining that the target object is located at the candidate position included in the combination based on the sum of the first scores; is possible. [Effects of the Invention]

[0011] According to at least one embodiment of the present disclosure, it is possible to provide an object detection device, an object detection method, and a program that can accurately detect an object regardless of the arrangement state of the object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. [Figure 2] FIG. 2 is a plan view showing a template corresponding to the pallet of FIG. 1. [Figure 3] 1 is a block diagram showing a configuration of an object detection device according to an embodiment. [Figure 4] 4 is a flowchart showing an object detection method that can be performed by the object detection device of FIG. 3. [Figure 5] 5 is an example of point cloud data acquired in step S100 of FIG. 4. [Figure 6] 10 is an example of a plurality of candidate positions extracted in step S102. [Figure 7] 7 is a diagram showing a process of finding a combination that satisfies a constraint condition from the plurality of candidate positions shown in FIG. 6. FIG. [Figure 8] FIG. 10 is a schematic diagram showing a template corresponding to a candidate position together with point cloud data. [Figure 9] FIG. 10 is a diagram schematically illustrating the distance between two candidate positions. [Figure 10] 5 is a flowchart showing a method for extracting candidate positions in step S102 of FIG. 4. [Figure 11] Fig. 6 is a diagram showing line segments extracted from the point cloud data of Fig. 5. In this example, a state in which a plurality of line segments passing through at least two pieces of data are extracted is shown. [Figure 12] FIG. 10 is a diagram showing adjacent regions set for line segments. [Figure 13] 11 is a schematic diagram showing how each piece of target data extracted in step S201 in FIG. 10 is applied to each feature position set in the template in FIG. 2 to tally up tentative candidate positions. FIG. [Figure 14] 10 is an example of a normal vector indicating a line segment extracted by a line segment extraction unit. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, several embodiments will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of components described as embodiments or shown in the drawings are merely illustrative examples and are not intended to limit the scope of the invention.

[0014] First, an object to be detected by the object detection device 100 according to at least one embodiment will be described. In the following, to specifically explain the embodiment, a pallet 1 that can be handled by a forklift vehicle (not shown) will be taken as an example of the object, but the object is not limited to this. Fig. 1 is a perspective view of the pallet 1, and Fig. 2 is a plan view showing a template 6 corresponding to the pallet 1 in Fig. 1.

[0015] As shown in Fig. 1, when placed on a substantially horizontal floor, the pallet 1, which is the object, is substantially parallel to the floor and has a pair of plate members 2a, 2b positioned above and below the floor, and multiple pillars 2c provided between the pair of plate members 2a, 2b. The multiple pillars 2c each extend in the vertical direction, thereby supporting the pair of plate members 2a, 2b.

[0016] When viewed from above (or below), the pair of plate members 2a, 2b each have a substantially square shape with four equal sides, and pillars 2c are located at each vertex of the pair of plate members 2a, 2b, the midpoint of each side of the pair of plate members 2a, 2b, and the center of the pair of plate members 2a, 2b. Spaces 4 are provided between the pillars 2c into which the claws of a forklift vehicle can be inserted.

[0017] In the object detection device 100, a template 6 is prepared that is used to detect a pallet 1 having such a configuration. The template 6 has a shape that corresponds to the shape of the actual pallet 1 when viewed from above. Fig. 2 shows the template 6 having a shape that corresponds to the layout in which multiple pillars 2c of the pallet 1 are arranged with spaces 4 between them (note that in Fig. 2, common symbols are used at positions that correspond to the pillars 2c and spaces 4 in Fig. 1 to show the correspondence with Fig. 1).

[0018] FIG. 3 is a block diagram showing the configuration of an object detection device 100 according to an embodiment, and FIG. 4 is a flowchart showing an object detection method that can be performed by the object detection device 100 of FIG.

[0019] The object detection device 100 is composed of, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and a computer-readable storage medium. A series of processes for realizing various functions is stored in a storage medium or the like in the form of a program, for example. The CPU reads this program into the RAM or the like and executes information processing and arithmetic operations to realize various functions. The program may be pre-installed in a ROM or other storage medium, provided in a state stored in a computer-readable storage medium, or distributed via wired or wireless communication means. Examples of computer-readable storage media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, and semiconductor memories.

[0020] Specifically, as shown in FIG. 3, the object detection device 100 includes a point cloud data acquisition unit 102, a candidate position extraction unit 104, a candidate position combination identification unit 106, a score calculation unit 108, and an object identification unit 110. The point cloud data acquisition unit 102 is configured to acquire point cloud data of a pallet 1, which is an object. The candidate position extraction unit 104 is configured to extract candidate positions of the pallet 1 from the point cloud data. The candidate position combination identification unit 106 is configured to identify a combination of candidate positions that satisfies a constraint condition. The score calculation unit 108 is configured to calculate a total value of first scores for each combination of candidate positions. The object identification unit 110 is configured to identify that the object is located at a candidate position included in the combination based on the total value of the first scores.

[0021] Further, the candidate position extraction unit 104 includes a line segment extraction unit 111, a target data extraction unit 112, a tentative candidate position calculation unit 114, and a candidate position selection unit 116. The line segment extraction unit 111 is configured to extract at least one line segment from the point cloud data. The target data extraction unit 112 is configured to extract target data included in an adjacent area including the line segment from the point cloud data. The tentative candidate position calculation unit 114 is configured to calculate the target data as a tentative candidate position by applying it to a template 6. The candidate position selection unit 116 is configured to select a candidate position from the tentative candidate positions.

[0022] Note that the block configuration in FIG. 3 is merely an example of the functions corresponding to each step of the object detection method performed by the object detection device 100, and each block may be integrated or further subdivided.

[0023] In the object detection method, first, point cloud data of the pallet 1 is acquired by the point cloud data acquisition unit 102 (step S100). The point cloud data is acquired, for example, by Lidar (Light Detection and Ranging). In this embodiment, the range in which the pallet 1 to be detected is placed is specified in advance on the field, and point cloud data corresponding to the sides of the pallet 1 is acquired by irradiating this range with laser light.

[0024] Figure 5 is an example of the point cloud data acquired in step S100 of Figure 4. Figure 5 shows point cloud data based on the reflected light of a laser beam irradiated onto the pallet 1 of Figure 1 from approximately the side. In FIG. 5, each data point included in the acquired point cloud data is shown together with a guide mark (broken line) corresponding to each pillar portion 2c of the pallet 1.

[0025] Next, the candidate position extraction unit 104 extracts candidate positions P of the object from the point cloud data acquired in step S100 (step S101). The point cloud data includes data points attributable to any object within the measurement range, but if a pallet 1 is present within the measurement range, the data attributable to the pallet 1 will be dominant. The candidate position extraction unit 104 can extract candidate positions P of the pallet 1 by analyzing such point cloud data (note that a specific method for extracting candidate positions P in step S101 will be described later).

[0026] Next, the candidate position combination identification unit 106 identifies a combination of candidate positions CP that satisfies the constraint condition based on the multiple candidate positions P extracted in step S101 (step S102). The constraint condition stipulates that the degree of interference between pallets 1 assumed to be at candidate position P must be less than a reference value. Since actual pallets 1 are tangible objects, they are not allowed to interfere with each other (i.e., the degree of interference is zero). However, in this method, by appropriately setting the reference value to allow slight interference as a constraint condition, significant candidate positions P are not eliminated, thereby improving the final detection accuracy.

[0027] Here, the method for identifying the combination CP of candidate positions in step S102 will be described in detail with reference to Fig. 6 and Fig. 7. Fig. 6 shows an example of the plurality of candidate positions P1 to P4 extracted in step S102, and Fig. 7 shows the process of finding a combination that satisfies the constraint conditions from the plurality of candidate positions P1 to P4 shown in Fig. 6.

[0028] 6 shows multiple candidate positions P1 to P4 extracted in step S101. For ease of explanation, it is assumed that the reference value of the interference degree in the constraint condition is zero (i.e., interference between candidate positions is not permitted). In this case, combinations including adjacent candidate positions "P1, P2," "P2, P3," and "P3, P4" are not valid. Therefore, as shown in FIG. 7, the combinations CP of candidate positions identified in step S102 are "P1, P4," "P1, P3," "P1, P4," "P2, P4," and "P4."

[0029] Next, for each of the combinations CP identified in step S102, the score calculation unit 108 calculates a first score for each of the candidate positions P included in the combination CP, and calculates a total value TS of the first scores S1 for each combination CP (step S103). The first score S1 is an evaluation index for evaluating the degree of match between the template 6 assumed to be at each candidate position P and the point cloud data, and is calculated using an arbitrary function so that the higher the degree of match between the template 6 and the point cloud data, the larger the value.

[0030] An example of the calculation of the first score S1 in step S103 will now be described with reference to Fig. 8. Fig. 8 is a schematic diagram showing a template corresponding to a candidate position P together with point cloud data. In Fig. 8, a template 6 is placed at the candidate position P for which the first score S1 is to be calculated, and the degree of match between the template 6 and the point cloud data is calculated as the first score S1. As shown in Fig. 8, at least a portion of the data included in the point cloud data is distributed so as to overlap with the template 6, and the greater the number of data included in the template 6, the higher the calculated first score S1.

[0031] Next, the object identification unit 110 identifies the position of the pallet 1 from each combination identified in step S102 based on the total value TS of the first scores S1 calculated in step S103 (step S104). In this embodiment, the object identification unit 110 identifies the combination that maximizes the total value TS of the first scores S1 in step S104, thereby identifying the position of the pallet 1.

[0032] In step S200, the candidate position extraction unit 104 may exclude one of the two candidate positions P5, P6 whose mutual distance L is smaller than a first threshold L1. FIG. 9 is a diagram schematically showing the distance L between the two candidate positions P5, P6. The first threshold L1, which is used to compare the distance L, is set as a reference value for determining whether the two candidate positions P5, P6 are sufficiently close to each other and therefore can be considered to be substantially the same. By excluding one of the two candidate positions P5, P6 whose mutual distance L is smaller than the first threshold L1 and therefore considered to be substantially the same, the number of candidate positions P to be calculated can be reduced, thereby reducing the calculation load, without reducing detection accuracy.

[0033] In this case, the candidate positions to be excluded may be selected by calculating the first score S1 for each of the two candidate positions P5 and P6 and selecting the one with the smaller first score S1. This allows candidate positions with a high degree of match with the template 6 to be retained preferentially by excluding candidate positions with a low degree of match with the template 6, thereby more effectively suppressing a decrease in detection accuracy while reducing the computational load.

[0034] Next, a more detailed description will be given of the method for extracting candidate positions P in step S200 described above. Fig. 10 is a flowchart showing the method for extracting candidate positions P in step S102 of Fig. 4.

[0035] First, the line segment extraction unit 111 extracts at least one line segment LE from the point cloud data (step S200). Here, by analyzing the point cloud data acquired by the point cloud data acquisition unit 102, elements that are latent in the point cloud data and that could be a line segment LE that constitutes at least a part of the contour of the pallet 1 are extracted. Figure 11 is a diagram showing the line segment LE extracted from the point cloud data of Figure 5. This example shows how multiple line segments LE that pass through at least two pieces of data are extracted.

[0036] Next, the target data extraction unit 112 extracts target data included in an adjacent region NR of the point cloud data for each line segment LE extracted in step S200 (step S201). Here, an adjacent region NR is set for the line segment LE extracted in step S200. For example, as shown in FIG. 12, the adjacent region NR has a substantially rectangular shape set along the line segment LE, and is defined by a long side NR1 along the line segment and a short side NR2 perpendicular to the line segment. In step S201, of the data included in the point cloud data, data included in the adjacent region NR set in this way is extracted as target data.

[0037] Next, the tentative candidate position calculation unit 114 calculates the position of the template when each of the target data extracted in step S201 is applied to each of the multiple characteristic positions p of the template 6 as tentative candidate positions KP (step S202). Here, the characteristic positions p of the template are positions that are likely to be detected as point cloud data when the pallet 1 is measured using a point cloud data acquisition device such as Lidar, and are set according to the shape of the template 6.

[0038] In this embodiment, the template 6 has the shape described above with reference to Figure 2, and the following are set as multiple characteristic positions p: a position on the left side of the pillar portion 2c1 located on the left front side of the template 6 (hereinafter referred to as the "first characteristic position p1" as appropriate), a position on the right side of the pillar portion 2c1 located on the left front side of the template 6 (hereinafter referred to as the "second characteristic position p2" as appropriate), a position on the left side of the pillar portion 2c2 located in the center of the template 6 (hereinafter referred to as the "third characteristic position P3" as appropriate), a position on the right side of the pillar portion 2c2 located in the center of the template 6 (hereinafter referred to as the "fourth characteristic position p4" as appropriate), a position on the left side of the pillar portion 2c3 located on the right side of the template 6 (hereinafter referred to as the "fifth characteristic position p5" as appropriate), and a position on the right side of the pillar portion 2c3 located on the right side of the template 6 (hereinafter referred to as the "sixth characteristic position p6" as appropriate).

[0039] Next, the candidate position selecting unit 116 tabulates the tentative candidate positions KP for each of the target data and the feature position p, and selects the tentative candidate position KP having the peak number of tabulations as the candidate position P (step S203). FIG. 13 is a schematic diagram showing how the target data extracted in step S201 is applied to the first feature position p1 to the sixth feature position p6 set in the template 6 of FIG. 2 to tabulate the tentative candidate positions KP. As shown in FIG. 13, the target data extracted in step S201 is applied to the first feature position p1 to the sixth feature position p6 of the template 6, respectively, and the positions of the template 6 in each case are tabulated as the tentative candidate positions KP. As a result, a distribution of the tabulated numbers is obtained, as shown in the lower graph of FIG. 13. The candidate position selecting unit 116 extracts peaks from this distribution and selects the tentative candidate position KP having the peak number of tabulations as the candidate position P. The method for extracting peaks from the distribution is not limited, but for example, a peak detection threshold may be set in advance and the peak may be extracted based on whether or not the peak detection threshold is exceeded, or the peak may be extracted by identifying a maximum value included in the distribution.

[0040] In the object detection method described above, the template 6 is applied to the selected candidate position P to detect the pallet 1. This allows the candidate position P, which is the target for calculating the first score S1, to be efficiently extracted from point cloud data containing a large amount of data, thereby making it possible to detect the pallet 1 with good accuracy while reducing the computational load.

[0041] In step S200, the line segment extraction unit 111 may also calculate a second score S2 corresponding to the number of data points included in the adjacent region NR (see FIG. 12) for each of the multiple line segments LE extracted by analyzing the point cloud data, and select line segments LE for which the second score S2 is equal to or greater than a predetermined third threshold. As a result, line segments LE for which the number of data points included in the adjacent region NR is small are excluded from extraction because the second score S2 is less than the third threshold. The line segments LE excluded in this way have a small number of data points included in the adjacent region NR, and therefore have little impact on the detection results of palette 1. This effectively reduces the computational load while maintaining the detection accuracy of palette 1.

[0042] In this case, the third threshold value serving as the selection criterion may be set based on the distance or angle relative to a reference position where data belonging to each line segment LE to be evaluated is acquired. That is, the third threshold value is variably set based on the distance or angle relative to a measurement device such as Lidar used to acquire the point cloud data for the data that forms the basis of each line segment LE extracted by the line segment extraction unit 111. For example, when acquiring point cloud data, the number of data items tends to decrease as the distance or angle relative to the reference position increases. Therefore, by making the third threshold value variable depending on the distance or angle, line segments that have a significant impact on the accuracy of object detection can be suitably extracted from the point cloud data.

[0043] Additionally, in step S200, the line segment extraction unit 111 may calculate, for each of the multiple line segments LE extracted by analyzing the point cloud data, the angle θ (see FIG. 12) with respect to a reference line SL that defines the target area in which the object is placed, and extract line segments for which the angle θ is less than a fourth threshold. For example, in a situation in which the pallet 1 is rarely placed at an angle with respect to the reference line SL that defines the target area in which the pallet 1 is placed, by excluding from the extraction targets line segments LE for which the angle θ with respect to the reference line SL is equal to or greater than the fourth threshold, it is possible to preferably extract line segments LE that have a significant impact on the detection accuracy of the pallet 1 from the point cloud data.

[0044] In step S200, the line segment extraction unit 111 may extract the line segment LE based on at least two pieces of data from the point cloud data whose distance from each other is within a predetermined range. The predetermined range is set based on the feature position p of the template 6. For example, the template 6 shown in FIG. 2 has the first feature position p1 to the sixth feature position p6 as described above, and the lower limit of the predetermined range is set to correspond to the width of the column portion 2c, which is the shortest distance between any two of the first feature position p1 to the sixth feature position p6 (the distance between the first feature position p1 and the second feature position p2, the distance between the third feature position p3 and the fourth feature position p4, and the distance between the fifth feature position p5 and the sixth feature position p6). The upper limit of the predetermined range is set to correspond to the longest distance between any two of the first feature position p1 to the sixth feature position p6 (the distance between the first feature position p1 and the sixth feature position p6).

[0045] In this way, by extracting line segments LE that are identified based on at least two pieces of point cloud data whose distance from each other is within a predetermined range, it is possible to preferably extract line segments that have a significant impact on the detection accuracy of pallet 1 from the point cloud data.

[0046] Furthermore, in step S200, the line segment extraction unit 111 may extract each of the multiple line segments LE obtained by analyzing the point cloud data by identifying it with a normal vector V that is based on the reference point O and has a quantized length l and phase φ. FIG. 14 shows an example of a normal vector V indicating a line segment LE extracted by the line segment extraction unit 111. As a result, since the length l and phase φ of the normal vector V are quantized, similar line segments LE whose normal vectors V are slightly different are considered to be identical. As a result, by excluding similar line segments LE from the extraction result, it is possible to reduce the computational load and preferably extract line segments LE that have a significant impact on the detection accuracy of palette 1 from the point cloud data.

[0047] In addition, within the scope of the present disclosure, the components in the above-described embodiments may be replaced with well-known components as appropriate, and the above-described embodiments may be combined as appropriate.

[0048] The contents described in each of the above embodiments can be understood, for example, as follows.

[0049] (1) An object detection device (100) according to one aspect includes: a point cloud data acquisition unit (102) for acquiring point cloud data of at least one object (1); a candidate position extraction unit (104) for extracting a plurality of candidate positions (P) of the object from the point cloud data; a candidate position combination specifying unit (106) for specifying a plurality of candidate position combinations (CPs) in which the degree of interference between the objects assumed to be at the candidate positions is less than a reference value; For each of the candidate positions included in the combination, a degree of match between a template of the object assumed to be at the candidate position and the point cloud data is evaluated. The evaluation index is set so that the higher the number of data included in the template among the data included in the point cloud data, the higher the evaluation index. First Score (S1) and calculating the first score calculated for each of the candidate positions. a score calculation unit (108) for calculating the sum (TS) of the above; an object identification unit (110) for identifying that the object is present at the candidate position included in the combination based on the total value of the first scores; Equipped with.

[0050] According to the above aspect (1), it is possible to narrow down candidate positions for which the first score is to be calculated from candidate positions of objects extracted from point cloud data using constraint conditions. This makes it possible to reduce the number of candidate positions for which the first score is to be calculated and effectively reduce the computational burden. Then, a total value of the first scores is calculated for each combination including the narrowed-down candidate positions, and it is possible to accurately detect objects with various arrangements, orientations, and numbers based on the total value. It is desirable that the reference value for identifying a combination of candidate positions is, for example, less than 6 cm.

[0051] (2) In another embodiment, in the above embodiment (1), The object identification unit identifies the object as being located at the candidate position included in the combination with the largest total value.

[0052] According to the above aspect (2), the object identification unit can accurately detect the object by identifying the combination that maximizes the total value of the first scores.

[0053] (3) In another aspect, in the above aspect (1) or (2), The candidate position combination specifying unit specifies a plurality of combinations of the candidate positions based on the presence or absence of the degree of interference.

[0054] According to the above aspect (3), a plurality of combinations of candidate positions are identified based on the presence or absence of interference.

[0055] (4) In another aspect, in any one of the above aspects (1) to (3), The candidate position extraction unit excludes the candidate position having the smaller first score from two candidate positions whose distance from each other is smaller than a first threshold value.

[0056] According to the above aspect (4), among the plurality of candidate positions extracted from the point cloud data, for two candidate positions whose distance from each other is less than the first threshold, the candidate position with the smaller first score is excluded. As a result, by omitting nearly overlapping candidate positions that are close to each other and therefore can be considered to be the same or similar, the number of candidate positions to be calculated can be reduced and the calculation load can be reduced without reducing the detection accuracy. The first threshold may be set based on the dimensions of the object. For example, if the object is the pallet 1 shown in the above embodiment, the first threshold can be set to a value that does not exceed the sum of the width of one space 4 (hole width) and the width of one column 2c (face width) of the pallet 1. More specifically, depending on the dimensions of the pallet 1, the first threshold can be set to, for example, 40 cm, preferably 10 cm.

[0057] (5) In another embodiment, in any one of the above (1) to (4), The candidate position extraction unit a line segment extraction unit (111) for extracting at least one line segment (LE) from the point cloud data; a target data extraction unit (112) for extracting target data included in a neighboring region (NR) including the line segment from the point cloud data; a tentative candidate position calculation unit (114) for calculating a position of the template as a tentative candidate position (KP) when the target data is applied to each of a plurality of feature positions (p) that the template has; a candidate position selection unit (116) for selecting, as the candidate positions, a plurality of tentative candidate positions having a peak in the number of tabulated tentative candidate positions for each of the target data and the feature positions; Equipped with.

[0058] According to the above aspect (5), for each line segment extracted from the point cloud data, target data included in an adjacent region is further extracted. Each piece of target data is fitted to a characteristic position of a template corresponding to the target, and the template positions at that time are tallied as tentative candidate positions. Then, based on the tallied results, multiple tentative candidate positions with a peak in the tallied number are selected as candidate positions. This allows efficient extraction of candidate positions for calculation of the first score from point cloud data containing a large amount of data, thereby enabling target detection with high accuracy while reducing the computational load.

[0059] (6) In another embodiment, in the above embodiment (5), The line segment extraction unit extracts the line segments for which a second score (S2) corresponding to the number of data included in the adjacent region is equal to or greater than a third threshold.

[0060] According to the above aspect (6), for line segments extracted from point cloud data, a second score corresponding to the number of data points included in the adjacent region is calculated, and line segments whose second score is equal to or greater than a third threshold value are selected as extraction targets. As a result, line segments whose second score is less than the third threshold value and whose adjacent region contains a small number of data points are excluded from extraction targets. Since the line segments excluded in this way have a small number of data points included in the adjacent region, they have little impact on the object detection results, and therefore the computational load can be effectively reduced while maintaining the object detection accuracy.

[0061] (7) In another embodiment, in the above embodiment (6), The line segment extraction unit sets the third threshold based on the distance or angle of the line segment relative to a reference position for acquiring the point cloud data.

[0062] According to the above aspect (7), the third threshold value for determining whether to exclude a line segment extracted from point cloud data is set based on the distance or angle of the line segment relative to a reference position. For example, since the number of data tends to decrease as the distance or angle relative to the reference position where point cloud data is acquired increases, by making the third threshold value variable depending on the distance or angle, it is possible to suitably extract line segments that have a large effect on the accuracy of object detection from point cloud data.

[0063] (8) In another embodiment, in any one of the above (5) to (7), The line segment extraction unit extracts the line segment whose angle (θ) with respect to a reference line (SL) that defines an object region in which the at least one object is located is less than a fourth threshold value.

[0064] According to the above aspect (8), for example, in a situation where an object is rarely positioned at an angle relative to a reference line that defines the object area in which the object is positioned, by excluding line segments whose angle relative to the reference line is equal to or greater than the fourth threshold from the extraction target, it is possible to preferably extract line segments that have a large impact on the detection accuracy of the object from the point cloud data.

[0065] (9) In another embodiment, in any one of the above (5) to (8), The line segment extraction unit extracts the line segment based on at least two pieces of data from the point cloud data, the distance between which is within a predetermined range.

[0066] According to the above aspect (9), by extracting a line segment identified based on at least two pieces of point cloud data whose distance from each other is within a predetermined range, it is possible to preferably extract line segments that have a large impact on the detection accuracy of the object from the point cloud data.

[0067] (10) In another embodiment, in any one of the above (5) to (9), The line segment extraction unit extracts the line segment identified by a normal vector (V) whose length (l) and phase (φ) are quantized, using a reference point (O) set in a target area in which the at least one target object is located as a reference.

[0068] According to the above aspect (10), when a line segment extracted from point cloud data is specified as the length and phase of a normal vector relative to a reference point, the length and phase are quantized and handled discretely. As a result, line segments with similar lengths and phases are considered to be the same, and other similar line segments are excluded from extraction targets, thereby reducing the computational load and making it possible to suitably extract line segments that have a large impact on the accuracy of object detection from point cloud data.

[0069] (11) In another embodiment, in any one of the above (1) to (10), The object is a pallet (1) in which a pair of plate members (2a, 2b) are supported by a plurality of pillars (2c).

[0070] According to the above aspect (11), a pallet having a pair of plate members supported by a plurality of pillars can be suitably detected as the object.

[0071] (12) An object detection method according to one aspect includes: acquiring point cloud data of at least one object (1); extracting a plurality of candidate positions (P) of the object from the point cloud data; Identifying a plurality of combinations (CPs) of the candidate positions that make the interference between the objects assumed to be at the candidate positions less than a reference value; For each of the candidate positions included in the combination, a template (6) of the object assumed to be at the candidate position is evaluated for a degree of match between the point cloud data. The evaluation index is set so that the higher the number of data included in the template among the data included in the point cloud data, the higher the evaluation index. First Score (S1) and calculating the first score calculated for each of the candidate positions. Calculating the sum (TS) of determining that the target object is located at the candidate position included in the combination based on the sum of the first scores; Equipped with.

[0072] According to the above aspect (12), it is possible to narrow down candidate positions for which the first score is to be calculated from candidate positions of objects extracted from point cloud data using constraint conditions. This makes it possible to reduce the number of candidate positions for which the first score is to be calculated, and effectively reduce the computational burden. Then, a total value of the first scores is calculated for each combination including the narrowed-down candidate positions, and it is possible to accurately detect objects with various arrangements, orientations, and numbers based on the total value.

[0073] (13) A program according to one aspect includes: In a computer device, acquiring point cloud data of at least one object (1); extracting a plurality of candidate positions (P) of the object from the point cloud data; Identifying a plurality of combinations (CPs) of the candidate positions that make the interference between the objects assumed to be at the candidate positions less than a reference value; For each of the candidate positions included in the combination, a template (6) of the object assumed to be at the candidate position is evaluated for a degree of match between the point cloud data. The evaluation index is set so that the higher the number of data included in the template among the data included in the point cloud data, the higher the evaluation index. First Score (S1) and calculating the first score calculated for each of the candidate positions. Calculating the sum (TS) of determining that the target object is located at the candidate position included in the combination based on the sum of the first scores; is possible.

[0074] According to the above aspect (13), it is possible to narrow down candidate positions for which the first score is to be calculated from candidate positions of objects extracted from point cloud data using constraint conditions. This makes it possible to reduce the number of candidate positions for which the first score is to be calculated and effectively reduce the computational burden. Then, a total value of the first scores is calculated for each combination including the narrowed-down candidate positions, and it is possible to accurately detect objects with various arrangements, orientations, and numbers based on the total value. [Explanation of symbols]

[0075] 1 palette 2a Plate member 2c Pillar 4 Space 6 Templates 100 Object detection device 102 Point cloud data acquisition unit 104 Candidate position extraction unit 106 Candidate position combination identification unit 108 Score Calculation Section 110 Object Identification Unit 111 Line segment extraction section 112 Target data extraction unit 114 Temporary candidate position calculation unit 116 Candidate position selection section Combination of CP candidate positions KP tentative candidate position LE line segment NR flanking region O reference point

Claims

1. a point cloud data acquisition unit for acquiring point cloud data of at least one object; a candidate position extraction unit for extracting a plurality of candidate positions of the object from the point cloud data; a candidate position combination specifying unit for specifying a plurality of combinations of the candidate positions in which the degree of interference between the objects assumed to be at the candidate positions is less than a reference value; a score calculation unit that calculates a first score as an evaluation index for evaluating a degree of match between a template of the object hypothesized to be at the candidate position and the point cloud data for each of the candidate positions included in the combination, the first score being higher as the number of data included in the template among the data included in the point cloud data increases, and calculates a total value of the first scores calculated for each of the candidate positions; an object identification unit for identifying the object as being present at the candidate position included in the combination based on the total value of the first scores; An object detection device comprising:

2. The object detection device according to claim 1 , wherein the object identification unit identifies the object as being located at the candidate position included in the combination with the largest sum.

3. The object detection device according to claim 1 , wherein the candidate position combination specifying unit specifies a plurality of combinations of the candidate positions based on the presence or absence of the degree of interference.

4. The object detection device according to claim 1 , wherein the candidate position extraction unit excludes the candidate position having the smaller first score from two candidate positions whose distance from each other is smaller than a first threshold value.

5. The candidate position extraction unit a line segment extraction unit for extracting at least one line segment from the point cloud data; a target data extraction unit for extracting target data included in an adjacent area including the line segment from the point cloud data; a tentative candidate position calculation unit for calculating, as a tentative candidate position, a position of the template when the target data is applied to each of a plurality of feature positions of the template; a candidate position selection unit for selecting, as the candidate positions, a plurality of the tentative candidate positions having a peak in the number of tabulated tentative candidate positions for each of the target data and the characteristic positions; The object detection device according to claim 1 , comprising:

6. The object detection device according to claim 5 , wherein the line segment extraction unit extracts the line segments for which a second score corresponding to the number of data included in the adjacent region is equal to or greater than a third threshold value.

7. The object detection device according to claim 6 , wherein the line segment extraction unit sets the third threshold value based on a distance or an angle of the line segment relative to a reference position where the point cloud data is acquired.

8. The object detection device according to claim 5 , wherein the line segment extraction unit extracts the line segment whose angle with respect to a reference line that defines an object region in which the at least one object is located is less than a fourth threshold value.

9. The object detection device according to claim 5 , wherein the line segment extraction unit extracts the line segment based on at least two pieces of data from the point cloud data, the distance between which is within a predetermined range.

10. 10. The object detection device according to claim 5, wherein the line segment extraction unit extracts the line segment identified by a normal vector whose length and phase are quantized, using a reference point set in the object region in which the at least one object is located as a reference.

11. The object detection device according to claim 1 , wherein the object is a pallet having a pair of plate members supported by a plurality of columns.

12. acquiring point cloud data of at least one object; extracting a plurality of candidate positions of the object from the point cloud data; identifying a plurality of combinations of the candidate positions that make the degree of interference between the objects assumed to be at the candidate positions less than a reference value; a step of calculating, for each of the candidate positions included in the combination, a first score as an evaluation index for evaluating the degree of match between the template of the object hypothesized to be at the candidate position and the point cloud data, the first score being higher as the number of data included in the template among the data included in the point cloud data increases, and calculating a total value of the first scores calculated for each of the candidate positions; determining that the object is located at the candidate position included in the combination based on the sum of the first scores; An object detection method comprising:

13. In a computer device, acquiring point cloud data of at least one object; extracting a plurality of candidate positions of the object from the point cloud data; identifying a plurality of combinations of the candidate positions that make the degree of interference between the objects assumed to be at the candidate positions less than a reference value; a step of calculating, for each of the candidate positions included in the combination, a first score as an evaluation index for evaluating the degree of match between the template of the object hypothesized to be at the candidate position and the point cloud data, the first score being higher as the number of data included in the template among the data included in the point cloud data increases, and calculating a total value of the first scores calculated for each of the candidate positions; determining that the object is located at the candidate position included in the combination based on the sum of the first scores; A program that can be executed.

Images