OBJECT DETECTION DEVICE AND OBJECT DETECTION METHOD

Abstract

One object of the invention is to accurately estimate the shape and position of a vehicle in a case where a corner section cannot be observed. An object detection device comprises a detection unit that detects an object using environmental information obtained from an external sensor, a shape information management unit that processes shape information indicating the shape of the object, a shape estimation unit that estimates the shape of the object based on the shape information and partial shape information obtained from the external information, which indicates the shape of at least part of the object, and a position estimation unit that estimates the position of the object using the estimated shape of the object.

Description

Technical field

[0001] The present invention relates to an object detection device and an object detection method for detecting the position of another vehicle in the vicinity of a host vehicle. Technical background

[0002] In implementing preventive safety functions, automated driving, and similar systems, it is effective to detect an object in the environment using an in-vehicle sensor and perform detection processing similar to image processing and point cloud processing. However, if the positional relationship between the host vehicle and another vehicle changes during vehicle position estimation using the in-vehicle sensor, the observed location shifts from the area in front of the vehicle to the side of the vehicle, making the position estimate unstable. Therefore, a position estimation method that takes the observation point into account is required.

[0003] PTL 1 discloses in the solution section of the abstract of the patent a method for identifying an observation point by observing a corner section of a vehicle and for estimating the vehicle position based on the corner section, wherein the method is described as "an object detection device 3 for detecting another vehicle that is in the vicinity of a vehicle is designed to determine whether there is a partially detected vehicle that is partially hidden by another vehicle that is located in front of it among several detected other vehicles, and in a case where a partially detected vehicle is present, if a corner section of the partially detected vehicle can be detected,to calculate the center position of the partially detected vehicle based on the corner section of the partially detected vehicle by taking into account the length of the partially detected vehicle in the front-to-back direction and the length of the partially detected vehicle in the vehicle width direction, which are used as predefined lengths in the front-to-back direction and the vehicle width direction. List of citations from patent literature

[0004] PTL 1: JP 2022-011745 A Summary of the invention: Technical problem

[0005] By using the method described above from PTL 1, the center position of a partially detected vehicle, which is partially obscured by another vehicle in front of it, can be easily calculated with respect to the corner segment of the partially detected vehicle. However, the problem is that an accurate estimate is only possible if the corner segment can be observed, which occurs, for example, when the vehicles are close to each other as they pass one another.

[0006] Therefore, in view of this problem, one object of the present invention is to provide an object recognition device and an object recognition method by which the shape and position of a vehicle can be accurately estimated even in a case where a corner of the vehicle cannot be observed. Solution to the problem

[0007] To solve the above problem, an object detection device according to the present invention comprises a detection unit that detects an object using environmental information obtained from an external sensor, a shape information management unit that handles shape information specifying the shape of the object, a shape estimation unit that estimates the shape of the object based on the shape information and partial shape information obtained from the external information, which specifies the shape of at least a part of the object, and a position estimation unit that estimates the position of the object using the estimated shape of the object. Advantageous effects of the invention

[0008] According to the present invention, the shape and position of a vehicle can be accurately estimated when a corner of the vehicle cannot be observed. Brief description of the drawings

[0009] They show: Fig. 1 a functional block diagram of a configuration of an object detection device according to an embodiment, Fig. 2 a diagram of a database of a shape information management unit, Fig. 3 a processing flow diagram according to an object recognition device according to an embodiment, Fig. 4 a diagram of an overtaking scene, Fig. 5 a diagram of a partial shape extraction by a partial shape extraction unit, Fig. 6A a diagram of a procedure for determining a corner section by principal component analysis, Fig. 6B a diagram of a procedure for determining a corner section by principal component analysis, Fig. 7 a diagram of a shape comparison by a shape comparison unit, Fig. 8 a diagram of a scene in which shape information is updated, Fig. 9 a diagram of an update of shape comparison information, Fig. 10 a diagram of a characteristic shape of a vehicle and Fig. 11 a diagram of an example of a comparison based on characteristic values. Description of embodiments

[0010] One embodiment of the present invention is described in detail below.

[0011] Fig. Figure 1 is a functional block diagram showing a configuration of an object detection device 1 according to an embodiment of the present invention. The object detection device 1 shown here is an in-vehicle device to which an external sensor 2, such as a LiDAR sensor, is connected, and it detects the type and position of an object in the vicinity of a host vehicle V0, in particular the type and position of another vehicle, based on the output of the external sensor 2. It should be noted that the object detection device 1 is, in particular, a computer comprising a computing device, a storage device, and the like, and operates as various functional units as described below when the computing device executes a predetermined control program.Additionally, the external sensor 2 is, for example, a LiDAR sensor located at four corners on the host vehicle V0: front left, front right, rear left and rear right.

[0012] Here, the shape of the vehicle, as viewed from above, is essentially rectangular, regardless of the vehicle type (passenger car, truck, two-wheeler, etc.). Therefore, the vehicle's position can be easily calculated based on the corner segment of the vehicle, either by detecting or estimating it. Generally, with a data-driven method such as deep learning, the computational effort is high, resulting in a slow processing speed for the object detection device 1. In contrast, the position estimation method, using the corner segment as a reference, prevents this slowdown in the detection processing due to its low computational effort. Therefore, in the object detection device 1, according to the present embodiment, the shape of the vehicle, as viewed from above, is approximated as a rectangle, and the determination of the vehicle's type and position is performed simply under this assumption. <Objekterkennungsvorrichtung 1>

[0013] As in Fig. As shown in Figure 1, the object recognition device 1 comprises an object detection unit 10, a shape information management unit 11, a shape estimation unit 12, and a position estimation unit 13. The shape estimation unit 12 comprises a partial shape extraction unit 12a, a corner section determination unit 12b, and a shape comparison unit 12c. The details of each part are described in the sequence below. <<Objekterfassungseinheit 10> >

[0014] The object detection unit 10 detects the type of object by using the measurement information from the external sensor 2. A detection target is a vehicle type such as a passenger car, a truck, or a two-wheeled vehicle. <<Forminformations-Managementeinheit 11> >

[0015] The shape information management unit 11 treats object shape information as a database. This database can be stored in a storage device inside the vehicle, or a database can be provided outside the vehicle, and information from the database can be retrieved through communication. Fig. 2 It is assumed that the shape information management unit 11 stores information in databases in the form of a passenger vehicle database 11a, a truck database 11b and a two-wheeled vehicle database 11c for each vehicle type to be recorded, and that shape information for each vehicle type is registered in each database.

[0016] It should be noted that the shape information comprises length information in the front-to-rear direction of the vehicle (hereinafter referred to as vehicle length), length information in the vehicle width direction (hereinafter referred to as vehicle width), and shape comparison information with the partial shape information. The shape comparison information can be a contour, a three-dimensional voxel, a 3D CAD model, or a feature value calculated by a mathematical model. These shape information components can be predefined, newly generated based on a capture result from object capture unit 10, or be information obtained from a previous tracking of the vehicle. The shape information can also be updated based on the capture result from object capture unit 10.

[0017] An update procedure is described below. <<Formschätzeinheit 12> >

[0018] The shape estimation unit 12 estimates an object shape using the partial shape information that specifies the shape of at least part of the object captured by the object capture unit 10, and the shape information handled by the shape information management unit 11.

[0019] The partial shape extraction unit 12a calculates partial shape information based on the object information acquired by the object acquisition unit 10. The partial shape information can be part of a contour, part of a three-dimensional voxel, part of a 3D CAD model, or part of the feature value calculated by a mathematical model.

[0020] The corner section determination unit 12b determines whether a corner section of the object is included in the sub-shape information calculated by the sub-shape extraction unit 12a. Corner section refers to a vertex and two distinct semi-lines emanating from the vertex. Specifically, in a case where the other vehicle V1 is in front of the host vehicle V0, corner section refers to the right or left end of the rear end face of the other vehicle V1, and in a case where the other vehicle V1 is behind the host vehicle V0, it refers to the right or left end of the front end face of the other vehicle V1.

[0021] If the corner section determination unit 12b determines that no corner section is present, the shape comparison unit 12c compares the partial shape information with the shape comparison information in the database. A comparison procedure is now described. First, the vehicle to which the partial shape information in the database should match is selected. The vehicle is selected, for example, using an index indicating the degree of similarity between the partial shape information and the shape comparison information. Accordingly, with reference to the database, the length and width of the vehicle are estimated in the front-to-back direction of the oncoming vehicle V1. Next, the position of at least one corner section of the vehicle is estimated by comparing the partial shape information and the shape comparison information. <<Positionsschätzeinheit 13> >

[0022] The position estimation unit 13 estimates the vehicle's position using the vehicle shape estimated by the shape estimation unit 12. Using the corner section estimated by the shape comparison unit 12c as a reference, the remaining corner section of the rectangle is calculated using the vehicle length and width as vehicle width and vehicle length information stored in the shape information management unit 11. <Beispiel des Betriebs der Objekterkennungsvorrichtung 1>

[0023] Below, an example of the operation of the object recognition device 1 according to the present embodiment is described in detail with reference to the flowchart from Fig. 3 described. The object detection device 1 executes this routine each time data is received from the external sensor 2. In the following example of operation, the external sensor 2 is a LiDAR sensor, and the object detection unit 10 performs vehicle detection using a three-dimensional point cloud d3. Additionally, the shape information management unit 11 processes shape information for a passenger car, a truck, and a two-wheeler in a database and predefines the vehicle width, vehicle length, and shape comparison information as shape information. The shape comparison information is an outline of the vehicle as viewed from the side. Additionally, as described in Fig. Figure 4 shows a scene in which the host vehicle V0, traveling from left to right in the drawing, and the other vehicle V1, approaching from the opposite direction, pass each other, defined as the operating scene.

[0024] In process P1, the object detection unit 10 of the object detection device 1 performs vehicle detection processing by clustering based on the data of the three-dimensional point cloud d3. Specifically, point clouds with small Euclidean distances between them are clustered. Next, using the latitude, longitude, and altitude information of the clustered point cloud, it is determined whether the clustered object is a vehicle.

[0025] In particular, it is determined that a vehicle exists if its width, length and height are within a certain threshold range.

[0026] In process P2, the partial shape extraction unit 12a of the object recognition device 1 calculates a partial shape based on the point cloud clustered in process P1. The partial shape is a contour of the opposite vehicle V1 as viewed from the side. Fig. Figure 5 shows an example of calculating a partial form. Fig. Reference symbol 50 denotes the three-dimensional point cloud d3 clustered in process P1, and reference symbol 51 denotes an outline of a two-dimensional point cloud d2a calculated in process P2.

[0027] A specific procedure for calculating a partial shape is described. First, the three-dimensional point cloud d3, clustered in process P1, is projected vertically onto a plane defined to coincide with the side surface of the vehicle and converted into a two-dimensional point cloud d2a. Next, a polygon containing all the point clouds is obtained by Delaunay triangulation using the projected two-dimensional point cloud d2a and defined as contour 51 of the two-dimensional point cloud d2a. Contour 51 of the point cloud is a partial shape.

[0028] In process P3, the corner section determination unit 12b of the object detection device 1 determines whether the corner section of the vehicle detected in process P1 can be detected. A specific determination procedure is now described. First, the three-dimensional point cloud d3 clustered in process P1 is projected vertically onto the ground and converted into a two-dimensional point cloud d2b. Next, a principal component analysis is performed on the projected two-dimensional point cloud d2b to calculate a first principal component 61 and a second principal component 62.

[0029] Fig. Figure 6A shows an example of a principal component analysis result when a corner was detected, and Fig. Figure 6B shows an example of a result when a corner was not detected. As can be seen from the comparison between the two drawings, the dispersion in the direction of the second principal component 62 is greater when the section under consideration has a corner than when the section under consideration does not. If the variance of the point cloud in the direction of the second principal component 62 is greater than that in Fig. If the threshold shown in 6A is lower, it is therefore determined that a corner has been detected, and if the variance is smaller than that shown in Fig. If the threshold shown in 6B is not detected, it is determined that a corner was not recognized.

[0030] If the corner section of the other vehicle V1 was detected, as in Fig. As shown in 6A, processing then continues with the processing from process P5. In contrast, as shown in Fig. Figure 6B shows that processing continues with the processing from process P4 if the corner section of the other vehicle V1 is not detected. Here, it is assumed that the corner section cannot be detected, and processing continues with the processing from process P4.

[0031] In process P4, the shape comparison unit 12c compares the partial shape calculated in process P2 with pre-stored shape comparison information. A specific processing procedure is described. First, the vehicle type is determined based on the size of the partial shape. Specifically, the vehicle height calculated from the partial shape is compared with the heights of the truck, passenger car, and two-wheeler calculated from the shape comparison information, and the type with the closest height is determined to be the type of the oncoming vehicle V1. Here, it is assumed that the vehicle is identified as a passenger car. Therefore, the length and width of the oncoming vehicle V1 can be estimated with reference to the passenger car information in the database. This allows the rectangular shape of the other vehicle V1, viewed from above, to be determined.Next, a comparison is performed between the partial shape and the shape comparison information.

[0032] Fig. Figure 7 shows an example of the comparison. In Fig. Reference 7 denotes a reference numeral 70 for shape comparison information of a passenger vehicle, reference numeral 71 denotes a partial shape and a windshield section of the shape comparison information, and reference numeral 72 denotes a corner section of the vehicle. As in Fig. As shown in Figure 7, the shape comparison information and the partial shape are displayed overlapping, and the partial shape is moved in the front-to-back direction of the vehicle. At this time, a comparison is performed by searching for a section of greatest similarity between the inclined shape of the vehicle's windshield section in the partial shape and the inclined shape of windshield section 71 from the shape comparison information. Finally, the position of the corner section of the other vehicle V1 is calculated by setting the front end section of the vehicle as corner section 72 based on the shape comparison information.

[0033] In process P5, the position estimation unit 13 calculates the vehicle's position based on the corner segment's position estimated in process P4. Specifically, the remaining corner segment of the vehicle is estimated using the vehicle's length and width estimated in process P4. Additionally, the center of the estimated rectangle is calculated as the vehicle's center position. If process P5 is reached without having passed through process P4, the position of the other vehicle, V1, can be calculated based on the corner segment under consideration.

[0034] The following advantageous effects can be obtained by the object recognition device 1 according to the embodiment described above. (1) In a case where the corner section of the other vehicle cannot be detected, it can be estimated even in a scene where the vehicle's rear cannot be detected, for example, in a case where the vehicles are passing each other, and the estimated corner section can be used as determining material to estimate the position and shape of the other vehicle by calculating the corner section using the partial shape and the shape information stored in the object information management unit. (2) By estimating the position of the other vehicle using the corner segment thus obtained, the position of the other vehicle can be accurately calculated even in a case where the corner segment cannot be estimated. In addition, a general machine learning object recognition method requires a large amount of training data, whereas using the present method, the vehicle position can be calculated without requiring any training data. (3) The shape information management unit 11 stores shape information for a passenger car, a truck, and a two-wheeled vehicle. Because the characteristics of the size and shape of the vehicle depend heavily on the type of vehicle being detected, the accuracy of the position estimation decreases when detection processing focuses on only one vehicle type. However, as in the case of the object detection device 1, the position of the other vehicle can be estimated very accurately by storing several types of shape information and using them selectively according to the type of vehicle being detected. (4) By storing the vehicle length, vehicle width, and shape comparison information as shape information of the vehicle processed by the shape information management unit 11, at least one corner segment of the object can be detected and estimated, and the vehicle position can be easily estimated with high accuracy by using the corner segment as a reference. This prevents the processing load of object detection and the slowdown of the processing speed. Additionally, by storing the shape comparison information, a comparison with the partial shape can be made if the corner segment cannot be detected, the position of the corner segment can be estimated, and the position of the other vehicle can be calculated accurately. (5) By storing the contour information of the object viewed from the side as shape comparison information, a comparison can be made to calculate the corner section of the other vehicle when only the side surface of the other vehicle is considered, as occurs in a case where the vehicles are passing each other. Because the contour information is two-dimensional, the amount of data is also smaller than with three-dimensional CAD information or voxel information. Therefore, processing errors in object recognition can be reduced compared to a case where shape comparison information is stored with a large amount of data. <datenbank-aktualisierungsverarbeitung>

[0035] This section describes a process for updating the shape information in the database based on information obtained from observation. The assumed scene occurs before passing into Fig. 4, where a positional relationship exists in which the host vehicle V0 and the other vehicle V1 are facing each other, as in Fig. 8 shown. A reference 80 in Fig. Figure 8 denotes a three-dimensional point cloud d3 acquired by the external sensor 2 of the host vehicle V0. It is assumed that the object type is estimated using the same procedure as in process P1 and determined to be a passenger vehicle. First, the passenger vehicle's shape information is duplicated in the database and registered therein as passenger vehicle A. At this time, the vehicle length, vehicle width, and shape comparison information inherit the information from the copy source. Next, the database values ​​are updated. Because an accurate value for the vehicle width can be observed based on the positional relationship between the vehicles, the width of passenger vehicle A in the database is updated to reflect the observed vehicle width. The vehicle length is not updated because an accurate value cannot be observed based on the positional relationship between the vehicles.

[0036] Part of the shape comparison information is provided by the in Fig. The procedures shown in section 9 have been updated. Reference numeral 90 in Fig. 9 denotes a clustered point cloud. First, as in process P2, a sub-shape is extracted. A reference symbol 91 in Fig. Item 9 specifies the extracted partial shape. Next, as in process P4, a comparison is made between the shape comparison information of passenger vehicle A and the partial shape. Finally, a section obtained as a partial shape is replaced in the shape comparison information in the database.

[0037] As described above, the shape information processed by the shape information management unit 11 is updated using the object information acquired by the detection unit. This allows the vehicle position to be estimated with higher accuracy than if the position were estimated using predefined shape information. Additionally, by updating the shape information through observation, the vehicle position can be estimated with high accuracy even if the amount of pre-stored shape information is small. This prevents an increase in the memory consumption of the object recognition device 1 and a slowdown in processing speed.

[0038] Even if the comparison between the outlines described so far is carried out with a focus on the shape of the windshield section, according to the present invention, it is also possible to determine, based on the point cloud information obtained through observation, which part of the vehicle shape should be the focus of the comparison. A specific implementation method is described.

[0039] A scene is assumed in which the vehicles are in Fig. 8 facing each other.

[0040] First, a characteristic form of the vehicle is defined, as in Fig. 10 shown and specified. In Fig. Reference numeral 10 denotes a section with low curvature in a sub-shape, and reference numeral 101 denotes a section with high curvature in the sub-shape. Here, it is assumed that a section with high curvature is suitable for comparison. Next, a section with high curvature, such as 101, is stored in the database along with the shape comparison information. Finally, when comparing the sub-shape and the shape comparison information, weighting is performed to increase the degree of similarity between the section with high curvature, as described above. This allows the position of the opposing vehicle V1 to be estimated with high accuracy compared to the case where no weighting is performed. Additionally, there is an infinite number of shapes of ordinary vehicles, and it is difficult to predetermine weights for all matching shapes.According to the present embodiment, however, the weighting can be carried out while the vehicle is driving, without having to predetermine the weights for comparison.

[0041] In addition to the embodiment described above, the comparison method according to the present invention can be changed according to the distance between the host vehicle V0 and the opposing vehicle V1. Here, a case is considered in which the vehicles are so close to each other that the outline cannot be detected. In this case, it is not possible to accurately estimate the vehicle position using the contour comparison method described so far. Therefore, in a case where the vehicles are close together, a comparison based on a feature value, as described in Fig. Figure 11 shows how this is done. The feature value can be an irregular shape of the vehicle or a luminance value from a point cloud obtained from the vehicle. A comparison using an irregular shape as the feature value is shown below. First, the feature value of the vehicle's side surface is calculated before the vehicles are close together, and the corresponding portion of the shape comparison information is compared and stored in the database.

[0042] A reference numeral 110 in Fig. 11 denotes a section with particularly high irregularities among the calculated irregularities. If vehicles are close together, feature values ​​are calculated similarly even for sub-forms. In Fig. Reference numeral 111 designates a sub-shape, and reference numeral 112 designates a section with particularly large irregularities among the irregularities in the sub-shape. In this case, a comparison is performed such that the irregular shape in the sub-shape matches the section of the irregular shape in the database. This allows the vehicle position estimation to be very accurate using the contour-based comparison method described above, regardless of the distance between the host vehicle V0 and the opposing vehicle V1.

[0043] In addition to the embodiment described above, the present invention allows vehicle speed information to be used for comparison. Consider a case where the vehicle is being tracked and its speed was estimated prior to the current time. At that time, the vehicle's current position can be narrowed down using the speed and operating cycle of the object detection device 1, assuming that the change in speed from the previous time to the present time is within a certain range. Using this approach, the comparison is performed within a narrowed range of vehicle positions. This prevents a reduction in processing speed by narrowing the search area during comparison.

[0044] Although the present invention has been described above with reference to one embodiment, it is not limited to or by that embodiment. Various modifications that might occur to those skilled in the art within the scope of protection of the present invention can be made to the configurations and details of the present invention. Reference symbol list 1 object recognition device 10 object detection units 11 Form Information Management Unit 12 shape estimation units 12a Partial Form Extraction Unit 12b Corner section determination unit 12c Form comparison unit 13 Position estimation unit 2 external sensors QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2022-011745 A

[0004]

Claims

[1] Object recognition device comprising: a detection unit that detects an object using environmental information obtained from an external sensor, a shape information management unit that handles shape information specifying the shape of the object, a shape estimation unit that estimates the shape of the object based on the shape information and partial shape information obtained from external information, which specifies the shape of at least part of the object, and A position estimation unit that estimates the position of the object using the estimated shape of the object. [2] Object recognition device according to claim 1, wherein the detection unit detects the object using a point cloud captured by the external sensor and The shape estimation unit estimates the shape of the object by comparing the shape information and the partial shape information, thereby specifying the shape of at least part of the object using the point cloud. [3] Object recognition device according to claim 1, wherein the shape estimation unit, in a case where the object is represented as a rectangle, receives the position of at least one corner section of the rectangle and uses the position of the corner section to estimate the shape of the object. [4] Object recognition device according to claim 1, wherein, in a case where the object is represented as a rectangle, a corner section of the rectangle is not included in the object detected by the detection unit, the shape estimation unit uses the shape information and the partial shape information to estimate the shape of the object. [5] Object recognition device according to claim 1, wherein the shape information management unit stores the predefined shape information and / or the shape information detected by the external sensor and uses the shape information detected by the external sensor to update the stored shape information. [6] Object recognition device according to claim 1, wherein the shape information management unit stores contour information of an object viewed from the side as shape information. [7] Object recognition method performed by a computing device, the method comprising: a detection step for capturing an object using information about the external environment obtained from an external sensor, a shape estimation step for estimating the shape of the object based on shape information specifying the shape of the object and on partial shape information obtained from external information, which specifies the shape of at least part of the object, and a position estimation step to estimate the position of the object using the estimated shape of the object.

Images