Object recognition device, object recognition method, and computer-readable recording medium

Abstract

The extraction unit (101) extracts detection points of stationary objects from among a plurality of detection points around a vehicle (200) detected by an external detection sensor (501) at a plurality of detection timings as stationary object detection points. The grouping unit (105) groups two or more stationary object detection points estimated to be detection points of the same stationary object from among the plurality of stationary object detection points extracted by the extraction unit (101) at a plurality of detection timings.

Description

Technical Field

[0001] The present invention relates to a technology for recognizing an environment around a vehicle. Background Art

[0002] In Patent Document 1, information on the position of an obstacle detected by an obstacle detection device when a driver moves a vehicle from an arbitrary location to a parking position is stored in a nonvolatile memory. In Patent Document 1, when the vehicle is parked at the same parking position next time, parking assistance is performed using the information on the position of the obstacle stored in the nonvolatile memory.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2008-207732 Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] In the technology of Patent Document 1, when using spatially sparse sensors as obstacle detection devices, an area that is actually not drivable may be mistakenly recognized as an area that is drivable. That is, in the technology of Patent Document 1, there is a problem that the environment around the vehicle cannot be accurately recognized.

[0008] The main object of the present invention is to solve the above-mentioned problems. More specifically, the main object of the present invention is to accurately recognize the environment around a vehicle even when using spatially sparse sensors.

[0009] Technical means for solving technical problems

[0010] The object recognition device involved in the present invention has:

[0011] an extracting unit that extracts, as the stationary object detection point, detection points of the stationary object from among a plurality of detection points around the vehicle detected by the sensor at a plurality of detection timings; and

[0012] A grouping unit groups two or more stationary object detection points estimated to be detection points of the same stationary object among the plurality of stationary object detection points extracted by the extraction unit at the plurality of detection timings.

[0013] Effects of the Invention

[0014] According to the present invention, even when spatially sparse sensors are used, the environment around the vehicle can be accurately recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1This is a diagram showing an overview of the operation of the object recognition device according to the first embodiment.

[0016] Figure 2 This is a diagram showing an example of a functional configuration of an object recognition device according to Embodiment 1.

[0017] Figure 3 This is a flowchart showing an example of the operation of the object recognition device according to the first embodiment.

[0018] Figure 4 This is a flowchart showing details of the process of updating the position and posture of the vehicle according to the first embodiment.

[0019] Figure 5 Detailed flowchart showing the process of deleting stationary object detection point information according to the first embodiment.

[0020] Figure 6 Detailed flowchart showing the stationary object determination process of the detection point information according to the first embodiment.

[0021] Figure 7 Detailed flowchart of the coordinate conversion process of the stationary object detection point information according to the first embodiment.

[0022] Figure 8 Detailed flowchart showing the storage process of the stationary object detection point information according to the first embodiment.

[0023] Fig. 9 This is a flowchart showing the details of the grouping process involved in Implementation Example 1.

[0024] Fig.10 This is a flowchart showing the details of the packet deletion process involved in the first embodiment.

[0025] Fig.11 This is a diagram for explaining the effects involved in the first embodiment.

[0026] Fig.12 This is a diagram for explaining the effects involved in the first embodiment.

[0027] Fig.13 This is a diagram for explaining the effects involved in the first embodiment.

[0028] Fig.14 This is a diagram showing an overview of the operation of the object recognition device according to the second embodiment.

[0029] Fig.15 This is a diagram showing an overview of the operation of the object recognition device according to the second embodiment.

[0030] Fig.16This is a flowchart showing an example of the operation of the object recognition device according to the second embodiment.

[0031] Fig.17 This is a flowchart showing the details of the packet concatenation process involved in the second embodiment.

[0032] Fig.18 This is a diagram showing a connection example when the number of packets connected to the packet j according to the second embodiment is 0.

[0033] Fig.19 This is a diagram showing a connection example when the number of packets connected to the packet j according to the second embodiment is M-1.

[0034] Fig. 20 This is a diagram showing a connection example when the number of packets connected to the packet j according to the second embodiment is M1.

[0035] Fig.21 This is a diagram for explaining the effects involved in the second embodiment.

[0036] Fig. 22 This is a diagram for explaining the effects involved in the second embodiment.

[0037] Fig.23 This is a diagram showing a hardware configuration example of the object recognition device according to Embodiment 1. DETAILED DESCRIPTION

[0038] Hereinafter, embodiments of the present invention will be described using the drawings. In the following description of the embodiments and the drawings, parts with the same reference numerals represent the same parts or corresponding parts.

[0039] Implementation method 1.

[0040] ***summary***

[0041] First, refer to Figure 1 , an overview of the operation of the object recognition device involved in this embodiment is described.

[0042] Figure 1 φ is map data showing a state where the vehicle 200 is traveling at a speed V on a road. Figure 1 In order to make the drawings clear, the roads are not drawn, but Fig.11 As shown, vehicle 200 is traveling on road 600 . Fig.11 In the figure, the vehicle 200 is traveling on a road 600 which is separated by guardrails 610 on both sides. Fig.11 In the figure, the road 600 is an area where the vehicle 200 can travel, and is a travelable area. The outer side of the guardrail 610 is an area where the vehicle 200 cannot travel, and is a non-travelable area.

[0043] Vehicle 200 is equipped with an object recognition device and an exterior detection sensor which will be described later.

[0044] The external detection sensor searches for objects (including stationary objects and moving objects) located around the vehicle 200 at each repeated detection timing. The detection timing arrives in a short period such as milliseconds or microseconds. The external detection sensor is, for example, a millimeter wave radar, a camera, a sonar, etc. The external detection sensor detects objects existing around the vehicle 200 as a plurality of detection points 300.

[0045] in addition, Figure 1 In the Fig.11 The illustration of the detection point 300 corresponding to the guardrail 610 on the right side is shown.

[0046] Figure 1 (a) shows that t=t n That is, Figure 1 (a) shows an example in which the detection timing is set to t n A plurality of detection points 300 detected by the external detection sensors are arranged on the mapping data. Since the external detection sensors are spaced sparsely, at time t n Objects existing around vehicle 200 are recognized as points.

[0047] The object recognition device according to this embodiment extracts detection points on stationary objects from among the plurality of detection points 300 as stationary object detection points (hereinafter referred to as stationary object detection points 350 ). The object recognition device according to this embodiment accumulates the positions of the extracted stationary object detection points 350 .

[0048] Figure 1 (b) shows that t=t n-m ~t n The cumulative status of the test results. That is, Figure 1 (b) shows the following example: n-m To time t n The stationary object detection points 350 detected by the external detection sensor at multiple detection timings are arranged on the mapping data.

[0049] The object recognition device is n-m To time t n The stationary object detection points 350 detected by the external detection sensor are accumulated.

[0050] Thus, in the present embodiment, a plurality of stationary object detection points 350 at a plurality of detection timings are mapped, so that a stationary object is recognized as a point group.

[0051] Figure 1 (c) shows an example of the grouped stationary object detection points 350 .

[0052] The object recognition device groups the stationary object detection points 350 estimated as detection points of the same stationary object among the accumulated stationary object detection points 350 . Figure 1 In (c), the stationary object detection points 350 within the frame line are processed as one group 400.

[0053] The grouping is performed using, for example, spatial density. A group whose spatial density is less than a threshold value is determined to be a false detection. The group determined to be a false detection is deleted from the mapping data.

[0054] Figure 1 In the example of (c), three groups 400 exist around the vehicle 200 . Each group 400 represents a stationary object that exists around the vehicle 200 .

[0055] Thus, in this embodiment, even when using spatially sparse sensors, a plurality of detection points 300 can be used. Therefore, according to this embodiment, the positions of stationary objects around the vehicle 200 can be accurately recognized, and the environment around the vehicle 200 can be accurately recognized.

[0056] ***Structure description***

[0057] Figure 2 The functional configuration of the object recognition device 100 according to the present embodiment and devices connected to the object recognition device 100 are shown.

[0058] The vehicle 200 is equipped with the object recognition device 100 , an external detection sensor 501 , a vehicle motion detection sensor 502 , and a vehicle position / posture sensor 503 .

[0059] Figure 2 In the figure, only elements directly related to the description of this embodiment are shown. For example, the vehicle 200 includes various elements such as an engine, brakes, and tires, but they are not directly related to the description of this embodiment, so they are omitted from the figure.

[0060] The object recognition device 100 is a computer mounted on the vehicle 200. The object recognition device 100 includes an extraction unit 101, a vehicle position / posture estimation unit 102, an information updating unit 103, an accumulation unit 104, and a grouping unit 105. The extraction unit 101, the vehicle position / posture estimation unit 102, the information updating unit 103, the accumulation unit 104, and the grouping unit 105 are described in detail later.

[0061] The operation procedure of the object recognition device 100 corresponds to the object recognition method. The program for realizing the operation of the object recognition device 100 corresponds to the object recognition program.

[0062] The object recognition device 100 is connected to an external detection sensor 501 , a vehicle motion detection sensor 502 , and a vehicle position / posture sensor 503 .

[0063] The external detection sensor 501 detects objects outside the vehicle 200, that is, objects around the vehicle 200. As described above, the external detection sensor 501 is a millimeter wave radar, a camera, a sonar, etc. The external detection sensor 501 detects objects existing around the vehicle 200 as a plurality of detection points 300.

[0064] Furthermore, the external detection sensor 501 outputs the detection point information 111 indicating the position and speed of the detection point 300 to the object recognition device 100 .

[0065] The vehicle motion detection sensor 502 acquires the speed and yaw rate of the vehicle 200. Then, the vehicle motion detection sensor 502 outputs the vehicle motion information 112 indicating the acquired speed and yaw rate to the object recognition device 100.

[0066] The vehicle position / attitude sensor 503 acquires the position and attitude (orientation) of the vehicle 200. Then, the vehicle position / attitude sensor 503 outputs the vehicle position / attitude information 113 indicating the acquired position and attitude (orientation) to the object recognition device 100. The vehicle position / attitude sensor 503 is, for example, a GNSS (Global Navigation Satellite System) or a geomagnetic sensor.

[0067] Fig.23 A hardware configuration example of the object recognition device 100 according to the present embodiment is shown.

[0068] As hardware, the object recognition device 100 includes a processor 901 , a main storage device 902 , an auxiliary storage device 903 , and a communication device 904 .

[0069] The auxiliary storage device 903 stores a program that implements Figure 2 The functions of the extraction unit 101, the vehicle position / posture estimation unit 102, the information updating unit 103 and the grouping unit 105 are shown.

[0070] These programs are loaded from the auxiliary storage device 903 to the main storage device 902. Then, the processor 901 executes these programs to perform the operations of the extraction unit 101, the vehicle position / posture estimation unit 102, the information update unit 103, and the grouping unit 105 described later.

[0071] Fig.23, a state in which the processor 901 executes a program is schematically shown, and the program realizes the functions of the extraction unit 101, the vehicle position / posture estimation unit 102, the information updating unit 103 and the grouping unit 105.

[0072] Figure 2 The accumulation unit 104 shown is realized by the main storage device 902 or the auxiliary storage device 903 .

[0073] The communication device 904 receives the detection point information 111 from the external detection sensor 501 . Furthermore, the communication device 904 receives the vehicle motion information 112 from the vehicle motion detection sensor 502 . Furthermore, the communication device 904 receives the vehicle position / posture information 113 from the vehicle position / posture sensor 503 .

[0074] Next, Figure 2 The extraction unit 101, the vehicle position / posture estimation unit 102, the information updating unit 103, the accumulation unit 104, and the grouping unit 105 shown will be described.

[0075] The extraction unit 101 extracts, as stationary object detection points 350 , stationary object detection points 300 of the plurality of detection points 300 around the vehicle 200 detected by the external detection sensor 501 at a plurality of detection timings. The periphery of the vehicle 200 refers to the scannable range of the external detection sensor 501 .

[0076] That is, the extraction unit 101 acquires the detection point information 111 from the external detection sensor 501 via the communication device 904 at each detection timing. The extraction unit 101 acquires the detection point information 111 at each detection point 300. The detection point information 111 shows the position coordinates of the detection point 300, the speed of the detection point 300, and the detection time of the external detection sensor 501. The position coordinates of the detection point 300 shown in the detection point information 111 are position coordinates in a coordinate system with the position of the vehicle 200 as the origin.

[0077] Furthermore, the extraction unit 101 acquires the vehicle motion information 112 from the vehicle motion detection sensor 502 via the communication device 904 .

[0078] The extraction unit 101 extracts the detection point 300 of the stationary object from the plurality of detection points 300 indicated by the detection point information 111 at each detection timing as the stationary object detection point 350. More specifically, the extraction unit 101 obtains the ground speed of each detection point 300 using the vehicle motion information 112, and extracts the stationary object detection point 350 based on the ground speed of each detection point 300.

[0079] Then, the extraction unit 101 generates the stationary object detection point information 114 indicating the position coordinates and detection time of the extracted stationary object detection point 350. The extraction unit 101 generates the stationary object detection point information 114 for each stationary object detection point 350. The position coordinates of the stationary object detection point 350 indicated by the stationary object detection point information 114 are position coordinates in a coordinate system with the position of the vehicle 200 as the origin. Then, the extraction unit 101 outputs the generated stationary object detection point information 114 to the information updating unit 103.

[0080] The processing performed by the extraction unit 101 corresponds to the extraction processing.

[0081] The vehicle position / posture estimating unit 102 obtains one or both of the vehicle motion information 112 and the vehicle position / posture information 113 via the communication device 904 at each repeatedly arriving estimation timing. Then, the vehicle position / posture estimating unit 102 estimates the position and posture of the vehicle 200 at the current moment in a coordinate system (hereinafter referred to as a mapping coordinate system) having a certain direction at a certain point on the map as an origin, using one or both of the vehicle motion information 112 and the vehicle position / posture information 113.

[0082] Then, the vehicle position / posture estimation unit 102 stores the estimation result in the integration unit 104 as the position / posture information 115. The position / posture information 115 indicates the estimated time (current time) of the position and posture and the estimated position and posture of the vehicle 200.

[0083] In addition, the estimated timing may not coincide with the detected timing.

[0084] The information updating unit 103 acquires the stationary object detection point information 114 from the extracting unit 101, and further acquires the position / posture information 115 from the integrating unit 104. Then, the information updating unit 103 converts the position of the stationary object detection point 305 into the position of the mapping coordinate system using the stationary object detection point information 114 and the position / posture information 115. The information updating unit 103 stores the stationary object detection point information 116 indicating the position of the stationary object detection point 350 in the mapping coordinate system in the integrating unit 104.

[0085] Furthermore, the information updating unit 103 instructs the integrating unit 104 to delete the stationary object detection point information 116 and the position / posture information 115 that meet the prescribed conditions.

[0086] The integrating unit 104 integrates the position / posture information 115 acquired at a plurality of estimated timings.

[0087] Furthermore, the integrating unit 104 integrates stationary object detection point information 116 acquired at a plurality of detection timings.

[0088] Furthermore, the integrating unit 104 deletes the stationary object detection point information 116 and the position / posture information 115 that meet the predetermined conditions according to the instruction of the information updating unit 103 .

[0089] The processing performed by the integrating unit 104 corresponds to integrating processing.

[0090] The grouping unit 105 groups two or more stationary object detection points 350 estimated to be detection points of the same stationary object among the plurality of stationary object detection points 350. Figure 1 As described in (c), the grouping unit 105 groups the stationary object detection points 350 using the spatial density.

[0091] In addition, after performing grouping, the grouping unit 105 calculates the spatial density of two or more stationary object detection points included in each group. Then, the grouping unit 105 discards the group whose calculated spatial density is below the spatial density threshold. The grouping unit 105 can change the spatial density threshold according to the distance between the group and the vehicle 200.

[0092] The grouping unit 105 can display group information 117 indicating the grouping result on, for example, a display device in the vehicle 200 (not shown).

[0093] The processing performed by the grouping unit 105 is equivalent to the grouping processing.

[0094] ***Action description***

[0095] Next, an operation example of the object recognition device 100 according to the present embodiment will be described.

[0096] Figure 3 An operation example of the object recognition device 100 according to this embodiment is shown.

[0097] [Step S1: Processing for updating the position and posture of the vehicle 200]

[0098] In step S1 , the vehicle position / posture estimating unit 102 updates the position / posture of the vehicle 200 in the mapping coordinate system.

[0099] Figure 4 The details of step S1 are shown below. Figure 4 Provide explanation.

[0100] In step S1-1, the vehicle position / posture estimation unit 102 determines the calculation method of the position / posture of the vehicle 200. The vehicle position / posture estimation unit 102 determines the calculation method based on the detection accuracy of the vehicle motion detection sensor 502 and the vehicle position / posture sensor 503. For example, in the case of receiving a fault notification from the vehicle motion detection sensor 502 or the vehicle position / posture sensor 503, the vehicle position / posture estimation unit 102 decides not to use the sensor that is the source of the fault notification. In addition, in the case where the detection accuracy of the vehicle motion detection sensor 502 or the vehicle position / posture sensor 503 is significantly reduced due to the influence of the driving environment, the vehicle position / posture estimation unit 102 decides not to use the sensor whose detection accuracy is significantly reduced.

[0101] That is, when the detection accuracy of the vehicle motion detection sensor 502 and the vehicle position / posture sensor 503 is good, the process moves to step S1-4.

[0102] On the other hand, when the detection accuracy of the vehicle position / posture sensor 503 decreases, such as when the vehicle 200 travels in a tunnel, the process moves to step S1 - 3 .

[0103] In addition, when the detection accuracy of the vehicle motion detection sensor 502 decreases, the process transfers to step S1-2.

[0104] In step S1 - 2 , the vehicle position / posture estimating unit 102 estimates the position / posture of the vehicle 200 using only the vehicle position / posture information 113 of the vehicle position / posture 503 .

[0105] First, the vehicle position / attitude estimation unit 102 calculates the difference between the latitude and longitude of the origin in the mapping coordinate system and the latest latitude and longitude indicated by the vehicle position / attitude information 113 from the vehicle position / attitude sensor 503. Next, the vehicle position / attitude estimation unit 102 uses the calculated difference value to convert the latest latitude and longitude indicated by the vehicle position / attitude information 113 into a value of the plane rectangular coordinate system. Then, the vehicle position / attitude estimation unit 102 obtains the orientation of the vehicle 200 using the geomagnetic sensor and ends the processing of step S1-2.

[0106] In step S1 - 3 , the vehicle position / posture estimating unit 102 estimates the position / posture of the vehicle 200 using only the vehicle motion information 112 from the vehicle motion detection sensor 502 .

[0107] First, the vehicle position / posture estimation unit 102 calculates the time from the last update time t n-1 Up to the update time t n Next, the vehicle position / posture estimation unit 102 assumes that at time tn-1 At time t n The vehicle 200 performs uniform circular motion at a speed V and a yaw rate ω, and calculates the changes (Δx, Δy, Δyaw) in the position and orientation of the vehicle 200 during the time difference Δt. Then, the vehicle position / attitude estimation unit 102 compares the changes (Δx, Δy, Δyaw) with the time t n-1 The values ​​of the position and orientation (x, y, yaw) of the vehicle 200 are added. With the above, the vehicle position / attitude estimation unit 102 ends the processing of step S1-3.

[0108] In step S1 - 4 , the vehicle position / attitude estimation unit 102 estimates the position / attitude of the vehicle 200 using the vehicle motion information 112 from the vehicle motion detection sensor 502 and the vehicle position / attitude information 113 from the vehicle position / attitude sensor 503 .

[0109] Here, the vehicle position / attitude estimation unit 102 establishes a model formula using the value of the vehicle motion information 112 and the value of the vehicle position / attitude information 113. Then, the vehicle position / attitude estimation unit 102 updates the value of the position and attitude of the vehicle 200 using a well-known technique, such as an extended Kalman filter. Above, the vehicle position / attitude estimation unit 102 ends the processing of step S1-4.

[0110] Finally, in step S1-5, the vehicle position / posture estimating unit 102 stores the position / posture information 115 indicating the position / posture of the vehicle 200 estimated in any of steps S1-2, S1-3, and S1-4 in the integrating unit 104. The vehicle position / posture estimating unit 102 terminates the processing of step S1.

[0111] [Step S2: Deletion Process of Stationary Object Detection Point Information]

[0112] In step S2 , the information updating unit 103 deletes the past stationary object detection point information 116 stored in the integrating unit 104 .

[0113] Figure 5 The details of step S2 are shown below. Figure 5 Provide explanation.

[0114] The information updating unit 103 performs the following processing on each stationary object detection point information 116 stored in the integrating unit 104 .

[0115] In step S2-1, the information updating unit 103 obtains the i-th stationary object detection point information obs among the N stationary object detection point information 116. i (i=1, 2, ..., N).

[0116] In step S2-2, the information updating unit 103 calculates the stationary object detection point information obs obtained in step S2-1 generated by the extraction unit 101 i The time t obs From the time t when this process is performed crnt The time elapsed up to (called the current time) is dt = t crnt -t obs Then, the information updating unit 103 determines whether the elapsed time dt is greater than or equal to a threshold value. If the elapsed time dt is greater than or equal to the threshold value, the process proceeds to step S2-4. If the elapsed time dt is less than the threshold value, the process proceeds to step S2-3.

[0117] In step S2-3, the information updating unit 103 calculates the position (x i ,y i ) and the position (x ego ,y ego ) i .

[0118] Then, the information updating unit 103 determines the calculated distance d i Is it above the threshold?

[0119] Distance d i If the distance d is greater than the threshold, the process proceeds to step S2-4. i When the value is less than the threshold value, if there is any unprocessed stationary object detection point information 116, the process proceeds to S2-1. If the process is completed for all stationary object detection point information 116, step S2 ends.

[0120] [Mathematical formula 1]

[0121]

[0122] In step S2-4, the stationary object detection point information obs i The deletion condition is satisfied, so the information updating unit 103 updates the stationary object detection point information obs i The data is deleted from the accumulating unit 104 .

[0123] Thus, in step S2-2, step S2-3, and step S2-4, the information updating unit 103 deletes the stationary object detection point 350 whose elapsed time after detection by the external detection sensor 501 is greater than the threshold value and whose distance from the vehicle 200 is greater than the threshold value. Therefore, in step S6 described later, the stationary object detection point 350 whose elapsed time after detection by the external detection sensor 501 is greater than the threshold value and whose distance from the vehicle 200 is greater than the threshold value is not grouped.

[0124] After step S2-4 is completed, if there is unprocessed stationary object detection point information 116, the process proceeds to S2-1. If the processing of all stationary object detection point information 116 is completed, step S2 is completed.

[0125] [Step S3: Stationary object determination processing of detection point information]

[0126] In step S3, the extraction unit 101 determines the detection point det_obs indicated by the detection point information 111 from the external detection sensor 501. i (i=1, 2, ..., N_DET_NUM) Whether it is a detection point of a stationary object.

[0127] Figure 6 The details of step S3 are shown below. Figure 6 Provide explanation.

[0128] In step S3-1, the extraction unit 101 obtains the detection point det_obs i The extraction unit 101 determines the traveling direction of the vehicle 200 based on the vehicle motion information 112 .

[0129] In step S3-2, the extraction unit 101 uses V ground =V x -V ego To calculate the detection point det_obs i The ground speed V ground (speed relative to the ground). V ego It is the speed of the vehicle 200 shown by the vehicle movement information 112 .

[0130] In step S3-3, the extraction unit 101 determines the ground speed V ground Is the absolute value of below the threshold? Ground speed V ground If the absolute value of is below the threshold, the process proceeds to step S3-4. ground When the absolute value of exceeds the threshold, the process proceeds to step S3-5.

[0131] In step S3-4, the extraction unit 101 extracts the detection point det_obs i It is determined to be a stationary object detection point 350. The stationary object detection point 350 is a detection point on a stationary object.

[0132] In step S3-5, the detection point det_obs i Determined as a moving object detection point. A moving object detection point refers to a detection point on a moving object.

[0133] In step S3-6, the extraction unit 101 generates the stationary object detection point information 114. More specifically, the extraction unit 101 obtains the information of the stationary object detection point 350 from the detection point information 111, and generates the stationary object detection point information 114. The stationary object detection point information 114 shows the position coordinates of the stationary object detection point 350 and the detection time of the external detection sensor 501.

[0134] Then, the extraction unit 101 outputs the generated stationary object detection point information 114 to the information updating unit 103 .

[0135] After step S3-6, step S3 ends.

[0136] [Step S4: Coordinate Conversion Processing of Stationary Object Detection Point Information]

[0137] In step S4 , the information updating unit 103 converts the position coordinates of the stationary object detection point 350 into a mapping coordinate system.

[0138] Figure 7 The details of step S4 are shown below. Figure 7 Provide explanation.

[0139] As described above, the position coordinates of the stationary object detection point indicated by the stationary object detection point information 114 are positions (xr, yr) with the position of the vehicle 200 at the time of detection of the stationary object detection point 350 as the origin. The information updating unit 103 converts the position (xr, yr) into a position (xm, ym) of the mapping coordinate system.

[0140] In step S4-1, the information updating unit 103 obtains two time points t closest to the detection time t indicated by the stationary object detection point information 114 from the position / posture information 115 stored in the accumulating unit 104. n ,t n+1 The position / posture information 115. n The position and posture contained in the position / posture information 115 are recorded as (x ego_n ,y ego_n , yaw ego_n ). In addition, the time point t n+1 The position and posture contained in the position / posture information 115 are recorded as (x ego_n+1 ,y ego_n+1 , yaw ego_n+1 ).

[0141] In step S4-2, the information updating unit 103 estimates the position and posture (x t) of the vehicle 200 at the detection time t based on the position / posture information 115 acquired in step S4-1. ego_t ,y ego_t , yaw ego_t ).

[0142] Specifically, the information updating unit 103 estimates the position and posture (x ) of the vehicle 200 at the detection time t using the following (Equation 2). ego_t ,y ego_t , yaw ego_t ).

[0143] [Mathematical formula 2]

[0144]

[0145] In step S4-3, the information updating unit 103 uses the estimated result of the position and posture of the vehicle 200 obtained in step S4-2 to convert the position coordinates of the stationary object detection point 350 indicated by the stationary object detection point information 114 into values ​​in the mapping coordinate system. Specifically, the information updating unit 103 converts the values ​​according to the following (Formula 3).

[0146] Among them, (x obs_rel ,y obs_rel ) is the position coordinate of the stationary object detection point 350 shown in the stationary object detection point information 114, (x obs_map ,y obs_map ) are the position coordinates of the stationary object detection point 350 in the mapping coordinate system.

[0147] After step S4-3, step S4 ends.

[0148] In addition, information obtained by performing coordinate conversion on the stationary object detection point information 114 is referred to as stationary object detection point information 116 .

[0149] [Mathematical formula 3]

[0150]

[0151] [Step S5: Storage Process of Stationary Object Detection Point Information]

[0152] In step S5 , the information updating unit 103 stores the stationary object detection point information 116 in the accumulating unit 104 .

[0153] Figure 8 The details of step S5 are shown below. Figure 8 Provide explanation.

[0154] In step S5 - 1 , the information updating unit 103 stores the stationary object detection point information 116 in the accumulating unit 104 .

[0155] [Step S6: Grouping Process]

[0156] In step S6 , the grouping unit 105 groups the stationary object detection points 350 .

[0157] Fig. 9 The details of step S6 are shown below. Fig. 9 Provide explanation.

[0158] In step S6-1, the grouping unit 105 groups two or more stationary object detection points 350 that are estimated to be detection points of the same stationary object among the plurality of stationary object detection points 350. Specifically, the grouping unit 105 uses a density-based clustering technique to group the stationary object detection points 350. As a density-based clustering technique, the grouping unit 105 can use, for example, the DBSCAN algorithm.

[0159] In step S6-2, the grouping unit 105 registers the group information 117 showing the details of the group 400 obtained in step S6-1 in the accumulation unit 104. The group information 117 includes the identifier of the group 400, the position coordinates of the frame line of the group 400, and the position coordinates of the stationary object detection point 350 included in the group 400.

[0160] In addition, the grouping unit 105 may output the grouping information 117 to the outside. For example, the grouping unit 105 may output the grouping information 117 to a display device mounted on the vehicle 200.

[0161] [Step S7: Group Deletion Processing]

[0162] In step S7, the grouping unit 105 deletes a part of the groups obtained in step S6.

[0163] Fig.10 The details of step S7 are shown below. Fig.10 Provide explanation.

[0164] In step S7-1, the packetization unit 105 obtains the i-th (i=1, 2, . . . , N_OBS_GROUP) packet 400 among the N_OBS_GROUP packets 400. Hereinafter, the i-th packet is referred to as packet i.

[0165] In step S7-2, the grouping unit 105 calculates a spatial density threshold thresh corresponding to the position of the group i.

[0166] Specifically, when the position of group i is (x, y), the grouping unit 105 calculates the threshold value thresh=f(x, y). The position (x, y) of group i is, for example, the position of the center point of the region corresponding to group i.

[0167] In addition, the threshold value thresh=f(x, y) is a function proportional to the distance from the vehicle 200 as shown in the following (Formula 4), for example.

[0168] [Formula 4]

[0169]

[0170] In step S7-3, the grouping unit 105 obtains the spatial density of the group i obtained during the grouping process in step S6. Then, the grouping unit 105 determines whether the obtained spatial density is less than the spatial density threshold thresh obtained in step S7-2. When the obtained spatial density is less than the spatial density threshold thresh, the process proceeds to step S7-4.

[0171] When the acquired spatial density exceeds the spatial density threshold value thresh, if there are unprocessed packets 400, the process proceeds to step S7-1. On the other hand, if all packets 400 have been processed, step S7 ends.

[0172] In step S7 - 4 , the packetizing unit 105 determines that the packet i is noise, and deletes the packet information 117 of the packet i.

[0173] After step S7-4, if there are unprocessed packets 400, the process proceeds to S7-1. On the other hand, if all packets 400 have been processed, step S7 ends.

[0174] **Description of the effects of the implementation method**

[0175] Thus, according to the present embodiment, even when spatially sparse sensors are used, stationary objects around the vehicle 200 can be accurately recognized, and the environment around the vehicle 200 can be accurately recognized.

[0176] The effects of this embodiment will be described in detail below.

[0177] Here, Fig.11 The surrounding environment shown is the premise.

[0178] As mentioned above, Fig.11 In the figure, the vehicle 200 is traveling on a road 600 which is separated by guardrails 610 on both sides. Fig.11 In the figure, the road 600 is an area where the vehicle 200 can travel, and is a travelable area. The outer side of the guardrail 610 is an area where the vehicle 200 cannot travel, and is a non-travelable area.

[0179] Fig.12 The detection results obtained by sensors that are spaced apart are shown. Fig.12In the example, only the detection point 300 is determined to be an obstacle around the vehicle 200, and only the detection point 300 is identified as a non-travelable area. Therefore, the area that is actually not traversable is mistakenly identified as a traversable area. When the technology of Patent Document 1 is used, Fig.12 The recognition results are shown.

[0180] Fig.13 The state after the stationary object detection points 350 are grouped by the grouping unit 105 is shown. Fig.13 In FIG. 4 , the stationary object detection points 350 are grouped, and there are a plurality of groups 400 . Fig.13 In the image, the group 400 is recognized along the shape of the guardrail 610. Therefore, it can be inferred that obstacles are continuously present on both sides of the vehicle 200. Thus, according to the present embodiment, even when spatially sparse sensors are used, the environment around the vehicle 200 can be accurately recognized.

[0181] Implementation method 2.

[0182] ***summary***

[0183] In this embodiment, an example in which a plurality of groups 400 obtained in Embodiment 1 are connected will be described.

[0184] In this embodiment, the differences from Embodiment 1 are mainly described.

[0185] In addition, matters not described below are the same as those in Implementation 1.

[0186] Fig.14 An example of connecting groups is shown.

[0187] Fig.14 Shown in Figure 1 (c) is a state where three packets 400 are connected in the mapping data.

[0188] Vehicle 200, stationary object detection point 350 and group 400 and Figure 1 Same as shown in (c).

[0189] In this embodiment, when the distance between the groups 400 is below the distance threshold, the grouping unit 105 connects the groups 400 with the connecting line 450. For example, in the group 400 located on the right side relative to the vehicle 200, the grouping unit 105 connects the left side of the frame line of each group 400 with the connecting line 450. On the other hand, in the group 400 located on the left side relative to the vehicle 200, the grouping unit 105 connects the right side of the frame line of each group 400 with the connecting line 450.

[0190] Furthermore, the grouping unit 105 can change the distance threshold value according to the running speed of the vehicle 200. Specifically, the grouping unit 105 increases the distance threshold value between the groups 400 as the running speed of the vehicle 200 increases.

[0191] Fig.15 (a) shows that the speed V of the vehicle 200 is within the speed threshold V thresh The following is the connection status between the groups 400. On the other hand, Fig.15 (b) shows that the speed V of the vehicle 200 is greater than the speed threshold V thresh When the speed V is greater than the speed threshold V thresh When the following Fig.15 As shown in (a), all the groups 400 are connected. On the other hand, when the speed V of the vehicle 200 is greater than the speed threshold V thresh When you want to be big, Fig.15 As shown in (b), even if the distance between the groups 400 is Fig.15 (a) is the same, and the group 400-1 and the group 400-2 are not connected.

[0192] ***Structure description***

[0193] The functional structure of the object recognition device 100 and the structure of the sensor mounted on the vehicle 200 according to this embodiment are as follows: Figure 2 As shown.

[0194] In addition, the hardware structure of the object recognition device 100 according to this embodiment is as follows: Fig.23 As shown.

[0195] ***Action description***

[0196] Fig.16 An operation example of the object recognition device 100 according to this embodiment is shown.

[0197] Fig.16 In the steps S1 to S7, Figure 3 Therefore, omit Fig.16 Description of steps S1 to S7.

[0198] [Step S8: Group concatenation processing]

[0199] In step S8 , the grouping unit 105 determines whether the vehicle 200 can pass between the groups 400 , and connects the groups 400 when it is determined that the vehicle 200 cannot pass through.

[0200] Fig.17 The details of step S16 are shown below. Fig.17 Provide explanation.

[0201] In step S8-1, the grouping unit 105 calculates a threshold value thresh for determining connection between groups based on the vehicle speed V of the vehicle 200. For example, the grouping unit 105 calculates using the threshold value thresh=a*V+b (a, b: constants). Thus, when the speed of the vehicle 200 is high, the threshold value thresh also increases.

[0202] In step S8-2, the grouping unit 105 selects the i-th group 400 (i=1, 2, . . . , N) among the N groups 400 generated in step S6-1. Hereinafter, the i-th group 400 is referred to as group i.

[0203] Furthermore, in step S8-3, the grouping unit 105 selects the j-th group 400 (j=1, 2, . . . , N, j≠i) among the N groups 400. Hereinafter, the j-th group 400 is referred to as group j.

[0204] In step S8-4, the distance d between the selected group i and group j is calculated using the following (Formula 5): ij In addition, "x i " represents the x coordinate of group i, "y i " represents the y coordinate of group i. Similarly, "x j " represents the x coordinate of group j, "y j " represents the y coordinate of group j. For example, "x i " represents the x coordinate of the center point of the area of ​​group i, "y i " represents the y coordinate of the center point of the area of ​​group i. Similarly, "x j " represents the x coordinate of the center point of the area of ​​group j, "y j ” represents the y coordinate of the center point of the area of ​​group j.

[0205] [Formula 5]

[0206]

[0207] In step S8-5, the grouping unit 105 divides the distance d between group i and group j into ij Compare with the threshold thresh.

[0208] At distance d ijIf the value is greater than the threshold value thresh, and if there is a packet 400 that has not been selected as the packet j, the process returns to step S8-3. If all packets 400 have been selected as the packet j, and if there is a packet 400 that has not been selected as the packet i, the process returns to step S8-2. If all packets 400 have been selected as the packet i, the process proceeds to step S8-13.

[0209] On the other hand, if the distance d ij If it is below the threshold thresh, the process proceeds to step S8-6.

[0210] In step S8-6, the grouping unit 105 obtains the number of packets 400 that have been linked to the packet j. In addition, the grouping unit 105 calculates the distance between each of the packets 400 linked to the packet j and the packet j.

[0211] In step S8-7, the grouping unit 105 determines whether the number of packets 400 connected to the packet j is M-1 or less.

[0212] Here, "M" represents the maximum number of connections that a group 400 can connect to (M=0, 1, 2, ..., N-1).

[0213] When the number of packets 400 concatenated with packet j is M-1 or less, the process proceeds to step S8-12.

[0214] On the other hand, when the number of packets connected to packet j exceeds M-1, the process proceeds to step S8-8.

[0215] In step S8-8, the packetization unit 105 determines whether the number of packets 400 concatenated with the packet j is M. When the number of packets 400 concatenated with the packet j is M, the process proceeds to step S8-9.

[0216] On the other hand, when the number of packets 400 connected to packet j exceeds M, if there is a packet 400 that is not selected as packet j, the process returns to step S8-3. When all packets 400 have been selected as packet j, if there is a packet 400 that is not selected as packet i, the process returns to step S8-2. When all packets 400 have been selected as packet i, the process proceeds to step S8-13.

[0217] In step S8-9, the grouping unit 105 obtains the packet k with the largest distance to the packet j among the M packets 400 connected to the packet j, and the distance d between the packet j and the packet k. jk .

[0218] In step S8-10, the grouping unit 105 divides the distance d between group i and group j into ij The distance d from group j and group k jk Make a comparison.

[0219] Distance d ij At distance d jk In the case of the following, the process proceeds to step S8-11. ij Specific distance d jk If the number of packets is greater than the number of packets, if there is a packet 400 that has not been selected as the packet j, the process returns to step S8-3. If all packets 400 have been selected as the packet j, if there is a packet 400 that has not been selected as the packet i, the process returns to step S8-2. If all packets 400 have been selected as the packet i, the process proceeds to step S8-13.

[0220] In step S8-11, the grouping unit 105 deletes the connection between the group j and the group k.

[0221] In step S8-12, the grouping unit 105 connects the group i and the group j.

[0222] When all the packets 400 have been selected as the packet i and the packet j, in step S8-13, the packetization unit 105 generates packet concatenation information reflecting the results of steps S8-11 and S8-12.

[0223] Then, the grouping unit 105 registers the generated group concatenation information in the accumulation unit 104. In addition, the grouping unit 105 may output the generated group concatenation information to the outside. For example, the grouping unit 105 may output the group concatenation information to a display device mounted on the vehicle 200.

[0224] Fig.18 and Fig.19 , an example of the operation of the grouping unit 105 when the judgment of step S8-7 is true is shown.

[0225] Fig.18 and Fig.19 In the above example, as a condition, "d ij <thresh”.

[0226] The judgment in step S8-5 is true, so the condition is satisfied. Therefore, in step S8-12, the packetizing unit 105 concatenates the packet i and the packet j.

[0227] Fig. 20An example of the operation of the grouping unit 105 is shown when the determination in step S8-7 is false and the processing in step S8-8 is true.

[0228] Fig. 20 In the above example, as a condition, "d ij <thresh” and “d ij <d jk ". In the determination of step S8-5, it is true (true), "d ij <thresh" holds true. If in step S8-10 the judgment is made that "d ij <d jk " If the condition is met, the grouping unit 105 deletes the connection between packet j and packet k in step S8-11, and connects packet i and packet j in step S8-12.

[0229] Furthermore, the grouping unit 105 may distinguish between a drivable area and a non-drivable area based on the connection result of the groups 400 .

[0230] For example, Fig.21 As shown, when the group 400 is connected, the grouping unit 105 can be based on the connection status of the group 400 and the connection line 450, such as Fig. 22 That is, the grouping unit 105 determines that the area inside the group 400 and the connecting line 450 is the drivable area. On the other hand, the grouping unit 105 determines that the area outside the group 400 and the connecting line 450 is the non-drivable area.

[0231] **Description of the effects of the implementation method**

[0232] Therefore, in this embodiment, two or more groups whose distances are below the threshold are connected. Then, in this embodiment, the drivable area and the non-drivable area can be distinguished based on the connection status of the groups. Therefore, according to this embodiment, when the vehicle 200 is an autonomous driving vehicle, the autonomous driving can be smoothly performed. That is, according to this embodiment, the situation where the autonomous driving vehicle enters the non-drivable area can be avoided, and the autonomous driving vehicle can be driven within the drivable area.

[0233] As mentioned above, although embodiment of this invention was demonstrated, these two embodiments can also be combined and implemented.

[0234] Alternatively, one of these two embodiments may be partially implemented.

[0235] Alternatively, these two embodiments may be partially combined for implementation.

[0236] In addition, the present invention is not limited to these embodiments, and various modifications can be made as needed.

[0237] ***Description of hardware structure***

[0238] Finally, the hardware structure of the object recognition device 100 is supplemented.

[0239] Fig.23 The processor 901 shown is an IC (Integrated Circuit) that performs processing.

[0240] The processor 901 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or the like.

[0241] Fig.23 The main storage device 902 shown is a RAM (Random Access Memory).

[0242] Fig.23 The auxiliary storage device 903 shown is a ROM (Read Only Memory), a flash memory, a HDD (Hard Disk Drive), or the like.

[0243] Fig.23 The communication device 904 shown is an electronic circuit that performs communication processing of data.

[0244] The communication device 904 is, for example, a communication chip or a NIC (Network Interface Card).

[0245] Furthermore, the auxiliary storage device 903 also stores an OS (Operating System).

[0246] Furthermore, at least a part of the OS is executed by the processor 901 .

[0247] The processor 901 executes at least a part of the OS, and executes a program that realizes the functions of the extraction unit 101 , the vehicle position / posture estimation unit 102 , the information update unit 103 , and the grouping unit 105 .

[0248] The processor 901 executes the OS, thereby performing task management, memory management, file management, communication control, and the like.

[0249] In addition, at least any one of the information, data, signal values ​​and variable values ​​representing the results of processing by the extraction unit 101, the vehicle position / posture estimation unit 102, the information updating unit 103 and the grouping unit 105 is stored in at least one of the main storage device 902, the auxiliary storage device 903, the registers within the processor 901 and the cache memory.

[0250] In addition, the program that realizes the functions of the extraction unit 101, the vehicle position / posture estimation unit 102, the information update unit 103, and the grouping unit 105 can be stored in a removable storage medium such as a magnetic disk, a floppy disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, a DVD, etc. Then, the removable storage medium storing the program that realizes the functions of the extraction unit 101, the vehicle position / posture estimation unit 102, the information update unit 103, and the grouping unit 105 can be circulated.

[0251] In addition, the word “unit” in the extraction unit 101 , the vehicle position / posture estimation unit 102 , the information updating unit 103 , and the grouping unit 105 may be replaced with “circuit,” “process,” “step,” or “processing.”

[0252] Furthermore, the object recognition device 100 may be implemented by a processing circuit, such as a logic IC (Integrated Circuit), a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).

[0253] In this case, the extraction unit 101 , the vehicle position / posture estimation unit 102 , the information updating unit 103 , and the grouping unit 105 are each realized as a part of a processing circuit.

[0254] In addition, in this specification, a processor and a processing circuit are collectively referred to as a "processing circuit".

[0255] That is, the processor and the processing circuit are respectively specific examples of the “processing circuit”.

[0256] Description of symbols

[0257] 100 Object recognition device

[0258] 101 Extraction Department

[0259] 102 Vehicle position / attitude estimation unit

[0260] 103 Information Update Department

[0261] 104 Accumulation

[0262] 105 Grouping Department

[0263] 111 Checkpoint Information

[0264] 112 Vehicle movement information

[0265] 113 Vehicle position / attitude information

[0266] 114 Stationary object detection point information

[0267] 115 Position / attitude information

[0268] 116 Stationary object detection point information

[0269] 117 Group Information

[0270] 200 vehicles

[0271] 300 detection points

[0272] 350 stationary object detection points

[0273] 400 Grouping

[0274] 450 Link Line

[0275] 501 External detection sensor

[0276] 502 Vehicle Motion Detection Sensor

[0277] 503 Vehicle Position / Posture Sensor

[0278] 600 Road

[0279] 610 Guardrail

[0280] 901 Processor

[0281] 902 Main storage device

[0282] 903 Auxiliary storage device

[0283] 904 Communication device.

Claims

1. An object recognition device, characterized in that: include: an extraction unit that extracts, as the stationary object detection point, a detection point of a stationary object from among a plurality of detection points around the vehicle detected by the sensor at a plurality of detection timings; as well as A grouping unit is configured to group two or more stationary object detection points, among the plurality of stationary object detection points extracted by the extraction unit at the plurality of detection timings, which are presumed to be detection points of the same stationary object and whose elapsed time after detection by the sensor is within a prescribed time and whose distance from the vehicle is within a prescribed distance.

2. The object recognition device according to claim 1, characterized in that: The extraction unit extracts the stationary object detection points based on the speed over the ground of each detection point.

3. The object recognition device according to claim 1, wherein: The grouping unit calculates, for each group, the spatial density of two or more stationary object detection points included in the group. The groups whose calculated spatial density is below the spatial density threshold are discarded.

4. The object recognition device according to claim 3, characterized in that: The grouping unit changes the spatial density threshold value according to a distance between a group and the vehicle.

5. The object recognition device according to claim 1, wherein: When a plurality of groups are obtained, the grouping unit connects any two or more of the plurality of groups.

6. The object recognition device according to claim 5, characterized in that: The grouping unit connects two or more groups whose distances to each other are less than a threshold.

7. The object recognition device according to claim 6, characterized in that: The grouping unit changes the threshold value according to the speed of the vehicle.

8. The object recognition device according to claim 6, characterized in that: The grouping unit determines a drivable area in which the vehicle can travel and a non-drivable area in which the vehicle cannot travel, based on a connection state of two or more groups.

9. The object recognition device according to claim 1, wherein: The object recognition device further comprises: a vehicle position / attitude estimating unit that estimates at least one of the position and attitude of the vehicle; and an information updating unit that uses the estimation result obtained by the vehicle position / posture estimating unit to perform coordinate conversion of the plurality of stationary object detection points, The grouping unit groups two or more stationary object detection points estimated to be detection points of the same stationary object among the plurality of stationary object detection points after coordinate conversion.

10. The object recognition device according to claim 1, characterized in that: The object recognition device further comprises: An information updating unit deletes the stationary object detection points that meet a deletion condition among the plurality of stationary object detection points.

11. An object recognition method, characterized in that: The computer extracts, as the stationary object detection point, a detection point of a stationary object from among a plurality of detection points around the vehicle detected by the sensor at a plurality of detection timings, The computer groups two or more stationary object detection points, among the multiple stationary object detection points extracted at the multiple detection timings, which are presumed to be detection points of the same stationary object and whose elapsed time after detection by the sensor is within a specified time and whose distance from the vehicle is within a specified distance.

12. A computer-readable recording medium, characterized in that: An object recognition program is recorded, and the object recognition program causes the computer to execute: Extraction processing of extracting, as the stationary object detection point, a detection point of a stationary object from among a plurality of detection points around the vehicle detected by the sensor at a plurality of detection timings; as well as A grouping process is provided for grouping two or more stationary object detection points which are inferred to be detection points of the same stationary object, among the plurality of stationary object detection points extracted by the extraction process at the plurality of detection timings, and whose elapsed time after detection by the sensor is within a prescribed time and whose distance from the vehicle is within a prescribed distance.

Images