Free space detection device and method of normalizing object position

Abstract

To set a free space with high precision.SOLUTION: The present invention relates to a free space detection device which arranges a free space end point, determined based upon a detection point of an object detected in a circumference of a predetermined center point, on a plurality of reference lines extending radially from the center point, and then connects free space end points on the plurality of reference lines to extract an outer edge of a free space. The free space detection device sets a border line in the middle between mutually adjacent reference lines, wherein when two border lines on both sides of a first reference line which is one of the reference lines are called first border lines respectively and a region sandwiched between the first reference lines around the first reference line is called a first angle region, positions of free space end points on the first reference lines are determined based upon the position closest to the center point among in in-region detection points which are detection points present in the first angle region.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a free space sensing device and a method for normalizing object positions. [Background technology]

[0002] Vehicles with driving assistance and autonomous driving functions that monitor the surroundings and perform some driving actions on behalf of humans have appeared to ensure safety and improve comfort. To realize driving assistance and autonomous driving functions, it is necessary to detect the free space around the vehicle, which is the drivable area, to understand the situation around the vehicle, and to make driving action decisions such as speed control based on the detection results. The free space around the vehicle is detected by detecting the relative positions of obstacles as seen from the vehicle using sensors such as cameras, radar, LIDAR (Light Detection and Ranging), and sonar mounted on the vehicle, and detecting areas without obstacles as seen from the vehicle. Driving action decisions are made by computer software that takes human operations and detection results by sensors as inputs and outputs control instructions such as target speed and target position, acceleration / deceleration, and steering to the vehicle.

[0003] Software that judges driving behavior is created for each driving assistance function and autonomous driving function, such as lane keeping assistance, following the vehicle in front, and emergency braking, and multiple functions are integrated into the vehicle as a whole. The output format of the detection results by the sensor, i.e., the format, varies from sensor to sensor depending on the operating principle and economic and technical constraints, and the output format may differ depending on the vehicle model and age. On the other hand, preparing software that judges driving behavior for each driving assistance function and autonomous driving function for different sensor combinations for each vehicle model will result in a large number of combinations and will be inefficient. Therefore, it is desirable to express the detection results by the sensor in a common format, create software that judges driving behavior based on that common format, and then make it possible to adjust for differences in vehicle models by parameter settings, etc.

[0004] One way to express the results of detection by a sensor is to use polar coordinates, which express the obstacle position, i.e. the boundary of the free space, in two dimensions as the vehicle moves on the road plane, and express it as the direction and distance seen from the vehicle. If the positions of obstacles around the vehicle expressed in polar coordinates are used in a common format, expressing them as distances to arbitrary angles and directions would increase the amount of information to be handled and the processing load would increase, so the entire circumference of the vehicle is divided into certain angles and expressed as the distance to the obstacle in each divided direction. However, since the divided angles in the common format do not necessarily match the angles detected by the sensor, it is necessary to convert the detected positions expressed in polar coordinates using arbitrary angles detected by the sensor into an expression of distances in each divided direction in the common format.

[0005] Patent document 1 discloses an information processing device that includes a processing unit that calculates a first presence probability of an object existing around a moving body using position information of the object measured by a plurality of sensors having different characteristics, obtains unmeasured information indicating that the position information was not obtained for each of the plurality of sensors, and determines a second presence probability of the object based on the first presence probability and the unmeasured information. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2017-215939 A Summary of the Invention [Problem to be solved by the invention]

[0007] The invention described in Patent Document 1 does not allow for accurate setting of the distance to the position of a detected obstacle. [Means for solving the problem]

[0008] A free space detection device according to a first aspect of the present invention is a free space detection device that arranges free space endpoints, which are determined based on detection points of objects detected around a predetermined central point, on a plurality of reference lines extending radially from the central point, and extracts the outer edge of the free space by connecting the free space endpoints on the plurality of reference lines. A boundary line is set midway between adjacent reference lines, one of the reference lines is designated as a first reference line, and each of the two boundary lines that sandwich the first reference line is designated as a first boundary line. When a region sandwiched between the first boundary lines and the first reference line is referred to as a first angle region, the position of the free space endpoint on the first reference line is determined based on the position closest to the central point among the detection points within the region that are the detection points present in the first angle region. A method for normalizing an object position according to a second aspect of the present invention is a method for normalizing an object position executed by a free space detection device that arranges free space end points, which are determined based on detection points of an object detected around a predetermined center point, on a plurality of reference lines extending radially from the center point, and extracts an outer edge of a free space by connecting the free space end points on the plurality of reference lines, and includes the steps of: setting a boundary line midway between adjacent reference lines; designating one of the reference lines as a first reference line; designating each of the two boundary lines sandwiching the first reference line as a first boundary line; and referring to a region sandwiched between the first boundary lines and centered on the first reference line as a first angle region, determining the position of the free space end point on the first reference line based on the position closest to the center point among intra-region detection points which are the detection points present in the first angle region. Effect of the Invention

[0009] According to the present invention, a free space can be set with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] A hardware configuration diagram of a vehicle equipped with a free space detection device according to a first embodiment. [Diagram 2]Functional block diagram of a free space detection device according to a first embodiment. [Diagram 3] A flowchart showing the process of the normalization unit in the first embodiment. [Figure 4] FIG. 13 is a diagram showing an example of an obstacle information array; [Diagram 5] A flowchart showing details of step S306 in FIG. 3. [Figure 6] FIG. 6 is a diagram showing a specific example of the process shown in FIG. 5. [Figure 7] Free space detector hardware configuration diagram [Figure 8] 11 is a flowchart showing the process of the normalization unit in the second embodiment. [Figure 9] Flowchart showing details of step S306 in FIG. 8 [Figure 10] FIG. 10 is a diagram showing a specific example of the process shown in FIG. [Figure 11] FIG. 1 is a diagram showing a comparison between the method according to the second embodiment and the method according to the first embodiment. [Figure 12] FIG. 11 is a diagram showing a comparison between the technique according to the second embodiment and the technique of a second comparative example. [Figure 13] FIG. 11 is a diagram showing a comparison between the technique according to the second embodiment and the technique of a second comparative example. [Figure 14] FIG. 11 is a diagram showing a comparison between the technique according to the second embodiment and the technique of a second comparative example. [Figure 15] A hardware configuration diagram of a vehicle equipped with a free space detection device according to a third embodiment. [Figure 16] A functional block diagram of a free space detection device according to a third embodiment. [Figure 17] 13 is a flowchart showing the process of the normalization unit in the fourth embodiment. [Figure 18] FIG. 23 is a diagram showing a specific example of processing by a normalization unit in the fourth embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] --First embodiment-- A first embodiment of a free space detection device will be described below with reference to Figs. 1 to 7. In this embodiment, the configuration and operation of a free space detection device mounted on a vehicle system that detects the vehicle's situation and controls the vehicle will be described. Vehicle control includes indirect control such as presenting information to the driver, and the vehicle system does not necessarily need to realize an automatic driving function that operates autonomously. For example, the vehicle system may have a driving assistance function that conveys information about free spaces to the driver by voice, image, etc.

[0012] FIG. 1 is a hardware configuration diagram of a vehicle 101 equipped with a free space detection device 102. However, in FIG. 1, the illustration of hardware that is not closely related to the free space detection device 102 is omitted. In the following, the vehicle 101 may be called the "own vehicle" or "own vehicle" to distinguish it from other vehicles. The vehicle 101 includes a free space detection device 102, a common bus 103, a driving action planning device 104, and a camera sensor 105. The free space detection device 102, the driving action planning device 104, and the camera sensor 105 are connected by the common bus 103 and can transmit information to each other.

[0013] The camera sensor 105 is an obstacle detection sensor that captures images of the surroundings of the vehicle 101 and detects obstacles from the captured images. When the camera sensor 105 detects an obstacle from the captured images, it estimates the position of the detection point of the obstacle based on the vehicle 101. The estimation of the position of the detection point can be realized, for example, by projecting the obstacle position onto the road surface using image recognition technology and road surface estimation, but it may also be realized by other methods. For example, the position of the detection point may be estimated by distance measurement using stereo vision technology using multiple image pickup elements, or the position of the detection point may be estimated with higher accuracy by measuring the distance in combination with other sensors such as radar.

[0014] Furthermore, although the name "camera" is used for convenience, it is not essential that the camera sensor 105 has an imaging element, and the camera sensor 105 may have any sensor and processing system capable of detecting obstacles. For example, the camera sensor 105 may use a radar sensor to enable detection at night, or may use a LIDAR (Light Detection and Ranging) sensor to detect obstacle positions with high accuracy.

[0015] Furthermore, the object to be detected by the camera sensor 105 does not necessarily have to be an obstacle with a physical form. For example, it may be an area where the presence of a vehicle is estimated based on headlights, shadows, sounds, etc., or an end point close to the vehicle where the intrusion of the vehicle is determined to be undesirable based on the predicted movement of the obstacle, traffic rules, etc. By using such expressions other than the actual positions of obstacles, it is possible to increase the number of events that can be handled, such as dangers that the sensor cannot directly detect and future predictions.

[0016] The free space detection device 102 detects free spaces using information on detection points detected by the camera sensor 105, as described below, and passes the free space information to the driving behavior planning device 104. The driving behavior planning device 104 uses the free space information to decide what action the vehicle should take, and controls the vehicle in accordance with the decided action. The free space detection device 102 and the driving behavior planning device 104 are ECUs (Electronic Control Units), which receive information from the common bus 103, process the information using software, and transmit the processing results to the common bus 103. Each ECU executes software stored in a storage device in a central processing unit, and transmits and receives information to the common bus 103 using an input / output device.

[0017] The method of realizing the functions of the free space detection device 102 and the driving behavior planning device 104 is not limited to the ECU. For example, a cheap computer such as a single board computer may be used to reduce costs. The driving behavior planning device 104 may obtain information from the common bus 103, and directly control the vehicle 101 without outputting the processed results to the common bus 103.

[0018] The method by which the driving action planning device 104 directly controls the vehicle 101 is not particularly limited. For example, the driving action planning device 104 may indirectly control the vehicle by converting information on a target speed or steering into voice or the like and conveying it to the driver, or may directly convert the information into a target hydraulic pressure for mechanical control and instruct the accelerator, brake, steering, or the like. These methods by which the driving action planning device 104 controls the vehicle 101 are common, and therefore detailed explanations will be omitted. In addition, the processing contents of the driving action planning device 104 are not the focus of the present invention, and are common techniques for following the vehicle in front, emergency braking, and the like, and therefore detailed explanations will be omitted.

[0019] The common bus 103 is a communication bus that can transmit at least the information output by the camera sensor 105 to the free space detection device 102, and can transmit the information output by the free space detection device 102 to the driving action planning device 104. The common bus 103 may be, for example, a bus-type communication bus such as a Controller Area Network (CAN), or a star-type communication bus such as IEEE802.3.

[0020] The free space detection device 102, the driving action planning device 104, and the camera sensor 105 do not need to be divided into hardware. For example, the functions of the free space detection device 102 and the driving action planning device 104 may be executed by a central processing unit inside the camera sensor 105, and the device may be configured to be the camera sensor 105 alone, thereby saving installation space. In this case, the function of the common bus 103 is realized by memory copy or the like in the central processing unit.

[0021] 2 is a functional block diagram of the free space detection device 102. The free space detection device 102 includes a receiving unit 201, a normalizing unit 202, and a transmitting unit 203 as its functions. The receiving unit 201 takes in the detection result of an obstacle by the camera sensor 105 obtained from the camera sensor 105 via the common bus 103 into the free space detection device 102 and outputs it to the normalizing unit 202. The normalizing unit 202 converts the detection result of the obstacle into a common format as a free space, and outputs it to the transmitting unit 203 as free space information. In this embodiment, converting the obstacle information into a common format as a free space is called normalization. The information transmitting block transmits the free space information converted into the common format by the normalizing unit 202 to the driving action planning device 104 via the common bus 103.

[0022] The normalization unit 202 may include an end point attribute assigning unit 2021 and a line segment attribute assigning unit 2022 as shown in FIG. 2. However, the end point attribute assigning unit 2021 and the line segment attribute assigning unit 2022 are preliminary configurations, and the normalization unit 202 may not include at least one of the end point attribute assigning unit 2021 and the line segment attribute assigning unit 2022. The end point attribute assigning unit 2021 assigns object attributes to the normalized position information of the obstacle. The line segment attribute assigning unit 2022 assigns attributes to the line segments connecting the normalized positions of the obstacles. The attributes are information indicating at least the presence or absence of an obstacle, and may be information that, if an obstacle exists, can identify whether the obstacle is a moving object or a stationary object. Furthermore, at least one of the speed and acceleration of the obstacle may be included as an attribute.

[0023] Since each functional block of the free space detection device 102 is configured as a software function, multiple blocks may be integrated and treated as one block. For example, the receiving unit 201, the normalizing unit 202, and the transmitting unit 203 may be configured as one functional block to reduce overhead associated with information transmission between the blocks. In addition, various methods can be used for the method of transmitting information between the functional blocks. For example, information may be transmitted by storing information in a shared memory, or each functional block may be arranged in a different core or device and the functional blocks may be connected and transmitted through a communication path such as serial communication, thereby simplifying the configuration of each device.

[0024] The common format adopted by the normalization unit 202 is the distance to an obstacle in each direction obtained by dividing the circumference of the vehicle 101 by a predetermined interval angle, i.e., the reference angle. For example, when the interval angle is "0.1 degrees", the front of the vehicle 101 is set to 0 degrees, and reference angles that differ by 0.1 degrees clockwise, i.e., 3600 reference angles from 0 degrees to 359.9 degrees, are defined, and the distance to the position where an obstacle exists is set for each reference angle. In this case, the center point may be the center of the vehicle 101 or another position. In the following, the center point is also referred to as the "origin". Since there is a possibility that no obstacle exists, a value representing the absence of an obstacle, for example a negative value, is set for a direction in which no obstacle is detected.

[0025] In the following, each point defined by the above-mentioned angle and distance is called a "free space end point." The area connecting the free space end points that indicate the presence of an obstacle means the outer edge of the free space, and the vehicle 101 can travel within the free space. In the following, the straight line that passes through the origin and indicates the reference angle is also called a "reference line." Therefore, the common format in this embodiment can be said to be the position information of the free space end points placed on each reference line. One end of all reference lines is always at the origin. For example, the first reference line is a half line that faces the vehicle 101 in the traveling direction at 0 degrees, and the second reference line is a half line that passes through the origin, is on the left side of the first reference line, and makes an angle with the first reference line at a pitch angle.

[0026] Ideally, the free space should completely match an area where no obstacles exist. It is also acceptable that the free space is expressed narrower than ideal and there is a gap between the free space and the obstacle. However, it is not acceptable that the free space is expressed wider than ideal and that an obstacle exists but is recognized as a free space. This is because if the driving action planning device 104 uses such free space information, there is a possibility that the vehicle 101 will collide with the obstacle.

[0027] In this embodiment, the index of the appropriateness of the free space is called "accuracy." For example, if a free space includes an area where an obstacle exists, the accuracy of the free space is very low. On the other hand, if a free space does not include any area where an obstacle exists, the accuracy is higher as the distance between the obstacle and the free space, i.e., the gap area, is smaller.

[0028] The common format for expressing the free space described here is a specific example in this embodiment, and other definitions may be used. For example, the increment angle may be set to 0 degrees except for the front of the vehicle 101, or the increment angle may be changed. The increment angle may not be constant over the entire area, and for example, different increment angles may be set for the front and rear of the vehicle 101. In other words, the angle between the reference lines may not be a constant value. In this case, it is preferable to set the increment angle for the front, where the collision risk is high, to be small, and the increment angle for the rear to be large. Such settings reduce the data size and the processing load. Furthermore, instead of setting attribute values ​​to points, attribute values ​​may be set to line segments connecting consecutive points in the angle direction, and consecutive points having the same attribute may be omitted to reduce the processing load.

[0029] FIG. 3 is a flowchart showing the processing of the normalization unit 202. However, in FIG. 3, the processing of the end point attribute assignment unit 2021 and the line segment attribute assignment unit 2022 is omitted. First, in step S302, the normalization unit 202 acquires information on the detection points output by the camera sensor 105. There may be only one detection point, but more often there is a plurality of detection points, so hereinafter, the plurality of detection points are also referred to as an "input point sequence." The input point sequence, which is the processing result of step S302, is expressed as an array of positions that are detection points of obstacles as seen from the vehicle 101, that is, as a point sequence. The obstacle position is expressed, for example, as two-dimensional coordinates on an orthogonal coordinate system with the center of the vehicle 101 as the origin, the front of the vehicle 101 as the positive x-axis, and the left of the vehicle 101 as the positive y-axis. However, three-dimensional coordinates may be adopted to add information in the height direction so that the risk of collision with the vehicle can be determined more precisely.

[0030] In the next step S303, the normalization unit 202 converts each of the detection points constituting the input point sequence into polar coordinates. The center of this polar coordinate system is, for example, the center of the vehicle 101. In the next step S304, the normalization unit 202 initializes an obstacle information array that stores information about obstacles. In the next step S355, the normalization unit 202 changes the reference angle to be processed in order and repeats the processes of steps S356 to S358. Specifically, the normalization unit 202 calculates all the reference angles and stores them in an array in ascending order, and sequentially increases the array index to sequentially change the processing target.

[0031] In this embodiment, the reference angles are sorted in ascending order to limit the processing range of the input point sequence and save processing, but the processing may be simplified without sorting in ascending order. Note that the reference angle, the reference line, and the reference angle region described next have a 1:1:1 correspondence, so it can be said that the reference line to be processed and the reference angle region to be processed are changed in order and the processing of steps S356 to S358 are repeated.

[0032] The reference angle region is a region whose boundaries are the midlines of adjacent reference lines on the right and left sides of the reference line corresponding to the reference angle to be processed. For example, when the step angle is 0.1 degrees, the region between -0.05 degrees or more and less than +0.05 degrees from the reference angle to be processed is the reference angle region. Note that in this embodiment, the midline is set as the boundary between the adjacent reference angles, but it does not necessarily have to be the midline. For example, the front side as viewed from the vehicle may be set wider, and obstacle detection may be moved forward of the vehicle to emphasize obstacles with a high collision risk for the vehicle, making it easier to make a safer collision judgment.

[0033] In step S356, the normalization unit 202 identifies all detection points that exist within the reference angle region to be processed. For example, if the reference angle to be processed is "12.5 degrees" and the interval angle is "0.1 degrees", detection points whose angles are equal to or greater than "12.45 degrees" and less than "12.55 degrees" are enumerated. In the following step S357, the normalization unit 202 identifies the detection point that is the shortest distance to the origin among the detection points identified in step S356. Hereinafter, the distance to the origin of the detection point identified in this step is referred to as the "closest approach distance". In other words, the closest approach distance is the distance between the origin and the detection point closest to the origin among the detection points in the reference angle region.

[0034] In the next step S358, the normalization unit 202 adds a combination of the reference angle targeted in step S305 and the closest distance obtained in step S307 to the obstacle information array initialized in step S304. This is information indicating the position of a free space end point on a certain reference line. If a value indicating that the closest distance was not obtained is set in step S307, the information is converted to information indicating that no obstacle exists in step S308 and then added to the obstacle information array. In step S309, if the normalization unit 202 determines that an unprocessed reference angle exists, it changes the processing target and returns to step S306, or if the normalization unit 202 has performed the processing from step S306 to S308 with all reference angles as processing targets, it ends the processing shown in FIG. 3.

[0035] 3 is completed, the obstacle information array initialized in step S304 stores the closest distances corresponding to all reference angles, i.e., all reference lines. Note that, in the initialization of step S304, information indicating that the closest distance could not be obtained in step S307 may be set in advance, and if the closest distance could not be obtained in step S307, the processing time of step S308 may be saved by skipping the processing of step S308.

[0036] The end point attribute assigning unit 2021 assigns an attribute to each point included in the obstacle information array. For example, the end point attribute assigning unit 2021 may determine that there is no obstacle when the closest approach distance is farther than a predetermined threshold. The end point attribute assigning unit 2021 may also distinguish between stationary objects and moving objects based on whether the value of each point included in the obstacle information array changes over time, or may set attributes by referring to additional information received by the receiving unit 201, such as information indicating the type of obstacle. The line segment attribute assigning unit 2022 assigns attributes to the line segment based on the attributes of the points at both ends of the line segment. For example, the line segment attribute assigning unit 2022 may determine that the line segment is a moving object when at least one of the points at both ends of the line segment changes over time, and assign the attribute of a stationary object to the line segment only when both points are stationary objects.

[0037] FIG. 4 is a diagram showing an example of an obstacle information array. The elements of the array are angle 401, distance 402, and attribute 403. The reference angle is stored in angle 401 in the order of processing in steps S305 to S309, the closest distance calculated in step S308 is stored as distance 402, and whether or not the closest distance was obtained in step S307 is set as attribute 403. Attribute 403 is set to "stationary" or "moving" according to the information held by the original input point sequence, and is set to "none" if the closest distance was not obtained in step S307. Note that if the original input point sequence does not have any particular information, it is treated as "moving".

[0038] By setting the presence or absence of an obstacle and the presence or absence of movement in attribute 403, the possibility of the shape of the free space changing is expressed, and this is used as reference information when planning driving behavior based on the free space information. Note that instead of determining whether the object is moving or stationary in attribute 403, the presence or absence of an obstacle may be simply set and the processing and information may be written simply. Furthermore, by setting distance 402 to 0 or more if there is an obstacle and to a negative distance if there is no obstacle, attribute 403 may be omitted to save memory and communication bandwidth.

[0039] In this case, a value equal to or greater than the sensor detection distance may be set instead of a negative value as a value indicating that no obstacle is present. Also, appropriate modifications may be made to accommodate limitations of the programming language used. For example, if the programming language used itself or the type of a specific variable cannot handle negative values, "0" may be used instead of a negative value. Furthermore, if a reference angle is designed in advance, the reference angle can be calculated from the array number, so that the angle 401 may be omitted to save memory and communication bandwidth. In any case, in step S308, information used by the driving action planning device 104 is set in the obstacle information array.

[0040] Fig. 5 is a diagram showing a method of conversion to a common format according to this embodiment. Fig. 5 shows the center of the vehicle 101 as the origin, and the detection points at which the sensor detects the obstacle 602 are represented by black circles, reference numerals 608, 609, 610, and 611. Note that the origin is not limited to the center of the vehicle 101, and may be, for example, the center of the rear wheel axle of the vehicle 101 to make it easier to calculate the vehicle's motion. Since the obstacle 602 is large, it is detected at multiple detection points, as indicated by reference numerals 608 to 611.

[0041] In Fig. 5, four reference lines C614-C617 are indicated by dashed lines, and the midlines 603-606 of these reference lines are indicated by broken lines. Each of the regions 614-617 centered on each reference line and sandwiched between the midlines is a reference angle region. The white stars on each reference line represent the free space end points.

[0042] In the method of this embodiment, the distance of the detection point closest to the vehicle 101 in each reference angle region is set as the closest approach distance of that region. That is, the closest approach distance is the distance of detection point 608 in region 614, the distance of detection point 610 in region 615, and the distance of detection point 611 in region 616. Therefore, the positions indicated by the white stars in FIG. 5 are free space end points. It can be said that the free space end points indicated by the white stars according to this embodiment are arranged at positions where the points closest to the origin among the partial point sequences in each reference angle region are moved on an arc centered on the origin and intersect with the reference line. A negative value is set for region 617 since there is no detection point. In FIG. 5, the outer edge of the free space region obtained by the method of this embodiment is indicated by a solid arc.

[0043] FIG. 6 is a diagram showing a comparative example method. In the comparative example shown in FIG. 6, a grid is created by dividing not only the angle but also the distance at a predetermined interval, and the area closer to the origin than the grid of the obstacle closest to the origin for each angle area is set as a free space area. In FIG. 6, the obstacles detected by the sensor are shown as white squares, and the boundaries of the free space for each angle area are shown as solid lines. For example, in the angle area θ1, obstacles exist in the fourth grid and the sixth grid from the center, so the area closer to the origin than the fourth grid closest to the center, that is, the third grid from the center, is the free space. In the angle area θ2, obstacles exist between the third and fourth grids from the center, and the fifth to seventh grids. In this case, the area closer to the origin than the third grid closest to the center, that is, the second grid from the center, is the free space. In this comparative example, not only the angle of the free space but also the distance to the origin take discrete values, so the accuracy is lower than that of this embodiment.

[0044] 7 is a hardware configuration diagram of the free space detection device 102. The free space detection device 102 includes a CPU 41 which is a central processing unit, a ROM 42 which is a read-only storage device, a RAM 43 which is a readable and writable storage device, and a communication device 45. The CPU 41 loads a program stored in the ROM 42 into the RAM 43 and executes it to perform the various calculations described above.

[0045] The free space detection device 102 may be realized by a field programmable gate array (FPGA) that is a rewritable logic circuit or an application specific integrated circuit (ASIC) that is an integrated circuit for a specific application, instead of the combination of the CPU 41, the ROM 42, and the RAM 43. The free space detection device 102 may be realized by a combination of different configurations, for example, a combination of the CPU 41, the ROM 42, the RAM 43, and an FPGA, instead of the combination of the CPU 41, the ROM 42, and the RAM 43.

[0046] According to the above-described first embodiment, the following advantageous effects can be obtained. (1) The free space detection device 102 arranges free space end points determined based on objects detected around a predetermined center point, such as the center of the vehicle 101, on a plurality of reference lines extending radially from the center point, such as the straight line L101 in Fig. 6, and extracts the outer edge of the free space by connecting the free space end points on each reference line. The normalization unit 202 sets a boundary line in the middle of adjacent reference lines, and when one of the reference lines is set as a first reference line (L101 in Fig. 6), two boundaries (L102 and L103 in Fig. 6) sandwiching the first reference line are each set as a first boundary line, and the region (hatched region in Fig. 6) sandwiched between the first boundaries with the first reference line as the center is called a first angle region, the position of the free space end point (star mark in Fig. 6) on the first reference line is determined based on the position closest to the center point among the detection points within the region (circled "C" to "E") that are detection points present in the first angle region. Therefore, as described with reference to FIG. 6, the method of this embodiment can set a free space with higher accuracy than the comparative example.

[0047] (2) The normalization unit 202 includes an end point attribute assignment unit 2021 that assigns attributes to the free space end points based on the object detection state. This makes it possible to more appropriately use the free space information.

[0048] (3) The normalization unit 202 includes a line segment attribute assigning unit 2022 that assigns attributes to line segments connecting the free space end points based on the attributes assigned to the free space end points. This makes it possible to more appropriately use free space information.

[0049] (Variation 1) The output of camera sensor 105 does not have to match the expression of the processing result in step S302. For example, the output of camera sensor 105 may be a coordinate system other than a Cartesian coordinate system, or a Cartesian coordinate system whose origin and axes do not match, in which case normalization unit 202 performs coordinate conversion in step S302. An example of a case in which the origin does not match is when the coordinate system of camera sensor 105 has the attachment position of camera sensor 105 as its origin. Also, the output of camera sensor 105 may be a combination of the center position and size of an obstacle, in which case the peripheral position of the obstacle is calculated in step S302.

[0050] Furthermore, in step S302, detection correction may be performed based on the attitude of the vehicle 101, such as roll or pitch, associated with acceleration or deceleration of the vehicle 101. In addition, movement of the vehicle 101 associated with the time lag between the detection time and the processing time may be reflected in the output of the camera sensor 105, and the accuracy and reliability of the detected position may be improved by using statistical processing such as a Kalman filter.

[0051] --Second embodiment-- A second embodiment of a free space detection device will be described with reference to Figs. 8 to 14. In the following description, the same components as in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that information on detection points in adjacent reference angle regions is also used.

[0052] FIG. 8 is a flowchart showing the process of the normalization unit 202 in the second embodiment. The same process as that in the flowchart shown in FIG. 3 in the first embodiment is assigned the same step number and the description is omitted. The process of the first step S302 is the same as that in FIG. 3. In step S393 executed after step S302, the normalization unit 202 sorts the input point sequence in the order of angles. The angle expression may be obtained using the atan2 function, or, for example, the Euclidean distance to the coordinates of the point and the coordinate value may be used to obtain the sine value and cosine value, and the positive and negative coordinate values ​​may be used to obtain only the magnitude relationship of the angle. In the latter case, the process time required for the atan2 function can be saved. Note that the sorting in step S393 is performed for the purpose of reducing the processing load in step S306 described later, so step S393 may be deleted and the process of step S306 may be increased.

[0053] After step S393, the normalization unit 202 executes step S304. The process of step S304 is the same as that of FIG. 3. In the following step S305, the normalization unit 202 sequentially changes the reference angle to be processed and repeats the processes of steps S306 to S308. Specifically, the normalization unit 202 calculates all the reference angles and stores them in an array in ascending order, and sequentially increases the array index to sequentially change the processing target. In this embodiment, the reference angles are arranged in ascending order to limit the processing range of the input point sequence and save processing, but the processing may be simplified without ascending order. Note that the reference angle and the reference line have a 1:1 correspondence relationship, so it can also be said that the reference line to be processed is sequentially changed and the processes of steps S306 to S308 are repeated.

[0054] In step S306, the normalization unit 202 executes a process of generating a partial point sequence that overlaps with the reference angle region at the reference angle to be processed. An overview of the process in this step will be described in detail later with reference to FIG. 5. In step S307, the normalization unit 202 calculates the closest distance, which is the distance to the point closest to the vehicle 101 on the line segment connecting the partial point sequence generated in step S306. Here, the line segment connecting the partial point sequence is a line segment obtained by connecting the points that make up the partial point sequence in the order of the point sequence. If the partial point sequence is not defined, a value indicating that the closest distance could not be obtained, for example a negative value, is set.

[0055] In the next step S308, the normalization unit 202 adds a combination of the reference angle targeted in step S305 and the closest distance obtained in step S307 to the obstacle information array initialized in step S304. This is information indicating a free space end point on a certain reference line. If a value indicating that the closest distance was not obtained is set in step S307, the information is converted to information indicating that no obstacle exists in step S308 and then added to the obstacle information array. In step S309, if the normalization unit 202 determines that an unprocessed reference angle exists, it changes the processing target and returns to step S306, or if the normalization unit 202 has performed the processing from step S306 to S308 with all reference angles as processing targets, it ends the processing shown in FIG. 8.

[0056] (Details of partial point sequence generation process) Fig. 9 is a flowchart showing details of the partial point sequence generation process in step S306 in Fig. 8. A specific example of the process shown in Fig. 5 will be described later. Before the process shown in Fig. 9 is started, a reference angle to be processed, in other words, a reference line to be processed, is set in step S305. In the range shown in Fig. 9, the reference angle and reference line to be processed do not change, so the reference angle region does not change either.

[0057] First, in step S502, the normalization unit 202 initializes the partial point sequence array, the start number, and the end number. The elements of the partial point sequence array have the same format as the input point sequence, and have at least the coordinate values ​​of the points. The partial point sequence array is initialized to a coordinate value indicating that the point is invalid, for example, a huge value. The number of array elements may be made variable to save memory required for the array, in which case the number of elements may be set to "0" during initialization. The start number and end number will store the array number of the input point sequence in later processing. Therefore, the start number and end number are initialized to a value that shows that they are not an array number, such as a negative value.

[0058] In the next step S503, the normalization unit 202 calculates a reference angle region. Specifically, the normalization unit 202 calculates a start angle and an end angle of the reference angle region. For example, if the interval angle is "0.1 degrees" and the reference angle to be processed is "30 degrees", the start angle is 29.95 degrees and the end angle is 30.05 degrees. As described above, the reference angle region is a region centered on the reference line. In step S504, the normalization unit 202 starts a loop in which the array element number of the input point sequence is increased sequentially from "0". Specifically, the normalization unit 202 repeats the processes of steps S505 to S509 while increasing the array element number to be processed by "1" at a time.

[0059] In step S505, normalization unit 202 acquires the angle of the input point corresponding to the array number to be processed. The angle may be expressed as a value calculated by atan2 as described above, or as a sin / cos pair, as long as the start angle and end angle can be compared. In the following step S506, normalization unit 202 determines whether the angle of the input point acquired in step S505 is equal to or greater than the angle of the start point calculated in step S503. If normalization unit 202 determines that the angle of the input point is equal to or greater than the angle of the start point, it proceeds to step S507, and if it determines that the angle of the input point is less than the angle of the start point, it proceeds to step S510.

[0060] In step S507, if the start number initialized in step S502 remains initialized, i.e., is not set, normalization unit 202 sets the array element number to be processed as the start number and proceeds to step S508. If the start number is not not set, normalization unit 202 proceeds to step S508 without doing anything. In step S508, normalization unit 202 determines whether the angle of the point acquired in step S505 is smaller than the end angle calculated in step S503. If normalization unit 202 determines that the angle of the point acquired in step S505 is smaller than the end angle calculated in step S503, it proceeds to step S509, and if normalization unit 202 determines that the angle of the point acquired in step S505 is equal to or larger than the end angle calculated in step S503, it proceeds to step S510.

[0061] In step S509, if the end number initialized in step S502 remains initialized, i.e., is not set, the normalization unit 202 sets the end number to a value obtained by subtracting "1" from the array number of the processing target, and proceeds to step S510. If the end number is not not set, the normalization unit 202 proceeds to step S510 without doing anything. In step S510, if the array number of the processing target is the last array number of the input point sequence, the normalization unit 202 proceeds to step S511, and if the array number of the processing target is not the last array number of the input point sequence, the normalization unit 202 sets the array number of the processing target to the next array number in the input point sequence, and returns to step S505. Note that in the angle comparison in steps S506 and S508, the comparison is performed using a value obtained by adding 360 degrees to the angle or subtracting 360 degrees from the angle so that the absolute value of the angle difference is 180 degrees or less. This is intended to prevent the magnitude relationship from being reversed due to crossing 0 degrees or 360 degrees.

[0062] In step S511, the normalization unit 202 judges whether or not some values ​​have been set for both the start number and the end number. If it is judged that some values ​​have been set for both, the process proceeds to step S512, and if it is judged that at least one of them has not been set, the process proceeds to step S513. Whether or not the start number has been set is synonymous with whether or not step S507 has been executed. If step S507 or step S509 has not been executed because, for example, no points included in the input point sequence exist within the reference angle region, a negative judgment is made in step S511.

[0063] In step S512, the normalization unit 202 adds an intersection of the line segment connecting the input point sequences with the start angle (hereinafter referred to as the "start intersection") to the top of the partial point sequence array initialized in step S502, and proceeds to step S514. Here, the start intersection is an intersection of a line segment connecting the points of the input point sequence in order with a half line extending from the center of the host vehicle 101 in the start angle direction calculated in step S503. If the start intersection does not exist, the normalization unit 202 does not add the start intersection to the partial point sequence array in the processing of step S512.

[0064] In step S514, the normalization unit 202 adds the input point sequence indicated by the array numbers from the start number to the end number, including the start number and the end number, to the end of the partial point sequence array initialized in step S502. In the following step S515, the normalization unit 202 adds an intersection of the input point sequence and the end angle (hereinafter referred to as an "end intersection") to the partial point sequence array, and ends the process shown in Fig. 9. The end intersection is an intersection between a line segment that sequentially connects the points included in the input point sequence and a half line that is extended from the host vehicle in the direction of the end angle calculated in step S503. If the end intersection does not exist, the normalization unit 202 does not add an intersection to the partial point sequence array in step S515.

[0065] In step S513, which is executed if a negative judgment is made in step S511, the normalization unit 202 does not add anything to the partial point sequence array, leaves the partial point sequence array empty, and ends the process shown in Fig. 9. The process shown in Fig. 9 described above aims to generate the partial point sequence array initialized in step S502, and the process from step S307 onwards shown in Fig. 8 is executed using the partial point sequence array.

[0066] (Example of partial point sequence generation process) FIG. 10 is a diagram showing a specific example of the process shown in FIG. 9. As shown in the upper part of FIG. 10, in this example, seven points from a circled "A" to a circled "G" are stored in the input point sequence in order. In step S503, the normalization unit 202 calculates L102 indicating the start angle and L103 indicating the end angle. The area indicated by hatching with dots and sandwiched between the straight lines L102 and L103 is the reference angle area. Note that the positions of the input point sequence, the triangular "P" and "Q", and the star marks shown in FIG. 10 are shown for the sake of convenience, and this information is not obtained at the time when step S503 is completed. Specifically, the circled "C" to "E" are included in the reference angle area.

[0067] As described above, steps S504 to S510 are repeated for the number of input point sequences. In the following, the number of repeated processes is expressed as a "loop." For example, in the first loop, the input point sequence to be processed is the circled "A," and since the input point sequence has a total of "7" elements, in this example, processing is performed up to the seventh loop.

[0068] In the first loop, the normalization unit 202 processes the circled "A" and makes a negative determination in step S506, terminating the process. In the second loop, the normalization unit 202 processes the circled "B" and, as in the first loop, makes a negative determination in step S506, terminating the process. In the third loop, the normalization unit 202 makes a positive determination in step S506 because the circled "C" to be processed is included in the reference angle region, and proceeds to step S507. In step S507, the normalization unit 202 stores the array number "3" in the start number, since it is the first time it is executed in this third loop. In the following step S508, the normalization unit 202 makes a negative determination and terminates the process, since it is located to the right of the line L103 in the figure.

[0069] In the fourth and fifth loops, the normalization unit 202 processes the circled "D" and "E", and makes a positive decision in step S506, but does not perform any particular processing since this is not the first time that step S507 is executed. Then, in step S508, the normalization unit 202 makes a negative decision and ends the processing. In the sixth loop, the normalization unit 202 makes a positive decision in step S506, but does not perform any particular processing since this is not the first time that step S507 is executed. Then, in step S508, the normalization unit 202 makes a positive decision and proceeds to step S509. In step S509, since this is the first time that the normalization unit 202 is executed in the sixth loop, the normalization unit 202 sets the end number to "5", which is the element number "6" minus "1", and ends the processing.

[0070] In the seventh loop, the normalization unit 202 processes the circled "F" and makes an affirmative determination in both step S506 and step S508. However, since neither step S507 nor step S509 is the first execution of the normalization unit 202, the normalization unit 202 ends the process without performing any specific process. When the process of the seventh loop ends, the processes of steps S504 to S510 are completed. In the following step S511, since both the start number and the end number have been set, the normalization unit 202 makes an affirmative determination and proceeds to step S512.

[0071] In step S512, the normalization unit 202 calculates a start intersection point, which is an intersection point between a line segment connecting the points of the input point sequence in order and a half line extending from the center of the vehicle 101 in the start angle direction calculated in step S503. In this example, since L102 indicating the start angle passes between the circled "B" and "C", the coordinates of the triangular "P" which is the intersection point between the line segment connecting the circled "B" and "C" and the straight line L102 are calculated. Then, the normalization unit 202 adds this triangular "P" to the partial point sequence. In step S514, the normalization unit 202 adds the circled "C" to "E", which are elements of the input point sequence from the start number "3" to the end number "5", to the partial point sequence.

[0072] In step S515, the normalization unit 202 calculates a start intersection point, which is an intersection point between a line segment that sequentially connects the points of the input point sequence and a half line that is extended from the center of the vehicle 101 in the direction of the end angle calculated in step S503. In this example, since L103 indicating the end angle passes between the circled "E" and "F", the coordinates of the triangle enclosure "Q" that is the intersection point between the line segment connecting the circled "E" and "F" and the straight line L103 are calculated. Then, the normalization unit 202 adds this triangle enclosure "Q" to the partial point sequence. Through the above process, the partial point sequence is calculated as the triangle enclosure "P", the circle enclosures "C", "D", and "E", and the triangle enclosure "Q". The above is a specific example of the process shown in FIG. 9.

[0073] Once these partial point sequences are obtained, the distance of the triangular enclosure "P" that is closest to the origin among the partial point sequences is calculated as the closest distance by the processes of steps S307 and S308 in Fig. 8. Therefore, the free space end point in the reference angle area shown in Fig. 10 is set to a position on the reference line L101 that is the same distance from the origin to the triangular enclosure "P", that is, the position of the star.

[0074] (effect) The method of conversion to a common format according to this embodiment is compared with the conversion method according to the first embodiment and the method of conversion to a common format according to the second comparative example method, which is a second comparative example method, with reference to Figure 11 and Figures 12 to 14. The conversion method according to the first embodiment is compared with the conversion method according to this embodiment, and Figures 12 to 14 compare the conversion method according to the second comparative example method with the conversion method according to this embodiment.

[0075] FIG. 11 is a diagram showing a comparison between the conversion method to the common format according to this embodiment and the conversion method in the first embodiment. However, since the configuration in FIG. 11 is the same as that in FIG. 5 in the first embodiment, a description of the overlapping configuration will be omitted. The conversion method in the first embodiment uses the closest detection point within the reference angle region. The white stars on each reference line represent the free space end points. The meaning of the white circles will be described later. When the detection points 608 to 611 are converted into the common format of the free space as an input point sequence by the method in the first embodiment, the outer edge of the free space region is set to the position indicated by the solid arc, as described with reference to FIG. 5.

[0076] On the other hand, when the method of this embodiment is used, points 612 and 613 indicated by white circles are obtained in the process of calculating the partial point sequence. The distances to point 612 in region 614, point 610 in region 615, and point 613 in region 616 are set as the closest points to the vehicle on the detection point sequence of points 608 to 611 where the obstacle 602 is detected, respectively, and no detection point exists in region 617. In FIG. 11, the outer edge of the free space region obtained by the method of this embodiment is indicated by a dashed arc. However, in region 615, the outer edge of the free space region by the method of the first embodiment and the outer edge of the free space region by the method of this embodiment overlap. It can be said that the free space end points according to this embodiment indicated by white stars are arranged at positions where the points closest to the origin among the partial point sequences in each reference angle region are moved on an arc centered on the origin and intersect with the reference line.

[0077] Comparing the method of the first embodiment with the method of this embodiment, the method of the first embodiment uses the distance from the origin of the actually detected detection point as the closest approach distance, and places importance on the accuracy of the detection point. On the other hand, the method of the second embodiment assumes that there are obstacles other than the detection points, and interpolates between the detection points to use the distance of the intersection with the midline of the reference line as a candidate for the closest approach distance. There is no superiority or inferiority between the method of the first embodiment and the method of the second embodiment, but the approach to the detection point detected by the sensor is different.

[0078] Next, using Figs. 12, 13, and 14, a comparison is made between the conversion method to a common format according to the present embodiment and the second comparative example method. In the second comparative example method, the conversion to a common format is simply performed by linear interpolation to obtain the intersections of the detection point sequence and each reference angle. Fig. 12 is a diagram showing obstacles and detection points before conversion to a common format. In Fig. 12, the host vehicle 101 is present at the bottom of the figure, and the sensor detects obstacles 702 and 703. The results of the sensor detection are detection points 704, 705, 706, and 707, and a point sequence 708 connecting these is set as an input point sequence. Note that the obstacle 702 is an obstacle having a size that produces multiple detection points, such as a side wall, and 703 is a small obstacle that produces only one detection point, such as a pole.

[0079] Fig. 13 is a diagram showing the result of converting the detection result of Fig. 12 by the method of the second comparative example. In the method of the second comparative example, linear interpolation is used to convert into a common format. In Fig. 13, a half line 801 extending radially from the vehicle 101 represents a reference line. For the input point sequence 708, points 802, 803, 804, 805, and 806 are obtained as intersections with the reference line 801 by linear interpolation, constituting a point sequence 807.

[0080] In the sequence of points 807, points 804 and 805 are on the left and right of the obstacle 703. This indicates that there is a free space farther away than the actual position of the obstacle 703, and the presence of the obstacle 703 is not fully reflected in the sequence of points 807. In other words, it indicates a free space in which the presence of the obstacle 703 has been overlooked. This is because each detection point in the sequence of detection points 708 exists between the half lines of the reference line 801.

[0081] If driving behavior is planned based on the free space in a state where the obstacle 703 is overlooked, there is a possibility that a plan to collide with the obstacle 703 will be determined, which increases the risk of collision. Normalization by linear interpolation is superior in that it requires less calculation as a method for calculating the distance at each reference angle, but it can be seen that the presence of an obstacle may be overlooked and the distance to the obstacle cannot be calculated accurately.

[0082] FIG. 14 is a diagram showing the result of converting the detection result of FIG. 12 by the method of this embodiment. In FIG. 14, a median line 901 shown by a dashed line is obtained for a reference line 801 shown by a solid line. That is, each region between the median lines 901 is a reference angle region. Then, a partial point sequence overlapping with the detection point sequence 708 in each reference region is obtained, and a point 902 closest to the vehicle on a line segment of the partial point sequence is shown by a cross, i.e., a plus marker. There is a maximum of one point 902 in each reference angle region. Then, the distance from the vehicle 101 to the point 902 is set as the distance to the obstacle on each reference line 801, and a point 903 is set at the distance set from the vehicle 101 on each reference line, and a point sequence connecting the points 903 is shown as a point sequence 904. Looking at the point sequence 904, it can be seen that the point 903 is set at a position closer to the obstacle 703 than the point sequence in FIG. 13, and a free space capturing the obstacle 703 more precisely can be expressed.

[0083] According to the above-described second embodiment, the following advantageous effects can be obtained. (4) The free space detection device 102 determines the positions of the free space end points (stars in FIG. 11) on the first reference line based on the position closest to the center point among the positions of the intra-area detection points (circled "C" to "E") that are detection points present in the first angle area, and the intersection points (triangles "P" and "Q") where a line connecting the intra-area detection points and detection points not included in the first angle area intersects with the first boundary line. Therefore, as described with reference to FIGS. 12 to 14, the method of this embodiment can set a free space with high accuracy.

[0084] --Third embodiment-- A third embodiment of a free space detection device will be described with reference to Figs. 15 and 16. In the following description, the same components as in the first embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that a plurality of sensors are mounted on the vehicle.

[0085] 15 is a hardware configuration diagram of a vehicle 101A equipped with a free space detection device 102A. The vehicle 101A includes a camera sensor 105, a common bus 103, and a driving action planning device 104, similar to the vehicle 101 in the first embodiment. The vehicle 101A further includes a radar sensor 1003 and a LIDAR 1004, and includes a free space detection device 102A instead of the free space detection device 102. The free space detection device 102A has the same hardware configuration as the free space detection device 102 in the first embodiment.

[0086] In FIG. 15, all the sensors are connected by the common bus 103, but the form of connection is not limited as long as the detection results of each sensor can be transmitted to the free space detection device 102A. For example, the camera sensor 105, the radar sensor 1003, and the LIDAR 1004 may transmit their detection results to the free space detection device 102A using different communication paths and communication protocols. For these sensors, for example, LVDS (Low voltage differential signaling), CAN, IEEE802.3, etc. can be used. By adopting such a configuration, it is possible to adopt a communication method suitable for the sensor to reduce the development cost of the sensor and to reduce the risk of communication interruption due to a failure of the common bus 103.

[0087] The free space detection device 102A synthesizes the detection results of a plurality of sensors into one piece of free space information, and transmits it to the driving action planning device 104. This embodiment has advantages such as making the sensors redundant to increase fault tolerance, obtaining free space information over a wider range by compensating between sensors with different detection principles and detection ranges, and obtaining more reliable free space information.

[0088] 16 is a functional block diagram of the free space detection device 102A. The free space detection device 102 includes a camera detection receiving unit 1101, a radar detection receiving unit 1102, a LIDAR detection receiving unit 1103, a sensor fusion unit 1104, a first normalization unit 202-1, a second normalization unit 202-2, a third normalization unit 202-3, a fourth normalization unit 202-4, and a transmission unit 203. The camera detection receiving unit 1101, the radar detection receiving unit 1102, and the LIDAR detection receiving unit 1103 have the same functions as the reception unit 201 in FIG. The camera detection receiving unit 1101, the radar detection receiving unit 1102, and the LIDAR detection receiving unit 1103 receive obstacle information detected by the camera sensor 105, the radar sensor 1003, and the LIDAR 1004, respectively, and output the information to the corresponding normalization unit 202 as an input point sequence.

[0089] The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 perform the same processing as the normalization unit 202 in the first embodiment. The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 only differ in the data to be processed, and there is no particular difference in operation. The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 convert the obstacle information into a common format as a free space, that is, perform normalization processing, and output the information to the sensor fusion unit 1104. At this time, the origin of the free space information output by each normalization unit 202 may not be the same.

[0090] The sensor fusion unit 1104 acquires each of the free space information output by the first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3, converts the information into values ​​of a coordinate system having a unified origin, for example, the center of the vehicle 101A, and rearranges the information by the angle seen from the origin. However, the method of integrating the three pieces of free space information is not limited to this. For example, the origins of the three pieces of free space information may be unified and rearranged, and multiple points that exist within a predetermined distance from each other may be aggregated to the center of gravity position to reduce the number of points. Also, as is common in a technology called sensor fusion, each point of different free spaces may be tracked by a Kalman filter or the like, and the points may be combined into a more likely point to increase the reliability of the point position.

[0091] The fourth normalization unit 202-4 performs the same process as the normalization unit 202 in the first embodiment on the obstacle information input from the sensor fusion unit 1104 as an input point sequence, converts it into a free space in a common format, and outputs it to the transmission unit 203. The transmission unit 203 transmits the free space information transmitted from the fourth normalization unit 202-4 to the transmission unit 203.

[0092] In this embodiment, there is no need to be aware of differences in the output format of the detection results from the sensors by arranging the normalization unit 202 on both the input and output of the sensor fusion unit 1104. Furthermore, there is no need for a mechanism to guarantee that the output conforms to a common format during processing, so the processing of the sensor fusion unit 1104 can be simplified.

[0093] (Modification of the third embodiment) In the above-described third embodiment, the normalization unit 202 is arranged at both the input and output of the sensor fusion unit 1104. However, in order to reduce the processing load, the normalization unit 202 on the input side of the sensor fusion unit 1104 may be omitted and only the normalization unit 202 on the output side of the sensor fusion unit 1104 may be provided, or the normalization unit 202 on the output side of the sensor fusion unit 1104 may be omitted. However, when the normalization unit 202 on the input side of the sensor fusion unit 1104 is omitted, the sensor fusion unit 1104 needs to handle detection results in different formats for each sensor. Also, when the normalization unit 202 on the output side of the sensor fusion unit 1104 is omitted, the sensor fusion unit 1104 needs to perform a process of making the free space output by the sensor fusion unit 1104 have a common format.

[0094] --Fourth embodiment-- A fourth embodiment of the free space detection device will be described with reference to Figs. 17 and 18. In the following description, the same components as those in the second embodiment are given the same reference numerals, and differences will be mainly described. Points that are not particularly described are the same as those in the second embodiment. In this embodiment, the operation of the normalization unit 202 is different from that in the second embodiment. Other configurations and operations are the same as those in the second embodiment, and therefore descriptions will be omitted.

[0095] Fig. 17 is a flowchart showing the process of the normalization unit 202 in the fourth embodiment. The flowchart shown in Fig. 17 corresponds to step S511 and subsequent steps in Fig. 5 in the second embodiment. That is, the whole process shown in Fig. 8 in the second embodiment and the process of steps S502 to S510 in Fig. 9 are the same as those in the first embodiment. When the loop process shown in steps S504 to S510 is completed, the normalization unit 202 executes step S1202.

[0096] In step S1202, the normalization unit 202 calculates the start intersection in the same manner as in the process of step S512 in the first embodiment. As described in the first embodiment, there are cases where the start intersection does not exist. In the following step S1203, if the start intersection exists, the normalization unit 202 calculates an input point (hereinafter referred to as the "start right point") that exists immediately to the right of the start intersection, and does nothing if the start intersection does not exist. The start right point is the point that is closest in angle to the start intersection among the input points that exist to the right of the start intersection as viewed from the host vehicle 101. However, there may be cases where the start right point does not exist depending on the value of the input point sequence.

[0097] In step S1204, the normalization unit 202 judges whether or not both the start intersection and the start right point exist and are sufficiently close to each other. The definition of "sufficiently close" will be described later. If the normalization unit 202 judges that both the start intersection and the start right point exist and are sufficiently close to each other, the process proceeds to step S1205, and if the normalization unit 202 judges that either the start intersection or the start right point does not exist or that the start intersection and the start right point are not sufficiently close to each other, the process proceeds to step S1206. In step S1205, the normalization unit 202 adds the start intersection to the partial point sequence array initialized in step S502, and then proceeds to step S1206.

[0098] In step S1206, the normalization unit 202 determines whether or not both the start number and the end number are set, similar to step S511 in the first embodiment. If the normalization unit 202 determines that both the start number and the end number are set, the process proceeds to step S1207, and if the normalization unit 202 determines that at least one of the start number and the end number is not set, the process proceeds to step S1208. In step S1207, the normalization unit 202 adds the input point sequence from the start number to the end number to the partial point sequence array, similar to step S514 in the first embodiment, and then proceeds to step S1208.

[0099] In step S1208, the normalization unit 202 calculates the end intersection in the same manner as in step S515 in the first embodiment. However, as described in the first embodiment, there are cases where the end intersection does not exist. In the following step S1209, if the end intersection exists, the normalization unit 202 calculates an input point (hereinafter referred to as the "end left point") that exists immediately to the left of the end intersection, and does nothing if the end intersection does not exist. Note that the end left point is the point that is closest in angle to the end intersection among the input points that exist to the left of the end intersection as viewed from the host vehicle 101, and there are cases where the end right point does not exist depending on the value of the input point sequence.

[0100] In the next step S1210, the normalization unit 202 judges whether the end intersection and the end left point exist and are sufficiently close to each other. If it is judged that both the end intersection and the end left point exist and are sufficiently close to each other, the process proceeds to step S1211, and if it is judged that either the end intersection or the end left point does not exist or that the end intersection and the end left point are not sufficiently close to each other, the process of Fig. 17 ends. In step S1211, the normalization unit 202 adds the end intersection to the partial point sequence array and ends the process of Fig. 17.

[0101] In step S1204 and step S1210, "sufficiently close" means that the distance between the two points is closer than a predetermined value, for example, 1 m. The definition of "sufficiently close" may be other. For example, the distance between the two points may be defined as being closer than 0.1 m, in which case the risk of erroneous interpolation to a position where no obstacle exists is reduced. In addition, the difference in angle between the two points may be defined as being equal to or less than a predetermined threshold, in which case the risk of erroneous interpolation is reduced. By adopting this embodiment, even if the input point is not included in the reference angle region, the free space end point can be interpolated as long as the condition in step S1204 or step S1208 is satisfied. Therefore, for example, even if the obstacle detection by the sensor is rougher than the common format, a free space in which the position of the obstacle is interpolated can be generated.

[0102] FIG. 18 is a diagram showing a specific example of the process of the normalization unit 202 in this embodiment. FIG. 18 is based on substantially the same preconditions as FIG. 10 in the second embodiment, and describes the calculation of free space end points in a reference angle region centered on the reference line L101. The difference from FIG. 10 is that the input point sequence is reduced, and only the circled "A", "B", "F", and "G" are included. Therefore, there is no input point sequence in the reference angle region shown by hatching. In this embodiment, step S1202 calculates the start intersection of the triangular enclosure "P", and step S1208 calculates the end intersection of the triangular enclosure "Q". Therefore, if the determinations in steps S1204, S1206, and S1210 are all affirmative, the triangular enclosure "P" and "Q" are set as partial point sequences.

[0103] Then, the distance between the origin and the triangular "P" that is closest to the origin is added as the closest distance to the obstacle information array by the processes in steps S307 and S308 in Fig. 8. Therefore, in the example shown in Fig. 15, the distance to the origin of the free space end point indicated by the white star on the reference line L101 is equal to the distance from the triangular "P" to the origin.

[0104] According to the above-described fourth embodiment, the following advantageous effects can be obtained. (5) When no object presence point exists within the first reference angle region, the normalization unit 202 sets a free space end point based on the position of the intersection between the first boundary line and a line connecting a pair of object presence points existing on both sides of the first reference angle region. Therefore, even when the input point sequence does not exist in the reference angle region, the free space end point can be formed.

[0105] (Modification of the fourth embodiment) 9 are replaced with the processes of steps S1201 to S1211, but step S511 may be left and step S512 and after may be replaced with the processes of steps S1201 to S1211. In this case, if there is no start number or end number, step S513 is executed, and if there is no input point sequence within the reference angle region, a partial point sequence is not generated, thereby reducing the risk of excessively interpolating free space end points in this embodiment.

[0106] In each of the above-described embodiments, the vehicle on which the free space detection device is mounted is assumed to be a four-wheeled vehicle, but the free space detection device may be mounted on a vehicle other than a four-wheeled vehicle. For example, the free space detection device may be mounted on a movable vehicle such as a two-wheeled vehicle, a bus, a truck, a three-wheeled motor vehicle, a cart, a caterpillar vehicle, or a railroad car. The free space detection device does not have to be mounted on a movable body. For example, the free space detection device may be installed in a warehouse where the cart moves, and the free space detection device may process information from a sensor installed in the warehouse. In this case, the degree of freedom in installing the sensors can be increased.

[0107] In addition, when the free space detection device processes the output of a plurality of sensors as in the second embodiment, the reliability of detection can be improved by arranging the plurality of sensors so that they complement each other's blind spots. In addition, a plurality of free space detection devices may be connected to each other so as to be able to communicate with each other, and free space information output by a certain free space detection device may be used by another free space detection device. Furthermore, in this case, the free space detection device may output not only the free space information but also the output of the sensor to another free space detection device, and the free space detection device can generate free space information using information from the sensor connected to the other free space detection device. When the free space information is output to another device, the normalization described in this embodiment is performed, so that it is not necessary to be aware of the output format of each sensor, and the design of the monitoring device can be simplified.

[0108] In each of the above-mentioned embodiments and modifications, the configuration of the functional blocks is merely an example. Some functional configurations shown as separate functional blocks may be integrated, or a configuration shown in one functional block diagram may be divided into two or more functions. In addition, some of the functions of each functional block may be provided by other functional blocks.

[0109] In the above-mentioned embodiments and modifications, the program is stored in the ROM 42 (not shown), but the program may be stored in a non-volatile storage device. The free space detection device may have an input / output interface (not shown), and the program may be read from another device via a medium available to the input / output interface and the free space detection device when necessary. The medium here refers to, for example, a storage medium that is detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal that propagates through the network. Some or all of the functions realized by the program may be realized by a hardware circuit or an FPGA.

[0110] The above-mentioned embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other aspects that are conceivable within the scope of the technical idea of ​​the present invention are also included in the scope of the present invention. [Explanation of symbols]

[0111] 102, 102A: Free space detector 201: Receiving unit 202: Normalization section 2021: End point attribute assignment section 2022: Line attribute assignment section

Claims

1. A free space detection device which arranges free space end points on a plurality of reference lines extending radially from a predetermined center point, the end points being determined based on detection points of an object detected around the center point, and extracts an outer edge of a free space by connecting the free space end points on the plurality of reference lines, A boundary line is set midway between adjacent reference lines, When one of the reference lines is a first reference line, each of the two boundary lines sandwiching the first reference line is a first boundary line, and a region sandwiched between the first boundary lines and centered on the first reference line is called a first angle region, A free space detection device, wherein the position of the free space end point on the first reference line is determined based on a position closest to the center point among the intra-region detection points that are the detection points present in the first angle region.

2. 2. The free space detection device according to claim 1, a position of the free space end point on the first reference line is determined based on the position of the detection point within the region and the position of an intersection where a straight line connecting the detection point within the region and the detection point not included in the first angle region intersects with the first boundary line, the position of the free space end point on the first reference line being closest to the center point.

3. 3. The free space detection device according to claim 2, a free space detection device that, when the detection point is not present within the first angle region, sets the free space end point based on the position of the intersection point with a line connecting a pair of the detection points present on both sides of the first angle region.

4. 2. The free space detection device according to claim 1, The free space detection device further comprises an end point attribute assigning unit that assigns attributes to the free space end points based on a detection state of the object.

5. 5. The free space detection device according to claim 4, The free space detection device further includes a line segment attribute assigning unit that assigns attributes to line segments connecting the free space endpoints based on the attributes assigned to the free space endpoints.

6. A method for normalizing an object position executed by a free space detection device, the method comprising: arranging free space end points, which are determined based on detection points of an object detected around a predetermined center point, on a plurality of reference lines extending radially from the center point; and extracting an outer edge of the free space by connecting the free space end points on the plurality of reference lines, the method comprising: In the case where a boundary line is set midway between adjacent reference lines, one of the reference lines is defined as a first reference line, each of the two boundary lines sandwiching the first reference line is defined as a first boundary line, and an area sandwiched between the first boundary lines and centered on the first reference line is called a first angle area, A method for normalizing an object position, comprising: determining a position of the free space end point on the first reference line based on a position of an intra-region detection point that is the detection point existing in the first angle region and is closest to the center point.

Images