Free space detection device, object position normalization method

Abstract

To set a free space with high precision.SOLUTION: The present invention relates to a free space detection device which arranges a free space end point, determined based upon a detection point of an object detected in a circumference of a predetermined center point, on a plurality of reference lines extending radially from the center point, and then connects free space end points on the plurality of reference lines to extract an outer edge of a free space. The free space detection device sets a border line in the middle between mutually adjacent reference lines, wherein when two border lines on both sides of a first reference line which is one of the reference lines are called first border lines respectively and a region sandwiched between the first reference lines around the first reference line is called a first angle region, positions of free space end points on the first reference lines are determined based upon the position closest to the center point among in in-region detection points which are detection points present in the first angle region.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a free space detection device and a method for normalizing the position of an object.

Background Art

[0002] For the purpose of ensuring the safety and improving the comfort of automobiles, vehicles having driving support functions and autonomous driving functions in which the vehicle performs some of the surrounding monitoring and driving actions on behalf of humans have emerged. In order to realize the driving support function and the autonomous driving function, it is necessary to detect the free space, which is the drivable area around the host vehicle, for understanding the situation around the host vehicle, and to make a driving action determination such as speed suppression based on the detection result. The detection of the free space around the host vehicle is performed by detecting the relative position of an obstacle seen from the host vehicle by sensors such as a camera, a radar, a LIDAR (Light Detection and Ranging), and a sonar mounted on the vehicle, and detecting an area without an obstacle seen from the host vehicle. The driving action determination is performed by computer software that takes as input the operation of a human or the detection result by a sensor, etc., and outputs a control instruction such as a target speed, a target position, or acceleration / deceleration / steering to the vehicle.

[0003] The software for making the driving action determination is created for each driving support function and autonomous driving function such as lane keeping support, following a preceding vehicle, and emergency braking, and integrates a plurality of functions as a whole vehicle. The output format of the detection result by the sensor, that is, the format, differs for each sensor depending on the operating principle, economic, and technical constraints, and may differ depending on the vehicle type and year. On the other hand, it is inefficient to prepare the software for making the driving action determination for each driving support function and autonomous driving function according to the combination of different sensors for each vehicle type because the number of combinations increases. Therefore, it is desirable to express the detection result by the sensor in a common format, create software for determining the driving action based on the common format, and make it possible to adjust the difference in vehicle type by parameter setting or the like.

[0004] One way to represent the detection results from sensors is to represent the location of obstacles, i.e., the boundaries of free space, in two dimensions, as the vehicle moves along the road plane. This is done using polar coordinates, which express the direction and distance from the vehicle. When using a common format for the position of obstacles around the vehicle expressed in polar coordinates, expressing them as distances for arbitrary angles and directions increases the amount of information to handle and thus the processing load. Therefore, the entire circumference of the vehicle is divided into fixed angles, and the distance to obstacles is expressed in each of the divided directions. However, the angles divided in the common format do not necessarily match the angles detected by the sensors. Therefore, it is necessary to convert the detection position expressed in polar coordinates using arbitrary angles detected by the sensors into the common format, which expresses the distance in each of the divided directions.

[0005] Patent Document 1 discloses an information processing device that, for each of a plurality of sensors with different characteristics, calculates a first probability of the presence of an object in the vicinity of a moving object using the position information of the object measured by the sensor, acquires unmeasured information indicating that the position information could not be obtained for each of the plurality of sensors, and determines a second probability of the presence of the object based on the first probability of presence and the unmeasured information. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2017-215939 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] In the invention described in Patent Document 1, it is not possible to accurately set the distance to the location of the detected obstacle. [Means for solving the problem]

[0008] A free space detection device according to a first aspect of the present invention is a free space detection device that arranges free space endpoints determined based on detection points of objects detected around a predetermined center point on a plurality of reference lines extending radially from a predetermined center point, and extracts the outer edge of a free space by connecting the free space endpoints on the plurality of reference lines, wherein a boundary line is set between adjacent reference lines, one of the reference lines is designated as a first reference line, each of the two boundary lines that sandwich the first reference line is designated as a first boundary line, and the region sandwiched by the first boundary lines with the first reference line as the center is called a first angular region, and the position of the free space endpoint on the first reference line is determined based on the position of the region detection points that are the detection points located in the first angular region that are closest to the center point. A second aspect of the present invention relates to a method for normalizing the position of an object, which is performed by a free space detection device that arranges free space endpoints determined based on detection points of an object detected around a predetermined center point on a plurality of reference lines extending radially from a predetermined center point, and extracts the outer edge of a free space by connecting the free space endpoints on the plurality of reference lines, wherein a boundary line is set between adjacent reference lines, one of the reference lines is designated as a first reference line, each of the two boundary lines that sandwich the first reference line is designated as a first boundary line, and the region sandwiched between the first boundary lines with the first reference line as the center is called a first angular region, and the position of the free space endpoint on the first reference line is determined based on the position of the region detection point that is closest to the center point among the detection points that exist in the first angular region. [Effects of the Invention]

[0009] According to the present invention, it is possible to set a free space with high precision. [Brief explanation of the drawing]

[0010] [Figure 1] Hardware configuration diagram of a vehicle equipped with a free space detection device in the first embodiment [Figure 2]Functional block diagram of the free space detection device in the first embodiment [Figure 3] Flowchart showing the processing of the normalization unit in the first embodiment [Figure 4] Diagram showing an example of the obstacle information array [Figure 5] Flowchart showing the details of step S306 in FIG. 3 [Figure 6] Diagram showing a specific example of the processing shown in FIG. 5 [Figure 7] Hardware configuration diagram of the free space detection device [Figure 8] Flowchart showing the processing of the normalization unit in the second embodiment [Figure 9] Flowchart showing the details of step S306 in FIG. 8 [Figure 10] Diagram showing a specific example of the processing shown in FIG. 9 [Figure 11] Diagram showing a comparison between the method according to the second embodiment and the method according to the first embodiment [Figure 12] Diagram showing a comparison between the method according to the second embodiment and the second comparative example method [Figure 13] Diagram showing a comparison between the method according to the second embodiment and the second comparative example method [Figure 14] Diagram showing a comparison between the method according to the second embodiment and the second comparative example method [Figure 15] Hardware configuration diagram of a vehicle equipped with the free space detection device in the third embodiment [Figure 16] Functional block diagram of the free space detection device in the third embodiment [Figure 17] Flowchart showing the processing of the normalization unit in the fourth embodiment [Figure 18] Diagram showing a specific example of the processing of the normalization unit in the fourth embodiment

Embodiments for Carrying Out the Invention

[0011] —First Embodiment— Hereinafter, a first embodiment of the free space detection device will be described with reference to FIGS. 1 to 7. This embodiment describes the configuration and operation of a free space detection device mounted in a vehicle system that detects the situation of the host vehicle and controls the vehicle in a vehicle. Vehicle control includes indirect ones such as presenting information to the driver, and the vehicle system does not necessarily need to implement an autonomous driving function that operates autonomously. For example, a vehicle system having a driving support function that transmits free space information to the driver by voice, image, or the like may be used.

[0012] FIG. 1 is a hardware configuration diagram of a vehicle 101 equipped with a free space detection device 102. However, in FIG. 1, hardware with little relevance to the free space detection device 102 is not shown. Hereinafter, the vehicle 101 may also be referred to as the "host vehicle" or the "host car" to distinguish it from other vehicles. The vehicle 101 includes a free space detection device 102, a common bus 103, a driving behavior planning device 104, and a camera sensor 105. The free space detection device 102, the driving behavior planning device 104, and the camera sensor 105 are connected by the common bus 103 and can transmit information to each other.

[0013] The camera sensor 105 is an obstacle detection sensor that photographs the surroundings of the vehicle 101 and detects obstacles from the photographed images obtained. When the camera sensor 105 detects an obstacle from the photographed image, it estimates the position of the detection point of the obstacle based on the host vehicle 101. The estimation of the position of the detection point can be realized, for example, by projecting the position of the obstacle on the road surface state by image recognition technology and road surface estimation, but it may also be realized by other methods. For example, a plurality of imaging elements may be prepared and the position of the detection point may be estimated by distance measurement using stereo vision technology, or the distance may be measured in combination with other sensors such as a radar to estimate the position of the detection point and improve the accuracy.

[0014] Furthermore, although the name "camera" is used for convenience, it is not essential that it has an image sensor; the camera sensor 105 can be equipped with any sensor and processing system capable of detecting obstacles. For example, the camera sensor 105 may use a radar sensor to enable detection at night, or it may use a LIDAR (Light Detection and Ranging) sensor to detect obstacle locations with high accuracy.

[0015] Furthermore, the object detected by the camera sensor 105 does not necessarily have to be a physical obstacle. For example, it could be an area where the presence of a vehicle can be estimated from headlights, shadows, or sounds, or the endpoint closest to the vehicle in an area where it is determined that the vehicle's entry is undesirable based on obstacle movement predictions, traffic rules, etc. By using such representations other than the actual location of an obstacle, the number of events that the sensor can handle can be increased, such as dangers that cannot be directly detected or future predictions.

[0016] The free space detection device 102 uses information about detection points detected by the camera sensor 105 to detect free space as described later, and passes the free space information to the driving action planning device 104. The driving action planning device 104 uses the free space information to determine the action the vehicle should take and controls the vehicle according to the determined action. The free space detection device 102 and the driving action planning device 104 are ECUs (Electronic Control Units), which receive information from the common bus 103, process it using software, and transmit the processing results to the common bus 103. Each ECU executes software stored in its storage device on the central processing unit and sends and receives information to and from the common bus 103 using input / output devices.

[0017] Furthermore, the implementation method for the functions of the free space detection device 102 and the driving action planning device 104 is not limited to an ECU. For example, an inexpensive computer such as a single-board computer may be used to reduce costs. In addition, the driving action planning device 104 may acquire information from the common bus 103 and not output the processed results to the common bus 103, and the driving action planning device 104 may directly control the vehicle 101.

[0018] The method by which the driving action planning device 104 directly controls the vehicle 101 is not particularly limited. For example, the driving action planning device 104 may indirectly control the vehicle by converting target speed and steering information into voice or other means and communicating it to the driver, or it may directly instruct the accelerator, brakes, steering wheel, etc., by converting it into target hydraulic pressure for mechanical control. Since these methods of how the driving action planning device 104 controls the vehicle 101 are common, a detailed explanation will be omitted. Similarly, the processing content of the driving action planning device 104 is not central to the present invention and is a common technique used in following a preceding vehicle and emergency braking, so a detailed explanation will also be omitted.

[0019] The common bus 103 is a communication bus that can transmit at least the information output by the camera sensor 105 to the free space detection device 102, and transmit the information output by the free space detection device 102 to the driving action planning device 104. The common bus 103 may be a bus-type communication bus such as CAN (Controller Area Network), or a star-type communication bus such as IEEE802.3.

[0020] The free space detection device 102, the driving action planning device 104, and the camera sensor 105 do not need to be separate hardware components. For example, the functions of the free space detection device 102 and the driving action planning device 104 could be performed by a central processing unit located inside the camera sensor 105, and the camera sensor 105 could be the only device, thus saving installation space. In this case, the functions of the common bus 103 would be implemented by memory copying or similar methods within the central processing unit.

[0021] Figure 2 is a functional block diagram of the free space detection device 102. The free space detection device 102 includes a receiving unit 201, a normalization unit 202, and a transmitting unit 203. The receiving unit 201 receives the obstacle detection results from the camera sensor 105, obtained via the common bus 103, into the free space detection device 102 and outputs them to the normalization unit 202. The normalization unit 202 uses the obstacle detection results to convert them into a common format for free space and outputs them as free space information to the transmitting unit 203. In this embodiment, the process of converting obstacle information into a common format for free space is called normalization. The information transmission block transmits the free space information converted into a common format by the normalization unit 202 to the driving action planning device 104 via the common bus 103.

[0022] The normalization unit 202 may include an endpoint attribute assignment unit 2021 and a line segment attribute assignment unit 2022, as shown in Figure 2. However, the endpoint attribute assignment unit 2021 and the line segment attribute assignment unit 2022 are preliminary configurations, and the normalization unit 202 does not need to include at least one of the endpoint attribute assignment unit 2021 and the line segment attribute assignment unit 2022. The endpoint attribute assignment unit 2021 assigns object attributes to the normalized position information of obstacles. The line segment attribute assignment unit 2022 assigns attributes to line segments connecting the positions of normalized obstacles. The attributes are at least information indicating the presence or absence of obstacles, and if obstacles exist, they may be information that can identify whether the obstacle is a moving object or a stationary object. Furthermore, at least one of the velocity and acceleration of the obstacles may be included as attributes.

[0023] Each functional block of the free space detection device 102 is configured as a software function, so multiple blocks can be integrated and treated as a single block. For example, the receiving unit 201, the normalization unit 202, and the transmitting unit 203 can be configured as a single functional block to reduce the overhead associated with information transmission between blocks. Furthermore, various methods can be used for information transmission between functional blocks. For example, information can be transmitted by storing it in a shared memory, or each functional block can be placed on a different core or device and connected to it via a communication path such as serial communication, resulting in a simpler configuration for each device.

[0024] The common format adopted by the normalization unit 202 is the distance to the obstacle in each direction, i.e., the reference angle, obtained by dividing the entire circumference of the vehicle 101 into predetermined increments of angle. For example, if the increment is "0.1 degrees", the front of the vehicle 101 is set as 0 degrees, and 3600 reference angles are defined clockwise, each differing by 0.1 degrees, i.e., from 0 degrees to 359.9 degrees, and the distance to the location where an obstacle exists is set for each reference angle. In this case, the center point may be the center of the vehicle 101 or another location. Hereafter, the center point will also be called the "origin". Note that there is a possibility that no obstacle exists, so a value indicating the absence of an obstacle, such as a negative value, is set for directions in which no obstacle was detected.

[0025] In the following, each point defined by the aforementioned angle and distance will be referred to as a "free space endpoint." The area connecting the free space endpoints, which indicate the presence of obstacles, represents the outer edge of the free space, and the vehicle 101 is able to travel within the free space. In the following, a straight line passing through the origin and indicating a reference angle will also be referred to as a "reference line." Therefore, the common format in this embodiment can be said to be the positional information of the free space endpoints placed on each reference line. Every reference line has one end at the origin. For example, the first reference line is a half-line pointing straight ahead in the direction of travel of the vehicle 101, with an angle of 0 degrees. The second reference line is a half-line passing through the origin, to the left of the first reference line, and the angle it makes with the first reference line is a stepped angle.

[0026] Ideally, the free space should perfectly correspond to an area where no obstacles are present. It is also acceptable for the free space to be represented as narrower than ideal, with gaps existing between it and obstacles. However, it is unacceptable for the free space to be represented as wider than ideal, and for obstacles to be present but still recognized as free space. This is because if the driving action planning device 104 uses such free space information, the vehicle 101 may collide with an obstacle.

[0027] In this embodiment, the indicator of the appropriateness of a free space is called "precision." For example, if a free space includes an area where an obstacle exists, the precision of that free space is very low. Conversely, if a free space does not include an area where an obstacle exists at all, the smaller the distance from the obstacle, i.e., the smaller the gap area, the higher the precision can be said to be.

[0028] The common format for representing free space described here is a specific example in this embodiment, and other definitions are also possible. For example, all areas except the front of the vehicle 101 may be set to 0 degrees, or the step angle may be changed. Also, the step angle does not have to be constant throughout the entire area; for example, different step angles may be set for the front and rear of the vehicle 101. That is, the angle between the reference lines does not have to be a constant value. In this case, it is preferable to set a smaller step angle for the front, where the risk of collision is high, and a larger step angle for the rear. Such settings reduce the data size and the processing load. Furthermore, instead of setting attribute values ​​for points, attribute values ​​may be set for line segments connecting consecutive points in the angular direction, and consecutive points with the same attribute may be omitted, thereby reducing the processing load.

[0029] Figure 3 is a flowchart showing the processing of the normalization unit 202. However, in Figure 3, the processing of the endpoint attribute assignment unit 2021 and the line segment attribute assignment unit 2022 is omitted. First, in step S302, the normalization unit 202 acquires information on the detection points output by the camera sensor 105. There may be only one detection point, but there are more than one, so below, multiple detection points will also be referred to as the "input point sequence". The input point sequence, which is the processing result of step S302, is represented as an array of positions that are detection points of obstacles as seen from the vehicle 101, i.e., as a sequence of points. The obstacle position is represented, for example, as a two-dimensional coordinate on a Cartesian coordinate system with the center of the vehicle 101 as the origin, the front of the vehicle 101 as the positive x-axis, and the left side of the vehicle 101 as the positive y-axis. However, a three-dimensional coordinate system may be used to add height information and allow for a more precise determination of the collision risk with the vehicle.

[0030] In the following step S303, the normalization unit 202 transforms each detection point constituting the input point sequence into polar coordinates. The center of this polar coordinate system is, for example, the center of the vehicle 101. In the following step S304, the normalization unit 202 initializes an obstacle information array that stores information about obstacles. In the following step S355, the normalization unit 202 repeats the processing in steps S356 to S358 by sequentially changing the reference angles to be processed. Specifically, the normalization unit 202 calculates all the reference angles and stores them in an array in ascending order, and sequentially increases the index of the array to sequentially change the processing targets.

[0031] In this embodiment, the processing range of the input point sequence is limited by ordering the reference angles in ascending order, thereby saving processing time. However, the processing can be simplified by not ordering in ascending order. Note that the reference angle, reference line, and reference angle region, which will be explained next, have a 1:1:1 correspondence. Therefore, it can also be said that the processing in steps S356 to S358 is repeated by sequentially changing the reference line and reference angle region to be processed.

[0032] The reference angle region is defined by the midline between the reference line corresponding to the reference angle of the object being processed and the adjacent reference lines to the right and left. For example, if the step angle is 0.1 degrees, the reference angle region is the area between -0.05 degrees and +0.05 degrees from the reference angle of the object being processed. In this embodiment, the midline is defined as the midline between adjacent reference angles, but it is not necessary to use the midline. For example, the forward side from the perspective of the vehicle may be widened to shift obstacle detection towards the front of the vehicle, emphasizing obstacles that pose a high collision risk to the vehicle and making it easier to make safer collision judgments.

[0033] In step S356, the normalization unit 202 identifies all detection points that exist within the reference angle region of the processing target. For example, if the reference angle of the processing target is "12.5 degrees" and the step angle is "0.1 degrees", it lists detection points whose angle is "12.45 degrees" or greater and less than "12.55 degrees". In the following step S357, the normalization unit 202 identifies the detection point with the shortest distance to the origin from among the detection points identified in step S356. Hereafter, the distance from the origin to the detection point identified in this step will be called the "closest approach distance". In other words, the closest approach distance is the distance between the detection point closest to the origin and the origin among the detection points within the reference angle region.

[0034] In the following step S358, the normalization unit 202 adds the combination of the reference angle targeted in step S305 and the nearest neighbor distance obtained in step S307 to the obstacle information array initialized in step S304. This information indicates the position of a free space endpoint on a certain reference line. If a value indicating that the nearest neighbor distance could not be obtained was set in step S307, it is converted to information indicating that no obstacles exist in step S308 and then added to the obstacle information array. In step S309, if the normalization unit 202 determines that there are unprocessed reference angles, it changes the processing target and returns to step S306. If it has processed all reference angles and executed the processes from steps S306 to S308, it terminates the process shown in Figure 3.

[0035] When the processing of the normalization block shown in Figure 3 is completed, the obstacle information array initialized in step S304 stores the nearest approach distances corresponding to all reference angles, i.e., all reference lines. Note that in the initialization of step S304, information indicating that the nearest approach distance could not be obtained in step S307 may be set in advance, and if the nearest approach distance could not be obtained in step S307, the processing of step S308 may be skipped to save processing time in step S308.

[0036] The endpoint attribute assignment unit 2021 assigns attributes to each point included in the obstacle information array. For example, the endpoint attribute assignment unit 2021 may determine that there is no obstacle if the closest approach distance is farther than a predetermined threshold. The endpoint attribute assignment unit 2021 may also distinguish between stationary and moving objects based on whether the value of each point included in the obstacle information array changes over time, or it may set attributes by referring to additional information received by the receiving unit 201, such as information indicating the type of obstacle. The line segment attribute assignment unit 2022 assigns attributes to line segments based on the attributes of the points at both ends of the line segment. For example, the line segment attribute assignment unit 2022 may determine that a line segment is a moving object if at least one of the points at both ends of the line segment changes over time, and assign the stationary attribute to the line segment only if both points are stationary.

[0037] Figure 4 shows an example of an obstacle information array. Each element of the array is an angle 401, a distance 402, and an attribute 403. In steps S305 to S309, the reference angle is stored in angle 401 in the order in which it was processed, the nearest distance calculated in step S308 is stored as distance 402, and in step S307, whether or not the nearest distance was obtained is set as attribute 403. Attribute 403 is set to "stationary" or "moving" according to the information contained in the original input point sequence, and is set to "none" if the nearest distance could not be obtained in step S307. If the original input point sequence does not contain any particular information, it is treated as "moving".

[0038] By setting the presence or absence of obstacles and whether or not they are moving in attribute 403, the possibility of changes in the shape of the free space is represented, and this serves as reference information when planning driving actions based on free space information. Alternatively, instead of determining whether an obstacle is moving or stationary in attribute 403, it is also possible to simply set whether or not an obstacle is present and write the processing and information more simply. Furthermore, by setting the distance to 0 or greater if an obstacle is present, and a negative distance to distance 402 if there is no obstacle, attribute 403 can be omitted to save memory and communication bandwidth.

[0039] In this case, instead of a negative value to indicate the absence of obstacles, a value greater than or equal to the sensor detection distance may be set. Furthermore, modifications may be made as appropriate to accommodate limitations of the programming language used. For example, if the programming language itself, or a specific variable type, cannot handle negative values, "0" may be used instead of a negative value. Additionally, if a reference angle is pre-designed, the reference angle can be calculated from the array number, allowing for the omission of angle 401 to save memory and communication bandwidth. In any case, step S308 sets the information used by the driving action planning device 104 into the obstacle information array.

[0040] Figure 5 shows the method for converting to a common format according to this embodiment. In Figure 5, the center of the vehicle 101 is used as the origin, and the detection points where the sensor detects the obstacle 602 are represented by black circles labeled 608, 609, 610, and 611. Note that the origin is not limited to the center of the vehicle 101; for example, it may be set to the center of the rear axle of the vehicle 101 to facilitate calculations of the vehicle's motion. Because the obstacle 602 is large, it is detected at multiple detection points as shown by labels 608 to 611.

[0041] In Figure 5, the four reference lines C614 to C617 are shown as dashed lines, and the median lines 603 to 606 of these reference lines are shown as dashed lines. Each of the regions 614 to 617, centered on each reference line and enclosed by the median lines, represents a reference angle region. The white stars on each reference line represent free space endpoints.

[0042] In the method of this embodiment, the closest distance to the detection point closest to the vehicle 101 in each reference angle region is defined as the closest distance to that region. That is, in region 614, the closest distance is the distance to detection point 608; in region 615, the closest distance is the distance to detection point 610; and in region 616, the closest distance is the distance to detection point 611. Therefore, the positions indicated by the white stars in Figure 5 become the free space endpoints. The free space endpoints in this embodiment, indicated by the white stars, can also be described as the points closest to the origin among the sequence of partial points in each reference angle region, moved along an arc centered at the origin, and placed at the position where they intersect with the reference line. Since there are no detection points in region 617, a negative value is set. In Figure 5, the outer edge of the free space region obtained by the method of this embodiment is shown by a solid arc.

[0043] Figure 6 shows a comparative example method. In the comparative example shown in Figure 6, a grid is created by dividing not only the angle but also the distance at predetermined intervals, and the area closer to the origin than the grid of the obstacle closest to the origin for each angle region is designated as the free space region. In Figure 6, obstacles detected by the sensor are shown as white squares, and the boundaries of the free space for each angle region are shown as solid lines. For example, in angle region θ1, obstacles exist in the 4th and 6th grids from the center, so the area closer to the origin than the 4th grid (closest to the center), i.e., the grids from the center up to the 3rd grid, is the free space. In angle region θ2, obstacles exist between the 3rd and 4th grids from the center, and in the 5th to 7th grids. In this case, the area closer to the origin than the 3rd grid (closest to the center), i.e., the grids from the center up to the 2nd grid, is the free space. In this comparative example, not only the angle of the free space but also the distance to the origin takes discrete values, so the accuracy is lower than in this embodiment.

[0044] Figure 7 is a hardware configuration diagram of the free space detection device 102. The free space detection device 102 comprises a CPU 41, which is a central processing unit; a ROM 42, which is a read-only storage device; a RAM 43, which is a read-write storage device; and a communication device 45. The CPU 41 performs the various calculations mentioned above by loading the program stored in the ROM 42 into the RAM 43 and executing it.

[0045] The free space detection device 102 may be implemented using a rewritable logic circuit such as an FPGA (Field Programmable Gate Array) or an application-specific integrated circuit such as an ASIC (Application Specific Integrated Circuit) instead of the combination of CPU 41, ROM 42, and RAM 43. Alternatively, the free space detection device 102 may be implemented using a different configuration, such as a combination of CPU 41, ROM 42, RAM 43 and FPGA, instead of the combination of CPU 41, ROM 42, and RAM 43.

[0046] According to the first embodiment described above, the following effects and advantages can be obtained. (1) The free space detection device 102 places free space endpoints determined based on objects detected around the center point on multiple reference lines extending radially from a predetermined center point, such as the center of a vehicle 101, for example, the straight line L101 in Figure 6, and extracts the outer edge of the free space by connecting the free space endpoints on each reference line. The normalization unit 202 sets boundary lines between adjacent reference lines, designating one of the reference lines as the first reference line (L101 in Figure 6), and designating each of the two boundary lines (L102 and L103 in Figure 6) that sandwich the first reference line as the first boundary line, and calling the region sandwiched by the first boundary lines with the first reference line as the center (hatched region in Figure 6) the first angular region. The position of the free space endpoint (star in Figure 6) on the first reference line is determined based on the position closest to the center point among the detection points within the region (circled "C" to "E") that exist in the first angular region. Therefore, as explained with reference to Figure 6, the method of this embodiment can set a free space with higher accuracy than the comparative example.

[0047] (2) The normalization unit 202 includes an endpoint attribute assignment unit 2021 that sets attributes for free space endpoints based on the detection state of the object. Therefore, free space information can be used more appropriately.

[0048] (3) The normalization unit 202 includes a line segment attribute assignment unit 2022 that assigns attributes to line segments connecting free space endpoints based on the attributes assigned to the free space endpoints. Therefore, free space information can be used more appropriately.

[0049] (Variation 1) The output of the camera sensor 105 does not necessarily have to match the representation of the processing result in step S302. For example, the output of the camera sensor 105 may be in a coordinate system other than a Cartesian coordinate system, or in a Cartesian coordinate system where the origin and axes do not coincide. In this case, the normalization unit 202 performs a coordinate transformation in step S302. An example of a case where the origin does not coincide is when the coordinate system of the camera sensor 105 uses the mounting position of the camera sensor 105 as its origin. Also, the output of the camera sensor 105 may be a combination of the center position and size of an obstacle. In this case, the outer perimeter position of the obstacle is calculated in step S302.

[0050] Furthermore, in step S302, detection correction may be performed based on the attitude of the vehicle 101, such as roll and pitch, due to the acceleration and deceleration of the vehicle 101. In addition, the movement of the vehicle 101 due to the time difference between the detection time and the processing time may be reflected in the output of the camera sensor 105, or the accuracy and reliability of the detection position may be improved by using statistical processing such as a Kalman filter.

[0051] —Second Embodiment— A second embodiment of the free space detection device will be described with reference to Figures 8 to 14. In the following description, the same reference numerals are used for components that are the same as in the first embodiment, and the differences will be mainly explained. Points that are not specifically described are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that it also uses information from detection points within adjacent reference angle regions.

[0052] Figure 8 is a flowchart showing the processing of the normalization unit 202 in the second embodiment. Processes identical to those shown in the flowchart in Figure 3 in the first embodiment are given the same step numbers and their explanations are omitted. The processing of the first step S302 is the same as in Figure 3. In step S393, which is executed after step S302, the normalization unit 202 sorts the input point sequence in order of angle. The angle representation may be obtained using the atan2 function, or, for example, the sin and cos values ​​may be obtained using the Euclidean distance to the coordinates of the points and the coordinate values, and together with the sign of the coordinate values, only the magnitude relationship of the angles may be determined. In the latter case, the processing time required for the atan2 function can be saved. Note that the sorting in step S393 is performed for the purpose of reducing the processing load in step S306, which will be described later, so step S393 may be deleted and the processing in step S306 may be increased.

[0053] The normalization unit 202 executes step S304 after step S393. The processing in step S304 is the same as in Figure 3. In the following step S305, the normalization unit 202 changes the reference angles to be processed in order and repeats the processing in steps S306 to S308. Specifically, the normalization unit 202 calculates all the reference angles, stores them in an array in ascending order, and then sequentially increases the index of the array to change the processing target in order. In this embodiment, the processing range of the input point sequence is limited by ordering the reference angles in ascending order, thereby saving processing time, but the processing can be simplified by not ordering them in ascending order. Note that since there is a one-to-one correspondence between the reference angles and the reference lines, it can also be said that the processing in steps S306 to S308 is repeated by sequentially changing the reference lines to be processed.

[0054] In step S306, the normalization unit 202 performs the process of generating a sequence of partial points that overlap with the reference angle region at the reference angle to be processed. The outline of the process in this step will be described in detail later with reference to Figure 5. In step S307, the normalization unit 202 calculates the nearest nearest point, which is the distance to the point closest to the vehicle 101 on the line segment connecting the partial points generated in step S306. Here, the line segment connecting the partial points is the line segment obtained by connecting the points that make up the partial points in the order of the point sequence. If a partial point sequence is not defined, a value indicating that the nearest nearest point could not be obtained, such as a negative value, is set.

[0055] In the following step S308, the normalization unit 202 adds the combination of the reference angle targeted in step S305 and the nearest neighbor distance obtained in step S307 to the obstacle information array initialized in step S304. This information indicates the free space endpoint on a certain reference line. If a value indicating that the nearest neighbor distance could not be obtained was set in step S307, it is converted to information indicating that no obstacles exist in step S308 and then added to the obstacle information array. In step S309, if the normalization unit 202 determines that there are unprocessed reference angles, it changes the processing target and returns to step S306. If all reference angles are processed and the processing from steps S306 to S308 is executed, the process shown in Figure 8 is terminated.

[0056] (Details of the partial point generation process) Figure 9 is a flowchart detailing the partial point sequence generation process in step S306 of Figure 8. A specific example of the process shown in Figure 5 will be described later. Before the process shown in Figure 9 begins, the reference angle to be processed, or in other words, the reference line to be processed, is set in step S305. Within the range shown in Figure 9, the reference angle and reference line to be processed do not change, and therefore the reference angle region also does not change.

[0057] In step S502, the normalization unit 202 first initializes the array for the partial point sequence, the start number, and the end number. The elements of the array for the partial point sequence are in the same format as the input point sequence and contain at least the coordinate values ​​of the points. The array for the partial point sequence is initialized with coordinate values ​​that indicate invalid points, such as very large values. Note that the number of array elements may be made variable to save memory required for the array, in which case the number of elements may be set to "0" during initialization. The start number and end number will store the array index of the input point sequence in later processing. Therefore, the start number and end number are initialized with values ​​that indicate they are not array indexes, such as negative values.

[0058] In the following step S503, the normalization unit 202 calculates the reference angle region. Specifically, the normalization unit 202 calculates the start angle and end angle of the reference angle region. For example, if the step angle is "0.1 degrees" and the reference angle to be processed is "30 degrees", the start angle is 29.95 degrees and the end angle is 30.05 degrees. As mentioned above, the reference angle region is the region centered on the reference line. In step S504, the normalization unit 202 starts a loop that sequentially increases the array index of the input point sequence from "0". Specifically, the normalization unit 202 repeats the processing in steps S505 to S509 while increasing the array index of the data to be processed by "1" each time.

[0059] In step S505, the normalization unit 202 obtains the angle of the input point corresponding to the array number to be processed. The angle can be expressed as a value obtained using atan2, as explained earlier, or as a sin / cos pair, as long as the angle can be compared with the start angle and end angle. In the following step S506, the normalization unit 202 determines whether the angle of the input point obtained in step S505 is greater than or equal to the angle of the start point calculated in step S503. If the normalization unit 202 determines that the angle of the input point is greater than or equal to the angle of the start point, it proceeds to step S507; if it determines that the angle of the input point is less than the angle of the start point, it proceeds to step S510.

[0060] In step S507, if the start number initialized in step S502 remains initialized, i.e., unset, the normalization unit 202 sets the array number to be processed as the start number and proceeds to step S508. If the start number is not unset, the normalization unit 202 does nothing and proceeds to step S508. In step S508, the normalization unit 202 determines whether the angle of the point obtained in step S505 is smaller than the end angle calculated in step S503. If the normalization unit 202 determines that the angle of the point obtained in step S505 is smaller than the end angle calculated in step S503, it proceeds to step S509. If the normalization unit 202 determines that the angle of the point obtained in step S505 is greater than or equal to the end angle calculated in step S503, it proceeds to step S510.

[0061] In step S509, if the end number initialized in step S502 remains initialized, i.e., unset, the normalization unit 202 sets the end number to the array number of the item to be processed minus "1" and proceeds to step S510. If the end number is not unset, the normalization unit 202 does nothing and proceeds to step S510. In step S510, if the array number of the item to be processed is the last in the input point sequence, the normalization unit 202 proceeds to step S511. If the array number of the item to be processed is not the last in the input point sequence, it sets the array number of the item to be processed to the next array number in the input point sequence and returns to step S505. Note that in the angle comparison in steps S506 and S508, the comparison is performed using a value obtained by adding 360 degrees to the angle or subtracting 360 degrees, so that the absolute value of the angle difference is 180 degrees or less. This is to prevent the reversal of the greater-than / less-than relationship due to crossing 0 degrees or 360 degrees.

[0062] In step S511, the normalization unit 202 determines whether any values ​​have been set for both the start number and the end number. If it determines that some values ​​have been set for both, it proceeds to step S512; if it determines that at least one of them has not been set, it proceeds to step S513. Whether or not the start number has been set is equivalent to whether or not step S507 has been executed. If step S507 or step S509 has not been executed for reasons such as the absence of any points included in the input point sequence within the reference angle region, step S511 is deemed negative.

[0063] In step S512, the normalization unit 202 adds the intersection point of the line segment connecting the input point sequences and the starting angle (hereinafter referred to as the "starting intersection point") to the beginning of the partial point sequence array initialized in step S502, and proceeds to step S514. Here, the starting intersection point is the intersection point of the line segment formed by sequentially connecting the points in the input point sequence and the half-line extended from the center of the vehicle 101 in the direction of the starting angle calculated in step S503. If no starting intersection point exists, the normalization unit 202 does not add the starting intersection point to the partial point sequence array in the processing of step S512.

[0064] In step S514, the normalization unit 202 adds the input point sequence, which includes the start number and end number and is pointed to by the sequence numbers from the start number to the end number, to the end of the partial point sequence array initialized in step S502. In the following step S515, the normalization unit 202 adds the intersection point of the input point sequence and the end angle (hereinafter referred to as the "end intersection point") to the partial point sequence array and completes the process shown in Figure 9. The end intersection point is the intersection point of the line segment formed by sequentially connecting the points included in the input point sequence and the half-line extended from the vehicle in the direction of the end angle calculated in step S503. If no end intersection point exists, the normalization unit 202 does not add an intersection point to the partial point sequence array in step S515.

[0065] If a negative determination is made in step S511, in step S513, the normalization unit 202 does not add anything to the partial point sequence array, leaving the partial point sequence array empty, and terminates the process shown in Figure 9. The process shown in Figure 9, as described above, aims to generate the partial point sequence array initialized in step S502, and the processes from step S307 onwards shown in Figure 8 are executed using the partial point sequence array.

[0066] (Specific example of a partial point sequence generation process) Figure 10 shows a specific example of the process shown in Figure 9. As shown at the top of Figure 10, in this example, the input point sequence contains seven points in order, from circled "A" to circled "G". In step S503, the normalization unit 202 calculates L102, which indicates the start angle, and L103, which indicates the end angle. The area enclosed by the lines L102 and L103, indicated by the hatched dots, is the reference angle region. Note that the positions of the input point sequence, the triangles "P" and "Q", and the star marks shown in Figure 10 are shown for explanatory purposes only, and this information is not obtained when step S503 is completed. Specifically, the reference angle region includes points circled "C" through "E".

[0067] As mentioned above, steps S504 to S510 are executed repeatedly for each input point sequence. Below, the number of repetitions will be expressed as a "loop". For example, in the first loop, the input point sequence to be processed is the circled "A", and since there are a total of "7" elements in the input point sequence, the process will be performed up to the 7th loop in this example.

[0068] In the first loop, the normalization unit 202 processes the circled "A" and makes a negative judgment in step S506, ending the process. In the second loop, the normalization unit 202 processes the circled "B" and, as in the first loop, makes a negative judgment in step S506, ending the process. In the third loop, the normalization unit 202 determines that the circled "C," which is the target of processing, is included in the reference angle region, so it makes an affirmative judgment in step S506 and proceeds to step S507. In step S507, since the normalization unit 202 is executed for the first time in this third loop, it stores the array index "3" as the starting number. In the following step S508, the normalization unit 202 determines that it is located to the right of the line L103 in the diagram, so it makes a negative judgment and ends the process.

[0069] In the 4th and 5th loops, the normalization unit 202 processes the circled "D" and "E" and makes a positive judgment in step S506, but does not perform any special processing because it is not the first time step S507 is executed. Then in step S508, the normalization unit 202 makes a negative judgment and terminates processing. In the 6th loop, the normalization unit 202 makes a positive judgment in step S506, but does not perform any special processing because it is not the first time step S507 is executed. Then in step S508, the normalization unit 202 makes a positive judgment and proceeds to step S509. In step S509, since it is the first time the normalization unit 202 is executed in this 6th loop, it sets the termination number to "5", which is the element number "6" minus "1", and terminates processing.

[0070] In the seventh loop, the normalization unit 202 processes the circled "F" and makes a positive judgment in both step S506 and step S508. However, since neither step S507 nor step S509 is the first execution, the normalization unit 202 terminates without performing any specific processing. Once this seventh loop is completed, the processing in steps S504 to S510 is completed. In the following step S511, the normalization unit 202 makes a positive judgment because both the start number and end number have been set, and proceeds to step S512.

[0071] In step S512, the normalization unit 202 calculates the starting intersection point, which is the intersection of a line segment formed by sequentially connecting the points in the input point sequence and a half-line extended from the center of the vehicle 101 in the direction of the starting angle calculated in step S503. In this example, since L102, which indicates the starting angle, passes between the circled "B" and "C", the normalization unit calculates the coordinates of the triangled "P", which is the intersection point between the line segment connecting the circled "B" and "C" and the line L102. The normalization unit 202 then adds this triangled "P" to the sub-point sequence. In step S514, the normalization unit 202 adds the circled "C" to "E", which are elements of the input point sequence from the starting number "3" to the ending number "5", to the sub-point sequence.

[0072] In step S515, the normalization unit 202 is the intersection point of the line segment formed by sequentially connecting the points of the input point sequence and the half-line extended from the center of the vehicle 101 in the direction of the final angle calculated in step S503. end The intersection point is calculated. In this example, since L103, which indicates the end angle, passes between the circled "E" and "F", the coordinates of the triangled "Q", which is the intersection point between the line segment connecting the circled "E" and "F" and the line line L103, are calculated. The normalization unit 202 then adds this triangled "Q" to the sub-point sequence. Through the above process, the sub-point sequence is calculated as triangled "P", circled "C", "D", "E", and triangled "Q". The above is a concrete example of the process shown in Figure 9.

[0073] Once these partial point sequences are obtained, the distance to the triangularly enclosed "P" closest to the origin is calculated as the nearest neighbor distance through the processing in steps S307 and S308 of Figure 8. Therefore, the free space endpoints in the reference angle region shown in Figure 10 are set to the same distance from the origin to the triangularly enclosed "P" at the position of the star, which is reached by moving along the reference line L101 from the origin.

[0074] (effect) Referring to Figures 11 and 12-14, we will compare the conversion method to the common format according to this embodiment with the conversion method in the first embodiment and the conversion method to the common format according to the second comparative example method. Figure 11 compares the conversion method in the first embodiment with the conversion method according to this embodiment, and Figures 12-14 compare the second comparative example method with the conversion method according to this embodiment.

[0075] Figure 11 is a diagram comparing the conversion method to the common format according to this embodiment with the conversion method in the first embodiment. However, since the configuration of Figure 11 is the same as that of Figure 5 in the first embodiment, the explanation of the redundant configuration is omitted. The conversion method in the first embodiment uses the closest detection point within the reference angle region. The white stars on each reference line represent the free space endpoints. The meaning of the white circles will be explained later. When these detection points 608 to 611 are converted to the common format of the free space using the method of the first embodiment as an input point sequence, the outer edge of the free space region is set to the position indicated by the solid arc, as explained with reference to Figure 5.

[0076] On the other hand, using the method of this embodiment, points 612 and 613, indicated by white circles, are obtained in the process of calculating the partial point sequence. On the detection point sequence 608-611, where the obstacle 602 was detected, the nearest point is set as point 612 in region 614, point 610 in region 615, and point 613 in region 616, respectively, as the point closest to the vehicle. No detection points exist in region 617. In Figure 11, the outer edge of the free space region obtained by the method of this embodiment is shown by a dashed arc. However, in region 615, the outer edge of the free space region obtained by the method of the first embodiment and the outer edge of the free space region obtained by the method of this embodiment overlap. The free space endpoints of this embodiment, indicated by white stars, can also be said to be obtained by moving the point closest to the origin from the partial point sequence in each reference angle region along an arc centered on the origin and placing it at a position where it intersects the reference line.

[0077] Comparing the method of the first embodiment with the method of this embodiment, the method of the first embodiment uses the distance from the origin of the actually detected point as the closest approach distance, emphasizing the reliability of the detection point. On the other hand, the method of the second embodiment assumes that obstacles other than the detection point may exist, and interpolates the distance between the detection points to the intersection with the median of the reference line, using the distance to the intersection with the median line of the reference line as a candidate for the closest approach distance. There is no superiority or inferiority between the method of the first embodiment and the method of the second embodiment; they simply have different approaches to the detection points detected by the sensor.

[0078] Next, Figures 12, 13, and 14 illustrate a comparison between the method for converting to a common format according to this embodiment and the method of the second comparative example. The method of the second comparative example converts to a common format by simply finding the intersection points of the detection point sequence and each reference angle using linear interpolation. Figure 12 shows the obstacles and the detection points before conversion to a common format. In Figure 12, the vehicle 101 is located at the bottom of the figure, and the sensor detects obstacles 702 and 703. The results detected by the sensor are detection points 704, 705, 706, and 707, and the sequence of points 708 connecting these points is used as the input sequence of points. Obstacle 702 is an obstacle large enough to produce multiple detection points, such as a side wall, while 703 is a small obstacle such as a pole that produces only one detection point.

[0079] Figure 13 shows the result of converting the detection results from Figure 12 using the second comparative example method. In the second comparative example method, linear interpolation is used to convert to a common format. In Figure 13, the half-line 801 extending radially from the vehicle 101 represents the reference line. Points 802, 803, 804, 805, and 806 are obtained by linear interpolation at the intersections of the input point sequence 708 with the reference line 801, forming the point sequence 807.

[0080] In point sequence 807, points 804 and 805 are located to the left and right of obstacle 703. Therefore, it represents that there is free space extending further than the actual location of obstacle 703, and point sequence 807 does not adequately reflect the presence of obstacle 703. In other words, it represents free space where the presence of obstacle 703 has been overlooked. This is because each detection point in detection point sequence 708 lies between the half-lines of the reference line 801.

[0081] Planning driving actions based on free space while overlooking obstacle 703 may result in a plan that leads to a collision with obstacle 703, thus increasing the risk of collision. Normalization using linear interpolation is advantageous in that it requires less computation to determine distances at each reference angle, but it can overlook the presence of obstacles and therefore cannot accurately determine the distance to obstacles.

[0082] Figure 14 shows the result of converting the detection results in Figure 12 using the method of this embodiment. In Figure 14, a median line 901, shown as a dashed line, is determined relative to the reference line 801, shown as a solid line. That is, each region between the median lines 901 is a reference angle region. Then, a sub-point sequence that overlaps with the detection point sequence 708 within each reference region is determined, and the point 902 closest to the vehicle on the line segment of the sub-point sequence is indicated with a cross, or plus marker. There is a maximum of one point 902 in each reference angle region. Then, the distance from the vehicle 101 to point 902 is set as the distance to the obstacle on each reference line 801, and point 903 is set at the set distance from the vehicle 101 on each reference line, and the sequence of points connecting point 903 is shown as point sequence 904. Looking at point sequence 904, it can be seen that point 903 is set closer to the obstacle 703 compared to the point sequence in Figure 13, and the free space that captures the obstacle 703 is represented more precisely.

[0083] According to the second embodiment described above, the following effects and advantages can be obtained. (4) The free space detection device 102 determines the position of the free space endpoint (star in Figure 11) on the first reference line based on the position of the region detection points (circled "C" to "E") which are detection points that exist in the first angular region, and the intersection points (triangles "P" and "Q") where the straight lines connecting the region detection points and detection points not included in the first angular region intersect the first boundary line, which are the positions closest to the center point. Therefore, as explained with reference to Figures 12 to 14, the method of this embodiment can set a free space with high accuracy.

[0084] —Third Embodiment— A third embodiment of the free space detection device will be described with reference to Figures 15 and 16. In the following description, the same reference numerals are used for components that are the same as in the first embodiment, and the differences will be mainly explained. Points that are not specifically explained are the same as in the first embodiment. This embodiment differs from the first embodiment mainly in that multiple sensors are mounted on the vehicle.

[0085] Figure 15 is a hardware configuration diagram of vehicle 101A equipped with a free space detection device 102A. Vehicle 101A is equipped with a camera sensor 105, a common bus 103, and a driving action planning device 104, similar to vehicle 101 in the first embodiment. Vehicle 101A is further equipped with a radar sensor 1003 and a LIDAR 1004, and is equipped with a free space detection device 102A instead of the free space detection device 102. The free space detection device 102A has the same hardware configuration as the free space detection device 102 in the first embodiment.

[0086] In Figure 15, all sensors are connected via a common bus 103, but the connection configuration is not limited as long as the detection results of each sensor can be transmitted to the free space detection device 102A. For example, the camera sensor 105, radar sensor 1003, and LIDAR 1004 may each transmit their detection results to the free space detection device 102A using different communication paths or protocols. These sensors can use, for example, LVDS (Low Voltage Differential Signaling), CAN, IEEE 802.3, etc. By adopting such a configuration, it is possible to reduce the development cost of sensors by adopting a communication method suitable for the sensor, and to reduce the risk of communication interruption due to a failure of the common bus 103.

[0087] The free space detection device 102A synthesizes the detection results from multiple sensors into a single piece of free space information and transmits it to the driving action planning device 104. This embodiment has advantages such as increasing fault tolerance by making the sensors redundant, obtaining a wider range of free space information by having sensors with different detection principles and detection ranges complement each other, and obtaining more reliable free space information.

[0088] Figure 16 is a functional block diagram of the free space detection device 102A. A The system includes a camera detection receiver 1101, a radar detection receiver 1102, a LiDAR detection receiver 1103, a sensor fusion unit 1104, a first normalization unit 202-1, a second normalization unit 202-2, a third normalization unit 202-3, a fourth normalization unit 202-4, and a transmission unit 203. The camera detection receiver 1101, the radar detection receiver 1102, and the LiDAR detection receiver 1103 have the same functions as the receiver 201 in Figure 1. The camera detection receiver 1101, the radar detection receiver 1102, and the LiDAR detection receiver 1103 receive obstacle information detected by the camera sensor 105, the radar sensor 1003, and the LiDAR 1004, respectively, and output it as an input point sequence to the corresponding normalization unit 202.

[0089] The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 perform the same processing as the normalization unit 202 in the first embodiment. The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 differ only in the data they process, and there is no particular difference in their operation. The first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3 convert the obstacle information into a common format as free space, i.e., perform normalization processing, and output it to the sensor fusion unit 1104. At this time, the origins of the free space information output by each normalization unit 202 do not need to coincide.

[0090] The sensor fusion unit 1104 acquires the free space information output by the first normalization unit 202-1, the second normalization unit 202-2, and the third normalization unit 202-3, converts it into a coordinate system having a unified origin, for example, the center of the vehicle 101A, and then rearranges it by angle viewed from that origin. However, the method of integrating the three free space information is not limited to this. For example, after unifying and rearranging the origins of the three free space information, multiple points that are within a predetermined distance from each other may be aggregated at the centroid to reduce the number of points. Alternatively, as is common in a technique called sensor fusion, each point in different free spaces may be tracked using a Kalman filter or the like, and combined into a more likely point to improve the reliability of the point's position.

[0091] The fourth normalization unit 202-4 takes the obstacle information input from the sensor fusion unit 1104 as an input point sequence, processes it in the same way as the normalization unit 202 in the first embodiment, converts it into a free space in a common format, and outputs it to the transmission unit 203. The transmission unit 203 receives the free space information transmitted from the fourth normalization unit 202-4. sending I believe.

[0092] In this embodiment, by placing the normalization unit 202 at both the input and output of the sensor fusion unit 1104, it becomes unnecessary to be aware of differences in the output format of the detection results from the sensors. Furthermore, since there is no need for a mechanism to guarantee that the output will conform to a common format during processing, the processing of the sensor fusion unit 1104 can be simplified.

[0093] (Modified example of the third embodiment) In the third embodiment described above, the normalization unit 202 is placed at both the input and output of the sensor fusion unit 1104. However, in order to reduce the processing load, the normalization unit 202 on the input side of the sensor fusion unit 1104 may be omitted, leaving only the normalization unit 202 on the output side of the sensor fusion unit 1104, or the normalization unit 202 on the output side of the sensor fusion unit 1104 may be omitted. However, if the normalization unit 202 on the input side of the sensor fusion unit 1104 is omitted, the sensor fusion unit 1104 will need to handle detection results of different formats for each sensor. Also, if the normalization unit 202 on the output side of the sensor fusion unit 1104 is omitted, the sensor fusion unit 1104 will need to perform processing to make the free space output by the sensor fusion unit 1104 a common format.

[0094] —Fourth Embodiment— A fourth embodiment of the free space detection device will be described with reference to Figures 17 and 18. In the following description, the same reference numerals are used for components that are the same as in the second embodiment, and the differences will be explained primarily. Points that are not specifically described are the same as in the second embodiment. In this embodiment, the operation of the normalization unit 202 differs from that of the second embodiment. Other configurations and operations are the same as in the second embodiment, so their description will be omitted.

[0095] Figure 17 is a flowchart showing the processing of the normalization unit 202 in the fourth embodiment. The flowchart shown in Figure 17 corresponds to steps S511 onwards in Figure 5 of the second embodiment. That is, the entire processing shown in Figure 8 of the second embodiment, and the processing of steps S502 to S510 in Figure 9 are the same as in the first embodiment. When the loop processing shown in steps S504 to S510 is completed, the normalization unit 202 executes step S1202.

[0096] In step S1202, the normalization unit 202 calculates the starting intersection in the same way as the process in step S512 in the first embodiment. As explained in the first embodiment, there may be cases where there is no starting intersection. In the following step S1203, if there is a starting intersection, the normalization unit 202 calculates the input point located immediately to the right of the starting intersection (hereinafter referred to as the "starting right point"), and does nothing if there is no starting intersection. The starting right point is the input point located to the right of the starting intersection as viewed from the vehicle 101, which has the closest angle to the starting intersection. However, depending on the values ​​in the input point sequence, there may be cases where there is no starting right point.

[0097] In step S1204, the normalization unit 202 determines whether both the starting intersection and the starting right point exist and whether the starting intersection and the starting right point are sufficiently close. The definition of sufficiently close will be described later. If the normalization unit 202 determines that both the starting intersection and the starting right point exist and that the starting intersection and the starting right point are sufficiently close, it proceeds to step S1205. If it determines that either the starting intersection or the starting right point does not exist, or that the starting intersection and the starting right point are not sufficiently close, it proceeds to step S1206. In step S1205, the normalization unit 202 adds the starting intersection to the partial point sequence array initialized in step S502 and proceeds to step S1206.

[0098] In step S1206, the normalization unit 202 determines whether both the start number and the end number are set, similar to step S511 in the first embodiment. If the normalization unit 202 determines that both the start number and the end number are set, it proceeds to step S1207; if it determines that at least one of the start number and the end number is not set, it proceeds to step S120 8 The process continues. In step S1207, the normalization unit 202 adds the input sequence from the start number to the end number to the subsequence sequence array, similar to step S514 in the first embodiment, and proceeds to step S1208.

[0099] In step S1208, the normalization unit 202 calculates the end intersection in the same way as in step S515 in the first embodiment. However, as described in the first embodiment, there may be cases where no end intersection exists. In the following step S1209, if an end intersection exists, the normalization unit 202 calculates the input point located immediately to the left of the end intersection (hereinafter referred to as the "end left point"), and does nothing if no end intersection exists. The end left point is the input point located to the left of the end intersection as viewed from the vehicle 101, and is the point with the closest angle to the end intersection. Depending on the value of the input point sequence, the end left In some cases, no points exist.

[0100] In the following step S1210, the normalization unit 202 determines whether a termination intersection and a termination left point exist, and whether the termination intersection and the termination left point are sufficiently close. If both the termination intersection and the termination left point exist and it is determined that they are sufficiently close, the process proceeds to step S1211. If either the termination intersection or the termination left point does not exist, or if it is determined that the termination intersection and the termination left point are not sufficiently close, the process in Figure 17 is terminated. In step S1211, the normalization unit 202 adds the termination intersection to the partial point sequence array and terminates the process in Figure 17.

[0101] In steps S1204 and S1210, "sufficiently close" means that the distance between the two points is less than a predetermined value, for example, 1m. However, the definition of "sufficiently close" may be other. For example, it may be defined as the distance between the two points being less than 0.1m, in which case the risk of incorrect interpolation to a location where no obstacles exist is reduced. Alternatively, it may be defined as the difference in angle between the two points being less than or equal to a predetermined threshold, in which case the risk of incorrect interpolation is reduced. By adopting this embodiment, even if the input point is not included within the reference angle region, the free space endpoint can be interpolated if the conditions of step S1204 or step S1208 are met. Therefore, for example, even if the obstacle detection by the sensor is coarser than the common format, a free space with the position of the obstacle interpolated can be generated.

[0102] Figure 18 shows a specific example of the processing of the normalization unit 202 in this embodiment. Figure 18 has substantially the same preconditions as Figure 10 in the second embodiment and explains the calculation of free space endpoints in the reference angle region centered on the reference line L101. The difference from Figure 10 is that the input point sequence is smaller, consisting only of the circled "A", "B", "F", and "G". Therefore, there is no input point sequence in the reference angle region shown by hatching. In this embodiment, the starting intersection point "P" enclosed in a triangle is calculated in step S1202, and the ending intersection point "Q" enclosed in a triangle is calculated in step S1208. Therefore, if a positive judgment is made in all of steps S1204, S1206, and S1210, the triangled "P" and "Q" are set as partial point sequences.

[0103] Then, through the processing in steps S307 and S308 in Figure 8, the distance between the triangular-enclosed "P" closest to the origin and the origin is added to the obstacle information array as the closest approach distance. Therefore, in the example shown in Figure 15, the distance from the free space endpoint, indicated by a white star on the reference line L101, to the origin is equal to the distance from the triangular-enclosed "P" to the origin.

[0104] According to the fourth embodiment described above, the following effects and advantages can be obtained. (5) If no object existence points exist within the first reference angle region, the normalization unit 202 sets free space endpoints based on the position of the intersection point between the line connecting a pair of object existence points on both sides of the first reference angle region and the first boundary line. Therefore, free space endpoints can be formed even if no input point sequence exists in the reference angle region.

[0105] (Modified version of the fourth embodiment) In this embodiment, steps S511 onwards in Figure 9 are replaced with the processing from steps S1201 to S1211. However, it is also possible to retain step S511 and replace steps S512 onwards with the processing from steps S1201 to S1211. In this case, if there is no start number or end number, step S513 will be executed, and if there is no input point sequence within the reference angle region, a partial point sequence will not be generated, thereby reducing the risk of excessive interpolation of free space endpoints in this embodiment.

[0106] In the embodiments described above, the vehicle on which the free space detection device is mounted is assumed to be a four-wheeled vehicle, but the free space detection device may be mounted on vehicles other than four-wheeled vehicles. For example, the free space detection device may be mounted on a mobile vehicle such as a motorcycle, bus, truck, three-wheeled vehicle, cart, tracked vehicle, or railway vehicle. Furthermore, the free space detection device does not have to be mounted on a mobile body. For example, the free space detection device may be installed in a warehouse where carts move, and the free space detection device may process information from sensors installed in the warehouse. In this case, the degree of flexibility in sensor installation can be increased.

[0107] Furthermore, when a free space detection device processes the outputs of multiple sensors, as in the second embodiment, the reliability of detection can be increased by arranging the multiple sensors so that they compensate for each other's blind spots. Multiple free space detection devices may also be connected in a communicative manner, and free space information output by one free space detection device may be used by another. In this case, a free space detection device may output not only free space information but also sensor outputs to other free space detection devices, and a free space detection device can generate free space information using sensor information connected to other free space detection devices. By performing the normalization described in this embodiment when outputting free space information to other devices, it becomes unnecessary to be aware of the output format of each sensor, simplifying the design of the monitoring device.

[0108] In the embodiments and modifications described above, the configuration of the functional blocks is merely an example. Several functional configurations shown as separate functional blocks may be integrated, or a configuration represented in one functional block diagram may be divided into two or more functions. Furthermore, some of the functions of one functional block may be provided by other functional blocks.

[0109] In the embodiments and modifications described above, the program is stored in a ROM 42 (not shown), but the program may be stored in a non-volatile storage device. Furthermore, the free space detection device may have an input / output interface (not shown), and the program may be read from another device via a medium available to the input / output interface and the free space detection device when needed. Here, "medium" refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, i.e., a wired, wireless, or optical network, or a carrier wave or digital signal propagating through such a network. Also, some or all of the functions realized by the program may be realized by hardware circuits or FPGAs.

[0110] The embodiments and modifications described above may be combined in any way. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments that can be conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. [Explanation of Symbols]

[0111] 102, 102A: Free space detection device 201: Receiving Unit 202: Normalization section 2021: Endpoint attribute assignment section 2022: Line segment attribute assignment section

Claims

1. A free space detection device that places free space endpoints determined based on detection points of objects detected around a predetermined center point on a plurality of reference lines extending radially from the center point, and extracts the outer edge of the free space by connecting the free space endpoints on the plurality of reference lines, A boundary line is set midway between adjacent reference lines. When one of the aforementioned reference lines is called the first reference line, and each of the two boundary lines that enclose the first reference line is called the first boundary line, and the region enclosed by the first boundary lines with the first reference line as the center is called the first angular region, A free space detection device in which the position of the free space endpoint on the first reference line is determined based on the position of the detection point within the region that is closest to the center point among the detection points that exist in the first angular region.

2. A free space detection device according to claim 1, A free space detection device in which the position of the free space endpoint on the first reference line is determined based on the position of the detection point within the region, and the position of the intersection point where a straight line connecting the detection point within the region and the detection point not included in the first angular region intersects the first boundary line, the position closest to the center point.

3. A free space detection device according to claim 2, A free space detection device that, when no detection points exist within the first angular region, sets the free space endpoint based on the position of the intersection point of a line connecting a pair of detection points located on both sides of the first angular region.

4. A free space detection device according to claim 1, A free space detection device further comprising an endpoint attribute assignment unit that sets attributes for the free space endpoints based on the detection state of the object.

5. A free space detection device according to claim 4, A free space detection device further comprising a line segment attribute assignment unit that assigns attributes to line segments connecting the free space endpoints based on the attributes assigned to the free space endpoints.

6. A method for normalizing the position of an object, performed by a free space detection device, which places free space endpoints determined based on detection points of objects detected around a predetermined center point on a plurality of reference lines extending radially from the center point, and extracts the outer edge of the free space by connecting the free space endpoints on the plurality of reference lines, In a case where a boundary line is set midway between adjacent reference lines, one of the reference lines is designated as the first reference line, each of the two boundary lines flanking the first reference line is designated as the first boundary line, and the region enclosed by the first boundary lines with the first reference line as the center is called the first angular region, A method for normalizing the position of an object, comprising determining the position of the free space endpoint on the first reference line based on the position of the detection point within the region that is closest to the center point.

Images