Road surface state detection device and road surface state detection method

Abstract

To provide a road surface state detection device capable of improving position recognition accuracy of a recess region.SOLUTION: The road surface state detection device 1 detects a state of a road surface using information on a plurality of observation points obtained from the sensor 4. The attribute assignment unit 5 determines whether an observation point is the road surface 3 or a recess 8 recessed from the road surface 3, and assigns a road surface attribute indicating the road surface 3 or a recess attribute indicating the recess 8 to the observation point. The pseudo recess point generation unit 6 generates, as a pseudo recess point 14, an intersection point between a road surface virtual surface or a road surface virtual line 12 estimated from a road surface observation point 9 to which a road surface attribute is assigned among the observation points and a detection straight line 13 connecting a recess observation point 10 to which a recess attribute is assigned among the observation points and the sensor 4 by a straight line. The concave region recognition unit 7 recognizes a concave region 15 in which the concave portion 8 exists by using the concave portion observation points 10 and the pseudo concave points 14.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a road surface condition detection device and a road surface condition detection method. [Background technology]

[0002] Conventionally, road surface condition detection devices are known that detect the condition of the road surface on which a moving object moves. Objects to be detected by road surface condition detection devices include depressions on the road surface, such as potholes and road gutters. Accurately recognizing the positions of these objects is an important task for the safe and effective execution of autonomous driving and driver assistance applications.

[0003] Potholes occur on roads, especially in cold regions. When a vehicle's tires pass through a pothole, it can cause unpleasant vibrations for the driver. Furthermore, if the tire hits the wall of the pothole, the tire or part of the vehicle can be damaged. By detecting the location of the pothole using a road surface condition detection device and controlling the suspension, it is possible to suppress unpleasant vibrations and prevent the tire from passing through the pothole by encouraging braking or steering.

[0004] Furthermore, when a vehicle runs into a ditch, there is a risk that the tires may come off the road, etc. Therefore, apps such as autonomous driving and lane keep assist must accurately recognize the lateral position of the vehicle in relation to the ditch and determine the path to follow with a sufficient margin.

[0005] The road surface condition detection device described in Patent Document 1 uses a distance measurement sensor such as LiDAR mounted on a vehicle to calculate the distance between the vehicle and each of multiple observation points lined up in the vehicle's traveling direction, as well as the altitude of each of the multiple observation points.If the difference in distance or difference in altitude between two adjacent observation points in the vehicle's traveling direction is greater than a predetermined threshold, the observation point farther from the vehicle out of the two adjacent observation points is designated as a change point, and it is determined that a depression exists at that change point. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-15601 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the road surface condition detection device described in Patent Document 1 recognizes the wall of a depression on the road surface that is farther from the vehicle by using an observation point specified as a change point. Patent Document 1 does not describe or consider at all how to recognize the position of a depression that is closer to the vehicle. Hereinafter, the wall of the recess farther from the vehicle will be referred to as the "far wall of the recess" or the "far point of the recess area," and the position of the recess closer to the vehicle will be referred to as the "starting point of the recess area."

[0008] Here, when measuring distances using LiDAR or the like, observation points cannot be obtained for the wall on the starting point side of the recessed area or the bottom of the recessed area due to occlusion by the road surface on the vehicle side of the starting point of the recessed area. Therefore, the technology described in Patent Document 1 cannot accurately recognize the position of the starting point of the recessed area. Therefore, when the road surface condition detection device described in Patent Document 1 is used in autonomous driving or driving assistance applications, there is a risk that the timing of braking or steering to avoid the recessed area will be incorrect, causing the vehicle to travel through the recessed area and collide with the wall on the far side of the recessed area, or the vehicle to run off the road into the recessed area.

[0009] In view of the above, an object of the present disclosure is to provide a road surface condition detection device and a road surface condition detection method that can improve the accuracy of recognizing the position of a recessed portion area. [Means for solving the problem]

[0010] According to one aspect of the present disclosure, a road surface condition detection device detects the condition of a road surface on which a mobile body is moving using information from a plurality of observation points obtained from a sensor that observes the periphery of the mobile body using electromagnetic waves, an attribute assigning unit that determines whether an observation point is on a road surface or a depression recessed from the road surface, and assigns a road surface attribute indicating that the observation point is on a road surface or a depression attribute indicating that the observation point is a depression; a pseudo recess point generation unit that generates, as a pseudo recess point, an intersection between a virtual road surface surface or a virtual road surface line estimated from a road surface observation point that has been assigned a road surface attribute among the observation points, and a detection line that connects a recess observation point that has been assigned a recess attribute among the observation points and the sensor; The apparatus is provided with a recessed area recognition unit that recognizes recessed areas where recesses exist by using recess observation points and pseudo recess points.

[0011] According to this, the intersection of a virtual road surface or virtual road line and a detection line can be said to indicate the presence of a depression, because it is physically determined that electromagnetic waves such as light and radio waves used for sensor observation have passed through the space where the intersection exists. Therefore, by treating the intersection as a pseudo depression point and using it to recognize a depression area, the problem of inability to recognize the position of the start point of a depression area, which occurred in the technology of Patent Document 1, can be resolved. Therefore, this road surface condition detection device can improve the accuracy of position recognition of a depression area.

[0012] According to another aspect of the present disclosure, a road surface condition detection method detects a condition of a road surface on which a mobile object is moving using information from a plurality of observation points obtained from a sensor that observes a periphery of the mobile object using electromagnetic waves, determining whether the observation point is a road surface or a depression recessed from the road surface, and assigning a road surface attribute indicating that the observation point is a road surface or a depression attribute indicating that the observation point is a depression; generating, as a pseudo-depression point, a point where a virtual road surface surface or a virtual road surface line estimated from a road surface observation point to which a road surface attribute has been assigned among the observation points intersects with a detection line connecting a depression observation point to which a depression attribute has been assigned among the observation points and the sensor; This involves using recess observation points and pseudo recess points to recognize recess regions where recesses exist.

[0013] As a result, the road surface condition detection method according to another aspect of the present disclosure also achieves the same effects as the road surface condition detection device according to the above-described one aspect of the present disclosure.

[0014] The reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and the specific components described in the embodiments described below. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a schematic configuration of a road surface condition detection device according to a first embodiment. [Figure 2] 4 is a flowchart showing a road surface condition detection process executed by the road surface condition detection device. [Figure 3] FIG. 2 is an explanatory diagram for explaining an attribute assignment process executed by the road surface condition detection device. [Figure 4] 10A and 10B are explanatory diagrams for explaining a pseudo-depression-portion generating process executed by the road surface condition detection device; [Figure 5] 10A and 10B are explanatory diagrams for explaining a recessed portion area recognition process executed by the road surface condition detection device; [Figure 6] 10 is an explanatory diagram for explaining an ideal case in the pseudo depression generating process executed by the road surface condition detection device according to the second embodiment. FIG. [Figure 7] FIG. 7 is a view taken in the direction of an arrow VII in FIG. 6. [Figure 8] FIG. 2 is an explanatory diagram for explaining point cloud data of observation points. [Figure 9] 10 is an explanatory diagram for explaining a case where the inclination of a virtual road surface is incorrect in the pseudo-depression portion generation process executed by the road surface condition detection device according to the second embodiment. FIG. [Figure 10] 10 is a view taken in the direction of the arrow X in FIG. 9. [Figure 11] 4 is an explanatory diagram for explaining a recessed portion area recognition process executed by a road surface condition detection device of a first comparative example. FIG. [Figure 12] 12 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device of the first comparative example executes after a predetermined time has elapsed from FIG. 11. FIG. [Figure 13]13 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device of the first comparative example executes after a predetermined time has elapsed from FIG. 12. FIG. [Figure 14] 14 is a view taken in the direction of the arrow XIV in FIG. 11. [Figure 15] 13 is a view taken in the direction of the arrow XV in FIG. 12. [Figure 16] FIG. 16 is a view taken in the direction of the arrow XVI in FIG. [Figure 17] 15 is an explanatory diagram for explaining the recessed portion area recognition process executed by the road surface condition detection device of the third embodiment, and corresponds to FIG. 14. FIG. [Figure 18] 18 is an explanatory diagram for explaining the recessed portion area recognition process that the road surface condition detection device of the third embodiment executes after a predetermined time has elapsed from FIG. 17, and corresponds to FIG. 15. FIG. [Figure 19] 19 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device of the third embodiment executes after a predetermined time has elapsed from FIG. 18, and corresponds to FIG. 16. FIG. [Figure 20] 10 is an explanatory diagram for explaining a recessed portion area recognition process executed by a road surface condition detection device of a second comparative example. FIG. [Figure 21] 21 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device of the second comparative example executes after a predetermined time has elapsed from FIG. 20. FIG. [Figure 22] 22 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device of the second comparative example executes after a predetermined time has elapsed from FIG. 21. FIG. [Figure 23] FIG. 10 is an explanatory diagram for explaining a recessed portion area recognition process executed by a road surface condition detection device according to a fourth embodiment. [Figure 24] 24 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device according to the fourth embodiment executes after a predetermined time has elapsed from FIG. 23. FIG. [Figure 25] 25 is an explanatory diagram for explaining recessed portion area recognition processing that the road surface condition detection device according to the fourth embodiment executes after a predetermined time has elapsed from FIG. 24. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following embodiments, identical or equivalent parts will be denoted by the same reference numerals, and description thereof will be omitted.

[0017] (First embodiment) A first embodiment will be described with reference to Figs. 1 to 5. A road surface condition detection device 1 of the first embodiment detects the condition of a road surface 3 on which a vehicle 2 as a moving body moves (specifically, travels). Road surface condition information detected by the road surface condition detection device 1 is used for an autonomous driving ECU or a driving assistance application, etc. ECU stands for Electronic Control Unit. Based on the information, the autonomous driving ECU or driving assistance application controls the suspension device, braking device, steering device, etc. of the vehicle 2, or determines the driving path, etc.

[0018] 1, a road surface condition detection device 1 is connected to a sensor 4 mounted on a vehicle 2 via an in-vehicle LAN or a wire harness using, for example, CAN communication. CAN stands for Controller Area Network, and LAN stands for Local Area Network.

[0019] The sensor 4 is capable of observing the surroundings of the vehicle 2 using electromagnetic waves such as light or radio waves, and obtaining information on a plurality of observation points, i.e., point cloud information. Specifically, the sensor 4 is, for example, a LiDAR, a radar, a camera, or the like.

[0020] LiDAR is an abbreviation for Light Detection and Ranging, or Laser Imaging Detection and Ranging. LiDAR emits laser light around the vehicle, receives the light reflected from observation points around the vehicle, and analyzes the reflected light to obtain various information such as the distance to the observation points, relative speed, and direction.

[0021] For example, radar emits millimeter waves around the vehicle, receives the waves reflected from observation points around the vehicle, and can obtain various information such as the distance to the observation points, relative speed, and direction by analyzing the reflected waves.

[0022] The camera is a digital camera that periodically or irregularly captures images of the vehicle's surroundings. The camera can acquire point cloud information by converting each pixel in the captured camera image into Cartesian coordinates.

[0023] Hereinafter, the multiple observation points may be referred to as a "point cloud," and the information of the multiple observation points may be referred to as "point cloud information." The information of the multiple observation points acquired by the sensor 4, i.e., the point cloud information, is transmitted to the road surface condition detection device 1 via an in-vehicle LAN or the like.

[0024] The road surface condition detection device 1 is composed of an electronic control unit (i.e., ECU) that is made up of a microcomputer including a processor that performs control processing and arithmetic processing, and memories such as ROM and RAM that store programs, data, etc., and its peripheral circuits. The road surface condition detection device 1 can function as an attribute assignment unit 5, a pseudo depression point generation unit 6, a depression area recognition unit 7, etc., as the processor performs various control processing and arithmetic processing based on the programs stored in the memory. In other words, the road surface condition detection device 1 is equipped with the attribute assignment unit 5, the pseudo depression point generation unit 6, and the depression area recognition unit 7 as functional units.

[0025] The road surface condition detection process executed by the road surface condition detection device 1 will be described with reference to the flowchart of Fig. 2 and Fig. 3 to Fig. 5. This process is repeatedly executed at a predetermined control period (for example, 100 ms) when the main switch of the vehicle 2 is turned on.

[0026] First, in step S1 of FIG. 2, the attribute assigning unit 5 executes an attribute assigning process. Specifically, as shown in FIG. 3, the attribute assigning unit 5 determines whether the multiple observation points observed by the sensor 4 are the road surface 3 or depressions 8 recessed from the road surface 3. The attribute assigning unit 5 then assigns to these observation points a "road surface attribute" indicating that they are the road surface 3, or a "depression attribute" indicating that they are depressions 8. Hereinafter, an observation point among the multiple observation points that has been assigned a road surface attribute will be referred to as a "road surface observation point 9." Furthermore, an observation point among the multiple observation points that has been assigned a depression attribute will be referred to as a "depression observation point 10." Therefore, the attribute assigning unit 5 classifies the multiple observation points into road surface observation points 9 and depression observation points 10. 3, road surface observation points 9 are shown as completely black circles, and depression observation points 10 are shown as circles with a black outer edge and a white inner edge. Note that for multiple observation points, it is not necessarily necessary to make a determination regarding attributes other than the road surface 3 and depression 8 (for example, other vehicles, pedestrians, etc.), but such a determination may be made.

[0027] The following two methods are given as examples of techniques for determining and classifying the attributes of multiple observation points, but any method that achieves a similar purpose may be adopted.

[0028] As a first method, a method can be adopted in which the geometric positional relationship of the point cloud is used to classify the road surface observation points 9 and the depression observation points 10. For example, if the difference in distance between two adjacent observation points is greater than a predetermined threshold, the observation point farther from the vehicle 2 is determined to be the depression observation point 10, and the observation point closer to the vehicle 2 is determined to be the road surface observation point 9. Alternatively, if the difference in altitude between two adjacent observation points is greater than a predetermined threshold, the observation point with the lower altitude is determined to be the depression observation point 10, and the observation point with the higher altitude is determined to be the road surface observation point 9.

[0029] As a second method, a semantic segmentation technique using DNN (Deep Neural Network) can be used to classify road surface observation points 9 and depression observation points 10. Note that each observation point in the point cloud input from the sensor 4 to the road surface condition detection device 1 has location information such as two-dimensional or three-dimensional coordinates, and may also have accompanying information such as signal strength and Doppler velocity depending on the classification function. This improves the accuracy of classification of road surface observation points 9 and depression observation points 10 using semantic segmentation technique. Note that Doppler velocity can be obtained using LiDAR that uses the FMCW method (i.e., frequency modulated continuous wave method). FMCW stands for frequency modulated continuous wave.

[0030] Next, in step S2 of Fig. 2, the pseudo recess point generation unit 6 executes a road surface information generation process. Specifically, as shown in Fig. 4, the pseudo recess point generation unit 6 estimates an estimated road surface or an estimated road surface line 11 from a plurality of road surface observation points 9. In Fig. 4, the estimated road surface or the estimated road surface line 11 is indicated by a thick line.

[0031] The following three methods are exemplified as techniques for generating the road surface estimation surface or road surface estimation line 11, but any other method may be adopted as long as it has a function of achieving the same purpose.

[0032] As a first method, a method of estimation using a straight line in two dimensions (distance × height) for each horizontal direction can be adopted. For example, this method includes linear regression, in which a regression line of multiple road surface observation points 9 is used as the road surface estimation line 11.

[0033] The second method is to use a plane estimation method in a three-dimensional plane (X, Y, Z). One example of this method is RANSAC. RANSAC stands for Random Sample Consensus. The following paper describes RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography internet<URL:htpps: / / dl.acm.org / doi / 10.1145 / 358669.358692> Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography: Communications of the ACM: Vol 24, No 6

[0034] As a third method, it is possible to use a method in which the road surface estimation surface or road surface estimation line 11 is not necessarily a single straight line or plane, but is estimated by combining multiple planes or straight lines. An example of this method is the method proposed in the following document A. Document A: Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloud internet<URL:htpps: / / arxiv.org / abs / 2207.11919> [2207.11919] Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloud (arxiv.org)

[0035] Next, in step S3 of Fig. 2, the pseudo recessed portion point generation unit 6 executes a pseudo recessed portion point generation process. Specifically, as shown in Fig. 4, the pseudo recessed portion point generation unit 6 extrapolates (i.e., extends) the estimated road surface or estimated road surface line 11 to generate a virtual road surface surface or virtual road surface line 12. The virtual road surface or virtual road surface line 12 is extrapolated above the recessed portion 8. In Fig. 4, the virtual road surface surface or virtual road surface line 12 is generated by extrapolating, in a direction away from the vehicle 2, the estimated road surface surface or estimated road surface line 11 generated using a plurality of road surface observation points 9 that are closer to the vehicle 2 than the recessed portion observation point 10. Note that the virtual road surface or virtual road surface line 12 may also be generated by extrapolating, in a direction closer to the vehicle 2, the estimated road surface surface or estimated road surface line 11 (not shown) generated using a plurality of road surface observation points 9 that are farther from the vehicle 2 than the recessed portion observation point 10. The pseudo recess point generation unit 6 calculates the intersection of the straight line connecting the recess observation point 10 and the sensor 4 (hereinafter referred to as the "detection straight line 13") with the virtual road surface or virtual road line 12, and sets this intersection as a pseudo recess point 14. In Figure 4, the pseudo recess point 14 is shown as a square with a black outer edge and a white inner edge.

[0036] Note that Figure 4 is basically illustrated in a two-dimensional coordinate system of distance and height, and this technology can be used as long as there is at least two-dimensional point cloud information. However, actual on-board sensors 4 (e.g., LiDAR, Radar, camera) generally perform three-dimensional observations, and this technology can also be used with such sensors 4. For example, in the case of LiDAR that scans multiple beamlines vertically and horizontally, the process is first limited to one horizontal layer, and this processing is performed in two dimensions, distance and vertical, on that horizontal layer. Alternatively, it is possible to focus on a single beamline and perform processing to find the intersection between the virtual road surface that exists in the vicinity and the focused beamline.

[0037] Next, in step S4 of Fig. 2, recess area recognition unit 7 recognizes recess area 15 where recess 8 exists, using recess observation points 10 and pseudo recess points 14. Specifically, as shown in Fig. 5, recess observation points 10 and pseudo recess points 14 are grouped to recognize recess area 15. This makes it possible to make distance D1 between the start point of actual recess 8 and the start point of recess area 15 very small.

[0038] The road surface condition detection device 1 of the first embodiment described above has the following configuration and provides the following effects. Specifically, the road surface condition detection device 1 includes an attribute assigning unit 5, a pseudo depression point generating unit 6, and a depression area recognizing unit 7. The attribute assigning unit 5 assigns road surface attributes or depression attributes to a plurality of observation points transmitted from the sensor 4. The pseudo depression point generating unit 6 estimates a virtual road surface or a virtual road surface line 12 from the road surface observation points 9. The pseudo depression point generating unit 6 then generates, as pseudo depression points 14, the intersections of the virtual road surface or virtual road surface line 12 and a detection line 13 connecting the depression observation points 10 and the sensor 4. The depression area recognizing unit 7 uses the depression observation points 10 and the pseudo depression points 14 to recognize depression areas 15 where depressions 8 exist.

[0039] According to this, the intersection of the virtual road surface or virtual road surface line 12 and the detection line 13 is physically confirmed as having been passed by electromagnetic waves such as light and radio waves used for observation by the sensor 4 through the space where the intersection exists, and therefore it can be said that a depression 8 exists. Therefore, by making the intersection a pseudo depression point 14 and using it to recognize the depression area 15, it is possible to solve the problem of not being able to recognize the position of the start point of the depression area 15 that occurred in the technology of Patent Document 1. Therefore, this road surface condition detection device 1 can improve the accuracy of recognizing the position of the depression area 15.

[0040] (Second embodiment) The second embodiment will be described. In the second embodiment, the road surface information generation process in step S2 and the pseudo recess point generation process in step S3 are partially changed from the first embodiment, and the rest is the same as the first embodiment, so only the parts that are different from the first embodiment will be described.

[0041] First, ideal cases for the road surface information generation process in step S2 and the pseudo recess point generation process in step S3 are shown in FIGS.

[0042] 6, in an ideal case, a virtual road surface or virtual road line 12 is accurately formed on the upper surface of the recessed portion 8. Therefore, a pseudo recessed portion point 14, which is the intersection of the virtual road surface or virtual road line 12 and the detection line 13, is also accurately formed on the upper surface of the recessed portion 8.

[0043] In this case, as shown in Figure 7, the road surface observation points 9 adjacent to a given pseudo depression point 14 in the vehicle width direction (hereinafter referred to as "adjacent road surface observation points 9a, 9b") and the given pseudo depression point 14 are at similar distances from the sensor 4. In other words, the difference between the distance D2 between the given pseudo depression point 14 and the sensor 4 and the distance D3 between the adjacent road surface observation points 9a, 9b and the sensor 4 is approximately the difference in distance due to differences in position in the vehicle width direction. Note that the adjacent road surface observation points 9a, 9b can be said to be road surface observation points 9 observed with electromagnetic waves at the same angle vertically and adjacent angles horizontally with respect to the detection line 13 used to generate the given pseudo depression point 14.

[0044] As shown in FIG. 8, information on multiple observation points (i.e., point cloud information) is acquired by inputting information such as the distance of each observation point to each square of the matrix-shaped point cloud data as seen from the sensor 4. The direction in which the numbers in the vertical layer of the point cloud data are arranged corresponds to the vertical direction or the vehicle traveling direction (forward of the vehicle). In FIG. 8, the Nth vertical layer of the matrix-shaped point cloud data is indicated by hatching. Adjacent road surface observation points 9a and 9b can be considered to be road surface observation points 9 horizontally adjacent to a given pseudo-depression point 14 in the same Nth vertical layer. In FIG. 7, the adjacent road surface observation points 9a and 9b and the pseudo-depression point 14 included in the Nth vertical layer are surrounded by a dashed line 16. Furthermore, the adjacent road surface observation points 9a and 9b and the pseudo-depression point 14 included in the Mth vertical layer are surrounded by a dashed line 17.

[0045] Next, regarding the road surface information generation process in step S2 and the pseudo recess point generation process in step S3, cases in which the inclination of the estimated road surface or estimated road surface line 11 is incorrect are shown in FIGS.

[0046] In this case, as shown in Fig. 9, the slope of the estimated road surface or estimated road surface line 11 is significantly different from the shape of the actual road surface 3 due to undulations in the road surface 3 or increased detection errors at the observation points. Therefore, the virtual road surface or virtual road surface line 12 is formed at a position different from the top surface of the recessed portion 8. Therefore, the pseudo recessed portion point 14 is formed at a position where no recessed portion 8 exists. In the example shown in Fig. 9, the pseudo recessed portion point 14 is formed at a position closer to the vehicle 2 than the recessed portion 8.

[0047] In this case, as shown in Figure 10, the distances from the sensor 4 to the adjacent road surface observation points 9a, 9b and the predetermined pseudo-depression point 14 will be significantly different. In other words, the difference between the distance D4 between the predetermined pseudo-depression point 14 and the sensor 4 and the distance D5 between the adjacent road surface observation points 9a, 9b and the sensor 4 will be greater than the difference in distance due to differences in position in the vehicle width direction. In Figure 10, the adjacent road surface observation points 9a, 9b and the pseudo-depression point 14 included in the Nth vertical layer are surrounded by a dashed line 18. Furthermore, the adjacent road surface observation points 9a, 9b and the pseudo-depression point 14 included in the Mth vertical layer are surrounded by a dashed line 19.

[0048] 9 and 10, if the pseudo recess point 14 is generated in an incorrect position, the accuracy of position recognition of the start point of the recess region 15 decreases. Therefore, the pseudo recess point generation unit 6 executes a process of invalidating the pseudo recess point 14 once generated when the pseudo recess point 14 once generated satisfies the condition of the following (Equation 1).

[0049] |(average distance between the sensor 4 and two adjacent road surface observation points 9a, 9b in the same vertical layer as the given pseudo depression point 14 in the horizontal direction) - (distance between the given pseudo depression point 14 and the sensor 4) | > threshold (Equation 1) The threshold is set taking into consideration the difference in distance between a given pseudo depression point 14 and adjacent road surface observation points 9a, 9b in the vehicle width direction in an ideal case. This makes it possible to prevent a pseudo depression point 14 from being generated in an incorrect position. Note that the first term on the left side of the above equation 1 is "... the average distance between two adjacent road surface observation points 9a, 9b and the sensor 4", but is not limited to this and may also be "... the distance between one adjacent road surface observation point 9a and the sensor 4".

[0050] The road surface condition detection device 1 of the second embodiment described above has the following configuration and provides the following functions and effects. That is, in the second embodiment, if the difference between the distance between adjacent road surface observation points 9a, 9b and the sensor 4 and the distance between a predetermined pseudo recess point 14 and the sensor 4 is greater than a predetermined threshold, the pseudo recess point generation unit 6 invalidates the predetermined pseudo recess point 14 that it has once generated.

[0051] According to this, in an ideal case, the difference between the distance between adjacent road surface observation points 9a, 9b and sensor 4 and the distance between a predetermined pseudo recession point 14 and sensor 4 is considered to be approximately the same as the difference in distance caused by the difference in their horizontal positions. Therefore, if the difference between the distance between adjacent road surface observation points 9a, 9b and sensor 4 and the distance between a predetermined pseudo recession point 14 and sensor 4 is greater than a predetermined threshold, the predetermined pseudo recession point 14 is likely to be in an incorrect position. Therefore, by invalidating the predetermined pseudo recession point 14, the pseudo recession point generation unit 6 can prevent the pseudo recession point 14 from being generated in an incorrect position, and prevent a decrease in the accuracy of position recognition of the recession area 15.

[0052] (Third embodiment) The third embodiment will be described. The third embodiment is different from the first embodiment in that the road surface condition detection process is partially changed, and the remaining steps are the same as the first embodiment, so only the differences from the first embodiment will be described.

[0053] Before describing the third embodiment, a road surface condition detection process executed by a road surface condition detection device 1 of a first comparative example will be described with reference to Figures 11 to 16. Note that the first comparative example is similar to the first embodiment and is not a conventional technique.

[0054] Figures 11 to 13 show the recessed portion area recognition processing of step S4 in each road surface condition detection processing that is executed at predetermined control intervals (for example, 100 ms) while the vehicle 2 is traveling in the first comparative example. Time passes in the order of Figures 11, 12, and 13, and as the vehicle 2 moves forward, it approaches the recessed portion 8. Specifically, Figure 11 shows the recessed portion area recognition processing of step S4 in the first road surface condition detection processing, Figure 12 shows the recessed portion area recognition processing of step S4 in the second road surface condition detection processing, and Figure 13 shows the recessed portion area recognition processing of step S4 in the third road surface condition detection processing.

[0055] 14 to 16 are plan views corresponding to FIGS. 11 to 13, respectively. As shown in FIGS. 14 to 16, in the first comparative example, as in the first embodiment, recessed portion observation points 10 and pseudo recessed portion points 14 generated for each control cycle in steps S1 to S3 of the road surface condition detection process executed for each control cycle are grouped in step S4 to recognize recessed portion regions 15. This can solve the problem of inability to recognize the position of the starting point of recessed portion region 15, and can improve the accuracy of position recognition of recessed portion region 15, compared to the technology that recognizes recessed portion region 15 using only recessed portion observation points 10, as in Patent Document 1 mentioned above.

[0056] In contrast, as shown in FIGS. 17 to 19, the road surface condition detection device 1 of the third embodiment uses superimposed data of recession observation points 10 and pseudo recession points 14, which are generated for each control cycle in steps S1 to S3 of the road surface condition detection process, to group the superimposed data and recognize a recession region 15 in step S4. FIGS. 17 to 19 are plan views corresponding to FIGS. 11 to 13, respectively. Therefore, time passes in the order of FIGS. 17, 18, and 19, and as the vehicle 2 moves forward, it approaches a recession region 8. The recession region recognition unit 7 calculates the amount of movement of the vehicle 2 during the control cycle and superimposes the recession observation points 10 and pseudo recession points 14, which are generated for each predetermined control cycle, to generate superimposed data of the recession observation points 10 and pseudo recession points 14. This makes it possible to increase the number of recess observation points 10 and pseudo recess points 14 used in recognizing the position of recess region 15, compared to the method of recognizing recess region 15 using recess observation points 10 and pseudo recess points 14 generated for each control cycle as in the first comparative example. This improves the accuracy in recognizing the width and depth of recess region 15, and can more effectively suppress errors in the starting points of recess region 15.

[0057] The road surface condition detection device 1 of the third embodiment described above has the following configuration and provides the following functions and effects. That is, in the third embodiment, the recess area recognition unit 7 recognizes the recess area 15 using superimposed data of the recess observation points 10 and pseudo recess points 14, which are generated for each predetermined control cycle and are superimposed by calculating the amount of movement of the vehicle 2 during the control cycle.

[0058] According to this, the number of recession observation points 10 and pseudo recession points 14 used by recession area recognition unit 7 when recognizing recession areas 15 increases with each control cycle. As a result, recession area recognition unit 7 can recognize recession areas 15 using information on recession observation points 10 and pseudo recession points 14 recognized in past control cycles, in addition to information on recession observation points 10 and pseudo recession points 14 recognized in the current control cycle. Therefore, this road surface condition detection device 1 can further improve the accuracy of recognizing the position of recession areas 15, including the start point, far point, and width of recession areas 15.

[0059] (Fourth embodiment) The fourth embodiment will be described. The fourth embodiment is similar to the first embodiment in that the road surface condition detection process is partially changed, but the remaining steps are the same as the first embodiment, and therefore only the differences from the first embodiment will be described.

[0060] Before describing the fourth embodiment, the road surface condition detection process executed by the road surface condition detection device 1 of the second comparative example will be described with reference to FIGS. 20 to 22. FIG.

[0061] 20 to 22 show the recessed portion area recognition process in step S4 of the road surface condition detection process that is executed every predetermined control period (for example, 100 ms) while the vehicle 2 is traveling in the second comparative example. Time passes in the order of Fig. 20, Fig. 21, and Fig. 22, and as the vehicle 2 moves forward, it approaches the recessed portion 8. Specifically, Fig. 20 shows the first road surface condition detection process, Fig. 21 shows the second road surface condition detection process, and Fig. 22 shows the third road surface condition detection process.

[0062] In the second comparative example, a recessed area 15 is recognized using only recessed area observation points 10, as in Patent Document 1. However, the second comparative example differs from Patent Document 1 in that a recessed area 15 is recognized using an occupancy grid map formed on the predicted path of the vehicle 2, and is not a conventional technique. Hereinafter, the occupancy grid map will be referred to as OGM. OGM stands for Occupancy Grid Map. Figures 20 to 22 show the prior probability, observation, and posterior probability of the OGM corresponding to the road surface position. Note that each figure shows the OGM as viewed from the vehicle width direction, but in reality, the OGM extends horizontally. Hereinafter, the occupancy probability of a target on the road surface shown in each cell of the OGM will be simply referred to as the "occupancy probability." Furthermore, the occupancy probability of a recessed area 8 shown in each cell of the OGM will be referred to as the "recess probability."

[0063] FIG. 20 shows the first round of road surface condition detection processing. In the first round of road surface condition detection processing, the prior probabilities for all of cells 1 to 13 are set to "medium occupancy probability." In the following explanation, the information observed in that round of road surface condition detection processing will simply be referred to as "observation." In the observations of the first round of road surface condition detection processing, cells 5 to 7 are set to "lower than medium occupancy probability," cell 11 is set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probability is the sum of the prior probability and observation in that round of road surface condition detection processing. Therefore, the posterior probability of the first round of road surface condition detection processing is the same as the above observation.

[0064] FIG. 21 shows the second road surface condition detection process. The prior probabilities are the same as the posterior probabilities of the previous road surface condition detection process. However, the cell numbers are corrected according to the movement amount of vehicle 2 in the control cycle. Therefore, the prior probabilities of the second road surface condition detection process are obtained by shifting the cell numbers toward vehicle 2 compared to the posterior probabilities of the first road surface condition detection process. In the observations of the second road surface condition detection process, cells 5 and 6 are set to "lower than medium occupancy probability," cell 10 is set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probabilities are as follows: cell 4 is set to "lower than medium occupancy probability," cells 5 and 6 are set to "low occupancy probability," cell 10 is set to "higher than medium depression probability," and the other cells are set to "medium occupancy probability."

[0065] FIG. 22 shows the third road surface condition detection process. The prior probabilities for the third road surface condition detection process are obtained by shifting the cell numbers toward vehicle 2 compared to the posterior probabilities for the second road surface condition detection process. The observations for the third road surface condition detection process show that cell 5 is set to "lower than medium occupancy probability," cell 9 is set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probabilities for cell 3 are set to "lower than medium occupancy probability," cell 4 is set to "low occupancy probability," cell 5 is set to "lower than medium occupancy probability," cell 9 is set to "high depression probability," and the other cells are set to "medium occupancy probability."

[0066] In the second comparative example described above, as shown in the posterior probability for each time, the recess probability is high only at the far point of recess region 15, and the start point of recess region 15 cannot be recognized.

[0067] In contrast, as shown in Figures 23 to 25, the road surface condition detection device 1 of the fourth embodiment uses an OGM formed on the predicted path of a vehicle 2, and recognizes a recessed portion area 15 using recessed portion observation points 10 and pseudo recessed portion points 14. Figures 23 to 25 also show the prior probability, observation, and posterior probability of the OGM corresponding to the road surface position. Note that although each figure shows the OGM as viewed from the vehicle width direction, in reality the OGM extends horizontally.

[0068] Figure 23 shows the first road surface condition detection process. In the first road surface condition detection process, the prior probabilities for all cells 1 to 13 are set to "medium occupancy probability." The observations of the first road surface condition detection process are that cells 5 to 7 are set to "lower than medium occupancy probability," cells 9 to 11 are set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probabilities of the first road surface condition detection process are the same as the above observations.

[0069] Figure 24 shows the second road surface condition detection process. The prior probabilities for the second road surface condition detection process are obtained by shifting the cell numbers toward vehicle 2 compared to the posterior probabilities for the first road surface condition detection process. The observations for the second road surface condition detection process show that cells 5 and 6 are set to "lower than medium occupancy probability," cells 7 to 10 are set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probabilities are that cell 4 is set to "lower than medium occupancy probability," cells 5 and 6 are set to "low occupancy probability," cell 7 is set to "medium depression probability," cells 8 to 10 are set to "higher than medium depression probability," and the other cells are set to "medium occupancy probability."

[0070] FIG. 25 shows the third road surface condition detection process. The prior probabilities for the third road surface condition detection process are obtained by shifting the cell numbers toward the vehicle 2 compared to the posterior probabilities for the second road surface condition detection process. In the observations for the third road surface condition detection process, cell 5 is set to "lower than medium occupancy probability," cells 6 to 9 are set to "medium depression probability," and the other cells are set to "medium occupancy probability." The posterior probabilities for cell 3 are set to "lower than medium occupancy probability," cell 4 is set to "lower than medium occupancy probability," cell 5 is set to "lower than medium occupancy probability," cell 6 is set to "higher than medium depression probability," cells 7 to 9 are set to "high depression probability," and the other cells are set to "medium occupancy probability." In this way, in the road surface condition detection device 1 of the fourth embodiment, the depression probability increases from the start point of the depression region 15 to the distant point.

[0071] The road surface condition detection device 1 of the fourth embodiment described above has the following configuration and provides the following functions and effects. That is, in the fourth embodiment, the recessed area recognition unit 7 updates the probability of the recessed area 15 input into each cell of the occupancy grid map formed on the predicted path of the vehicle 2 based on the recessed area 15 recognized at each predetermined control cycle, and recognizes the recessed area 15.

[0072] According to this, since the gridded map is a framework that inherits past information within the framework of a Bayesian network, the recessed area recognition unit 7 can further improve the accuracy of position recognition of the recessed area 15, including the starting point of the recessed area 15, by using past information as in the third embodiment.

[0073] (Other embodiments) (1) In each of the above embodiments, the moving body has been described as a vehicle 2, but this is not limiting. The moving body may be anything that moves on a road surface 3, such as a train, a probe, or a drone.

[0074] (2) In the above embodiments, the road surface condition detection device 1 and the autonomous driving ECU or driving assistance application are described as separate components, but this is not limited thereto. For example, the road surface condition detection device 1 and the autonomous driving ECU or driving assistance application may be configured as an integrated unit.

[0075] The present disclosure is not limited to the above-described embodiments and can be modified as appropriate within the scope of the claims. Furthermore, the above-described embodiments and portions thereof are not unrelated to each other and can be combined as appropriate unless the combination is clearly impossible. It goes without saying that, in each of the above embodiments, the elements constituting the embodiments are not necessarily essential unless specifically stated as essential or clearly considered essential in principle. Furthermore, in each of the above embodiments, when numerical values ​​such as the number, values, amounts, and ranges of components of the embodiments are mentioned, they are not limited to the specific numbers unless specifically stated as essential or clearly limited to a specific number in principle. Furthermore, in each of the above embodiments, when the shape, positional relationship, etc. of components are mentioned, they are not limited to the shape, positional relationship, etc., unless specifically stated or limited to a specific shape, positional relationship, etc. in principle.

[0076] The control unit and the method described herein may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the control unit and the method described herein may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method described herein may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to perform one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory tangible storage medium. The memory is a non-transitory tangible storage medium. [Explanation of symbols]

[0077] 1; Road surface condition detection device, 2; Vehicle (moving object), 3; Road surface, 4; Sensor, 5; Attribute assignment unit, 6; Pseudo depression point generation unit, 7; Depression area recognition unit, 8; Depression, 9; Road surface observation point, 10; Depression observation point, 12; Virtual road surface or virtual road surface line, 13; Detected line, 14; Pseudo depression point, 15; Depression area

Claims

1. A road surface condition detection device for detecting the condition of a road surface on which a moving object (2) is moving using information from a plurality of observation points obtained from a sensor (4) that observes the periphery of the moving object (2) using electromagnetic waves, comprising: an attribute assigning unit (5) that determines whether the observation point is a road surface (3) or a recess (8) recessed from the road surface, and assigns to the observation point a road surface attribute indicating that it is the road surface or a recess attribute indicating that it is the recess; a pseudo recess point generation unit (6) that generates a pseudo recess point (14) as a point where a virtual road surface or virtual road surface line (12) estimated from a road surface observation point (9) among the observation points to which the road surface attribute has been assigned intersects with a detection line (13) that connects a recess observation point (10) among the observation points to which the recess attribute has been assigned and the sensor; A road surface condition detection device comprising a recess area recognition unit (7) that recognizes a recess area (15) where the recess exists using the recess observation points and the pseudo recess points.

2. 2. The road surface condition detection device according to claim 1, wherein the pseudo recess point generation unit invalidates the predetermined pseudo recess point that has been generated if a difference between a distance (D3, D5) between the sensor and the road surface observation point observed with the electromagnetic waves at the same angle vertically as seen from the sensor and an adjacent angle horizontally with respect to the detection line used when generating the predetermined pseudo recess point, and a distance (D2, D4) between the predetermined pseudo recess point and the sensor is greater than a predetermined threshold.

3. 3. The road surface condition detection device according to claim 1, wherein the recessed portion area recognition unit recognizes the recessed portion area using superimposed data of the recessed portion observation points and the pseudo recessed portion points, which are generated for each predetermined control cycle and are obtained by calculating a movement amount of the moving body that moves during the control cycle and superimposing the recessed portion observation points and the pseudo recessed portion points.

4. 3. The road surface condition detection device according to claim 1, wherein the recessed area recognition unit inputs a probability of the recessed area to each cell of an occupancy grid map formed on a predicted path of the moving body based on the recessed area recognized at each predetermined control cycle, and further updates the probability of the recessed area input to each cell based on the recessed area recognized at each predetermined control cycle, and recognizes the recessed area.

5. A road surface condition detection method for detecting the condition of a road surface on which a moving object (2) is moving using information from a plurality of observation points obtained from a sensor (4) that observes the periphery of the moving object (2) using electromagnetic waves, comprising: determining whether the observation point is a road surface (3) or a recess (8) recessed from the road surface, and assigning a road surface attribute indicating that the observation point is the road surface or a recess attribute indicating that the observation point is the recess (S1); generating a pseudo-depression point (14) at a point where a virtual road surface or virtual road surface line (12) estimated from a road surface observation point (10) assigned with the road surface attribute among the observation points intersects with a detection line (13) connecting a depression observation point (10) assigned with the depression attribute among the observation points and the sensor; and A road surface condition detection method including recognizing (S4) a recessed portion area (15) in which the recessed portion exists using the recessed portion observation points and the pseudo recessed portion points.

Images