Road surface damage detection device, road surface damage detection method, and storage medium

Abstract

The present application relates to a kind of road surface damage detection device, road surface damage detection method and storage medium.Computing server will be detected to indicate the average of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum value of the maximum

Description

Technical Field

[0001] This invention relates to a road damage detection device, a road damage detection method, and a storage medium for detecting road damage based on time-series data related to the behavior of multiple vehicles. Background Technology

[0002] Road surfaces deteriorate due to vehicle traffic, potentially resulting in ruts caused by tires rutting the road surface in the direction of travel, or potholes caused by partial peeling of the pavement. In particular, potholes can not only cause tire blowouts but also lead to accidents, thus requiring early detection and repair.

[0003] However, the detection of road surface damage such as potholes mainly relies on personnel to inspect the road, which is hardly efficient, and it is not easy to quickly detect road surface damage.

[0004] Japanese Patent Application Publication No. 2021-086476 discloses an invention that detects road surface damage based on changes in the wheel speeds of multiple vehicles, including a road surface damage detection device, a road surface damage detection method, and a procedure. Summary of the Invention

[0005] However, the invention described in Japanese Patent Application Publication No. 2021-086476 primarily detects road surface damage based on whether the maximum variation in wheel speed exceeds a threshold. The maximum variation in wheel speed can also be detected by temporary debris such as gravel, maintenance openings, side ditch covers, railway crossings, and vehicle avoidance maneuvers, which are distinct from road surface damage. Therefore, it may be difficult to distinguish between situations where road surface damage exists and situations where it does not.

[0006] The present invention takes into account the above facts and aims to provide a pavement damage detection device, a pavement damage detection method, and a storage medium that can appropriately detect pavement damage.

[0007] To achieve the above objectives, the road damage detection device described in Scheme 1 includes:

[0008] The physical quantity detection department detects physical quantities that represent the individual behaviors of multiple vehicles.

[0009] The calculation and statistics unit performs statistics on the average value (interval average variation) and the maximum value (interval maximum variation) of the physical quantity detected by the physical quantity detection unit per unit time over a first predetermined period, which is longer than the first predetermined period, respectively, over a second predetermined period.

[0010] The filtering unit removes noise components from the statistical results of the computational statistics unit; and

[0011] The road surface damage detection unit detects road surface damage locations based on the results output by the filtering unit.

[0012] According to the pavement damage detection device described in Scheme 1, by filtering the analysis data, pavement damage can be detected based on data that suppresses changes in significant physical quantities that may be detected as noise components, such as temporary falling objects like gravel, maintenance holes, side ditch covers, railway crossings, and vehicle avoidance behavior.

[0013] The road damage detection device described in Scheme 2 is based on the road damage detection device described in Scheme 1.

[0014] The filtering unit uses filters to remove noise components from the statistical results of the computational statistics unit. The filters include moving average, Gaussian filter, and deep learning-based filter.

[0015] According to the pavement damage detection device described in Scheme 2, by using filter processing to smooth the data before processing, false detection of pavement damage is suppressed.

[0016] In the pavement damage detection device described in Scheme 3, if the average variation of the interval included in the result output by the filtering unit is above a predetermined first threshold and the maximum variation of the interval included in the result output by the filtering unit is below a predetermined second threshold that is greater than the first threshold, the pavement damage detection unit determines that there is a possibility of pavement damage.

[0017] According to the road damage detection device described in Scheme 3, the possibility of road damage can be determined by comparing the interval average variation of a physical quantity representing the behavior of a vehicle with a predetermined threshold.

[0018] In the pavement damage detection device described in Scheme 4, if the pavement damage detection unit determines that abrupt pavement damage has occurred in either the case where the change in the average variation of the interval in the first specified period is above a specified threshold, or the case where the difference between the maximum variation of the interval and the average variation of the interval in the second specified period is above a specified third threshold, the pavement damage detection unit determines that abrupt pavement damage has occurred in either the case where the change in the average variation of the interval in the first specified period is above a specified threshold, or the case where the difference between the maximum variation of the interval and the average variation of the interval in the second specified period is above a specified third threshold.

[0019] According to the road damage detection device described in Scheme 4, it is able to detect the occurrence of abrupt road damage based on changes in physical quantities that represent vehicle behavior.

[0020] In the pavement damage detection device of Scheme 5, if the change of the time series of the interval average variation in the result output by the filter unit increases in a downward convex curve, it is determined that the pavement has deteriorated over time; if the change of the time series of the interval average variation is flat, it is determined that the state of the pavement has not changed; and if the change of the time series of the interval average variation changes in a step-like manner, it is determined that one of the following treatments has been carried out: pavement cutting, repair, and repaving.

[0021] According to the road damage detection device described in Scheme 5, the presence or absence of road damage or road construction can be determined by the time series changes of physical quantities representing vehicle behavior.

[0022] In the road damage detection device of Scheme 6, the physical quantity detection unit is a wheel speed sensor that detects the wheel speed of the vehicle as the physical quantity.

[0023] The road damage detection device according to Scheme 6 can detect road damage based on the wheel speed detected by the wheel speed sensor commonly found in vehicles.

[0024] In the pavement damage detection device of Scheme 7,

[0025] The wheel speed sensors detect the speed of each of the four wheels of the vehicle.

[0026] If the difference between the average value of the change in wheel speed per unit time of the left and right wheels with respect to the plurality of vehicles and the average value of the change in wheel speed per unit time of the left and right wheels with respect to the plurality of vehicles during the first specified period is a predetermined fourth threshold or higher, the road surface damage detection unit determines that road surface damage exists.

[0027] According to the road damage detection device described in Scheme 7, road damage can be detected based on the differences in wheel speeds detected independently on the four wheels and at the left and right wheels.

[0028] In the road damage detection device of Scheme 8, the physical quantity detection unit is an inertial measurement device that detects the angular velocity of the vehicle's attitude angle and the vehicle's acceleration as the physical quantities.

[0029] According to the road damage detection device described in Scheme 8, road damage can be detected based on the time series data of the angular velocity of the vehicle's attitude angle and the vehicle's acceleration.

[0030] In the road damage detection device of Scheme 9, the physical quantity detection unit is a steering angle sensor that detects the vehicle's steering angle as the physical quantity.

[0031] The road damage detection device according to Scheme 9 can detect road damage based on time-series data of the vehicle's steering angle.

[0032] In the road damage detection device of Scheme 10, the physical quantity detection unit is a throttle sensor that detects the throttle opening, which represents the deceleration of the vehicle, as the physical quantity.

[0033] The road damage detection device according to Scheme 10 can detect road damage based on the throttle opening, which represents the deceleration of the vehicle.

[0034] In the road damage detection device of Scheme 11, the physical quantity detection unit is a brake pedal sensor that detects the braking force of the brake pedal, which represents the deceleration of the vehicle, as the physical quantity.

[0035] According to the road damage detection device of Scheme 11, road damage can be detected based on the force applied to the brake pedal, which indicates the deceleration of the vehicle.

[0036] To achieve the above objectives, the pavement damage detection method in Scheme 12 includes:

[0037] The process of separately detecting physical quantities representing the individual behaviors of multiple vehicles;

[0038] The process of statistically analyzing the average value of the variation of the detected physical quantity per unit time within a first specified period, i.e., the interval average variation, and the maximum value within the first specified period, i.e., the interval maximum variation, respectively, for a second specified period that is longer than the first specified period.

[0039] The process of removing noise components from the statistical results; and

[0040] The process of detecting road surface damage based on the result after removing the noise components.

[0041] According to the pavement damage detection method described in Scheme 12, by filtering the analysis data, pavement damage can be detected based on data that suppresses changes in significant physical quantities that may be detected as noise components, such as temporary falling objects like gravel, maintenance holes, side ditch covers, railway crossings, and vehicle avoidance behavior.

[0042] The storage medium of Scheme 13, used to achieve the above objectives, stores a road surface damage detection program that enables the computer to function as the following structure.

[0043] The following structures include:

[0044] The calculation and statistics department performs statistics on the average value (interval average change) and the maximum value (interval maximum change) of the physical quantity representing the behavior of multiple vehicles per unit time over a first specified period, respectively, over a second specified period that is longer than the first specified period.

[0045] The filtering unit removes noise components from the statistical results of the computational statistics unit; and

[0046] The road surface damage detection unit detects road surface damage locations based on the results output by the filtering unit.

[0047] According to the pavement damage detection procedure described in Scheme 13, by filtering the analysis data, pavement damage can be detected based on data that suppresses changes in significant physical quantities that may be detected as noise components, such as temporary falling objects like gravel, maintenance holes, side ditch covers, railway crossings, and vehicle avoidance behavior.

[0048] As described above, the pavement damage detection device, pavement damage detection method, and storage medium according to the present invention can appropriately detect pavement damage. Attached Figure Description

[0049] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which the same reference numerals denote the same elements, wherein:

[0050] Figure 1 This is a schematic diagram showing the structure of the road damage detection device according to this embodiment;

[0051] Figure 2 It is a block diagram showing the structure of the vehicle;

[0052] Figure 3 This is a block diagram illustrating an example of the specific structure of the computing server according to this embodiment;

[0053] Figure 4 This is a functional block diagram of the CPU of the computing server involved in this embodiment;

[0054] Figure 5 This is a flowchart illustrating an example of the processing of a computing server according to this embodiment;

[0055] Figure 6 This is a schematic diagram illustrating an example of time series variation during the second specified period of the interval average variation;

[0056] Figure 7 This is a schematic diagram illustrating an example of time series variation during the second specified period of the interval with the maximum variation;

[0057] Figure 8 This is an illustration of how road surface damage is detected based on the different wheel speeds of the left and right wheels of a vehicle.

[0058] Figure 9 This is a sequence diagram illustrating an example of the processing of the road surface damage detection device according to this embodiment. Detailed Implementation

[0059] The following uses Figure 1 This embodiment describes the road surface damage detection device 100. Figure 1 The road damage detection device 100 shown includes a communication device 110, a data storage device 120, a computing server 10, and a terminal 130. The communication device 110 acquires data from multiple vehicles 200, which are so-called connected cars equipped with a continuous network connection. The data storage device 120 stores the data received by the communication device 110. The computing server 10 detects road damage based on the data stored in the data storage device 120. The terminal 130 is a terminal capable of viewing information about the road damage detected by the computing server 10.

[0060] As described below, data storage device 120 is a data server equipped with a database. Computing server 10 is a computer capable of performing high-speed, advanced computational processing. Data storage device 120 and computing server 10 can each be a standalone server, or they can be distributed across a cloud. Data storage device 120 and computing server 10 can also be the same server. Terminal 130 is not a necessary component. For example, if computing server 10 has input devices such as a keyboard and mouse and output devices such as a monitor, then terminal 130 can be omitted.

[0061] Figure 2This is a block diagram showing the structure of vehicle 200. Vehicle 200 consists of a storage device 18, a Global Navigation Satellite System (GNSS) device 20, an input device 12, a computing device 14, and an output device 16. The storage device 18 stores the data required for the computing device 14 and the computing results of the computing device 14. The Global Navigation Satellite System (GNSS) device 20 uses signals transmitted from artificial satellites (hereinafter referred to as "satellites") to perform position estimation. The input device 12 is input with the wheel speed detected by the vehicle speed sensor 24, the angular velocity and acceleration of the vehicle 200's attitude angle detected by the IMU (Inertial Measurement Unit) 26, the steering angle of the vehicle 200 detected by the steering angle sensor 28, the throttle opening of the vehicle 200 detected by the throttle sensor, the brake pedal force of the vehicle 200 detected by the brake pedal sensor 32, and information obtained by the V2X communication unit 34 through wireless communication. The arithmetic unit 14 calculates information representing the behavior of the vehicle 200, such as wheel speed, based on input data from the input device 12 and data stored in the storage device 18, and associates this information with the position information of the vehicle 200 detected by the GNSS device 20, etc., and outputs it. The output device 16 outputs the calculation result of the arithmetic unit 14 to the V2X communication unit 34. The vehicle speed sensor 24 is configured to detect the speeds of the four wheels of the vehicle 200 respectively. In addition, the vehicle 200 may also have a master cylinder sensor that detects the pressure in the master cylinder of the brake, in addition to the brake pedal sensor 32.

[0062] As mentioned earlier, vehicle 200 is a so-called connected car. However, vehicle 200 may not be a connected car, or it may be a vehicle equipped with aftermarket communication devices such as TransLog (data transmission dashcam) that analyze / utilize driving data sent from onboard devices installed in vehicle 200, as well as various sensors that acquire driving data.

[0063] Figure 3 This is a block diagram illustrating an example of the specific structure of a computing server 10 according to an embodiment of the present invention. The computing server 10 is configured to include a computer 40. The computer 40 includes a central processing unit (CPU) 42, a read-only memory (ROM) 44, a random access memory (RAM) 46, and an input / output port 48. As an example, the computer 40 is preferably a model capable of performing advanced computational processing at high speed.

[0064] In computer 40, CPU 42, ROM 44, RAM 46, and input / output port 48 are interconnected via various buses such as address bus, data bus, and control bus. On input / output port 48, a monitor 50, mouse 52, keyboard 54, hard disk (HDD) 56, and a disk drive 60 for reading information from various disks (e.g., CD-ROM, DVD, etc.) 58 are respectively connected as various input / output devices.

[0065] Additionally, a network 62 is connected to the input / output port 48. Information can be exchanged with various devices connected to the network 62. In this embodiment, a data storage device 120 is connected to the network 62; the data storage device 120 is a data server connected to a database (DB) 122. Information can be exchanged with respect to the DB 122.

[0066] DB 122 stores time-series data of multiple vehicles 200 obtained via communication device 110. Data can be stored in DB 122 not only via communication device 110, but also via computer 40 or other devices connected to network 62.

[0067] In this embodiment, it is described that time-series data of multiple vehicles 200 are stored in DB 122 connected to data storage device 120. However, the information in DB 122 may also be stored in external storage devices such as HDD 56 built into computer 40 or hard disk of peripheral device.

[0068] The computer 40 has a program for detecting road surface damage installed on its HDD 56. In this embodiment, the CPU 42 executes the program to detect road surface damage based on data obtained from a data storage device.

[0069] There are several methods for installing the road damage detection program of this embodiment onto the computer 40. For example, the program can be stored together with a setup program on a CD-ROM, DVD, or the like. Then, the disk can be mounted onto the disk drive 60. The program can be installed onto the HDD 46 by executing the setup program on the CPU 42. Alternatively, the program can be installed onto the HDD 46 by communicating with other information processing devices connected to the computer 40 via a public telephone line or network 62.

[0070] Figure 4A functional block diagram of the CPU 42 of the computing server 10 is shown. The various functions implemented by the CPU 42 of the computing server 10 in executing the program related to pavement damage detection are explained. The program related to pavement damage detection includes preprocessing, statistical, filtering, and decision functions. In the preprocessing function, analysis data is prepared. In the statistical function, the variation, average, and maximum values ​​of the obtained analysis data are calculated. In the filtering function, filters are applied to the data processed by the statistical function to remove noise, etc. In the decision function, pavement damage is detected. The CPU 42 executes the machine learning program, which has these functions, such as... Figure 4 As shown, the CPU 42 functions as a preprocessing unit 72, a statistics unit 74, a filtering unit 76, and a determination unit 78.

[0071] Figure 5 This is a flowchart illustrating an example of the processing of the computing server 10 that constitutes the road surface damage detection device 100 according to this embodiment. In step S100, analysis data is obtained from the data storage device 120.

[0072] In step S102, analysis data is prepared in the preprocessing unit 72. Specifically, the preparation of analysis data in step S102 involves the following process: obtaining information such as the wheel speeds of the four wheels of a vehicle 200, which are associated with the position information of the vehicle 200, from time-series data collected from a large number of unspecified vehicles 200, divided into a first predetermined period (e.g., 1 day), and then obtaining it for a second predetermined period longer than the first predetermined period (e.g., 30 days). Hereinafter, in this embodiment, the wheel speeds of the four wheels will be described as representative of the analysis data used for road surface damage detection. However, the analysis data used for road surface damage detection is not limited to this. For example, the analysis data used for road surface damage detection may also be the angular velocity and acceleration of the vehicle 200's attitude angle detected by the IMU 26. Furthermore, the analysis data used for road surface damage detection may also be information related to the behavior of the vehicle 200 detected by the steering angle sensor 28, the throttle sensor 30, and the brake pedal sensor 32. Additionally, the first predetermined period may not be 1 day but rather 2 to 7 days. The second stipulated period can also be 31 to 180 days instead of 30 days.

[0073] In step S104, the statistics unit 74 calculates the variation in wheel speed of the four wheels of the vehicle 200 for each wheel during each first specified period or for each trip of the vehicle 200. Wheel speeds at low speeds are difficult to be effective data for detecting road surface damage. Therefore, in this embodiment, information on wheel speeds above a specified speed is provided as analytical data for detecting road surface damage. The specified speed is, for example, 15 km / h. In the case of detecting road surface damage on highways or other roads with high speed limits, the specified speed can be even higher.

[0074] In this embodiment, road surface damage is detected based on wheel speed variation. As shown in Equation (1) below, wheel speed variation is the absolute value (ABS) of the wheel speed change Δ (wheel speed) per unit time Δt after processing with a high-pass filter (HPF) to remove noise.

[0075] Wheel speed variation = ABS(HPF(Δ(wheel speed) / vehicle t))…(1)

[0076] Furthermore, in step S104, the maximum wheel speed variation is defined as the time series data obtained by selecting the maximum value (among the four wheels) of multiple vehicles 200 at each time point in the wheel speed variation of the four wheels. The maximum wheel speed variation is then calculated.

[0077] In step S106, the statistics unit 74 establishes a correspondence between wheel speed changes and location. As described above, in this embodiment, the wheel speed information of vehicle 200 is associated with the location information of vehicle 200 detected by the GNSS device 20 or similar device equipped on vehicle 200. In step S106, road section information for establishing the correspondence between wheel speed changes and location is obtained. In step S106, the road section information is obtained mainly through the following steps.

[0078] (1) Divide the target area (e.g., the entire Toyota City area) into evaluation intervals (e.g., 10m × 10m).

[0079] (2) Obtain road link information (coordinates of the start and end points of the road) for the target area (e.g., the entire area of ​​Toyota City).

[0080] GNSS-based positioning often contains errors. Therefore, in step S108 described later, the positioning errors of GNSS are corrected by referring to the road link information in (2). The road link information can be pre-stored in the data storage device 120. Alternatively, the road link information can be obtained from an external server via a communication device 110 or the like. In addition, to reduce the computational load, the target area is divided into an evaluation interval of approximately 10m × 10m in (1). If the computing power of the computing server 10 is high, the evaluation interval can be expanded beyond 10m × 10m.

[0081] In step S108, the statistics unit 74 calculates the road surface condition index. Specifically, it uses the latitude and longitude coordinates included in the road link information obtained in step S106 to establish a correlation between wheel speed variation and road intervals. Furthermore, a road surface condition index is calculated for each evaluation interval. In this embodiment, the road surface condition index is the average of the maximum wheel speed variations among the multiple vehicles 200 (i.e., the interval average variation) and the maximum of the maximum wheel speed variations among the multiple vehicles 200 (i.e., the interval maximum variation). Specifically, the interval average variation is the quotient obtained by dividing the sum of the maximum wheel speed variations of each of the multiple vehicles 200 by the number of the multiple vehicles 200. The interval maximum variation is the maximum of the maximum wheel speed variations of all vehicles 200.

[0082] In step S108, as a road surface condition indicator, the average value of the wheel speed variation of each of the four wheels of each of the multiple vehicles 200 and the maximum value of the wheel speed variation of each of the four wheels of each of the multiple vehicles 200 are calculated with respect to the multiple vehicles 200.

[0083] In step S108, the calculation of the road surface condition index as described above is repeated "second specified period ÷ first specified period (e.g., 30)" times.

[0084] In step S110, the statistics unit 74 generates historical change data. Specifically, the road surface condition index calculated in step S108 for, for example, 30 days is saved as historical change data to a storage device such as HDD 56 or data storage device 120.

[0085] In step S112, the filtering unit 76 performs filter processing to remove or suppress noise components in the historical variation data of the time series data, making it a physical quantity corresponding to the true road surface condition. The filters applied in step S112 include, for example, moving averages, Gaussian filters, and deep learning methods such as Long Short-Term Memory (LSTM). Moving averages include simple moving averages, weighted moving averages, and exponential moving averages. Any moving average, whether simple, weighted, or exponential, is acceptable as long as it removes noise components from the time series data and smooths it. Furthermore, in simple, weighted, and exponential moving averages, the average value is calculated for the most recent predetermined number (n) of data. The predetermined number n is set in a way that makes the format of the time series data after moving average processing suitable for detecting road surface damage. Figure 6 and Figure 7 As shown, a suitable state for detecting pavement loss is when the time series data is smoothed and the degree of change in each first specified period can be identified.

[0086] In step S112, in addition to the moving average mentioned above, Gaussian filters or LSTM can also be used. The results of applying each filter to the time series data can be used to determine if they are as described above. Figure 6 and Figure 7 As shown, the time series data is smoothed and the degree of change in each first specified period can be identified to determine which of the following should be used: moving average, Gaussian filter, or LSTM.

[0087] Additionally, the filters mentioned above can be selected based on road specifications (different road standards such as whether it is a main road or a residential road, or the unevenness of the road surface, etc.) and road usage environment (weekly changes such as increased or decreased traffic volume on weekends, or monthly changes such as increased or decreased traffic volume at the end of the month, etc.). Moreover, the parameters of the selected filter can be set according to road specifications or usage environment (a specified number n for moving averages, and a specified number n or weight for weighted moving averages).

[0088] Figure 6 This is a schematic diagram illustrating an example of time series variation during the second specified period of interval average variation. Figure 6 The dashed line shown represents the original data 202 before filter processing of the interval average variation during the second specified period. Figure 6 The solid line shown represents the filtered data 204, representing the interval average variation during the second specified period. Figure 6 In the example shown, a weighted moving average was used for the filter. Figure 6 As shown, the filtered data 204 is smoothed relative to the original data 202. Furthermore, the filtered data 204 has noise components removed or suppressed.

[0089] Figure 7 This is a schematic diagram illustrating an example of time series variation during the second specified period of the interval with the maximum variation. Figure 7 The dashed line shown represents the original data 300 before filter processing, representing the maximum variation in the interval during the second specified period. Figure 7 The solid line shown represents the filtered data 302, representing the maximum variation within the interval during the second specified period. Figure 7 In the example shown, a weighted moving average was used for the filter. Figure 7 As shown, the filtered data 302 is smoothed relative to the original data 300. Furthermore, the filtered data 302 has noise components removed or suppressed.

[0090] In step S114, the determination unit 78 determines the road surface damage area. Specifically, in Figure 6 The data 204 after filtering by the interval average variation shown is above the specified first threshold 212 and Figure 7 If the filtered data 302, which represents the maximum variation in the interval shown, is below the predetermined second threshold 310 (which is greater than the first threshold 212), it is determined that there is a possibility of road surface damage.

[0091] If the filtered data 204, representing the average variation within the interval, is less than the predetermined first threshold 212, it can be determined that the road surface is sufficiently smooth. Furthermore, the risk of potholes or other unevenness is considered low. Conversely, if the filtered data 302, representing the maximum variation within the interval, is greater than the predetermined second threshold 310, which is greater than the first threshold 212, it is presumed that the road surface is uneven due to road construction or is temporarily covered with gravel. Therefore, this situation does not include cases where road surface damage such as potholes has occurred. For example, the first threshold 212 is specifically determined based on the filtered data 204, representing the average variation within the interval, under the condition of a smooth road surface. The second threshold 310 is specifically determined based on the filtered data 302, representing the maximum variation within the interval, under the condition of an uneven road surface due to road construction or other unevenness.

[0092] exist Figure 6 In either case where the change in the average variation of the shown interval relative to the previous day (i.e., the change within a unit of the first specified period) is above the specified change threshold 210, or in either case where the difference between the maximum variation of the interval and the average variation of the interval on the same day (within the same first specified period) within the second specified period is above the specified third threshold, the determination unit 78 determines that a sudden road surface damage has occurred. As an example, the change threshold 210 and the third threshold are specifically determined based on the changes in data when a sudden road surface damage has occurred.

[0093] Furthermore, the determination unit 78 can also determine road surface damage based on the waveform of the interval average variation. For example, if the change in the time series of the interval average variation is continuously increasing or increasing exponentially, it is determined that road surface damage has occurred over time. A continuously increasing change occurs when the curve representing the change in the time series of the interval average variation becomes increasingly convex. A convex curve indicates that the second derivative of the function representing that curve is positive. However, Figure 6 The interval average variation shown is discrete data. Therefore, Figure 6 The average variation over the interval shown is not differentiable. As an example, in this embodiment, an investigation was conducted on... Figure 6 The second derivative of the differentiateable function is obtained by curve fitting of the time series under the interval average variation, and is used to determine whether the curve representing the change under the interval average variation is convex downward.

[0094] Furthermore, if the curve representing the change in the time series of interval average variation is flat, the determination unit 78 determines that the road surface state has not changed. If the curve representing the change in the time series of interval average variation changes in a stepped manner, the determination unit 78 determines that one of the following treatments has been performed: road surface cutting, repair, or repaving. The above waveform-based determination mechanism can be constructed, for example, using deep learning such as LSTM.

[0095] Furthermore, the determination unit 78 can also detect road surface damage based on the difference in wheel speed variation at the left and right wheels of the vehicle 200. Figure 8 This diagram illustrates how road surface damage is detected based on the different wheel speeds of approximately 200 km / h wheels of a vehicle. (For example...) Figure 8 As shown, in the case of road surface damage, the wheel speed variation is larger compared to the case without road surface damage. For example, in this embodiment, road surface damage is determined to exist when the difference between the average of the wheel speed variation of one wheel on each side with respect to multiple vehicles 200 and the average of the wheel speed variation of the other wheel on each side with respect to multiple vehicles 200 during a first predetermined period is a predetermined fourth threshold. Since potholes and other road surface damage are approximately 15cm to 20cm in diameter, they can sometimes only be detected at the wheels of one wheel on each side; therefore, road surface damage is determined to exist as described above. For example, the fourth threshold is specifically determined based on the data changes when either wheel on each side drives onto a pothole or other road surface damage area.

[0096] As explained above, in this embodiment, as an example, road surface damage is detected based on the variation of the wheel speed of vehicle 200. The analysis data used for road surface damage detection is not limited to the variation of the wheel speed of vehicle 200. As mentioned earlier, time-series data of the angular velocity and acceleration of the vehicle 200's attitude angle detected by IMU 26 can also be processed as analysis data. Hereinafter, we will describe the case where, as a data point of angular velocity and acceleration of the vehicle 200's attitude angle when the wheels of vehicle 200 hit potholes in the road surface, data that is easily significantly affected by road surface damage is used, for example, the acceleration of vehicle 200 in the Z-axis direction (the vertical direction of vehicle 200) as analysis data.

[0097] In this case, the statistics department 74 calculates the change in acceleration in the Z-axis direction of the vehicle 200 (hereinafter referred to as "acceleration variation") for each first specified period or each trip of the vehicle 200. As shown in Equation (2) below, the acceleration variation is the absolute value (ABS) of the value after processing the change in acceleration Δ (acceleration) per unit time Δt using a high-pass filter (HPF) to remove noise.

[0098] Acceleration variation = ABS(HPF(Δ(acceleration) / acceleration t))…(2)

[0099] In addition, the statistics department 74 defines the maximum value of acceleration variation as the time series data obtained by selecting the maximum value of multiple vehicles 200 at each time point in the acceleration variation and calculates it.

[0100] Similar to the case of wheel speed variation, the acceleration variation and maximum acceleration variation calculated as described above are correlated with the location (road link information) in the statistics unit 74. Furthermore, road surface condition indicators are calculated. Specifically, the acceleration variation and road interval are associated using the latitude and longitude coordinates included in the road link information. Furthermore, road surface condition indicators are calculated for each evaluation interval. In this embodiment, the road surface condition indicators are the average of the maximum acceleration variation values ​​among the multiple vehicles 200, i.e., the interval average acceleration variation, and the maximum of the maximum acceleration variation values ​​among the multiple vehicles 200, i.e., the interval maximum acceleration variation. Specifically, the interval average acceleration variation is the quotient obtained by dividing the sum of the maximum acceleration variation values ​​of each of the multiple vehicles 200 by the number of the multiple vehicles 200. The interval maximum acceleration variation is the maximum of the maximum wheel speed variation values ​​of all vehicles 200.

[0101] The calculated interval average acceleration variation and interval maximum acceleration variation are processed by the filtering unit to remove or suppress noise components. As mentioned earlier, the filters used are moving average, Gaussian filter, and deep learning filters such as LSTM.

[0102] The result of filter processing is the interval average acceleration variation, such as Figure 6 The filtered data 204, as shown, is smoothed and noise components are removed or suppressed. The maximum acceleration variation in the interval is as follows: Figure 7 The data 302, as shown in the filter, is smoothed and noise components are removed or suppressed.

[0103] Then, the determination unit 78 determines the road damage zone based on the smoothed interval average acceleration variation and interval maximum acceleration variation. Specifically, if the filtered data of the interval average acceleration variation is above a predetermined first threshold and the filtered data of the interval maximum acceleration variation is below a predetermined second threshold that is greater than the first threshold, it is determined that there is a possibility of road damage.

[0104] If the data after filtering the interval average acceleration variation is less than a predetermined first threshold, the road surface can be judged to be in a sufficiently smooth state. Furthermore, the risk of potholes or other road surface damage can be determined to be low. Conversely, if the data after filtering the interval maximum acceleration variation is greater than a predetermined second threshold (which is greater than the first threshold), it is presumed that the road surface is uneven due to road construction or is temporarily covered with gravel. Therefore, this situation does not include cases where road surface damage such as potholes has occurred.

[0105] In either case where the change in the interval average acceleration relative to the previous day (i.e., the change in the first specified period unit) is above a specified change threshold, or in either case where the difference between the interval maximum acceleration change and the interval average acceleration change on the same day (within the same first specified period) within the second specified period is above a specified third threshold, the determination unit 78 determines that a sudden road surface damage has occurred.

[0106] In addition, the determination unit 78 can also determine road surface damage based on the waveform of the interval average acceleration variation. For example, if the change in the interval average variation over a time series is continuously increasing or increases exponentially, it is determined that time-related changes in the road surface have occurred.

[0107] Furthermore, if the curve representing the change in the time series of interval average acceleration is flat, the determination unit 78 determines that the road surface condition has not changed. If the curve representing the change in the time series of interval average acceleration changes in a stepped manner, the determination unit 78 determines that any of the following has been carried out: road surface cutting, repair, or repaving.

[0108] As described above, road surface damage can be detected based on the acceleration of vehicle 200 in the Z-axis direction detected by IMU 26. In addition to the Z-axis acceleration, IMU 26 can also detect acceleration in the X-axis direction (the longitudinal direction of vehicle 200), acceleration in the Y-axis direction (the lateral direction of vehicle 200), angular velocity in the longitudinal, lateral, and yaw directions of vehicle 200. Therefore, road surface damage can also be detected using these variations in acceleration and angular velocity.

[0109] Furthermore, for road surface damage detection, as mentioned earlier, information related to the behavior of the vehicle 200 detected by the steering angle sensor 28, throttle sensor 30, and brake pedal sensor 32 can also be provided. In this case, for example, road surface damage can be detected based on changes in the steering angle of the vehicle 200 detected by the steering angle sensor 28. This is because: if the driver sees road surface damage ahead, in many cases, the driver will attempt to avoid it. Additionally, if the driver sees road surface damage ahead, in many cases, the driver will slow down the vehicle 200. Therefore, road surface damage can also be detected based on changes in the throttle opening direction (detected by the throttle sensor 30) or the brake pedal force direction (detected by the brake pedal sensor 32).

[0110] Figure 9 This is a sequence diagram illustrating an example of the processing of the road damage detection device 100 according to this embodiment. In step S1, data such as wheel speed detected by vehicle speed sensor 24, angular velocity and acceleration of vehicle attitude angle detected by IMU 26, steering angle of vehicle 200 detected by steering angle sensor 28, throttle opening of vehicle 200 detected by throttle sensor 30, and brake pedal force detected by brake pedal sensor 32 are transmitted from multiple vehicles 200.

[0111] In step S2, the communication device 110 receives data transmitted from multiple vehicles 200. In step S3, the received data is transmitted to the data storage device 120.

[0112] In step S4, the data storage device 120 accumulates the data received. In step S5, the data storage device 120 sends the accumulated data to the computing server 10.

[0113] In step S6, the computing server 10 receives the accumulated data. Then, in step S7, the computing server 10 calculates, for example, the wheel speed variation. If the data detected by the IMU 26 is used in the road damage detection, the variation of the angular velocity of the vehicle 200's attitude angle or the variation of the vehicle 200's acceleration is calculated. In addition, the variations of the time-series data detected by the vehicle 200's steering angle sensor 28, throttle sensor 30, or brake pedal sensor 32 are calculated respectively.

[0114] In step S8, the calculation server 10 establishes a correspondence between wheel speed changes and location (road link information). In step S8, a correspondence can also be established between the angular velocity changes of the vehicle 200's attitude angle and location, the acceleration changes of the vehicle 200 and location, and the changes in time series data detected by the vehicle 200's steering angle sensor 28, throttle sensor 30, or brake pedal sensor 32, respectively.

[0115] In step S9, the computing server 10 calculates the road surface condition indicators. Specifically, it uses the latitude and longitude coordinates contained in the road link information obtained in step S8 to establish a correlation between wheel speed changes (or changes in the angular velocity of the vehicle 200's attitude angle, changes in the vehicle 200's acceleration, or changes in the vehicle 200's behavior) and road intervals. Furthermore, the road surface condition indicators are calculated for each evaluation interval.

[0116] In step S10, the computing server 10 generates historical change data. The generated historical change data is saved to an HDD 56 or a data storage device 120, which serves as a storage device.

[0117] In step S11, the computing server 10 performs filter processing to remove or suppress noise components in the epochal variation data that is time series data.

[0118] In step S12, the calculation server 10 determines the road damage area. In step S13, the road damage area data is sent to the data storage device 120.

[0119] In step S14, the data storage device 120 receives pavement damage zone data. In step S15, the received pavement damage zone data is stored.

[0120] In step S16, a pavement damage interval data viewing request is sent from terminal 130. The pavement damage interval data viewing request can also be sent via computing server 10. If pavement damage interval data is stored in the computing server 10's HDD 56 or similar device, the pavement damage interval data viewing request can also be sent to the computing server 10.

[0121] In step S17, the data storage device 120 sends the road damage interval data according to the road damage interval data viewing request from the terminal 130.

[0122] In step S18, terminal 130 receives road damage zone data from data storage device 120.

[0123] As explained above, this embodiment detects road surface damage based on so-called big data obtained from a large number of vehicles 200. In this embodiment, the big data used for road surface damage detection comprises physical quantities related to the behavior of each of the multiple vehicles 200, including variations in wheel speed of the four wheels of the vehicle 200 detected by vehicle speed sensor 24, variations in angular velocity and acceleration of the vehicle 200's attitude angle detected by IMU 26, variations in steering angle of the vehicle 200 detected by steering angle sensor 28, variations in throttle opening detected by throttle sensor 30, and variations in brake pedal force detected by brake pedal sensor 32. In this embodiment, road surface damage is detected by comparing the variations in these physical quantities with predetermined threshold values.

[0124] The raw data of physical quantities such as wheel speed variations calculated from the outputs of various sensors includes noise components that could lead to false detections of road surface damage. In this embodiment, the raw data is smoothed using filters such as moving averages, Gaussian filters, and LSTM, thereby suppressing noise components. Through this filtering process, in this embodiment, significant changes in physical quantities that might be detected due to temporary debris such as gravel, maintenance holes, side ditch covers, railway crossings, or vehicle avoidance behavior are suppressed.

[0125] Furthermore, temporary variations in physical quantities caused by vehicle 200 avoidance maneuvers or objects falling onto the road surface, and permanent variations in physical quantities caused by structures such as maintenance holes, side ditch covers, and railway crossings, are considered to be factors contributing to noise generation. In this embodiment, the influence of noise components during road surface damage detection is suppressed by comparing the physical quantity with multiple different thresholds or studying the waveform changes of the physical quantity over time.

[0126] For example, if the average variation within a range of the filtered data is above a predetermined first threshold and the maximum variation within a range of the filtered data is below a predetermined second threshold, which is greater than the first threshold, it is determined that there is a possibility of road surface damage. The average variation within a range is the average of multiple maximum values ​​obtained from multiple vehicles over a first predetermined period, representing the variation of a detected physical quantity per unit time. The maximum variation within a range is the maximum value among the multiple vehicles 200 representing the variation of a physical quantity indicating the behavior of vehicle 200 over the first predetermined period. The first threshold assumes a smooth road surface. If the variation of the physical quantity indicating the behavior of vehicle 200 is smaller than the first threshold, it can be determined that there is no road surface damage. Furthermore, the second threshold assumes a situation where the road surface is extremely uneven due to road construction, etc. If the variation of the physical quantity indicating the behavior of vehicle 200 is larger than the second threshold, it can be presumed that the variation is caused by road construction rather than road surface damage.

[0127] Furthermore, in either case where the change in the average variation within a first predetermined period of the filtered data is above a predetermined threshold, or where the difference between the maximum variation within the interval and the average variation within the same first predetermined period is above a predetermined third threshold, abrupt road surface damage is determined to have occurred. Thus, in this embodiment, the occurrence of abrupt road surface damage can be detected based on changes in physical quantities representing the behavior of the vehicle 200.

[0128] Furthermore, if the change in the time series of the interval average variation increases in a downward-convex curve, it is determined that the road surface has deteriorated over time. If the change in the time series of the interval average variation is flat, it is determined that the road surface condition has not changed. If the change in the time series of the interval average variation changes in a step-like manner, it is determined that any of the following has been carried out: road surface cutting, repair, or repaving. Thus, in this embodiment, the presence or absence of road surface damage or road surface construction can be determined by the time series change of a physical quantity representing the behavior of the vehicle 200.

[0129] Furthermore, in this embodiment, when the physical quantity representing the behavior of the vehicle 200 is the average value of the wheel speed variations detected independently by the four wheels, road surface damage can be detected based on the differences at the left and right wheels.

[0130] It should be noted that the "physical quantity detection unit" described in the claims corresponds to the "vehicle speed sensor 24," "IMU 26," "steering angle sensor 28," "throttle sensor 30," and "brake pedal sensor 32" described in the invention description. Furthermore, the "calculation and statistics unit" described in the claims corresponds to the "statistics unit 74" described in the invention description. The "road surface damage detection unit" described in the claims corresponds to the "determination unit 78" described in the invention description.

[0131] It should be noted that the processing performed by the CPU reading software (program) in the above embodiments can also be executed by various processors other than the CPU. Examples of processors in this case include field-programmable gate arrays (FPGAs) and other programmable logic devices (PLDs) whose circuit structure can be modified after manufacturing, as well as application-specific integrated circuits (ASICs) and other processors with circuit structures specifically designed for performing specific processes, i.e., dedicated circuits. Furthermore, the processing can be executed by one of these various processors. The processing can also be executed by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit composed of circuit elements such as semiconductor elements.

[0132] Furthermore, while the above embodiments describe a method where the program is pre-stored (installed) on a disk drive 60, etc., this is not a limitation. The program may also be provided as a non-transitory storage medium such as a CD-ROM, DVD-ROM, or USB memory. Alternatively, the program may be downloaded from an external device via a network.

[0133] Note 1

[0134] The information processing device includes a memory and at least one processor connected to the memory.

[0135] The processor is configured to: statistically analyze the changes in physical quantities representing the behavior of multiple vehicles per unit time, the average value (interval average change) within a first specified period, and the maximum value (interval maximum change) within the first specified period, respectively, for a second specified period longer than the first specified period; remove noise components from the statistical results; and detect road damage locations based on the results after removing the noise components.

Claims

1. A road surface damage detection device, comprising: The physical quantity detection department detects physical quantities that represent the individual behaviors of multiple vehicles. The calculation and statistics unit performs statistics on the average value (interval average variation) and the maximum value (interval maximum variation) of the physical quantity detected by the physical quantity detection unit per unit time over a first predetermined period, which is longer than the first predetermined period, respectively, over a second predetermined period. The filtering unit removes noise components from the statistical results of the computational statistics unit; and The road surface damage detection unit detects road surface damage locations based on the results output by the filtering unit.

2. The road surface damage detection device according to claim 1, wherein, The filtering unit uses filters to remove noise components from the statistical results of the computational statistics unit. The filters include moving average, Gaussian filter, and deep learning-based filter.

3. The road surface damage detection device according to claim 2, wherein, If the average variation of the interval included in the result output by the filtering unit is above a predetermined first threshold and the maximum variation of the interval included in the result output by the filtering unit is below a predetermined second threshold that is greater than the first threshold, the road surface damage detection unit determines that there is a possibility of road surface damage.

4. The road surface damage detection device according to claim 3, wherein, If the pavement damage detection unit determines that abrupt pavement damage has occurred in either of the following cases: the change in the average variation of the interval, measured in units of the first specified period, is above a specified threshold; or the difference between the maximum variation of the interval and the average variation of the interval in the second specified period is above a specified third threshold.

5. The road surface damage detection device according to claim 3, wherein, If the change in the time series of the interval average variation in the result output by the filter unit increases in a downward convex curve, the road surface damage detection unit determines that the road surface has deteriorated over time. If the change in the time series of the interval average variation is flat, the road surface condition is determined to be unchanged. If the change in the time series of the interval average variation is stepped, the road surface is determined to have undergone one of the following treatments: cutting construction, repair, and repaving.

6. The road surface damage detection device according to claim 4 or 5, wherein, The physical quantity detection unit is a wheel speed sensor that detects the wheel speed of the vehicle as the physical quantity.

7. The road surface damage detection device according to claim 6, wherein, The wheel speed sensors detect the speed of each of the four wheels of the vehicle. If the difference between the average value of the change in wheel speed per unit time of the left and right wheels with respect to the plurality of vehicles and the average value of the change in wheel speed per unit time of the left and right wheels with respect to the plurality of vehicles during the first specified period is a predetermined fourth threshold or higher, the road surface damage detection unit determines that road surface damage exists.

8. The road surface damage detection device according to claim 4 or 5, wherein, The physical quantity detection unit is an inertial measurement device that detects the angular velocity of the vehicle's attitude angle and the vehicle's acceleration as physical quantities.

9. The road surface damage detection device according to claim 4 or 5, wherein, The physical quantity detection unit is a steering angle sensor that detects the vehicle's steering angle as the physical quantity.

10. The pavement damage detection device according to claim 4 or 5, wherein, The physical quantity detection unit is a throttle sensor that detects the throttle opening, which represents the deceleration of the vehicle, as the physical quantity.

11. The road surface damage detection device according to claim 4 or 5, wherein, The physical quantity detection unit is a brake pedal sensor that detects the force applied to the brake pedal, which represents the deceleration of the vehicle, as the physical quantity.

12. A method for detecting pavement damage, comprising: The process of separately detecting physical quantities representing the individual behaviors of multiple vehicles; The process of statistically analyzing the average value of the variation of the detected physical quantity per unit time within a first specified period, i.e., the interval average variation, and the maximum value within the first specified period, i.e., the interval maximum variation, respectively, for a second specified period that is longer than the first specified period. The process of removing noise components from the statistical results; and The process of detecting road surface damage based on the result after removing the noise components.

13. A storage medium storing a road surface damage detection program that enables a computer to function as a structure comprising: The calculation and statistics department performs statistics on the average value (interval average change) and the maximum value (interval maximum change) of the physical quantity representing the behavior of multiple vehicles per unit time over a first specified period, respectively, over a second specified period that is longer than the first specified period. The filtering unit removes noise components from the statistical results of the computational statistics unit; and The road surface damage detection unit detects road surface damage locations based on the results output by the filtering unit.

Images